Method and device for assisting microscope zoom body assembly, storage medium and program product

Abstract

The invention discloses a method and equipment for assisting microscope zoom body assembly, a storage medium and a program product, and belongs to the field of microscope assembly and image processing. According to the method, the imaging effect is scored by using the deep learning model, and the engineer is guided to assemble the zoom body based on the score, so that the problems of non-uniform judgment standards, incapability of observing nuisance and the like due to the fact that the engineer performs adjustment by relying on naked eye observation at present are solved, the adjustment accuracy can be improved, and the adjustment efficiency can also be improved.

Description

Technical Field

[0001] This invention belongs to the field of microscope assembly and image processing, and specifically relates to a microscope zoom assembly technology. Background Technology

[0002] The zoom imaging system of a microscope is simply called a zoom lens. Assembling a zoom lens involves fitting pre-fabricated lenses into corresponding pre-fabricated structures. Due to the limited machining precision of structural components, it's impossible to fully meet the theoretical design requirements. Therefore, when designing structural components, manually adjustable settings (such as using knobs to adjust the left and right movement of lenses) are usually included to fine-tune the position of each lens within the structure. During assembly, the assembly engineer continuously and manually fine-tunes these settings, slightly moving each lens to achieve the best imaging effect. This "best imaging effect" requires human observation. Typically, a zoom lens contains a dozen or so lenses, and even a slight movement of one lens may necessitate corresponding adjustments to other lenses, a process that usually takes a considerable amount of time.

[0003] The imaging performance of a zoom lens is mainly affected by factors such as spherical aberration, chromatic aberration, and field curvature in the optical imaging system. After the design is finalized, these effects are minimized and optimized during the assembly and calibration process. Figure 1 As shown, (a) is the imaging effect when the calibration is poor. It can be seen that there are obvious color differences and blurring in the surrounding areas of the image. (b) is the imaging effect when the calibration is good, showing a better imaging effect. Existing assembly methods mainly rely on the experience of assembly engineers, who continuously observe the imaging effect in the image with the naked eye during calibration and then manually fine-tune it. The main problems with this method are: first, naked-eye observation is not objective, and subjective evaluation is not standardized; second, when fine-tuning after naked-eye observation, it is difficult to detect subtle differences, thus lacking a good guiding direction for manual fine-tuning; third, based on the first two reasons, the method of fine-tuning based on naked-eye observation consumes a lot of the engineer's time, resulting in a long overall calibration time and low efficiency; fourth, the inaccuracy of naked-eye observation means that the final calibrated zoom lens may not be in its optimal state, thus failing to achieve the best imaging effect. Summary of the Invention

[0004] To address the technical problems mentioned in the background section, this invention proposes a method, apparatus, storage medium, and program product for assisting in the assembly of microscope variadic bodies.

[0005] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows: A method for assisting microscope varistor assembly involves constructing and training a deep learning model to evaluate the imaging quality of images input into the model. During varistor calibration, the camera acquires images in real time and inputs them into the trained deep learning model. The deep learning model scores the imaging quality of each input image, where the score for a given frame is denoted as . S n+1 By calculating the score of the image frame S n+1 Rating of the previous frame S n The difference between two consecutive frames is used to determine whether a change has occurred: if If the two frames show a change, it means that the two images have changed; otherwise, it means that the two images have not changed. The set change threshold is used; if the current frame image changes, the score of the current frame image is recorded as follows. S e1 And so on, record them. S e2 , S e3 , S e4 ...; Let the score of the most recently recorded image that has changed be... S en ,Compare S en , S en-1 , S en-2 ,like If the three values ​​show the same trend, it indicates that the variogram adjustment needs to be continued in the current direction. If the three values ​​show different trends, it indicates that the adjustment of the zoom lens is close to the optimal imaging effect and needs to be changed to a more refined fine-tuning at the current position.

[0006] Furthermore, the deep learning model includes a feature extraction module and a score estimation module connected in sequence. The feature extraction module extracts features from the input image and transmits them to the score estimation module. The score estimation module uses two fully connected layers and outputs a node as the score output.

[0007] Furthermore, N images with different imaging effects are collected, and the imaging effects of these images are scored. Each image is randomly sent to M vision engineers for imaging effect scoring. The highest and lowest scores are removed, and the average of the remaining scores is taken as the final score of the image. The set of images with labeled scores is used as the training set of the deep learning model.

[0008] Furthermore, during the varimax calibration process, the real-time imaging images captured by the camera, the image scores from the deep learning model, and the trends of the scores for three consecutive images that have changed are displayed on the screen.

[0009] Secondly, the present invention also provides a computer device, which includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the above-described method for assembling auxiliary microscope variadic bodies.

[0010] Furthermore, the present invention also provides a computer storage medium storing a computer program that, when executed on a processor, implements the above-described method for assembling auxiliary microscope varistor bodies.

[0011] Finally, the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the above-described method for assembling an auxiliary microscope varistor.

[0012] The beneficial effects of adopting the above technical solution are as follows: This invention utilizes a deep learning model to evaluate imaging performance and provide a score, eliminating reliance on engineers' visual observation experience. The provided score is relatively objective and consistent, remaining unchanged over time or from other external influences. Subtle differences in the image are reflected in the score, and the scoring trend of consecutive frames guides engineers' assembly, significantly improving the efficiency and accuracy of zoom lens calibration. Attached Figure Description

[0013] Figure 1 These are comparison images of the imaging effects under different calibrations; Figure 2 This is a structural diagram of the deep learning model in the embodiment; Figure 3 This is a flowchart of the process for obtaining the model training set in this embodiment; Figure 4 This is a flowchart illustrating how a trained model guides the adjustment of variopy in an embodiment. Detailed Implementation

[0014] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0015] This embodiment proposes a method for assisting in the assembly of microscope zoom bodies. Its basic framework involves using a deep learning model to score imaging performance and then using that score to guide engineers in assembling the zoom body. Therefore, this embodiment mainly introduces two parts: how to construct and train the deep learning model, and how to use the model to guide zoom body calibration.

[0016] like Figure 2As shown, the deep learning model constructs a feature extraction module and a scoring estimation module. The feature extraction module can adopt CNN-based structures such as EfficientNet and MobileNet, or transformer-based structures such as ViT. The scoring estimation module uses two fully connected layers, outputting a node as the scoring output. The goal of the prepared training set is to prepare a large number of images, each with a relatively objective imaging score. On a zoom lens, the manual tuning settings are continuously adjusted to obtain images with different imaging effects (from the worst to the best), thus collecting N images with different imaging effects. To objectively score these images, each image is randomly sent to M vision engineers for imaging effect scoring. Finally, the highest and lowest values ​​are removed, and the average value is taken as the final score of the image. In practice, N=10000 and M=5 can be used. The score can be 0~100, where 0 represents the worst imaging effect and 100 represents the best imaging effect. The score can be a decimal value. The process of obtaining the training set is as follows. Figure 3 As shown, the steps are as follows: S1: Construct a variable-ploid assembly experimental platform.

[0017] S2: Set the initial manual fine-tuning lens settings. This position can be set to the far left of the knob, i.e., when the lens is at the far left. (In the next step, S3, you can move it a little to the right each time you take an image, for example, 20µm).

[0018] S3: Capture the current image and save it.

[0019] S4: Have all fine-tuning settings been captured? If not, manually continue fine-tuning the lens to the next position and continue shooting; if yes, it indicates that images of all situations have been captured.

[0020] S5: End data collection and summarize all collected images.

[0021] S6: Shuffle the images and score them for 5 visual engineers in turn.

[0022] S7: Based on the scoring rules, calculate the average score for each image as the final score. This yields the training set, where each image corresponds to a score.

[0023] The deep learning model is trained using these training set images. The loss function is MSE (Least Squared Error). The training and optimization process follows a relatively standard deep learning model training procedure, such as using Adam as the optimizer, setting the batch size to 8, and the learning rate to 0.001.

[0024] During zoom lens calibration, after the camera acquires an image, in addition to displaying it on a monitor for visual observation, the image is also input into a deep learning model for image scoring estimation. The model outputs the image score, which is displayed on the monitor in real time. Based on the camera's frame rate, images are continuously transmitted to the model for prediction, and the trend of score changes is recorded to better guide the calibration process. Specifically, assuming the score of the nth frame is S... n Then compare the scores S of frames n+1. n+1 With S n The difference is used to determine whether the current image has changed. The image changes only when a certain threshold of change is reached; otherwise, it remains unchanged. The core idea is to remove repetitive, unchanging images and retain only valuable, changing ones. When an image changes, the time S is recorded. n+1 For S e1 And so on, we can obtain S. e2 S e3 S e4 ...; Assuming the latest frame obtained is S en Compared to S en S en-1 S en-2 When these three values ​​show the same trend, it is considered that the adjustment needs to continue towards that trend. The specific method for determining the same trend is: when... When the trend is the same, it indicates that the current manual adjustment needs to continue in the current direction. The different trends indicate that the current manual adjustment is approaching the optimal imaging position and requires finer adjustments at the current position. This trend can assist engineers in making manual adjustments, indicating the direction of adjustment. The complete process described above is as follows: Figure 4 As shown, the steps are as follows: S1: Build a variable-sized body assembly experimental platform and deploy the trained deep learning model.

[0025] S2: The camera captures images.

[0026] S3: The image is input into the deep learning model to obtain a score.

[0027] S4: Is the difference between the current frame score and the previous frame greater than [value missing]? If no, it means the current frame image is no different from the previous frame, and we continue to return to S2 to collect images; if yes, we proceed to the next step.

[0028] S5: Record the current frame score to S en in sequence.

[0029] S6: Calculation .

[0030] S7: Displays trend information and ratings on the screen.

[0031] S8: Manually determine if the score is optimal. If yes, end the process; otherwise, manually adjust the lens based on the information in S7 and repeat the above cycle until the optimal position is found.

[0032] This embodiment also relates to a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for assembling an auxiliary microscope zoom lens. This embodiment also relates to a computer storage medium storing a computer program, which, when executed on a processor, implements the above-described method for assembling an auxiliary microscope zoom lens. This embodiment also relates to a computer program product, including a computer program or instructions, which, when executed by a processor, implements the above-described method for assembling an auxiliary microscope zoom lens.

[0033] The embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for assembling an auxiliary microscope zoom body, characterized in that: A deep learning model is constructed and trained to evaluate the imaging quality of images input into the model. During zoom adjustment, the camera acquires images in real time and inputs them into the trained deep learning model. The deep learning model scores the imaging quality of each input image. Let the score of a certain frame be denoted as . S n+1 By calculating the score of the image frame S n+1 Rating of the previous frame S n The difference between two consecutive frames is used to determine whether a change has occurred: if If the two frames show a change, it means that the two images have changed; otherwise, it means that the two images have not changed. The set change threshold is used; if the current frame image changes, the score of the current frame image is recorded as follows. S e1 And so on, record them. S e2 , S e3 , S e4 ...; Let the score of the most recently recorded image that has changed be... S en ,Compare S en , S en-1 , S en-2 ,like If the three values ​​show the same trend, it indicates that the variogram adjustment needs to be continued in the current direction. If the three values ​​show different trends, it indicates that the adjustment of the zoom lens is close to the optimal imaging effect and needs to be changed to a more refined fine-tuning at the current position.

2. The method for assembling an auxiliary microscope zoom body according to claim 1, characterized in that: The deep learning model includes a feature extraction module and a score estimation module connected in sequence. The feature extraction module extracts features from the input image and sends them to the score estimation module. The score estimation module uses two fully connected layers and outputs a node as the score output.

3. The method for assembling an auxiliary microscope zoom body according to claim 1, characterized in that: N images with different imaging effects are collected. The imaging effects of these images are scored. Each image is randomly sent to M vision engineers to score the imaging effects. The highest and lowest scores are removed, and the average of the remaining scores is taken as the final score of the image. The set of images with the scores is used as the training set of the deep learning model.

4. The method for assembling an auxiliary microscope zoom body according to claim 1, characterized in that: During the zoom calibration process, the real-time images captured by the camera, the image scores from the deep learning model, and the trends of the scores for three consecutive images that have changed are displayed on the screen.

5. A computer device comprising a processor and a memory, the memory storing a computer program, the processor being configured to execute the computer program to implement the method for assembling an auxiliary microscope variadic body as described in any one of claims 1-4.

6. A computer storage medium storing a computer program, which, when executed on a processor, implements the method for assembling an auxiliary microscope varistor as described in any one of claims 1-4.

7. A computer program product comprising a computer program or instructions that, when executed by a processor, implement the method for assembling an auxiliary microscope varistor as described in any one of claims 1-4.

Images