Microscope camera imaging out-of-focus detection method and system based on a calibration plate

Abstract

This invention relates to the field of image analysis and processing technology, and discloses a method and system for detecting defocus in microscope camera imaging based on a calibration plate. This addresses the technical problem that existing methods for determining whether defocus has occurred lack sufficient automation and accuracy. The method includes: capturing images of the calibration plate at different distances using a microscope camera to obtain several image samples; calculating a focus index sample for each image sample at a first target region; constructing an initial relationship model between the focus index value and the shooting position, and fitting this model based on the focus index samples and their corresponding distances to obtain a focus relationship model; calculating a focus threshold within the depth of field based on the focus relationship model; during calibration, capturing images of the calibration plate using the microscope camera to obtain a calibration image; calculating the calibration focus index value at a second target region of the calibration image; and determining whether the calibration focus index value is within the focus threshold range. If not, a defocus condition has occurred.

Description

Technical Field

[0001] This invention relates to the field of image analysis and processing technology, and in particular to a method, system, electronic device, computer storage medium, and computer program product for detecting defocus in microscope camera imaging based on a calibration plate. Background Technology

[0002] Optical calibration is the process of determining the input-output relationship of an optical imaging device (such as a camera). Its main purpose is to obtain the parameters of the camera's imaging geometric model, thereby acquiring information such as the correspondence between points in three-dimensional space and points in two-dimensional images. As it is the initial step in visual recognition and detection technology, it is the foundation and prerequisite for precision vision applications and a key technology for high-end applications such as precision measurement, sensor fusion, and 3D reconstruction. Without high-precision calibration, subsequent visual processing cannot be carried out.

[0003] For example, in the wafer inspection process of semiconductor manufacturing, precision vision inspection technology is required to ensure product quality and process consistency. Before performing specific wafer inspection, a calibration board is needed to determine the imaging parameters of the camera, and the accuracy of the calibration directly determines the reliability of subsequent inspection, measurement and analysis.

[0004] Achieving high-precision calibration requires high-standard calibration board fabrication, high-quality optical system imaging, and high-performance image algorithms. In addition, defocus detection is also a core step in the calibration process. Its purpose is to accurately measure and compensate for focus shifts caused by changes in object distance, temperature drift, or mechanical vibration, ensuring that the imaging plane remains consistent with the surface of the object to be inspected, thereby improving the accuracy of calibration and ensuring the accuracy of subsequent visual processing.

[0005] In existing technologies, out-of-focus issues frequently occur during calibration, leading to poor image quality and affecting calibration accuracy. While some out-of-focus detection methods exist, such as adjusting the camera position with high-precision movement until the image is clear, or training a built-in out-of-focus recognition model using manually labeled image samples, these methods are computationally complex and cannot achieve highly automated out-of-focus detection; furthermore, the detection results are not accurate enough.

[0006] Therefore, there is an urgent need for a detection method that can accurately determine whether the image of a microscope camera is out of focus, so as to automatically and accurately detect the focusing effect during visual recognition detection. Summary of the Invention

[0007] The main objective of this invention is to solve the technical problem in the prior art that when performing optical calibration, it is necessary to determine whether the calibration image is out of focus, but the automation and accuracy of the scheme for determining whether the image is out of focus are insufficient, which in turn leads to insufficient accuracy in subsequent calibration.

[0008] The first aspect of this invention provides a method for detecting defocus in microscope camera imaging based on a calibration plate, comprising:

[0009] The calibration plate was photographed at different distances using a microscope camera, resulting in several image samples.

[0010] Calculate the focus index value at the first target region of each image sample to obtain a focus index sample;

[0011] An initial relationship model between focus index values ​​and shooting positions is constructed, and the initial relationship model is fitted based on the focus index samples and the distance positions corresponding to the focus index samples to obtain a focus relationship model;

[0012] Based on the focusing relationship model, the focusing threshold is calculated within the depth of field of the microscope camera;

[0013] In response to the microscope camera imaging calibration request, the microscope camera is invoked to capture a calibration image of the calibration plate;

[0014] Calculate the calibration focus index value at the second target region of the calibration image;

[0015] Determine whether the calibrated focus index value is within the focus threshold range. If yes, the microscope camera image is not out of focus. If no, the microscope camera image is out of focus.

[0016] Optionally, in a first implementation of the first aspect of the present invention, after taking pictures of the calibration plate at different distance positions using a microscope camera to obtain several image samples, the method further includes:

[0017] Each image sample is divided into multiple image area regions, and the image center area region containing the image midpoint and the area regions containing the four corners of the image are extracted as the first target region.

[0018] After responding to the microscope camera imaging calibration request and calling the microscope camera to capture a calibration image of the calibration board, the method further includes:

[0019] Each of the calibration images is divided into multiple image area regions, and the image center area region containing the image midpoint and the area regions containing the four corners of the image are extracted as the second target region.

[0020] Optionally, in a second implementation of the first aspect of the present invention, calculating the focus index value at the first target region of each of the image samples includes:

[0021] Corner detection is performed on the image information within each of the first target regions, and the feature response values ​​of the image corners contained within each of the first target regions are calculated respectively;

[0022] Calculate the average value of the image corner feature response values ​​in each of the first target regions, and use the average value as the focus index value of the first target region.

[0023] Optionally, in a third implementation of the first aspect of the present invention, the step of using a microscope camera to photograph the calibration plate at different distances to obtain several image samples includes:

[0024] The step size coefficient is obtained based on the required computational accuracy, the depth of field of the microscope camera is obtained, and the movement step size is calculated based on the step size coefficient and the depth of field.

[0025] The distance between the microscope camera and the calibration plate is adjusted based on the moving step size, and an image is captured after each distance adjustment to obtain several image samples.

[0026] Optionally, in a fourth implementation of the first aspect of the present invention, the construction of the initial relationship model between the focus index value and the shooting position includes:

[0027] Based on the Gaussian distribution, a relationship function between the focus index value and the shooting position is constructed to obtain an initial relationship model;

[0028] The step of fitting the initial relationship model based on the focus index samples and the distance positions corresponding to the focus index samples to obtain the focus relationship model includes:

[0029] Obtain and set the initial values ​​of the model parameters in the initial relationship model. Based on the focusing index sample and the corresponding distance position, call the solver to perform optimization solution near the initial values ​​to obtain the fitted model parameters.

[0030] The focusing relationship model is obtained based on the fitted model parameters.

[0031] Optionally, in a fifth implementation of the first aspect of the present invention, the mathematical expression of the initial relational model is:

[0032] ;

[0033] in, This represents the focus index value at the first target region of the image sample. The area code representing the first target area. Indicates the coking index value The range of variation, This represents the distance parameter between the microscope camera and the calibration plate when capturing the image sample. Indicates the location of the Gaussian extremum. express The weighted mean square error, It is a constant.

[0034] The second aspect of the present invention provides a microscope camera imaging defocus detection system based on a calibration plate, comprising: an image acquisition module, used to call the microscope camera to take pictures of the calibration plate at different distance positions to obtain several image samples;

[0035] The index calculation module is used to calculate the focus index value at the first target region of each image sample to obtain a focus index sample.

[0036] The model fitting module is used to construct an initial relationship model between the focus index value and the shooting position, and to fit the initial relationship model based on the focus index sample and the distance position corresponding to the focus index sample to obtain the focus relationship model.

[0037] The threshold calculation module is used to calculate the focus threshold within the depth of field of the microscope camera based on the focus relationship model.

[0038] The image acquisition module is also used to respond to the microscope camera imaging calibration request by calling the microscope camera to take pictures of the calibration plate to obtain a calibration image;

[0039] The index calculation module is also used to calculate the calibration focus index value at the second target region of the calibration image;

[0040] The defocusing detection module is used to determine whether the calibrated focus index value is within the focus threshold range. If it is, the current microscope camera image is not in a defocused state; otherwise, the current microscope camera image is in a defocused state.

[0041] A third aspect of the present invention provides a microscope camera imaging defocus detection device based on a calibration plate, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the microscope camera imaging defocus detection device based on the calibration plate to perform the steps of the microscope camera imaging defocus detection method based on the calibration plate described above.

[0042] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described calibration plate-based microscope camera imaging defocus detection method.

[0043] A fifth aspect of the present invention provides a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the calibration plate-based microscope camera imaging defocus detection method described above.

[0044] The technical solution provided by this invention involves using a microscope camera to capture images of a calibration board at different distances, obtaining several image samples; calculating the focus index value at the first target region of each image sample, obtaining focus index samples; constructing an initial relationship model between the focus index value and the shooting position, and fitting the initial relationship model based on the focus index samples and their corresponding distance positions, obtaining a focus relationship model; calculating a focus threshold within the depth of field of the microscope camera based on the focus relationship model; responding to a microscope camera imaging calibration request, capturing images of the calibration board to obtain a calibration image; calculating the calibration focus index value at the second target region of the calibration image; and determining whether the calibration focus index value is within the focus threshold range. If it is, the current microscope camera image is not out of focus; otherwise, the current microscope camera image is out of focus. This method can improve the automation and accuracy of detecting camera defocus during optical calibration.

[0045] Furthermore, the system, electronic device, computer-readable storage medium, and computer program product provided by this invention are also used to solve corresponding technical problems. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 This is a schematic flowchart of an embodiment of the microscope camera imaging defocus detection method based on a calibration plate according to the present invention.

[0048] Figure 2 This is a schematic diagram of image partitioning in an embodiment of the microscope camera imaging defocus detection method based on a calibration plate according to the present invention.

[0049] Figure 3 This is a schematic diagram of an embodiment of the microscope camera imaging defocus detection system based on a calibration plate according to the present invention;

[0050] Figure 4 This is a schematic diagram of an embodiment of a microscope camera imaging defocus detection device based on a calibration plate according to the present invention;

[0051] Figure 5This is a schematic diagram illustrating the principle of a computer-readable medium according to an embodiment of the present invention. Detailed Implementation

[0052] Exemplary embodiments of the invention will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the drawings denote the same or similar elements, components, or parts, and therefore repeated descriptions of them will be omitted.

[0053] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0054] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.

[0055] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0056] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0057] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0058] Please see Figure 1 and Figure 2 An embodiment of the microscope camera imaging defocus detection method based on a calibration plate in this invention is as follows:

[0059] It is understood that the executing entity of this invention can be a microscope camera imaging defocus detection system based on a calibration plate, or it can be a terminal or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0060] The solution in this embodiment can be specifically applied to the calibration of microscope cameras to detect whether the acquired image is out of focus during the calibration process, thereby avoiding poor calibration results caused by calibration under out-of-focus conditions. "In focus" means that the concentric beams emitted from the object point, after passing through the optical system, precisely converge on the imaging plane (such as a camera sensor) to form an ideal image point with geometric dimensions approaching the diffraction limit. At this point, the system satisfies the object-image conjugate relationship in Gaussian optics, achieving faithful transmission of geometric and radiometric information. "Out of focus" means that the beams emitted from the object point, after passing through the optical system, have an axial deviation between their convergence point (or divergence point) and the imaging plane, resulting in a diffuse spot with a certain diameter on the imaging plane. This state disrupts the object-image conjugate relationship, causing attenuation of spatial frequency information and confusion of neighboring information. Since microscope systems are precision vision applications, their application scenarios are crucial. Furthermore, due to their very small depth of field and extremely short working distance, their focusing tolerance is very low; even slight out-of-focus can lead to data failure. Therefore, this embodiment requires a high degree of accuracy in detecting out-of-focus conditions. Furthermore, due to the short working distance of the microscope, even minor influences can affect the imaging distance of the calibration plate and the camera, leading to inaccurate calibration.

[0061] Therefore, to achieve highly automated detection of out-of-focus conditions, it is necessary to obtain a defocus feature threshold and determine whether the acquired image feature values ​​fall within the defocus feature threshold range to identify whether a defocused condition has occurred. Accordingly, before the imaging system performs specific optical calibration steps, it is necessary to collect sample data of the current imaging system and calculate the image feature threshold under focused conditions (hereinafter referred to as the focus threshold). Each time calibration is required, the defocus detection method described in this embodiment can be used to obtain the focus threshold and perform calibration, thereby improving the accuracy of the image detection system.

[0062] For example, changes in camera lens, lighting conditions, or slight vibrations can all alter the imaging system. In such cases, it is necessary to automatically perform focus threshold detection in order to correct the specific parameters of the out-of-focus detection scheme in this embodiment.

[0063] S101. Use the microscope camera to take pictures of the calibration plate at different distances and positions to obtain several image samples;

[0064] In this embodiment, before acquiring image samples, the microscope camera is first adjusted to a preset position. The camera and calibration plate are kept horizontal. The relative positions of the calibration plate and the camera are fixed so that the calibration plate fills the entire image range of the camera's field of vision. Adjust the vertical direction between the camera and the calibration plate. The distance is such that the calibration plate is roughly at the center of the camera's depth of field to produce a clear image; this is the preset position.

[0065] Then, taking the current position of the camera perpendicular to the calibration plate in the z-direction as a reference, the depth of field in the camera's z-direction is denoted as... (Units can be) (With the same step size) The camera moves vertically to take pictures, and at the same time, the z-axis reading on the camera's movement axis is recorded. The z-axis reading is used as the coordinates of the camera's shooting position.

[0066] Taking pictures and samples based on the current position of the camera perpendicular to the calibration plate, and then taking... Using the step size as a guide, the system moves several times from the reference position in both the positive and negative z-axis directions to take pictures and sample images, performing a total of n samplings to obtain n sample images. Simultaneously, the z-axis position reading corresponding to each sampling is recorded. ( The camera can be a microscope camera with a depth of field at the micrometer level. This represents the step size coefficient. The smaller the value, the higher the accuracy of the defocus feature threshold obtained from subsequent fitting and calculation; however, as the step size coefficient increases... Decreasing the step size increases the number of steps required for taking photos, leading to higher shooting costs and computational burden, and may also cause data overfitting. Therefore, in a preferred embodiment, the step size coefficient... The value range is from 0.05 to 0.1.

[0067] During the acquisition of sample images, the travel distance of the camera in the z-direction can be measured by... This means that, to ensure that sampling simultaneously covers both in-focus images within the depth of field and out-of-focus images outside the depth of field, the sampling distance in a single sampling operation should be at least three times the camera's depth of field, i.e.:

[0068] ;

[0069] It can be seen that, ;when When the value ranges from 0.05 to 0.1, it can be seen that at least 60 to 120 sample images are required in one sampling process.

[0070] S102. Calculate the focus index value at the first target region of each image sample to obtain the focus index sample.

[0071] In this embodiment, each image sample is divided into multiple image area regions, and the image center area region containing the image midpoint and the area regions containing the four corners of the image are extracted as the first target region.

[0072] For each sample image, let its image size be denoted as . ,in Indicates the image width. This represents the image height. First, the image is divided into regions, with... Indicates the average number of divisions in the width direction. The image can be divided into the average number of divisions along the height direction. There are 10 regions, and the width of each region is 100. The height of each area is Each region is numbered in pairs, and each partitioned region is represented as follows: ,in Take an integer value, with a range of 100%. , .

[0073] Please see Figure 3 ,by , To illustrate the situation, the area is divided into 9 regions. Each region is numbered using pairs of numbers, and each region is represented as... ,in, , In this embodiment, the top left corner of the image is selected. Top right Bottom left Bottom right and the middle Five regions were selected as the first target regions for corner detection and subsequent feature value calculation.

[0074] Because the edges of the image are more prone to out-of-focus issues due to lens distortion and other factors, the proposed solution uses a method of selecting specific regions for focus detection. Compared to the commonly used method of calculating the focus index statistics for the entire image, this method of dividing the image into regions and detecting them separately can effectively avoid misjudgments of local out-of-focus issues caused by sample or lens tilt, while also reducing the amount of data computation and increasing computational efficiency.

[0075] For each of the five sampled first target regions, corner detection is first performed within the region. The Harris corner feature response value at each corner coordinate is calculated, and the average value of the corner feature response values ​​at all corner positions is taken, denoted as . ,in, Indicates the area code. ; and will As the present A set of indices for the focus of the image at the axial coordinates. The average corner feature response value is calculated for each image sample to obtain different... The set of focus index samples corresponding to axis coordinates ;in, Indicates the number of the image sample. The number of sample images collected. ,and Take the integer part; For area code, ,and Take the integer part.

[0076] S103. Construct an initial relationship model between the focus index value and the shooting position, and fit the initial relationship model based on the focus index sample and the distance position corresponding to the focus index sample to obtain the focus relationship model.

[0077] Due to the characteristics of camera imaging and focusing, the focus index values ​​for the same area at different shooting distances follow a Gaussian distribution. Therefore, based on the Gaussian distribution, a relationship function between the focus index value and the shooting position is constructed to obtain an initial relationship model. The mathematical expression of the initial relationship model is as follows:

[0078]

[0079] in, This represents the focus index value at the first target region of the image sample. The area code representing the first target area. Indicates the coking index value The range of variation, This represents the distance parameter between the microscope camera and the calibration plate when capturing the image sample. Indicates the location of the Gaussian extremum. express The weighted mean square error, It is a constant.

[0080] Based on the focus index values ​​obtained above for different shooting distances The above expression is fitted to solve for the model parameters, where the parameters to be solved are as follows: , , , .

[0081] In one specific implementation, when solving for the model parameters, initial values ​​of the model parameters in the initial relationship model are obtained and set. Based on the focusing index sample and the corresponding distance position, the solver is called to perform optimization solving near the initial values ​​to obtain the fitted model parameters. The focusing relationship model is then obtained based on the fitted model parameters.

[0082] The specific method for setting the initial value is as follows: Indicates the coking index value The range of variation, i.e., the initial value ;when That is, when it is at a Gaussian extreme value, There are constants initial value ; express The weighted average has an initial value. ; express The weighted mean squared error has an initial value. After setting the initial parameter values, you can use a solver (such as the Levenberg-Marquardt solver) to perform optimization around the initial values.

[0083] This invention estimates the initial values ​​of Gaussian parameters using input sample data, and then uses a nonlinear optimization solver to optimize the solution near the initial values. The introduction of the initial values ​​reduces the number of iterations in the solver, increases the accuracy of the solution parameters, and enables the solution to be optimized. The Gaussian expression obtained by solving for values ​​that are not equal to zero is more general.

[0084] S104. Based on the focus relationship model, calculate the focus threshold within the depth of field of the microscope camera.

[0085] In solving about After establishing the relation, only images within the camera's depth of field are accepted as in-focus images. Therefore, given the camera's depth of field... Solve At that time, the boundary values ​​of the five focus thresholds corresponding to the five sampling regions are:

[0086] ;

[0087] Wherein, the focus threshold is greater than or equal to the boundary value. .

[0088] S105. In response to the microscope camera imaging calibration request, the microscope camera is called to take a picture of the calibration plate to obtain a calibration image, and the calibration focus index value at the second target area of ​​the calibration image is calculated.

[0089] The above steps are the steps for automatically calculating the focus threshold in this embodiment. Once the automatically calculated focus threshold is obtained, the image can be automatically calibrated based on the focus threshold.

[0090] When image calibration is required, the camera is used to capture an image of the calibration board on the plane to be calibrated. The image is then divided into regions. Based on the partitioning description in the previous steps, each calibration image is divided into multiple image area regions. The image center area region containing the image midpoint and the area regions containing the four corners of the image are extracted as the second target region.

[0091] Specifically, its image size can be recorded as follows: Where W represents the image width and H represents the image height. First, the image is divided into regions, with... Indicates the average number of divisions in the width direction. The image can be divided into the average number of divisions along the height direction. Each region. Similarly, with , Taking an image divided into nine parts as an example, the regions at the four corners and the middle part of the image are selected as the second target regions. Corner detection is performed on each second target region, and the average value of the corner feature response values ​​at all corner locations is calculated as the calibration focus index value, denoted as... .

[0092] S106. Determine whether the calibrated focus index value is within the focus threshold range;

[0093] S107. If so, then the current microscope camera image is not in focus.

[0094] S108. If not, the current microscope camera image is out of focus;

[0095] If it exists And all of them have If the focus index value of each region in the acquired calibration image is greater than the corresponding focus threshold, the current calibration image is considered to be in focus, and the subsequent calibration process is initiated. The camera parameters are calibrated using the position parameters corresponding to the calibration image, and subsequent image detection is performed based on the calibrated parameters.

[0096] If the calibration focus index value is not within the focus threshold range, the image is considered out of focus. The position parameters or camera parameters need to be adjusted, the calibration image needs to be reacquired, and out-of-focus detection and subsequent calibration need to be performed.

[0097] The solution provided in this embodiment of the invention can improve the automation and accuracy of detecting camera defocus during optical calibration. Furthermore, this solution avoids misjudgment of local defocus by partitioning the checkerboard calibration board and calculating the response values ​​of feature values ​​at the corner positions within each partition. The partitioning and averaging of feature values ​​comprehensively considers the differences in pixel values, resulting in better robustness of the detection. Moreover, this solution, through establishing and fitting a mathematical model, has lower requirements for the movement accuracy of the microscope camera, reducing costs, shortening calibration time, and improving calibration accuracy. The collected samples do not need to be pre-labeled with defocus judgment tags, and model training and parameter tuning are unnecessary; instead, data is automatically acquired for training and inference to obtain the defocus detection threshold and make a judgment, resulting in a high degree of automation.

[0098] The above describes the method for detecting defocus in microscope camera imaging based on a calibration plate in embodiments of the present invention. The following describes the system for detecting defocus in microscope camera imaging based on a calibration plate in embodiments of the present invention. Please refer to [link to documentation]. Figure 3 One embodiment of the microscope camera imaging defocus detection system based on a calibration plate in this invention includes:

[0099] The image acquisition module 301 is used to call the microscope camera to take pictures of the calibration plate at different distances and positions to obtain several image samples;

[0100] The index calculation module 302 is used to calculate the focus index value at the first target region of each image sample to obtain a focus index sample.

[0101] The model fitting module 303 is used to construct an initial relationship model between the focus index value and the shooting position, and to fit the initial relationship model based on the focus index sample and the distance position corresponding to the focus index sample to obtain the focus relationship model.

[0102] The threshold calculation module 304 is used to calculate the focus threshold within the depth of field of the microscope camera based on the focus relationship model.

[0103] The image acquisition module 301 is also used to respond to the microscope camera imaging calibration request by calling the microscope camera to take pictures of the calibration plate to obtain a calibration image;

[0104] The index calculation module 302 is also used to calculate the calibration focus index value at the second target region of the calibration image;

[0105] The defocus judgment module 305 is used to determine whether the calibrated focus index value is within the focus threshold range. If it is, the current microscope camera image is not in a defocus state; otherwise, the current microscope camera image is in a defocus state.

[0106] The system provided in this embodiment of the invention can achieve the technical effect of improving the automation and accuracy of detecting camera defocus during optical calibration.

[0107] In another embodiment of this application, the index calculation module 302 is further configured to divide each of the image samples into multiple image area regions, extract the image center area region containing the image midpoint and the area regions containing the four corners of the image as a first target region; and to divide each of the calibration images into multiple image area regions, extract the image center area region containing the image midpoint and the area regions containing the four corners of the image as a second target region.

[0108] In another embodiment of this application, the index calculation module 302 is further configured to perform corner detection on the image information in each of the first target areas and calculate the image corner feature response values ​​contained in each of the first target areas respectively;

[0109] Calculate the average value of the image corner feature response values ​​in each of the first target regions, and use the average value as the focus index value of the first target region.

[0110] In another embodiment of this application, the image acquisition module 301 is further configured to:

[0111] The step size coefficient is obtained based on the required computational accuracy, the depth of field of the microscope camera is obtained, and the movement step size is calculated based on the step size coefficient and the depth of field.

[0112] The distance between the microscope camera and the calibration plate is adjusted based on the moving step size, and an image is captured after each distance adjustment to obtain several image samples.

[0113] In another embodiment of this application, the model fitting module 303 is specifically used for:

[0114] Based on the Gaussian distribution, a relationship function between the focus index value and the shooting position is constructed to obtain an initial relationship model;

[0115] Obtain and set the initial values ​​of the model parameters in the initial relationship model. Based on the focusing index sample and the corresponding distance position, call the solver to perform optimization solution near the initial values ​​to obtain the fitted model parameters.

[0116] The focusing relationship model is obtained based on the fitted model parameters.

[0117] In another embodiment of this application, the mathematical expression of the initial relational model is:

[0118] ;

[0119] in, This represents the focus index value at the first target region of the image sample. The area code representing the first target area. Indicates the coking index value The range of variation, This represents the distance parameter between the microscope camera and the calibration plate when capturing the image sample. Indicates the location of the Gaussian extremum. express The weighted mean square error, It is a constant.

[0120] The system provided in this invention avoids local defocusing misjudgments by partitioning and sampling the checkerboard calibration board and calculating the response values ​​of feature values ​​at the corner positions within each partition. Furthermore, by partitioning and averaging the feature values, the differences in pixel values ​​are comprehensively considered, resulting in better detection robustness. Moreover, this solution, through the establishment and fitting of a mathematical model, has low requirements for the movement accuracy of the microscope camera, reducing costs, shortening the calibration time, and improving calibration accuracy. The collected samples do not need to be pre-labeled with defocusing judgment labels, and there is no need for model training and parameter tuning. Instead, the system automatically acquires data, performs training and inference, obtains the defocusing detection threshold, and makes a judgment, resulting in a high degree of automation.

[0121] Based on the same inventive concept, this specification also provides an electronic device for detecting defocus in microscope camera imaging based on a calibration plate. The electronic device for detecting defocus in microscope camera imaging based on a calibration plate in this embodiment of the invention will be described in detail below from the perspective of hardware processing.

[0122] Figure 4 This is a schematic diagram of an electronic device provided as an embodiment of this specification. Refer to the following... Figure 4 To describe the electronic device 400 according to this embodiment of the invention. Figure 4 The electronic device 400 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0123] like Figure 4 As shown, the electronic device 400 is presented in the form of a general-purpose computing device. The components of the electronic device 400 may include, but are not limited to: at least one processing unit 410, at least one storage unit 420, a bus 430 connecting different system components (including storage unit 420 and processing unit 410), a display unit 440, etc.

[0124] The storage unit stores program code that can be executed by the processing unit 410, causing the processing unit 410 to perform the steps described in the processing method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 410 can perform, for example... Figure 1 The steps are shown.

[0125] The storage unit 420 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 4201 and / or a cache storage unit 4202, and may further include a read-only memory unit (ROM) 4203.

[0126] The storage unit 420 may also include a program / utility 4204 having a set (at least one) program module 4205, such program module 4205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0127] Bus 430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0128] Electronic device 400 can also communicate with one or more external devices 100 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 400, and / or with any device that enables electronic device 400 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 450. Furthermore, electronic device 400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 460. Network adapter 460 can communicate with other modules of electronic device 400 via bus 430. It should be understood that, although... Figure 4 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0129] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described in this invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the method described above according to this invention. When the computer program is executed by a data processing device, it enables the computer-readable medium to implement the method described above, i.e.: as... Figure 1 The steps of the method shown.

[0130] Figure 5 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification.

[0131] accomplish Figure 1 The computer program of the method shown can be stored on one or more computer-readable media. A computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0132] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0133] In addition, the present invention also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the calibration plate-based microscope camera imaging defocus detection method as described in any of the above embodiments.

[0134] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0135] In summary, this invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the invention. The invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the invention can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0136] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0137] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0138] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting defocus in microscope camera imaging based on a calibration plate, characterized in that, include: The calibration plate was photographed at different distances using a microscope camera, resulting in several image samples. Each image sample is divided into multiple image area regions, and the image center area region containing the image midpoint and the area regions containing the four corners of the image are extracted as the first target region. Calculate the focus index value at the first target region of each image sample to obtain a focus index sample; An initial relationship model between focus index values ​​and shooting positions is constructed, and the initial relationship model is fitted based on the focus index samples and the distance positions corresponding to the focus index samples to obtain a focus relationship model; Based on the focusing relationship model, the focusing threshold is calculated within the depth of field of the microscope camera; In response to the microscope camera imaging calibration request, the microscope camera is invoked to capture a calibration image of the calibration plate; Each of the calibration images is divided into multiple image area regions, and the image center area region containing the image midpoint and the area regions containing the four corners of the image are extracted as the second target region. Calculate the calibration focus index value at the second target region of the calibration image; Determine whether the calibrated focus index value is within the focus threshold range. If yes, the microscope camera image is not out of focus. If no, the microscope camera image is out of focus.

2. The method for detecting defocus in microscope camera imaging based on a calibration plate according to claim 1, characterized in that, The calculation of the focus index value at the first target region of each of the image samples includes: Corner detection is performed on the image information within each of the first target regions, and the feature response values ​​of the image corners contained within each of the first target regions are calculated respectively; Calculate the average value of the image corner feature response values ​​in each of the first target regions, and use the average value as the focus index value of the first target region.

3. The method for detecting defocus in microscope camera imaging based on a calibration plate according to claim 1, characterized in that, The process of using a microscope camera to photograph the calibration plate at different distances and positions to obtain several image samples includes: The step size coefficient is obtained based on the required computational accuracy, the depth of field of the microscope camera is obtained, and the movement step size is calculated based on the step size coefficient and the depth of field. The distance between the microscope camera and the calibration plate is adjusted based on the moving step size, and an image is captured after each distance adjustment to obtain several image samples.

4. The method for detecting defocus in microscope camera imaging based on a calibration plate according to claim 1, characterized in that, The initial relationship model between the focus index value and the shooting position includes: Based on the Gaussian distribution, a relationship function between the focus index value and the shooting position is constructed to obtain an initial relationship model; The step of fitting the initial relationship model based on the focus index samples and the distance positions corresponding to the focus index samples to obtain the focus relationship model includes: Obtain and set the initial values ​​of the model parameters in the initial relationship model. Based on the focusing index sample and the corresponding distance position, call the solver to perform optimization solution near the initial values ​​to obtain the fitted model parameters. The focusing relationship model is obtained based on the fitted model parameters.

5. The method for detecting defocus in microscope camera imaging based on a calibration plate according to claim 1, characterized in that, The mathematical expression for the initial relational model is: ； in, This represents the focus index value at the first target region of the image sample. The area code representing the first target area. Indicates the coking index value The range of variation, This represents the distance parameter between the microscope camera and the calibration plate when capturing the image sample. Indicates the location of the Gaussian extremum. express The weighted mean square error, It is a constant.

6. A microscope camera imaging defocus detection system based on a calibration plate, characterized in that, The calibration plate-based microscope camera imaging defocus detection system includes: The image acquisition module is used to call the microscope camera to take pictures of the calibration plate at different distances and positions to obtain several image samples; The index calculation module is used to calculate the focus index value at the first target region of each image sample to obtain the focus index sample; divide each image sample into multiple image area regions, and extract the image center area region containing the image midpoint and the area regions containing the four corners of the image as the first target region. The model fitting module is used to construct an initial relationship model between the focus index value and the shooting position, and to fit the initial relationship model based on the focus index sample and the distance position corresponding to the focus index sample to obtain the focus relationship model. The threshold calculation module is used to calculate the focus threshold within the depth of field of the microscope camera based on the focus relationship model. The image acquisition module is also used to respond to the microscope camera imaging calibration request by calling the microscope camera to take pictures of the calibration plate to obtain a calibration image; The index calculation module is further configured to: divide each calibration image into multiple image area regions, extract the image center area region containing the image midpoint and the area regions containing the four corners of the image as the second target region; and calculate the calibration focus index value at the second target region of the calibration image. The defocusing detection module is used to determine whether the calibrated focus index value is within the focus threshold range. If it is, the current microscope camera image is not in a defocused state; otherwise, the current microscope camera image is in a defocused state.

7. A microscope camera imaging defocus detection device based on a calibration plate, characterized in that, The calibration plate-based microscope camera imaging defocus detection device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the calibration plate-based microscope camera imaging defocus detection device to perform the steps of the calibration plate-based microscope camera imaging defocus detection method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program / instructions thereon, characterized in that, When the program / instruction is executed by the processor, it implements the steps of the calibration plate-based microscope camera imaging defocus detection method as described in any one of claims 1-5.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the microscope camera imaging defocus detection method based on the calibration plate as described in any one of claims 1-5 are implemented.

Images