Virtual focus detection method and device for scanning imaging system and virtual focus detection system

Abstract

The invention discloses a virtual focus detection method and device for a scanning imaging system and a virtual focus detection system, and relates to the field of optical engineering, the field of computer vision or other related fields. The method comprises the steps that a to-be-shot starting point of a shot target is determined, and the shot target is fixed to a motion platform of a scanning imaging system; moving a shooting target to a to-be-shot starting point, and obtaining a scanning image of the shooting target; the virtual focus detection instruction is responded, virtual focus detection is carried out on the scanned image, a detection result is obtained, and the instruction type of the virtual focus detection instruction comprises local virtual focus detection and global virtual focus detection; and when the detection result indicates that the virtual focus abnormity occurs in the scanned image, generating a shooting stopping instruction which is used for controlling the scanning imaging system to stop scanning imaging. According to the invention, the technical problems of hysteresis and low automation degree of a virtual focus detection technology in related technologies are solved.

Description

Technical Field

[0001] The present invention relates to the field of optical engineering, computer vision or other related fields, and in particular to a defocus detection method and device for a scanning imaging system, and a defocus detection system. Background Art

[0002] As digital technology becomes more mature, optical imaging systems based on machine vision technology have been widely used in industrial inspection, medicine and life sciences, security monitoring, aerospace remote sensing and other fields. In order to meet real-time requirements, imaging systems usually use staring imaging combined with stepping motion or TDI cameras combined with line scanning to complete the shooting of the target. At this time, if impurities such as water vapor and dust are mixed into the imaging light path, there is a certain probability that the image quality will degrade. The imaging system is required to automatically detect, judge and intercept the above abnormal conditions in real time to avoid the abnormal data corresponding to the image after quality degradation affecting the subsequent image information processing results.

[0003] In the related technology, the following two abnormality detection methods are generally used: subjective judgment and autofocus detection. Among them, subjective judgment refers to manually checking whether there are image quality degradation phenomena such as out-of-focus in the scanned image after the imaging system completes scanning and shooting, and then determining whether there are abnormalities in the imaging optical path; autofocus detection refers to using the autofocus device installed in the imaging system to form a closed loop with the motion platform, and adjusting the distance between the imaging module and the shooting target in real time during the scanning and imaging process to ensure that the shooting target is always in the focal position, and when impurities appear in the imaging optical path, causing the image to be out of focus, the autofocus device will feedback an abnormal electrical signal, and by processing the abnormal electrical signal, it can also be determined whether there are abnormalities in the imaging optical path.

[0004] However, the above two abnormal situation detection methods have obvious defects: the subjective judgment method has a lag and usually cannot detect the problem in the first time when the abnormal situation occurs, and the degree of automation is low and the ease of use is poor, so this method cannot be integrated into the imaging system control software; the autofocus device that forms a closed loop with the motion platform in the autofocus detection method is expensive, which makes it difficult to promote its use.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a defocus detection method and device, and a defocus detection system for a scanning imaging system, so as to at least solve the technical problems of lag and low automation of defocus detection technology in related technologies.

[0007] According to one aspect of an embodiment of the present invention, there is provided a defocus detection method for a scanning imaging system, comprising: determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of the scanning imaging system; moving the shooting target to the starting point to be shot, and acquiring a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform while the moving platform moves in a predetermined direction of motion; in response to a defocus detection instruction, performing a defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection; when the detection result indicates that the scanned image has a defocus abnormality, generating a stop shooting instruction, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0008] Optionally, the step of determining the starting point to be photographed of the shooting target includes: when it is detected that the entire shooting target is within the shooting range, collecting a positioning image of the shooting target through the motion platform, wherein the motion platform is in a stationary state during the process of collecting the positioning image; analyzing features of the shooting target in the positioning image to obtain a feature set, wherein the feature set includes at least one of the following features: edge features, corner features, and scale-invariant features; and solving the positioning image based on the feature set to obtain a solving result, wherein the solving result includes the starting point to be photographed of the shooting target.

[0009] Optionally, the step of performing out-of-focus detection on the scanned image to obtain the detection result includes: performing grayscale integration on the pixel points on the scanned image along a first preset direction to obtain a grayscale curve of the scanned image, wherein the grayscale curve is used to record the one-dimensional grayscale information of the scanned image; filtering the grayscale curve based on a preset filtering strategy and a preset morphological algorithm, wherein the preset morphological algorithm includes: an opening operation and a closing operation; performing a bubble peak feature operation on the filtered grayscale curve to obtain a bubble peak feature value set, and generating the detection result based on the bubble peak feature value set.

[0010] Optionally, the step of performing a bubble peak feature operation on the grayscale curve after filtering to obtain a bubble peak feature value set, and generating the detection result based on the bubble peak feature value set includes: performing a global search on the grayscale curve after filtering to obtain a global grayscale minimum value of the grayscale curve, and determining the bubble peak center based on the global grayscale minimum value; performing a feature operation on the grayscale curve and the bubble peak center to obtain the bubble peak feature value set, wherein the bubble peak feature values ​​in the bubble peak feature value set include at least: amplitude, kurtosis, peak slope and peak width value; performing a weighted operation on all the bubble peak feature values ​​in the bubble peak feature value set according to preset weights to obtain an operation result; when the operation result exceeds a preset threshold range, setting the detection result to a scanned image having a local out-of-focus abnormality; or, when the operation result does not exceed the preset threshold range, setting the detection result to a scanned image having no local out-of-focus abnormality.

[0011] Optionally, before performing grayscale integration on the pixel points on the scanned image along a first preset direction, it also includes: performing image enhancement processing on the scanned image, wherein the image enhancement processing includes at least one of the following: normalization processing, smoothing processing, differential calculation, contrast adjustment, image sharpening and histogram equalization.

[0012] Optionally, the step of performing defocus detection on the scanned image to obtain a detection result also includes: inputting the scanned image into a preset logistic regression model; performing global defocus detection on the scanned image through the preset logistic regression model, and outputting the detection result.

[0013] Optionally, the scanned image is subjected to global defocus detection through the preset logistic regression model, and the step of outputting the detection result includes: performing grayscale averaging processing on the pixel points on the scanned image along a second preset direction through the preset logistic regression model to obtain a one-dimensional grayscale average value set of the scanned image in a third preset direction, wherein the second preset direction is perpendicular to the third preset direction, and the one-dimensional grayscale average value set includes M one-dimensional grayscale average values, where M is a positive integer; performing a differential convolution operation on all the one-dimensional grayscale average values ​​along the third preset direction to obtain a grayscale variance value of the scanned image; when the grayscale variance value is less than or equal to a preset grayscale variance threshold, setting the detection result as a global defocus abnormality occurring in the scanned image; or, when the grayscale variance value is greater than the preset grayscale variance threshold, setting the detection result as a global defocus abnormality occurring in the scanned image.

[0014] Optionally, the preset logistic regression model is obtained by the following method: collecting N sample images to obtain a sample set, wherein each of the sample images carries an image label, and the image label is: normal image or global out-of-focus image, and N is a positive integer; inputting the sample set into the initial logistic regression model, and outputting a classification result, wherein the classification result includes N model classification labels corresponding to the N sample images, and the model classification labels are obtained by performing a differential convolution operation on each of the sample images by the initial logistic regression model to obtain a grayscale variance value, and then classifying the N sample images based on the N grayscale variance values; when it is determined that the model classification label of any of the sample images is inconsistent with the image label carried by the sample image, adjusting the model parameters of the initial logistic regression model; when the model classification label of each of the sample images in the sample set is consistent with the image label carried by the sample image, stopping adjusting the model parameters to obtain the preset logistic regression model.

[0015] According to another aspect of an embodiment of the present invention, a defocus detection system is also provided, including: a microscopic imaging platform, connected to an image processing module, for acquiring a positioning image of a shooting target, scanning the shooting target to obtain a scanned image, and transmitting the positioning image and the scanned image to the image processing module; the image processing module, connected to the microscopic imaging platform and a host computer control module, respectively, for calculating the starting point to be shot of the shooting target based on the positioning image, and performing defocus detection on the scanned image; a motion platform, connected to a motion control module, for fixing the shooting target; the motion control module, connected to the motion platform and the host computer control module, respectively, for performing motion control on the motion platform; the host computer control module, connected to the image processing module and the motion control module, respectively, for performing collaborative control on the image processing module and the motion control module.

[0016] According to another aspect of an embodiment of the present invention, a defocus detection device for a scanning imaging system is provided, comprising: a determination unit, for determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of the scanning imaging system; an acquisition unit, for moving the shooting target to the starting point to be shot, and acquiring a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform during the movement of the moving platform in a predetermined movement direction; a detection unit, for responding to a defocus detection instruction, performing defocus detection on the scanned image, and obtaining a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection; a generation unit, for generating a stop shooting instruction when the detection result indicates that the scanned image has a defocus abnormality, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0017] Optionally, the determination unit includes: an acquisition module, used to acquire a positioning image of the shooting target through the motion platform when it is detected that the entire shooting target is within the shooting range, wherein the motion platform is in a stationary state during the process of acquiring the positioning image; an analysis module, used to analyze features of the shooting target in the positioning image to obtain a feature set, wherein the feature set includes at least one of the following features: edge features, corner features, and scale-invariant features; and a solution module, used to solve the positioning image based on the feature set to obtain a solution result, wherein the solution result includes the starting point to be shot of the shooting target.

[0018] Optionally, the detection unit includes: a grayscale integration module, used to perform grayscale integration of pixel points on the scanned image along a first preset direction to obtain a grayscale curve of the scanned image, wherein the grayscale curve is used to record one-dimensional grayscale information of the scanned image; a filtering module, used to filter the grayscale curve based on a preset filtering strategy and a preset morphological algorithm, wherein the preset morphological algorithm includes: an opening operation and a closing operation; an operation module, used to perform a bubble peak feature operation on the filtered grayscale curve to obtain a bubble peak feature value set, and generate the detection result based on the bubble peak feature value set.

[0019] Optionally, the operation module includes: a global search submodule, which is used to perform a global search on the filtered grayscale curve to obtain the global grayscale minimum of the grayscale curve, and determine the bubble peak center based on the global grayscale minimum; a feature operation submodule, which is used to perform feature operation on the grayscale curve and the bubble peak center to obtain the bubble peak feature value set, wherein the bubble peak feature values ​​in the bubble peak feature value set include at least: amplitude, kurtosis, peak slope and peak width value; a weighted operation submodule, which is used to perform a weighted operation on all the bubble peak feature values ​​in the bubble peak feature value set according to preset weights to obtain an operation result; a first setting submodule, which is used to set the detection result as a local out-of-focus abnormality in the scanned image when the operation result exceeds a preset threshold range; and a second setting submodule, which is used to set the detection result as a local out-of-focus abnormality in the scanned image when the operation result does not exceed the preset threshold range.

[0020] Optionally, the detection unit also includes: an image enhancement processing module, used to perform image enhancement processing on the scanned image, wherein the image enhancement processing includes at least one of the following: normalization processing, smoothing processing, differential calculation, contrast adjustment, image sharpening and histogram equalization.

[0021] Optionally, the detection unit further includes: an input module, used to input the scanned image into a preset logistic regression model; and an output module, used to perform global defocus detection on the scanned image through the preset logistic regression model and output the detection result.

[0022] Optionally, the output module includes: a grayscale average processing submodule, used to perform grayscale average processing on the pixel points on the scanned image along the second preset direction through the preset logistic regression model, and obtain a one-dimensional grayscale average value set of the scanned image in a third preset direction, wherein the second preset direction is perpendicular to the third preset direction, and the one-dimensional grayscale average value set includes M one-dimensional grayscale average values, and M is a positive integer; a differential convolution operation submodule, used to perform a differential convolution operation on all the one-dimensional grayscale average values ​​along the third preset direction to obtain a grayscale variance value of the scanned image; a third setting submodule, used to set the detection result as a global out-of-focus abnormality in the scanned image when the grayscale variance value is less than or equal to a preset grayscale variance threshold; and a fourth setting submodule, used to set the detection result as a global out-of-focus abnormality in the scanned image when the grayscale variance value is greater than the preset grayscale variance threshold.

[0023] Optionally, the preset logistic regression model is obtained through the following submodules: an acquisition submodule, used to acquire N sample images to obtain a sample set, wherein each of the sample images carries an image label, and the image label is: normal image or global out-of-focus image, and N is a positive integer; an input submodule, used to input the sample set into the initial logistic regression model, and output a classification result, wherein the classification result includes N model classification labels corresponding to the N sample images, and the model classification labels are obtained by performing a differential convolution operation on each of the sample images by the initial logistic regression model to obtain a grayscale variance value, and then classifying the N sample images based on the N grayscale variance values; an adjustment submodule, used to adjust the model parameters of the initial logistic regression model when it is determined that the model classification label of any of the sample images is inconsistent with the image label carried by the sample image; a stop adjustment submodule, used to stop adjusting the model parameters when the model classification label of each of the sample images in the sample set is consistent with the image label carried by the sample image, so as to obtain the preset logistic regression model.

[0024] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned out-of-focus detection methods for a scanning imaging system.

[0025] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-described defocus detection methods for a scanning imaging system.

[0026] The present disclosure provides a defocus detection method for a scanning imaging system, which includes first determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of the scanning imaging system, then moving the shooting target to the starting point to be shot, and acquiring a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform during the movement of the moving platform in a predetermined movement direction, then responding to a defocus detection instruction, performing defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection, and finally generating a stop shooting instruction when the detection result indicates that the scanning image has a defocus abnormality, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0027] In the present disclosure, motion control technology and image processing technology are combined, a motion platform is used to fix the shooting target, and the motion control technology is used to scan the shooting target through a scanning imaging system during the process of the motion platform moving from the starting point to be shot in a predetermined direction of movement to obtain a scanned image, and local defocus detection and global defocus detection are performed on the scanned image to obtain a detection result. When the detection result indicates that the scanned image has a defocus abnormality (including local defocus abnormality or global defocus abnormality), a stop shooting instruction is generated, and the scanning imaging system automatically stops the current scanning imaging after receiving the stop shooting instruction. The defocus detection method proposed in the present disclosure realizes real-time defocus detection of the scanned image during the scanning imaging process, and automatically generates a stop shooting instruction when the scanned image has a defocus abnormality, which is convenient for shutting down the scanning imaging system for inspection, thereby solving the technical problems of lag and low degree of automation in the defocus detection technology in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0029] Figure 1 is a flow chart of an optional out-of-focus detection method for a scanning imaging system according to an embodiment of the present invention;

[0030] Figure 2 is a structural diagram of an optional TDI imaging system according to an embodiment of the present invention;

[0031] Figure 3 is a flow chart of an optional out-of-focus detection method according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of an optional local out-of-focus image according to an embodiment of the present invention;

[0033] Figure 5 is an optional grayscale average schematic diagram according to an embodiment of the present invention;

[0034] Figure 6 is a structural diagram of an optional out-of-focus detection system according to an embodiment of the present invention;

[0035] Figure 7 is a schematic diagram of an optional defocus detection device for a scanning imaging system according to an embodiment of the present invention;

[0036] Figure 8 It is a hardware structure block diagram of an electronic device (or mobile device) for a defocus detection method of a scanning imaging system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0040] The opening operation first erodes and then dilates the target image. The erosion operation can remove small details on the boundary of objects in the target image, and the dilation operation can restore the shape of the object and fill the holes. The opening operation can smooth the edges of the object while maintaining the overall shape of the object.

[0041] The closing operation first dilates and then erodes the target image. The dilation operation can fill small holes inside the object in the target image, and the erosion operation can restore the shape of the object and remove details on the boundary. The closing operation can make the object more connected and maintain the overall shape of the object.

[0042] Bubble peak refers to one or more local peaks that appear in the grayscale curve, which usually indicates the bright or dark areas in the image. It is used to reflect the distribution of different grayscale levels in the image and can be used to analyze the image contrast, brightness, details and other information.

[0043] Normalization processing refers to mapping the pixel values ​​of an image to a specific range, generally [0,1] or [0,255], by linearly scaling each pixel value, thereby improving the contrast of the image.

[0044] Smoothing refers to reducing the noise or details in the image through filtering operations such as mean filtering, Gaussian filtering, and median filtering, thereby making the image more blurred and reducing noise and details.

[0045] Differential calculation refers to enhancing the edges or details of an image by performing differential or convolution operations on the pixel values ​​of the image, thereby highlighting the edges or details in the image, improving the clarity of the image, and highlighting the target.

[0046] Adjusting contrast refers to changing the contrast level of an image by modifying the pixel value distribution of the image through linear or nonlinear transformation, thereby making the brightness level in the image more evenly distributed and enhancing the visual effect and readability of the image.

[0047] Sharpening an image means enhancing the edges and details of an image through Laplace filtering, Gaussian filtering, sharpening filtering, etc., thereby improving the recognition accuracy of the image.

[0048] Histogram equalization refers to the process of remapping the grayscale value of each pixel by calculating the number of pixels that appear in the image at each grayscale level, so that the histogram distribution of the image is more uniform, thereby enhancing the contrast and dynamic range of the image and making the brightness distribution of the image more uniform.

[0049] CNN, convolutional neural network, is suitable for image processing and out-of-focus detection tasks. It can extract features from original images and learn the spatial structure of images.

[0050] SVM, support vector machine, is suitable for image processing and out-of-focus detection tasks. It can effectively process high-dimensional feature space and has good classification performance.

[0051] Grid Search, grid search, finds the best model parameters by traversing given parameter combinations, and cross-validation can be used to evaluate the performance of each parameter combination.

[0052] Random Search, which randomly selects parameter combinations for training and evaluation to find the best model parameters, is more efficient than grid search.

[0053] Bayesian Optimization, Bayesian optimization, can efficiently find the best parameter combination by considering historical parameter combinations and historical performance to select the next parameter combination.

[0054] Genetic Algorithm, genetic algorithm, optimizes parameters by simulating the process of biological evolution, and then searches for the best parameter combination.

[0055] It should be noted that the defocus detection method and device for a scanning imaging system in the present disclosure can be used in the field of optical engineering when real-time defocus detection is performed on the scanned image of the scanning imaging system, and can also be used in any field except the field of optical engineering when real-time defocus detection is performed on the scanned image of the scanning imaging system. The present disclosure does not limit the application field of the defocus detection method and device for a scanning imaging system.

[0056] The following embodiments of the present invention can be applied to various systems / applications / devices that require real-time out-of-focus detection of scanned images of a scanning imaging system. It can combine motion control technology and image processing technology to perform real-time out-of-focus detection of scanned images during the scanning imaging process, and automatically generate a stop shooting instruction when the scanned image has an out-of-focus abnormality, thereby facilitating shutdown inspection of the scanning imaging system.

[0057] Embodiment 1

[0058] According to an embodiment of the present invention, an embodiment of a defocus detection method for a scanning imaging system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0059] Figure 1 is a flow chart of an optional out-of-focus detection method for a scanning imaging system according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0060] Step S101, determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of a scanning imaging system.

[0061] Step S102, moving the photographing target to the starting point to be photographed, and acquiring a scanned image of the photographing target, wherein the scanned image is obtained by scanning the photographing target on the moving platform while the moving platform moves in a predetermined moving direction.

[0062] Step S103, responding to the defocus detection instruction, performing defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection.

[0063] Step S104, when the detection result indicates that the scanned image has a defocus abnormality, a stop shooting instruction is generated, wherein the defocus abnormality includes: a local defocus abnormality or a global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0064] Through the above steps, it is possible to first determine the starting point of the shooting target to be shot, wherein the shooting target is fixed on the moving platform of the scanning imaging system, then move the shooting target to the starting point to be shot, and obtain a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform while the moving platform moves in a predetermined direction of motion, and then respond to a defocus detection instruction, perform defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection, and finally, when the detection result indicates that the scanned image has a defocus abnormality, generate a stop shooting instruction, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning and imaging.

[0065] In an embodiment of the present invention, motion control technology and image processing technology are combined, a motion platform is used to fix the shooting target, and the motion control technology is used to scan the shooting target through a scanning imaging system during the process of the motion platform moving from the starting point to be shot in a predetermined direction of motion to obtain a scanned image, and local defocus detection and global defocus detection are performed on the scanned image to obtain a detection result. When the detection result indicates that the scanned image has a defocus abnormality (including local defocus abnormality or global defocus abnormality), a stop shooting instruction is generated, and the scanning imaging system automatically stops the current scanning imaging after receiving the stop shooting instruction. The defocus detection method proposed in an embodiment of the present invention realizes real-time defocus detection of the scanned image during the scanning imaging process, and automatically generates a stop shooting instruction when the scanned image has a defocus abnormality, which is convenient for shutting down the scanning imaging system for inspection, thereby solving the technical problems of lag and low degree of automation in the defocus detection technology in the related art.

[0066] The embodiment of the present invention is described in detail below in combination with the above steps.

[0067] The implementing body of the embodiment of the present invention may be a defocus detection system, which controls the motion platform through motion control technology to drive the shooting target to move on a predetermined scanning track, so that the scanning imaging system connected to the defocus detection system can capture the imaging details of the shooting target in real time to form a scanned image, and perform real-time processing and defocus detection on the scanned image through image processing technology. When it is determined that the scanned image has a defocus abnormality, a stop shooting instruction is issued to the scanning imaging system, and the abnormal situation is checked and eliminated after the scanning imaging system is shut down.

[0068] Step S101, determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of a scanning imaging system.

[0069] Optionally, step S101 includes: when it is detected that the entire shooting target is within the shooting range, collecting a positioning image of the shooting target through a motion platform, wherein the motion platform is in a stationary state during the process of collecting the positioning image; analyzing features of the shooting target in the positioning image to obtain a feature set, wherein the feature set includes at least one of the following features: edge features, corner features, and scale-invariant features; solving the positioning image based on the feature set to obtain a solving result, wherein the solving result includes the starting point to be shot of the shooting target.

[0070] It should be noted that the higher the clarity of the captured positioning image, the more accurate the calculated starting point to be shot. Therefore, a high-resolution camera or sensor is required for shooting during the capture of the positioning image. In addition, it should be ensured that the shooting conditions of the positioning image (such as the installation angle, installation distance, lighting conditions, etc. of the acquisition device) are consistent with the scanning conditions of the scanning imaging to reduce errors caused by environmental changes.

[0071] Optionally, in an embodiment of the present invention, one positioning image can be collected for solution to determine the starting point to be shot, or multiple positioning images can be collected for joint solution to improve the accuracy of positioning. Each positioning image can also be preprocessed before solution (for example, removing noise, adjusting contrast, etc.) to improve the accuracy of the solution.

[0072] It should be noted that edge features help to mark and locate the places where the intensity changes are most obvious in the image. Usually, the boundaries or contours of the target can be directly located. The position and shape of the target can be determined by extracting places with larger intensity gradients through edge detection algorithms (such as the Canny edge detection algorithm).

[0073] It should be noted that the corner feature is the position of the corner or sharp angle in the image, which is rotation invariant and scale invariant. The position and direction of the captured target can be determined through the corner detection algorithm (such as Harris corner detection or Shi-Tomasi corner detection).

[0074] It should be noted that scale-invariant features refer to image features that can remain stable at different scales, such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features), which detect key points at different scales by using Gaussian pyramids and scale space extreme points, and represent the feature points of the captured target through local image descriptors.

[0075] Step S102, moving the photographing target to the starting point to be photographed, and acquiring a scanned image of the photographing target, wherein the scanned image is obtained by scanning the photographing target on the moving platform while the moving platform moves in a predetermined moving direction.

[0076] Step S103, responding to the defocus detection instruction, performing defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection.

[0077] It should be noted that, when the instruction type of the defocus detection instruction is local defocus detection, image processing and local defocus detection are performed on the scanned image. Optionally, step S103 includes: performing grayscale integration on the pixel points on the scanned image along a first preset direction to obtain a grayscale curve of the scanned image, wherein the grayscale curve is used to record the one-dimensional grayscale information of the scanned image; filtering the grayscale curve based on a preset filtering strategy and a preset morphological algorithm, wherein the preset morphological algorithm includes: an opening operation and a closing operation; performing a bubble peak feature operation on the filtered grayscale curve to obtain a bubble peak feature value set, and generating a detection result based on the bubble peak feature value set.

[0078] It should be noted that if the scanned image is a color image, it is necessary to convert the color image into a grayscale image before grayscale integration. An embodiment of the present invention provides a common grayscale conversion formula: Y=0.299*R+0.587*G+0.114*B, where R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively, and Y is used to represent the grayscale value of the converted pixel point.

[0079] In the embodiment of the present invention, an orthogonal coordinate system can be determined in advance in the scanned image, and the X and Y axes of the coordinate system are used as reference directions, one of the reference directions is selected as the first preset direction (in the embodiment of the present invention, the Y direction is selected as the first preset direction for schematic illustration), and the integral starting point is determined, an integral graph with the same size as the scanned image is created, and the grayscale values ​​of all pixels are accumulated along the Y direction, that is, the grayscale information in the Y direction of the scanned image is simplified to obtain one-dimensional grayscale information in the X direction, and a grayscale curve is drawn based on all one-dimensional grayscale information in the X direction.

[0080] It should be noted that the morphological algorithm is an image processing method based on image shape and structure, and is often used for image segmentation, filtering, denoising, etc. The morphological algorithm includes but is not limited to opening and closing operations. The opening operation is used to remove small noise points in the image or separate adjacent objects, and the closing operation is used to fill the holes inside the object or connect adjacent objects.

[0081] The purpose of filtering the grayscale curve by using the morphological algorithm in the embodiment of the present invention is to remove noise in the curve or smooth the curve so as to better perform feature analysis on the curve (including but not limited to bubble peak feature calculation).

[0082] Optionally, the steps of performing bubble peak feature operation on the filtered grayscale curve to obtain a bubble peak feature value set, and generating a detection result based on the bubble peak feature value set include: performing a global search on the filtered grayscale curve to obtain the global grayscale minimum of the grayscale curve, and determining the bubble peak center based on the global grayscale minimum; performing feature operation on the grayscale curve and the bubble peak center to obtain a bubble peak feature value set, wherein the bubble peak feature values ​​in the bubble peak feature value set include at least: amplitude, kurtosis, peak slope and peak width value; performing weighted operation on all bubble peak feature values ​​in the bubble peak feature value set according to preset weights to obtain an operation result; when the operation result exceeds a preset threshold range, setting the detection result to a local out-of-focus abnormality in the scanned image; or, when the operation result does not exceed the preset threshold range, setting the detection result to a local out-of-focus abnormality in the scanned image.

[0083] The embodiment of the present invention uses the following method for calculating the characteristic value of the bubble peak as a reference.

[0084] The amplitude of the bubble peak represents the height / intensity of the bubble peak. After the center of the bubble peak is determined, the grayscale value of the center of the bubble peak in the grayscale curve is obtained as the amplitude.

[0085] The kurtosis of the bubble peak indicates the sharpness / flatness of the bubble peak, which can be obtained by calculating the deviation of the grayscale values ​​around the peak, calculating the grayscale values ​​of the left and right neighborhoods of the peak, and calculating their standard deviation. The larger the standard deviation, the higher the kurtosis, and the smaller the standard deviation, the lower the kurtosis.

[0086] The peak slope of the bubble peak represents the rising speed of the bubble peak, which can be obtained by calculating the slope of the left and right neighborhoods of the peak. The slope of each point in the left and right neighborhoods of the peak can be calculated using methods such as differential approximation or derivative.

[0087] The peak width of the bubble peak represents the width or duration of the bubble peak, which can be obtained by calculating the width of the left and right neighborhoods of the peak, finding the half-peak points on the left and right sides of the peak (i.e., the points with grayscale values ​​half the peak amplitude), and calculating the distance between the two points as the peak width.

[0088] Optionally, before performing grayscale integration on the pixel points on the scanned image along a first preset direction, it also includes: performing image enhancement processing on the scanned image, wherein the image enhancement processing includes at least one of the following: normalization processing, smoothing processing, differential calculation, contrast adjustment, image sharpening and histogram equalization.

[0089] It should be noted that the role of image enhancement is to improve the quality and clarity of the image by increasing the brightness, contrast, sharpness, etc. of the image, while enhancing the details and contrast of the image, so that the out-of-focus parts in the image can be detected more accurately.

[0090] In an embodiment of the present invention, when the instruction type of the defocus detection instruction is global defocus detection, image processing and global defocus detection are performed on the scanned image, and the detection process is as follows.

[0091] Optionally, step S103 further includes: inputting the scanned image into a preset logistic regression model; performing global defocus detection on the scanned image through the preset logistic regression model, and outputting the detection result.

[0092] It should be noted that before executing step S103, a preset logistic regression model for global out-of-focus detection must be prepared in advance. The preset logistic regression model is obtained by the following method: collect N sample images to obtain a sample set, wherein each sample image carries an image label, and the image label is: normal image or global out-of-focus image, and N is a positive integer; input the sample set into the initial logistic regression model, and output a classification result, wherein the classification result includes N model classification labels corresponding to the N sample images, and the model classification label is obtained by the initial logistic regression model performing a differential convolution operation on each sample image to obtain a grayscale variance value, and then classifying the N sample images based on the N grayscale variance values; when it is determined that the model classification label of any sample image is inconsistent with the image label carried by the sample image, adjust the model parameters of the initial logistic regression model; when the model classification label of each sample image in the sample set is consistent with the image label carried by the sample image, stop adjusting the model parameters to obtain the preset logistic regression model.

[0093] In the embodiment of the present invention, any of the following models can be selected as the initial logistic regression model: the basic logistic regression algorithm model in the machine learning library (such as scikit-learn), convolutional neural network, support vector machine; in the embodiment of the present invention, any of the following model modulation methods can be selected to train the initial logistic regression model: grid search, random search, Bayesian optimization, genetic algorithm. After adjusting the parameters of the initial logistic regression model using a suitable model modulation method, a preset logistic regression model that can be used for global out-of-focus detection is obtained.

[0094] Optionally, the scanned image is subjected to global defocus detection through a preset logistic regression model, and the step of outputting the detection result includes: performing grayscale averaging processing on the pixel points on the scanned image along a second preset direction through a preset logistic regression model to obtain a one-dimensional grayscale average value set of the scanned image in a third preset direction, wherein the second preset direction is perpendicular to the third preset direction, and the one-dimensional grayscale average value set includes M one-dimensional grayscale average values, and M is a positive integer; performing a differential convolution operation on all one-dimensional grayscale average values ​​along the third preset direction to obtain a grayscale variance value of the scanned image; when the grayscale variance value is less than or equal to a preset grayscale variance threshold, setting the detection result to indicate that the scanned image has a global defocus abnormality; or, when the grayscale variance value is greater than the preset grayscale variance threshold, setting the detection result to indicate that the scanned image does not have a global defocus abnormality.

[0095] It should be noted that the scanning imaging system performs angle calibration on the shooting target during the acquisition of the scanning image. During the calibration process, the scanning imaging system provides a default orthogonal coordinate system, with the Y direction and X direction of the coordinate system as the second preset direction and the third preset direction.

[0096] The grayscale values ​​of the pixels in the scanned image are averaged along the Y direction of the scanned image to obtain the one-dimensional grayscale average value, and all the one-dimensional grayscale average values ​​of the entire scanned image are integrated in the original order, and all the one-dimensional grayscale average values ​​are searched and convolved along the X direction to obtain the grayscale variance value used to characterize the overall defocus degree of the scanned image. The more serious the overall defocus of the scanned image, the blurrier the image, the smoother the grayscale change between adjacent pixels, and the smaller the grayscale variance value. On the contrary, if the scanned image does not have overall defocus, the clearer the image, the more drastic the grayscale change between adjacent pixels, and the larger the grayscale variance value. After calculating the grayscale variance value of the scanned image, the preset logistic regression model automatically determines the grayscale variance value according to the pre-set grayscale variance threshold, and then determines whether the scanned image has global defocus abnormality.

[0097] Step S104, when the detection result indicates that the scanned image has a defocus abnormality, a stop shooting instruction is generated, wherein the defocus abnormality includes: a local defocus abnormality or a global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0098] In an embodiment of the present invention, local defocus detection is performed on the scanned image through a grayscale morphological algorithm, the image contour is detected and the boundary information is used to quantify the structural features of the dark area of ​​the image, and a classification algorithm is used to determine in real time whether there is a local defocus abnormality in the image, thereby facilitating the generation of a stop shooting instruction and troubleshooting abnormalities in the scanning imaging system that may cause local defocus (small-sized water vapor or dust mixed in the imaging light path).

[0099] In an embodiment of the present invention, global defocus detection is performed on the scanned image through machine learning and grayscale variance feature extraction methods, the normalized one-dimensional grayscale variance value is selected as the image feature, and a logistic regression model is introduced. A supervised classification algorithm is used to perform real-time judgment on whether there is a global defocus abnormality in the image, and then a stop shooting instruction is generated to check for abnormal conditions that may cause global defocus in the scanning imaging system (water vapor or dust of slightly larger size mixed in the imaging optical path).

[0100] The defocus detection method provided in the embodiment of the present invention can perform real-time processing and real-time detection on the scanned image during the operation of the scanning imaging system, and immediately generate a stop shooting instruction after an image with defocus abnormality is found, which is conducive to stopping shooting as soon as the imaging light path is blocked to eliminate the blockage. The above process is fully automatic and without lag, which solves the problem of low automation and lag in the prior art of using defocus detection to eliminate imaging system failures.

[0101] In the embodiment of the present invention, there is no need to additionally install an autofocus device linked to the motion platform to perform automatic focus detection and closed-loop control, and real-time detection and interception of out-of-focus anomalies can be achieved during the imaging stage, saving hardware costs.

[0102] The present invention is described below in conjunction with another specific implementation manner.

[0103] The embodiments of the present invention realize real-time image acquisition and real-time defocus detection of a shooting target through a TDI imaging system.

[0104] Figure 2 is a structural diagram of an optional TDI imaging system according to an embodiment of the present invention, such as Figure 2 As shown, the TDI imaging system includes the following basic components: TDI microscopic imaging platform, motion platform, image processing module, host computer control module, motion control module and data processing system.

[0105] Among them, the motion platform is used to fix the shooting target and drive the shooting target to move according to the predetermined scanning trajectory, so that the fixed-position TDI microscopic imaging platform can perform a complete scanning and imaging of the shooting target; the movement process of the motion platform is controlled by the motion control module; after the TDI microscopic imaging platform collects the complete scanned image, it is transmitted to the image processing module for image processing and image analysis to determine whether the image has a defocus abnormality; the motion control platform and the image processing module are both controlled by the host computer control module; when the image has a defocus abnormality, the host computer control module transmits the defocus abnormality to the data processing system and generates a stop shooting instruction.

[0106] It should be noted that Figure 2The circuit / electrical signal transmission is represented by solid arrows, and the optical path / optical signal transmission is represented by dotted arrows.

[0107] In addition, the image processing module can also be linked with the motion control module to perform operation compensation on the motion control module according to the specific situation of the received scanned image to improve the predetermined scanning trajectory.

[0108] Figure 3 is a flow chart of an optional out-of-focus detection method according to an embodiment of the present invention. Figure 3 As shown, the method comprises the following steps:

[0109] S301, fixing the shooting target on the moving platform;

[0110] S302, the host computer control module controls the motion platform through the motion control module to take a photo of the shooting target, and processes the photographed image through the image processing module to calculate the scanning starting point of the shooting target;

[0111] S303, the host computer control module controls the motion platform through the motion control module to drive the shooting target to move to the scanning starting point;

[0112] S304, the host computer control module controls the motion platform through the motion control module to drive the imaging chip to move along a predetermined trajectory, completes the scanning imaging of the photographed object through the TDI microscopic imaging module, and sends the scanned image and interrupt signal to the image processing module at fixed periodic intervals;

[0113] S305, the image processing module responds to the interrupt signal, performs defocus detection on the received scanned image, and transmits the defocus detection result to the host computer control module;

[0114] S306, the host computer control module reads the out-of-focus detection result, contacts the data processing system to generate a stop shooting instruction when out-of-focus abnormality occurs in the scanned image, controls the TDI imaging system to stop shooting, and transmits the abnormal situation to the user terminal.

[0115] It should be noted that, in the process of the image processing module performing defocus detection on the received scanned image, the types of defocus detection should include at least two categories: local defocus detection and global defocus detection. The following describes these two types of defocus detection in detail.

[0116] 1. Local out-of-focus detection.

[0117] Figure 4 is a schematic diagram of an optional local out-of-focus image according to an embodiment of the present invention, such as Figure 4 As shown in the figure, there are usually dark strips with unfixed positions and widths in the local out-of-focus image ( Figure 4There are obvious boundary features between the dark area and other areas (indicated by A in the figure), and a strong contrast is formed between different areas. The target information originally in the dark area will be covered, showing a partial out-of-focus state.

[0118] The embodiment of the present invention detects and intercepts the above-mentioned local out-of-focus state, and the interception process can be roughly divided into four stages.

[0119] The first stage: image enhancement stage, image enhancement processing is performed on the scanned images that have not been image enhanced to increase the accuracy of subsequent image discrimination;

[0120] The second stage: grayscale integration stage, the grayscale values ​​of all pixels in the vertical direction of the image are integrated to obtain one-dimensional grayscale information and draw a one-dimensional grayscale curve. Considering local interference, some over-bright and over-dark data can be filtered out during integration;

[0121] The third stage: filtering processing stage, using grayscale morphological operators to perform closing operations on the one-dimensional grayscale curve to filter out dark line interference, and then perform opening operations to filter out bright spot interference;

[0122] The fourth stage: the bubble feature calculation stage, searches for the global grayscale minimum in the one-dimensional grayscale curve, determines it as the bubble peak center, calculates the bubble peak amplitude and steepness and other information, comprehensively quantifies the bubble peak, and if the bubble peak feature exceeds the limited threshold, it is judged as local defocus.

[0123] Second, global out-of-focus detection.

[0124] When slightly larger water vapor and dust are mixed into the optical path of the imaging system, the image quality will be severely degraded, and all available information in the image will be covered, resulting in global out-of-focus.

[0125] In the embodiment of the present invention, a logistic regression model is introduced to perform global defocus detection on the scanned image, and the grayscale variance value of the scanned image is calculated by the model, and then the grayscale variance value is used to determine whether the scanned image has global defocus abnormality. The embodiment of the present invention provides an optional reference method for calculating the grayscale variance value used by the logistic regression model as follows.

[0126] 1. Determine the orthogonal reference coordinate system on the scanned image, determine the Y direction and X direction of the coordinate system, average the grayscale of the pixels on the scanned image along the Y direction, and obtain the grayscale average value evenly arranged based on the X direction;

[0127] Figure 5 is an optional grayscale average schematic diagram according to an embodiment of the present invention, such as Figure 5As shown, the grayscale values ​​of all pixels are averaged along the TDI integration direction (ie, the Y direction). This process is equivalent to folding the grayscale information in the Y direction of the image and arranging it evenly along the optical direction (ie, the X direction) to obtain a one-dimensional grayscale average value.

[0128] 2. Perform differential convolution operation on the grayscale average values ​​evenly arranged based on the X direction to obtain the grayscale variance value;

[0129] The embodiment of the present invention provides an optional differential convolution operation formula: Among them, Smd X refers to the grayscale variance value along the X direction, k refers to the number of pixels along the X direction, f(x i ,y) refers to the one-dimensional grayscale average of the i-th pixel along the X direction after the Y direction is fixed.

[0130] Optionally, since the calculated value of grayscale variance may change when the brightness is different, in order to more accurately measure the grayscale variance of the scanned image, an optional brightness normalization formula is provided: Among them, Average refers to the number of brightness involved in the normalization calculation, ∑Smd X Refers to the sum of the grayscale variance values ​​involved in the normalization calculation, Smd X _avr refers to the grayscale variance value after brightness normalization.

[0131] 3. Determine whether the scanned image has global out-of-focus abnormality based on the grayscale variance value.

[0132] When the image is clearer, the grayscale changes between adjacent pixels are more drastic, Smd X The larger the value, the more blurred the image is, and the smoother the grayscale change between adjacent pixels is. X The smaller the value.

[0133] In an embodiment of the present invention, local defocus detection is performed on the scanned image through a grayscale morphological algorithm, the image contour is detected and the boundary information is used to quantify the structural features of the dark area of ​​the image, and a classification algorithm is used to determine in real time whether there is a local defocus abnormality in the image, thereby facilitating the generation of a stop shooting instruction and troubleshooting abnormalities in the scanning imaging system that may cause local defocus (small-sized water vapor or dust mixed in the imaging light path).

[0134] In an embodiment of the present invention, global defocus detection is performed on the scanned image through machine learning and grayscale variance feature extraction methods, the normalized one-dimensional grayscale variance value is selected as the image feature, and a logistic regression model is introduced. A supervised classification algorithm is used to perform real-time judgment on whether there is a global defocus anomaly in the image, thereby facilitating the generation of a stop shooting instruction and troubleshooting abnormal conditions in the scanning imaging system that may cause global defocus (water vapor or dust of slightly larger size mixed in the imaging optical path).

[0135] The defocus detection method provided in the embodiment of the present invention can perform real-time processing and real-time detection on the scanned image during the operation of the scanning imaging system, and immediately generate a stop shooting instruction after an image with defocus abnormality is found, which is conducive to stopping shooting as soon as the imaging light path is blocked to eliminate the blockage. The above process is fully automatic and without lag, which solves the problem of low automation and lag in the prior art of using defocus detection to eliminate imaging system failures.

[0136] In the embodiment of the present invention, there is no need to additionally install an autofocus device linked to the motion platform to perform automatic focus detection and closed-loop control, and real-time detection and interception of out-of-focus anomalies can be achieved during the imaging stage, saving hardware costs.

[0137] The present invention is described below in conjunction with another optional embodiment.

[0138] Embodiment 2

[0139] A defocus detection system provided in this embodiment is used to implement the various implementation steps of the defocus detection method for a scanning imaging system in the first embodiment above.

[0140] Figure 6 is a structural diagram of an optional out-of-focus detection system according to an embodiment of the present invention, such as Figure 6 As shown, the out-of-focus detection system includes: a microscopic imaging platform 61, connected to the image processing module, for collecting a positioning image of the shooting target, scanning the shooting target to obtain a scanned image, and transmitting the positioning image and the scanned image to the image processing module; an image processing module 62, connected to the microscopic imaging platform and the host computer control module respectively, for calculating the starting point to be shot of the shooting target based on the positioning image, and performing out-of-focus detection on the scanned image; a motion platform 63, connected to the motion control module, for fixing the shooting target; a motion control module 64, connected to the motion platform and the host computer control module respectively, for performing motion control on the motion platform; and a host computer control module 65, connected to the image processing module and the motion control module respectively, for collaboratively controlling the image processing module and the motion control module.

[0141] It should be noted that the out-of-focus detection system can at least implement the following evaluation steps:

[0142] Step S101, determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of a scanning imaging system.

[0143] Step S102, moving the photographing target to the starting point to be photographed, and acquiring a scanned image of the photographing target, wherein the scanned image is obtained by scanning the photographing target on the moving platform while the moving platform moves in a predetermined moving direction.

[0144] Step S103, responding to the defocus detection instruction, performing defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection.

[0145] Step S104, when the detection result indicates that the scanned image has a defocus abnormality, a stop shooting instruction is generated, wherein the defocus abnormality includes: a local defocus abnormality or a global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0146] The defocus detection system in the embodiment of the present invention can first determine the starting point of the shooting target to be shot by implementing the above-mentioned evaluation steps, wherein the shooting target is fixed on the moving platform of the scanning imaging system, and then move the shooting target to the starting point to be shot, and obtain a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform while the moving platform moves in a predetermined direction of motion, and then respond to the defocus detection instruction, perform defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection, and finally, when the detection result indicates that the scanned image has a defocus abnormality, a stop shooting instruction is generated, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning and imaging.

[0147] In the embodiment of the present invention, the motion control technology and the image processing technology are combined, and the upper computer control module controls the motion platform to drive the shooting target through the motion control module. The shooting target is scanned by the microscopic imaging platform during the movement from the starting point to be shot in a predetermined direction of motion through the motion control technology to obtain a scanned image. The microscopic imaging platform transmits the scanned image to the image processing module, and performs local defocus detection and global defocus detection on the scanned image to obtain a detection result. When the detection result indicates that the scanned image has a defocus abnormality (including a local defocus abnormality or a global defocus abnormality), the upper computer control module generates a stop shooting instruction, and the scanning imaging system automatically stops the current scanning imaging after receiving the stop shooting instruction. The defocus detection method proposed in the embodiment of the present invention realizes real-time defocus detection of the scanned image during the scanning imaging process, and automatically generates a stop shooting instruction when the scanned image has a defocus abnormality, so as to facilitate the shutdown inspection of the scanning imaging system, thereby solving the technical problems of lag and low degree of automation in the defocus detection technology in the related art.

[0148] The present invention is described below in conjunction with another optional embodiment.

[0149] Embodiment 3

[0150] A defocus detection device for a scanning imaging system provided in this embodiment includes a plurality of implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1.

[0151] Figure 7 is a schematic diagram of an optional out-of-focus detection device for a scanning imaging system according to an embodiment of the present invention. Figure 7 As shown, the device may include: a determination unit 71, an acquisition unit 72, a detection unit 73, and a generation unit 74.

[0152] The determination unit 71 is used to determine a starting point of a target to be photographed, wherein the target is fixed on a moving platform of the scanning imaging system.

[0153] The acquisition unit 72 is used to move the shooting target to the starting point to be shot and acquire a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform while the moving platform moves in a predetermined moving direction.

[0154] The detection unit 73 is used to respond to the defocus detection instruction, perform defocus detection on the scanned image, and obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection.

[0155] The generating unit 74 is used to generate a stop shooting instruction when the detection result indicates that the scanned image has a defocus abnormality, wherein the defocus abnormality includes: a local defocus abnormality or a global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0156] The above-mentioned defocus detection device for the scanning imaging system can first determine the starting point of the shooting target to be shot through the determination unit 71, wherein the shooting target is fixed on the moving platform of the scanning imaging system, and then move the shooting target to the starting point to be shot through the acquisition unit 72, and acquire the scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform during the movement of the moving platform in a predetermined movement direction, and then respond to the defocus detection instruction through the detection unit 73, perform defocus detection on the scanned image, and obtain the detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection, and finally generate a stop shooting instruction through the generation unit 74 when the detection result indicates that the scanning image has a defocus abnormality, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

[0157] In an embodiment of the present invention, motion control technology and image processing technology are combined, a motion platform is used to fix the shooting target, and the motion control technology is used to scan the shooting target through a scanning imaging system during the process of the motion platform moving from the starting point to be shot in a predetermined direction of motion to obtain a scanned image, and local defocus detection and global defocus detection are performed on the scanned image to obtain a detection result. When the detection result indicates that the scanned image has a defocus abnormality (including local defocus abnormality or global defocus abnormality), a stop shooting instruction is generated, and the scanning imaging system automatically stops the current scanning imaging after receiving the stop shooting instruction. The defocus detection method proposed in an embodiment of the present invention realizes real-time defocus detection of the scanned image during the scanning imaging process, and automatically generates a stop shooting instruction when the scanned image has a defocus abnormality, which is convenient for shutting down the scanning imaging system for inspection, thereby solving the technical problems of lag and low degree of automation in the defocus detection technology in the related art.

[0158] Optionally, the determination unit includes: an acquisition module, which is used to acquire a positioning image of the shooting target through a moving platform when it is detected that the entire shooting target is within the shooting range, wherein the moving platform is in a stationary state during the process of acquiring the positioning image; an analysis module, which is used to analyze the features of the shooting target in the positioning image to obtain a feature set, wherein the feature set includes at least one of the following features: edge features, corner point features, and scale-invariant features; and a solution module, which is used to solve the positioning image based on the feature set to obtain a solution result, wherein the solution result includes the starting point to be shot of the shooting target.

[0159] Optionally, the detection unit includes: a grayscale integration module, used to perform grayscale integration of pixel points on the scanned image along a first preset direction to obtain a grayscale curve of the scanned image, wherein the grayscale curve is used to record one-dimensional grayscale information of the scanned image; a filtering module, used to filter the grayscale curve based on a preset filtering strategy and a preset morphological algorithm, wherein the preset morphological algorithm includes: an opening operation and a closing operation; an operation module, used to perform a bubble peak feature operation on the filtered grayscale curve to obtain a bubble peak feature value set, and generate a detection result based on the bubble peak feature value set.

[0160] Optionally, the operation module includes: a global search submodule, which is used to perform a global search on the filtered grayscale curve to obtain the global grayscale minimum of the grayscale curve, and determine the bubble peak center based on the global grayscale minimum; a feature operation submodule, which is used to perform feature operation on the grayscale curve and the bubble peak center to obtain a bubble peak feature value set, wherein the bubble peak feature values ​​in the bubble peak feature value set include at least: amplitude, kurtosis, peak slope and peak width value; a weighted operation submodule, which is used to perform weighted operation on all bubble peak feature values ​​in the bubble peak feature value set according to preset weights to obtain an operation result; a first setting submodule, which is used to set the detection result as a local out-of-focus abnormality in the scanned image when the operation result exceeds a preset threshold range; and a second setting submodule, which is used to set the detection result as a local out-of-focus abnormality in the scanned image when the operation result does not exceed the preset threshold range.

[0161] Optionally, the detection unit also includes: an image enhancement processing module, used to perform image enhancement processing on the scanned image, wherein the image enhancement processing includes at least one of the following: normalization processing, smoothing processing, difference calculation, contrast adjustment, image sharpening and histogram equalization.

[0162] Optionally, the detection unit further includes: an input module, used to input the scanned image into a preset logistic regression model; and an output module, used to perform global defocus detection on the scanned image through the preset logistic regression model and output the detection result.

[0163] Optionally, the output module includes: a grayscale average processing submodule, which is used to perform grayscale average processing on pixel points on the scanned image along a second preset direction through a preset logistic regression model to obtain a one-dimensional grayscale average value set of the scanned image in a third preset direction, wherein the second preset direction is perpendicular to the third preset direction, and the one-dimensional grayscale average value set includes M one-dimensional grayscale average values, and M is a positive integer; a differential convolution operation submodule, which is used to perform a differential convolution operation on all one-dimensional grayscale average values ​​along the third preset direction to obtain a grayscale variance value of the scanned image; a third setting submodule, which is used to set the detection result as a global out-of-focus abnormality in the scanned image when the grayscale variance value is less than or equal to a preset grayscale variance threshold; and a fourth setting submodule, which is used to set the detection result as a global out-of-focus abnormality in the scanned image when the grayscale variance value is greater than a preset grayscale variance threshold.

[0164] Optionally, the preset logistic regression model is obtained through the following submodules: an acquisition submodule, used to acquire N sample images to obtain a sample set, wherein each sample image carries an image label, and the image label is: normal image or global out-of-focus image, and N is a positive integer; an input submodule, used to input the sample set into the initial logistic regression model, and output a classification result, wherein the classification result includes N model classification labels corresponding to the N sample images, and the model classification label is obtained by the initial logistic regression model performing a differential convolution operation on each sample image to obtain a grayscale variance value, and then classifying the N sample images based on the N grayscale variance values; an adjustment submodule, used to adjust the model parameters of the initial logistic regression model when it is determined that the model classification label of any sample image is inconsistent with the image label carried by the sample image; a stop adjustment submodule, used to stop adjusting the model parameters when the model classification label of each sample image in the sample set is consistent with the image label carried by the sample image, so as to obtain the preset logistic regression model.

[0165] The above-mentioned out-of-focus detection device for a scanning imaging system may also include a processor and a memory. The above-mentioned determination unit 71, acquisition unit 72, detection unit 73, generation unit 74, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0166] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and a stop shooting instruction is generated by adjusting kernel parameters when the detection result indicates that the scanned image is out of focus abnormal.

[0167] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0168] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: determining a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of a scanning imaging system; moving the shooting target to the starting point to be shot, and acquiring a scanned image of the shooting target, wherein the scanned image is obtained by scanning the shooting target on the moving platform during the movement of the moving platform in a predetermined movement direction; responding to a defocus detection instruction, performing a defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection; when the detection result indicates that the scanned image has a defocus abnormality, generating a stop shooting instruction, wherein the defocus abnormality includes: local defocus abnormality or global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning and imaging.

[0169] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned out-of-focus detection methods for a scanning imaging system.

[0170] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned out-of-focus detection methods for a scanning imaging system.

[0171] Figure 8 1 is a hardware structure block diagram of an electronic device (or mobile device) for a defocus detection method for a scanning imaging system according to an embodiment of the present invention. Figure 8 As shown, the electronic device may include one or more ( Figure 8 802a, 802b, ..., 802n are used to illustrate) processor 802 (processor 802 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), memory 804 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 8 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 8 More or fewer components as shown, or with Figure 8 Different configurations are shown.

[0172] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0173] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0176] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0177] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0178] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting defocus of a scanning imaging system, characterized in that: include: Determine a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of a scanning imaging system; The shooting target is moved to the starting point to be shot, and a scanned image of the shooting target is obtained, wherein the scanned image is obtained by scanning the shooting target on the moving platform while the moving platform moves in a predetermined moving direction; In response to a defocus detection instruction, performing defocus detection on the scanned image to obtain a detection result, wherein the instruction types of the defocus detection instruction include: local defocus detection and global defocus detection; When the detection result indicates that the scanned image has a defocus abnormality, a stop shooting instruction is generated, wherein the defocus abnormality includes: a local defocus abnormality or a global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

2. The out-of-focus detection method according to claim 1, characterized in that: The step of determining a starting point of a shooting target to be shot comprises: When it is detected that the entirety of the photographed target is within the photographing range, a positioning image of the photographed target is acquired by the motion platform, wherein the motion platform is in a stationary state during the process of acquiring the positioning image; Analyze the features of the photographed target in the positioning image to obtain a feature set, wherein the feature set includes at least one of the following features: edge features, corner features, and scale-invariant features; The positioning image is solved based on the feature set to obtain a solution result, wherein the solution result includes the starting point to be photographed of the shooting target.

3. The out-of-focus detection method according to claim 1, characterized in that: The step of performing out-of-focus detection on the scanned image to obtain a detection result comprises: Performing grayscale integration on pixel points on the scanned image along a first preset direction to obtain a grayscale curve of the scanned image, wherein the grayscale curve is used to record one-dimensional grayscale information of the scanned image; Filtering the grayscale curve based on a preset filtering strategy and a preset morphological algorithm, wherein the preset morphological algorithm includes: an opening operation and a closing operation; A bubble peak feature operation is performed on the filtered grayscale curve to obtain a bubble peak feature value set, and the detection result is generated based on the bubble peak feature value set.

4. The out-of-focus detection method according to claim 3, characterized in that: The step of performing bubble peak feature calculation on the grayscale curve after filtering to obtain a bubble peak feature value set, and generating the detection result based on the bubble peak feature value set includes: Performing a global search on the filtered grayscale curve to obtain a global grayscale minimum value of the grayscale curve, and determining the bubble peak center based on the global grayscale minimum value; Performing feature operations on the grayscale curve and the bubble peak center to obtain the bubble peak feature value set, wherein the bubble peak feature values ​​in the bubble peak feature value set at least include: amplitude, kurtosis, peak slope, and peak width value; Performing a weighted operation on all the bubble peak characteristic values ​​in the bubble peak characteristic value set according to a preset weight value to obtain an operation result; When the calculation result exceeds the preset threshold range, the detection result is set as a partial out-of-focus abnormality of the scanned image; or, When the calculation result does not exceed the preset threshold range, the detection result is set as no local out-of-focus abnormality occurs in the scanned image.

5. The out-of-focus detection method according to claim 3, characterized in that: Before performing grayscale integration on the pixel points on the scanned image along the first preset direction, the method further includes: The scanned image is subjected to image enhancement processing, wherein the image enhancement processing comprises at least one of the following: normalization processing, smoothing processing, difference calculation, contrast adjustment, image sharpening and histogram equalization.

6. The out-of-focus detection method according to claim 1, characterized in that: The step of performing out-of-focus detection on the scanned image to obtain a detection result also includes: Inputting the scanned image into a preset logistic regression model; The scanned image is subjected to global out-of-focus detection through the preset logistic regression model, and the detection result is output.

7. The out-of-focus detection method according to claim 6, characterized in that: The step of performing global defocus detection on the scanned image by using the preset logistic regression model and outputting the detection result comprises: Performing grayscale averaging processing on the pixel points on the scanned image along a second preset direction by using the preset logistic regression model to obtain a one-dimensional grayscale average value set of the scanned image in a third preset direction, wherein the second preset direction is perpendicular to the third preset direction, and the one-dimensional grayscale average value set includes M one-dimensional grayscale average values, where M is a positive integer; Performing a differential convolution operation on all the one-dimensional grayscale average values ​​along the third preset direction to obtain a grayscale variance value of the scanned image; When the grayscale variance value is less than or equal to the preset grayscale variance threshold, the detection result is set as a global out-of-focus abnormality of the scanned image; or, When the grayscale variance value is greater than the preset grayscale variance threshold, the detection result is set as no global out-of-focus abnormality occurs in the scanned image.

8. The out-of-focus detection method according to claim 6, characterized in that: The preset logistic regression model is obtained by the following method: Collect N sample images to obtain a sample set, wherein each sample image carries an image label, and the image label is: a normal image or a global defocused image, and N is a positive integer; Input the sample set into the initial logistic regression model and output a classification result, wherein the classification result includes N model classification labels corresponding to the N sample images, and the model classification labels are obtained by performing a differential convolution operation on each of the sample images by the initial logistic regression model to obtain a grayscale variance value, and then classifying the N sample images based on the N grayscale variance values; When it is determined that the model classification label of any of the sample images is inconsistent with the image label carried by the sample image, adjusting the model parameters of the initial logistic regression model; When the model classification label of each of the sample images in the sample set is consistent with the image label carried by the sample image, the adjustment of the model parameters is stopped to obtain the preset logistic regression model.

9. A defocus detection system, characterized in that: Used to implement the defocus detection method for a scanning imaging system according to any one of claims 1 to 8, the defocus detection system comprising: A microscopic imaging platform, connected to the image processing module, for acquiring a positioning image of a photographed target, scanning the photographed target to obtain a scanned image, and transmitting the positioning image and the scanned image to the image processing module; The image processing module is connected to the microscopic imaging platform and the host computer control module respectively, and is used to calculate the starting point of the shooting target to be shot based on the positioning image, and to perform out-of-focus detection on the scanned image; A motion platform, connected to the motion control module, and used to fix the shooting target; The motion control module is connected to the motion platform and the host computer control module respectively, and is used to perform motion control on the motion platform; The host computer control module is connected to the image processing module and the motion control module respectively, and is used to coordinately control the image processing module and the motion control module.

10. A defocus detection device for a scanning imaging system, characterized in that: include: A determination unit, used to determine a starting point of a shooting target to be shot, wherein the shooting target is fixed on a moving platform of the scanning imaging system; an acquisition unit, used to move the photographing target to the starting point to be photographed, and acquire a scanned image of the photographing target, wherein the scanned image is obtained by scanning the photographing target on the moving platform while the moving platform moves in a predetermined moving direction; A detection unit, configured to respond to a defocus detection instruction, perform defocus detection on the scanned image, and obtain a detection result, wherein the defocus detection instruction may include: local defocus detection and global defocus detection; A generating unit is used to generate a stop shooting instruction when the detection result indicates that the scanned image has a defocus abnormality, wherein the defocus abnormality includes: a local defocus abnormality or a global defocus abnormality, and the stop shooting instruction is used to control the scanning imaging system to stop scanning imaging.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the out-of-focus detection method for a scanning imaging system according to any one of claims 1 to 8.

12. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the out-of-focus detection method for a scanning imaging system as described in any one of claims 1 to 8.

Images