ROI positioning method and system based on laser automatic focusing, medium and electronic equipment

Abstract

The invention discloses an ROI positioning method and system based on laser automatic focusing, a medium and electronic equipment, and the method comprises the steps: positioning a pixel line where edge pixel points, farthest from a reference line, at two ends of a hot spot in the vertical direction of the pixel line in a current frame of laser image are located, so as to set the ROI of a next frame of laser image; acquiring a pixel row with a larger spacing with a reference row in the pixel rows where the edge pixel points at the two ends are located, so as to set a row boundary of the ROI; according to the invention, the upper and lower boundaries in the vertical direction of the pixel row are determined through the position of the light spot in the current frame image, the row boundary of the ROI is set based on one of the upper and lower boundaries having a larger spacing with the reference row, the ROI is formed by vertical symmetry with the reference row, and the size and position of the ROI are adjusted in real time so as to capture the light spot in the next frame image. The continuity of effective laser spot image acquisition is ensured, the refresh rate for calculating the defocusing amount through the spot mass center is improved, and high-speed real-time focusing is realized.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a ROI positioning method, system, medium and electronic equipment based on laser autofocus. Background Art

[0002] Autofocus devices use an area array sensor to capture a laser image reflected from the target object (such as a wafer or screen). By calculating the difference between the center of mass of each laser image frame and a reference center of mass, the sensor determines the distance between the current position and the focal point, known as the defocus value. This sensor then controls the Z-axis to move the corresponding distance to achieve real-time focusing. High-speed focus tracking requires a sufficiently fast refresh rate (on the order of 10 kHz) for the defocus value. Typical full-frame area scan cameras only have a frame rate of 400-500 fps, which is insufficient for such a high refresh rate. Therefore, the ROI function is essential.

[0003] In the actual focus tracking process, as the defocus amount changes, the size and offset of the laser image also change continuously. A fixed ROI cannot always capture the complete laser image, which affects the defocus amount calculation.

[0004] Patent CN108024065A discloses a terminal photography method, terminal, and computer-readable storage medium. The method includes: obtaining an initial image of a shooting scene; identifying a subject in the initial image and determining the subject's positional information in the initial image; adjusting a region of interest (ROI) in the initial image based on the subject's positional information; ensuring that the percentage of the subject's area in the adjusted ROI is greater than a first threshold; and capturing the scene using the adjusted ROI as the focus area to obtain a final image. In this way, the ROI size is adaptively adjusted based on the subject's size, improving the accuracy of focusing on the object.

[0005] Patent CN115546316A discloses an automatic ROI setting method for an industrial camera, which relates to the technical field of industrial inspection. The method includes: aiming the industrial camera lens at the upper surface of a conveyor belt, capturing an image of the conveyor belt with an arbitrary exposure time and excluding a test object as a calibration image, and recording the row mean, column mean, and exposure time of the calibration image as calibration values; after starting the conveyor belt, the test object on the conveyor belt enters the field of view of the industrial camera, and the industrial camera continuously captures images at a certain frame rate as test images, and records the row mean, column mean, and exposure time corresponding to all test images; subtracting the row mean and column mean of the test image from the row mean and column mean of the calibration image to obtain a row change curve and a column change curve; searching for all inflection points of the above row change curve and column change curve as the corner points of the ROI; and cropping the image into multiple rectangular intervals based on all ROI corner points and merging them into a large rectangle as the ROI output.

[0006] None of the above patents can cope with sudden bright spots or other environmental interference in the image, which causes the ROI area to be unable to cover the target subject of the next frame image, affecting the image acquisition frame rate.

[0007] Therefore, in order to solve the above problems, the present invention provides an ROI positioning method based on laser autofocus, which can adjust the size and offset of the area array sensor ROI frame by frame, achieve a high refresh rate while ensuring the capture of complete laser images, and improve the focus tracking performance. Summary of the Invention

[0008] The purpose of the present invention is to overcome the above problems existing in the prior art and provide a ROI positioning method, system, medium and electronic equipment based on laser autofocus.

[0009] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions: A ROI positioning method based on laser autofocus achieves real-time autofocus by dynamically adjusting the ROI to adapt to and capture the laser spot, including: Locate the pixel rows where the edge pixels of the bright spot in the vertical direction of the pixel row are farthest from the reference row at both ends of the pixel row in the current frame laser image to set the ROI of the next frame laser image; Obtaining a pixel row with a larger distance from a reference row among the pixel rows where the edge pixel points at both ends are located, so as to set a row boundary of the ROI; The row boundary is formed as another row boundary of the ROI by being axially symmetrical with the reference row, so that when an abnormal bright spot exists in the bright spot, the ROI located can accommodate a smaller laser spot in the next frame image approaching the focus; Wherein, the reference line is the pixel row in the image where the centroid position of the laser line formed in the focused state of the laser beam is rounded off;

[0010] The bright spot includes a laser spot or an abnormal bright spot.

[0011] Furthermore, locating the pixel rows where the edge pixel points at both ends of the bright spot in the vertical direction of the pixel row are farthest from the reference row in the current frame laser image includes: Binarize the image using a first threshold, and search the upper and lower regions of the reference row respectively, searching from the first row and the last row of the image towards the reference row, using the first binary image intersection as one of the boundaries of the bright spot, and determining the upper and lower boundaries of the bright spot based on the intersection line; The first threshold is obtained based on background noise calibration.

[0012] Furthermore, the method of using the first binary image boundary line as one of the boundaries of the bright spot includes: In the area where no boundary line is found, the row obtained by multiplying the row spacing from the first row or the last row of the image in the area to the reference row by the first preset coefficient is used as one of the boundaries of the bright spot.

[0013] Furthermore, locating the pixel row where the edge pixel points at both ends of the light spot in the vertical direction of the pixel row are farthest from the reference row in the current frame laser image includes: Obtain the image row mean curve, and locate the rows where the corresponding points on both sides of the highest peak have row means less than the second threshold value, to obtain the absolute value of the row difference; Multiplying the absolute value of the row difference by a second preset coefficient to obtain the bright spot row height; Based on the row height of the bright spot, the upper and lower boundaries of the bright spot are obtained symmetrically with the row where the highest peak vertex is located.

[0014] Furthermore, the second threshold is greater than the second highest peak vertex and less than the row mean corresponding to the highest peak vertex.

[0015] Furthermore, the two column boundaries of the ROI are the pixel columns where the positions of the two end points of the laser line are rounded off in the image.

[0016] Furthermore, the first preset coefficient is greater than 1, so as to reserve space for the change of the laser spot in the next frame.

[0017] A ROI positioning system based on laser autofocus, comprising: The positioning module locates the pixel rows where the edge pixels of the bright spot in the current frame laser image are farthest from the reference row in the vertical direction of the pixel row, so as to set the ROI of the next frame laser image; An acquisition module is configured to acquire a pixel row with a larger distance from a reference row in the pixel rows where the edge pixels at both ends are located, so as to set a row boundary of the ROI; A processing module forms another row boundary of the ROI by axially symmetricly taking the reference row as the row boundary, so that when an abnormal bright spot exists in the bright spot, the ROI located can accommodate a smaller laser spot in the next frame image approaching the focus; Wherein, the reference line is the pixel row in the image where the centroid position of the laser line formed in the focused state of the laser beam is rounded off; The bright spot includes a laser spot or an abnormal bright spot.

[0018] A storage medium stores a computer program thereon, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned ROI positioning method based on laser autofocus.

[0019] An electronic device, comprising: processor; and a memory for storing executable instructions of the processor; The processor is used to execute the executable instructions to implement the above-mentioned ROI positioning method based on laser autofocus.

[0020] The beneficial effects of the present invention are: (1) In the present invention, the upper and lower boundaries of the pixel row in the vertical direction are determined by the position of the bright spot in the current frame image, and the row boundary of the ROI is set based on the upper and lower boundaries with a larger distance from the reference row. The complete ROI area is formed by upper and lower symmetry with the reference row, and the size and position of the ROI are adjusted in real time to capture the laser spot in the next frame image, ensuring the continuity of the acquisition of the effective laser spot image, improving the refresh rate of the defocus amount calculated by the laser spot centroid, and realizing high-speed real-time focusing.

[0021] By locating the upper and lower boundaries of the bright spot in the vertical direction of the pixel row in the current frame laser image, the ROI of the next frame image can be reasonably predicted by adding reserved space through the bright spot position of the current frame without consuming too much computing resources; By obtaining the boundary with the larger spacing from the reference line among the upper and lower boundaries to set a row boundary of the ROI, the laser spot position in the next frame image is accurately predicted, so that the ROI can accommodate the next frame laser spot image, ensuring that the spot centroid can be effectively calculated for each frame image, so as to achieve high-speed refresh of defocus calculation and realize real-time focusing; By symmetrically setting the row boundary to form another row boundary of the ROI, the ROI is symmetrically distributed about the reference row. On the one hand, the laser spot appearing near the reference row can be accurately predicted. On the other hand, the sudden change of the bright spot appearing at the edge of the image caused by the environmental signal can be overcome to ensure the continuity of the effective laser spot image.

[0022] (2) In the present invention, the image is binarized based on the first threshold value to filter the influence of background noise on the recognition of the light spot boundary, and the upper and lower boundaries of the bright spot are quickly and accurately identified by the binarized image boundary line. The bright spot position in the current frame image is adaptively scaled by the first preset coefficient to predict and accommodate the laser light spot in the next frame image. The bright spot boundary is located by the binarized image boundary line to improve the accuracy, and the ROI boundary range is limited for positioning, which greatly improves the image data processing speed, maintains a high refresh rate of the laser light spot centroid, and improves the focus tracking performance.

[0023] (3) In the present invention, the image row mean curve is obtained so that the laser spot image is mapped to the vicinity of the highest peak of the curve, and the row where the two points of the highest peak are located is indexed based on the second threshold to locate the peak width of the highest peak. The peak width is scaled by the second preset coefficient to estimate the row height of the bright spot, thereby overcoming the loss of spot information on one side caused by the asymmetry of the highest peak. Based on the row height of the bright spot, the row where the highest peak vertex is located is symmetrically set to obtain the estimated upper and lower boundaries of the spot. The second threshold is set to be greater than the second highest peak vertex and less than the row mean corresponding to the highest peak vertex, so as to eliminate the interference of bright spots or background noise outside the laser spot image, thereby having stronger anti-interference ability, more stable refresh rate and better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 It is a flow chart of the ROI positioning method based on laser autofocus in the present invention; Figure 2 It is a flowchart of the ROI positioning system based on laser autofocus in the present invention; Figure 3 Schematic diagram of the row mean curve of Example 2 of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0026] In laser autofocus, the defocus state of the objective lens is determined by the spot shape of a semi-elliptical laser beam focused through the objective lens onto the surface of the object being measured. The defocus amount is calculated by the centroid of the laser spot image formed by reflection back to the image sensor. Based on the defocus state and defocus amount, the distance from the objective lens to the surface of the object being measured is adjusted to ensure that the focal point of the objective lens is located on the surface of the object being measured, achieving real-time focusing. High-speed focus tracking requires a sufficiently fast refresh rate for the defocus amount (on the order of 10 kHz). Typical full-frame area scan cameras can only reach a frame rate of 400-500 fps, which cannot meet the high refresh rate requirements and must be combined with the ROI function.

[0027] However, the existing autofocus ROI settings have the following problems: 1) Using full-frame to shoot laser images results in a low refresh rate and is not suitable for demanding follow-focus situations. 2) Full-frame cameras are prone to capturing stray light, which affects the calculation of defocus. 3) Fixed ROI cannot capture the complete laser image in the far focus state, affecting the focusing performance.

[0028] In order to solve the above problems, Figure 1 As shown, the present invention first provides a ROI positioning method based on laser autofocus, which realizes real-time autofocus by dynamically adjusting the ROI to adapt to and capture the laser spot, including: Locate the pixel rows where the edge pixels of the bright spot in the vertical direction of the pixel row are farthest from the reference row at both ends of the pixel row in the current frame laser image to set the ROI of the next frame laser image; Obtaining a pixel row with a larger distance from a reference row among the pixel rows where the edge pixel points at both ends are located, so as to set a row boundary of the ROI; The row boundary is formed as another row boundary of the ROI by being axially symmetrical with the reference row, so that when an abnormal bright spot exists in the bright spot, the ROI located can accommodate a smaller laser spot in the next frame image approaching the focus; Wherein, the reference line is the pixel row in the image where the centroid position of the laser line formed in the focused state of the laser beam is rounded off; The bright spots include laser spots or abnormal bright spots.

[0029] In this embodiment, the ROI of the next frame of laser image is set by locating the upper and lower boundaries of the bright spot in the vertical direction of the pixel row in the current frame of laser image; the boundary with a larger distance from the reference row among the upper and lower boundaries is obtained to set a row boundary of the ROI, so that the ROI can completely accommodate the next frame of laser spot image; the row boundaries are symmetrically set to form another row boundary of the ROI, so that the ROI is symmetrically distributed about the reference row, thereby overcoming the sudden change of bright spots at the edge of the image caused by environmental signals.

[0030] In this embodiment, based on the laser autofocus application scenario, the spot shape includes an upper half ellipse or a lower half ellipse, indicating that the focused object is in different defocus directions. When the spot shape becomes a laser line, it indicates that the object to be measured is in focus, and in the focusing process, the change of the spot is often linear and gradual, and is concentrated near the laser line. From the image, the change of the laser spot image is concentrated near the pixel row where the laser line is located, that is, the reference row. Therefore, the ROI setting cannot be separated from the reference row so that the ROI positioning can accurately predict the position of the laser spot image in the next frame.

[0031] In this embodiment, the change of the laser spot image of the next frame is predicted based on the laser spot image of the current frame, and the corresponding ROI is set to reduce the image output data and data processing volume to improve the data processing speed, ensure the refresh rate of the laser spot image to increase the center of mass calculation speed, and realize real-time autofocus.

[0032] In this embodiment, the change of the laser spot occurs in the vertical direction of the pixel row, and there is only a slight change in the spot width in the column direction. Therefore, it is only necessary to locate the boundary of the bright spot in the vertical direction of the pixel row to predict and limit the vertical position of the pixel row of the next frame ROI. The reason for doing this is that the change of the spot shape during the autofocus process is basically linear and gradual, rather than sudden. Therefore, it is only necessary to base it on the laser spot image position of the current frame and expand the appropriate coefficient to obtain the ROI of the next frame. Specifically, the coefficient can be obtained through experimental calibration.

[0033] In this embodiment, the pixel row where the centroid of the laser line formed by the laser beam in the focused state is located is used as the reference row. If the solved centroid position is non-integer, it is rounded and the integer pixel row where it is located is used as the reference row. This is set based on the laser autofocus characteristics. During the focusing process, the light spot changes always around the laser line, and the defocus direction is determined by judging the upper and lower semi-ellipse directions. Therefore, the pixel row where the laser line is located is used as the reference row, which can maximize the accuracy of ROI position positioning.

[0034] In this embodiment, for the autofocus process of the same focus object, the laser spot shape in the next frame image is smaller than the laser spot shape in the current frame image, and the smaller laser spot in the next frame image is closer to the focus. Therefore, relative to the ROI, the ROI of the next frame converges frame by frame to continuously reduce hardware computing resources and improve the efficiency of laser spot centroid calculation.

[0035] In this embodiment, after locating the upper and lower boundaries of the bright spot in the vertical direction of the pixel row, it is necessary to further obtain the end of the upper and lower boundaries that is farther away from the reference row as one of the boundaries of the ROI. After reserving a certain space on the boundary, an accurate prediction of the next frame of laser spot image can be achieved, thereby avoiding the ROI being unable to capture the next frame of laser spot image.

[0036] In this embodiment, an ROI row boundary is formed based on the bright spot boundary farther away from the reference row and is symmetrical about the reference row to form another row boundary of the ROI. The area formed between the two row boundaries is used as the ROI of the next frame image. An ROI that is always symmetrical about the reference row can be formed to adapt to the upper semi-elliptical or lower semi-elliptical shape of the laser spot and satisfy the focusing state of different defocus directions without causing a situation where the ROI area cannot frame the laser spot image.

[0037] In this embodiment, the reason for setting the bright spot's farther boundary axially symmetrically with respect to the reference line is that: In laser autofocus, environmental signals include the background signal between the two focused objects when the focused object moves out of the observation range and re-enters the observation range, or the influence of lens stray light, which will cause abnormal bright spots to appear at the edge of the image, thereby affecting the recognition and judgment of the ROI area, causing the ROI to be set at the edge of the image, so that the next frame of the ROI image that converges frame by frame cannot accommodate the laser spot, making it impossible to determine the center of mass of the laser spot, thereby affecting the autofocus effect.

[0038] In this embodiment, the bright spot includes the laser spot formed by the laser beam focusing on the focused object and the abnormal bright spot. Therefore, by locating the upper and lower boundaries of the bright spot, when an abnormal bright spot appears that is farther away from the reference line than the laser spot, the boundary of the abnormal bright spot is used as the spot boundary to ensure that the laser spot of the next frame will not be lost in the located ROI, so that real-time focus tracking can be achieved by converging the ROI frame by frame.

[0039] Furthermore, unlike the prior art, by identifying the farther boundary of the light spot and limiting the row position of the ROI symmetrically about the reference row, when abnormal conditions such as abnormal bright spot mutation or environmental interference occur in the image, the abnormal bright spot boundary is used as the row boundary of the ROI. The image output range of the ROI formed after symmetry with the reference row is higher than that of the ROI formed by the light spot boundary. Therefore, the image of the next frame will still be contained in the ROI, thereby avoiding the influence of the bright spot mutation that occurs at the edge of the image when the focused object moves out and then moves back into the image.

[0040] In the present invention, the upper and lower boundaries of the pixel row in the vertical direction are determined by the position of the bright spot in the current frame image, and the row boundary of the ROI is set based on the upper and lower boundaries with a larger distance from the reference row. The complete ROI area is formed symmetrically with the reference row, and the ROI size and position are adjusted in real time to capture the laser spot in the next frame image, ensuring the continuity of the acquisition of the effective laser spot image, improving the refresh rate of the defocus amount calculated by the laser spot centroid, and realizing high-speed real-time focusing.

[0041] By locating the upper and lower boundaries of the bright spot in the vertical direction of the pixel row in the current frame laser image, the ROI of the next frame image can be reasonably predicted by adding reserved space through the bright spot position of the current frame without consuming too much computing resources; By obtaining the boundary with the larger spacing from the reference line among the upper and lower boundaries to set a row boundary of the ROI, the laser spot position in the next frame image is accurately predicted, so that the ROI can accommodate the next frame laser spot image, ensuring that the spot centroid can be effectively calculated for each frame image, so as to achieve high-speed refresh of defocus calculation and realize real-time focusing; By symmetrically setting the row boundary to form another row boundary of the ROI, the ROI is symmetrically distributed about the reference row. On the one hand, the laser spot appearing near the reference row can be accurately predicted. On the other hand, the sudden change of the bright spot appearing at the edge of the image caused by the environmental signal can be overcome to ensure the continuity of the effective laser spot image.

[0042] In fact, due to the influence of sudden situations, the positioned ROI area in the present invention cannot guarantee to completely accommodate the laser spot in the next frame image. However, based on the identification of the bright spot boundary including the abnormal bright spot and the laser spot in the present invention and the symmetrical setting of the ROI about the reference line, the next frame ROI image contains at least a part of the laser spot image. The ROI area is scaled frame by frame through the laser spot image, so that the laser spot that converges frame by frame can be finally completely accommodated.

[0043] In order to accurately locate the position of the bright spot in the image, in some embodiments, locating the pixel row where the edge pixels at both ends of the bright spot in the vertical direction of the pixel row are farthest from the reference row in the current frame laser image includes: Binarize the image using a first threshold, and search the upper and lower regions of the reference row respectively, searching from the first row and the last row of the image towards the reference row, using the first binary image intersection as one of the boundaries of the bright spot, and determining the upper and lower boundaries of the bright spot based on the intersection line; The first threshold is obtained based on background noise calibration.

[0044] In this embodiment, the pixel value of each pixel in the image is binarized to facilitate the distinction between the laser spot image and the background image in the image, so as to quickly identify the upper and lower boundaries of the bright spot in the vertical direction of the pixel row, and divide the image into upper and lower areas by the reference row to search for the bright spot boundary respectively, so as to facilitate the subsequent comparison of the distance from the boundary to the reference row and symmetrical processing. The binarized image can be set to black and white areas for distinction, or to be distinguished by setting 1 and 0. In this case, the boundary between the black and white areas or 1 and 0 is considered to be the boundary of the bright spot. Therefore, the upper and lower boundaries of the bright spot can be located by determining the pixel row where the boundary line is located.

[0045] In this embodiment, during background noise calibration, an image of the target object is collected in the absence of a laser beam to count pixel values, and a first threshold is obtained based on this to eliminate the influence of background noise.

[0046] In some embodiments, taking the first binary image boundary line as one of the boundaries of the bright spot includes: In the area where no boundary line is found, the row obtained by multiplying the row spacing from the first row or the last row of the image in the area to the reference row by the first preset coefficient is used as one of the boundaries of the bright spot.

[0047] In this embodiment, if the boundary line of the binary image cannot be identified in the upper and lower half areas of the reference row, it means that the boundary of the laser spot image is located outside the image. The half area where the boundary of the laser spot image is not identified is expanded by a multiple of the first preset coefficient to serve as one of the row boundaries of the ROI, so that the laser spot image can be completely accommodated in the next frame image.

[0048] In this embodiment, due to the characteristics of laser autofocus, the focus object needs to be in focus, that is, changing from a far-focus state to a near-focus state, and the height of the bright spot row continues to shrink. Therefore, the area where the intersection line is searched directly uses the spot boundary as one of the row boundaries of the ROI, which can fully accommodate the next frame of laser spot image.

[0049] In this embodiment, the first preset coefficient can be obtained through experimental calibration.

[0050] Example 1

[0051] In this embodiment, the light spot boundary is quickly and accurately found through the binarized image. The specific steps are as follows: A1. Initialize the row height to H0 and the column width to W0.

[0052] A2. Collect laser image I(h,w).

[0053] A3. Binarize the image using the first threshold TH1 to obtain a mask image M(h,w). Values ​​greater than the first threshold TH1 are assigned a value of 1, indicating that they belong to the laser spot; values ​​less than the first threshold TH1 are assigned a value of 0, indicating that they belong to the background.

[0054] A4. Start searching from row 0 and stop searching in two situations: (a) Stop searching when a laser spot is found in the row (the total mask value in the row is greater than 0); (b) Stop searching when the row H containing the zero point is found. ref0 , which is the base row. The record row index is H1.

[0055] A5. Determine whether H1=0: If it is, it means that the upper end of the upper half of the current laser image has contacted the laser spot. Calculate the adjusted height of the upper half as follows:

[0056] Where abs is the absolute value, r_expand represents the width magnification, i.e. the first preset coefficient, which is a coefficient greater than 1. A value that is too large will cause a drastic change in the image height, while a value that is too small will cause the image height to be adjusted untimely. For example, a value of about 1.2 can be selected through experiments. Otherwise, it means that the upper half of the current laser image can contain the laser spot. Calculate the height: D1=abs(H1-H ref0 )

[0057] A6, from Hmax Searching from row -1 upwards, the search stops in two cases: (a) when a laser spot is found in the row (the total mask of the row is greater than 0); (b) when the zero point is found in row H. ref0 The record row index is H2.

[0058] A7. Determine whether H2 = 0. If it is, it means that the lower end of the lower half of the current laser image has contacted the laser spot. Calculate the adjusted height of the lower half as follows:

[0059] Otherwise, it means that the lower half of the current laser image can contain the laser spot, and the height is calculated as: D2=abs(H2-H ref0 )

[0060] A8. Calculate the adjusted laser image half height D = max(D1, D2).

[0061] The corresponding starting line: H start =H ref0 -D, row height H=2D.

[0062] A9. Set the ROI row boundary symmetrically with the reference row according to the starting row and the row height, and go to step A2 to iteratively execute the above steps.

[0063] Specifically, in the above steps, an image sensor overflow judgment is performed before setting the final ROI area. When the image sensor range is exceeded, the ROI is set to the full image to obtain the next frame of laser spot image.

[0064] In the present invention, the image is binarized based on a first threshold to filter out the influence of background noise on the recognition of the light spot boundary, and the upper and lower boundaries of the bright spot are quickly and accurately identified by the binarized image boundary line. The bright spot position exceeding the current frame image is adaptively scaled by a first preset coefficient to predict and accommodate the laser spot in the next frame image. The bright spot boundary is located by the binarized image boundary line to improve accuracy, and the ROI boundary range is limited for positioning, which greatly improves the image data processing speed, maintains a high refresh rate of the laser spot centroid, and improves the focus tracking performance.

[0065] In order to further reduce the influence of noise on the determination of the light spot position, in some embodiments, locating the pixel row where the edge pixels at both ends of the light spot in the vertical direction of the pixel row are farthest from the reference row in the current frame laser image includes: Obtain the image row mean curve, and locate the rows where the corresponding points on both sides of the highest peak have row means less than the second threshold value, to obtain the absolute value of the row difference; Multiplying the absolute value of the row difference by a second preset coefficient to obtain the bright spot row height; Based on the row height of the bright spot, the upper and lower boundaries of the bright spot are obtained symmetrically with the row where the highest peak vertex is located.

[0066] In this embodiment, the average value or total value of each row of pixels is obtained based on the image pixel values ​​to form a row average curve, which is used to reflect the difference in pixel values ​​of each row in the image, where Figure 3 As shown, there are several peaks in the row average curve, among which the highest peak reflects the area where the laser spot image is located, and the other peaks represent abnormal conditions such as background noise or sudden bright spots. The second threshold is set to remove the influence of noise in the row average curve, and the height of the bright spot in the vertical direction of the pixel row is determined by locating the peak width of the highest peak under the second threshold. Based on the asymmetry of the highest peak in the row average curve, the bright spot is not symmetrical about the row where the highest peak vertex is located. Therefore, it is necessary to expand the peak width by a second preset coefficient to obtain an estimated bright spot row height to include the bright spot on one side of the asymmetric peak and prevent the bright spot from being lost. Then, the upper and lower boundaries of the bright spot are estimated based on the bright spot row height.

[0067] In order to highlight the difference between the laser spot and other abnormal bright spot noise in the image, in some embodiments, the second threshold is greater than the second peak vertex and less than the row mean corresponding to the highest peak vertex.

[0068] In this embodiment, the second threshold is used to remove the influence of noise on the identification of the bright spot boundary. Therefore, in the row mean curve, all peaks except the highest peak represent noise. Therefore, the second threshold is set to the row pixel mean that is greater than the second highest peak and less than the highest peak to eliminate the influence of noise other than the laser spot image on the positioning accuracy of the spot boundary. In actual process, the second threshold can be set through experimental calibration or experience.

[0069] Example 2

[0070] In this embodiment, the peak width is calculated based on the row mean curve to determine the spot height, thereby locating the upper and lower boundaries of the spot. The specific steps are as follows: B1. Initialize the row height to H0 and the column width to W0.

[0071] B2. Collect laser image I(h,w).

[0072] B3. Calculate the row mean curve:

[0073] B4. Set the second threshold. In this embodiment, the following method is used to obtain the maximum value V in the row mean curve in real time: max and its row H peak and minimum value V min , where H peak The row with the highest peak.

[0074] Then the peak width threshold is obtained:

[0075] r_peak is a ratio ranging from 0 to 1 and can be adjusted (recommended value = 0.5).

[0076] B5, select row H where the maximum value is located peak Search the upper and lower half graphs respectively and find the first row on both sides whose mean is less than TH peak The corresponding rows have record row indexes of H1 and H2 respectively.

[0077] B6. Calculate peak width D peak =H2-H1, calculate the height including the laser spot: D=r_multi×D peak

[0078] r_multi is the second preset coefficient, which is greater than 1 and is obtained through experiments.

[0079] B7. Calculate the index of the two end rows containing the laser spot:

[0080] B8. Calculate the distance from the row index at both ends to the row where the zero point is located:

[0081] B9. Calculate the adjusted laser image half height D = max(D1, D2). The corresponding starting row is: H start =H ref0 -D, row height H=2D.

[0082] B10. Set the ROI row boundary symmetrically with the reference row according to the starting row and the row height, and proceed to step B2 to iteratively execute the above steps.

[0083] In the present invention, the image row mean curve is obtained so that the laser spot image is mapped to the vicinity of the highest peak of the curve, and the row where the two points of the highest peak are indexed based on the second threshold to locate the peak width of the highest peak. The peak width is scaled by the second preset coefficient to estimate the bright spot row height, thereby overcoming the lack of spot information on one side caused by the asymmetry of the highest peak, and based on the bright spot row height, a symmetrical arrangement is made with the row where the highest peak vertex is located to obtain the estimated upper and lower boundaries of the spot. The second threshold is set to be greater than the secondary peak vertex and less than the row mean corresponding to the highest peak vertex, so as to eliminate the bright spot or background noise interference outside the laser spot image, thereby having stronger anti-interference ability, more stable refresh rate and better performance.

[0084] In order to further reduce hardware computing resources and improve the refresh rate, in some embodiments, the ROI column width is set to a fixed length greater than the laser line length, and its column boundary is located by calibration.

[0085] In this embodiment, based on the shape characteristics of the laser spot during the autofocus process, there are changes in row height in the vertical direction of the pixel row, while there are only slight differences in column width in the column direction. Therefore, the position of the laser line can be calibrated in advance and the ROI column width can be set to a fixed length greater than the laser line length to locate the two column boundaries of the ROI in the column direction.

[0086] In order to cope with the change of the laser spot image of the next frame, the first preset coefficient and the second preset coefficient are both greater than 1, so as to reserve space for the change of the laser spot of the next frame.

[0087] In this embodiment, the upper and lower boundaries of the spot located based on the laser spot image of the current frame need to be multiplied by a preset coefficient to appropriately expand the ROI, so as to adapt to the changes in the spot shape of the next frame to accommodate the laser spot image of the next frame. Therefore, the preset coefficient needs to be greater than 1 to achieve the expansion of the ROI by a preset multiple based on the spot boundary. The specific coefficient setting can be obtained based on experimental calibration.

[0088] Based on the same inventive concept, Figure 2 As shown, the present invention also provides a ROI positioning system based on laser autofocus, comprising: The positioning module locates the pixel rows where the edge pixels of the bright spot in the current frame laser image are farthest from the reference row in the vertical direction of the pixel row, so as to set the ROI of the next frame laser image; An acquisition module is configured to acquire a pixel row with a larger distance from a reference row in the pixel rows where the edge pixels at both ends are located, so as to set a row boundary of the ROI; A processing module forms another row boundary of the ROI by axially symmetricly taking the reference row as the row boundary, so that when an abnormal bright spot exists in the bright spot, the ROI located can accommodate a smaller laser spot in the next frame image approaching the focus; Wherein, the reference line is the pixel row in the image where the centroid position of the laser line formed in the focused state of the laser beam is rounded off; The bright spot includes a laser spot or an abnormal bright spot.

[0089] Based on the same inventive concept, the present invention further provides a storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned ROI positioning method based on laser autofocus.

[0090] Based on the same inventive concept, the present invention finally provides an electronic device, comprising: processor; and a memory for storing executable instructions of the processor; The processor is used to execute the executable instructions to implement the above-mentioned ROI positioning method based on laser autofocus.

[0091] Throughout this specification, references to terms such as "one embodiment," "example," and "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0092] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention, and such changes and modifications fall within the scope of the invention as claimed.

Claims

1. A ROI positioning method based on laser autofocus, which realizes real-time autofocus by dynamically adjusting the ROI to adapt to and capture the laser spot, characterized in that: include: Locate the pixel rows where the edge pixels of the bright spot in the vertical direction of the pixel row are farthest from the reference row at both ends of the pixel row in the current frame laser image to set the ROI of the next frame laser image; Obtaining a pixel row with a larger distance from a reference row among the pixel rows where the edge pixel points at both ends are located, so as to set a row boundary of the ROI; The row boundary is formed as another row boundary of the ROI by being axially symmetrical with the reference row, so that when an abnormal bright spot exists in the bright spot, the ROI located can accommodate a smaller laser spot in the next frame image approaching the focus; Wherein, the reference line is the pixel row in the image where the centroid position of the laser line formed in the focused state of the laser beam is rounded off; The bright spot includes a laser spot or an abnormal bright spot.

2. The ROI positioning method based on laser autofocus according to claim 1, characterized in that: Locate the pixel rows where the edge pixels of the bright spot in the current frame laser image are farthest from the reference row at both ends of the pixel row in the vertical direction, including: Binarize the image using a first threshold, and search the upper and lower regions of the reference row respectively, searching from the first row and the last row of the image towards the reference row, using the first binary image intersection as one of the boundaries of the bright spot, and determining the upper and lower boundaries of the bright spot based on the intersection line; The first threshold is obtained based on background noise calibration.

3. The ROI positioning method based on laser autofocus according to claim 2, characterized in that: The method of using the first binary image boundary line as one of the boundaries of the bright spot includes: In the area where no boundary line is found, the row obtained by multiplying the row spacing from the first row or the last row of the image in the area to the reference row by the first preset coefficient is used as one of the boundaries of the bright spot.

4. The ROI positioning method based on laser autofocus according to claim 1, characterized in that: The method of locating the pixel rows where the edge pixel points at both ends of the light spot in the vertical direction of the pixel row are farthest from the reference row in the current frame laser image include: Obtain the image row mean curve, and locate the rows where the corresponding points on both sides of the highest peak have row means less than the second threshold value, to obtain the absolute value of the row difference; Multiplying the absolute value of the row difference by a second preset coefficient to obtain the bright spot row height; Based on the row height of the bright spot, the upper and lower boundaries of the bright spot are obtained symmetrically with the row where the highest peak vertex is located.

5. The ROI positioning method based on laser autofocus according to claim 4, characterized in that: The second threshold is greater than the second highest peak vertex and less than the row mean corresponding to the highest peak vertex.

6. A ROI positioning method based on laser autofocus according to any one of claims 1 to 4, characterized in that: The two column boundaries of the ROI are the pixel columns where the positions of the two end points of the laser line are rounded in the image.

7. The ROI positioning method based on laser autofocus according to claim 3, characterized in that: The first preset coefficient is greater than 1 to reserve space for the change of the laser spot in the next frame.

8. A ROI positioning system based on laser autofocus, characterized in that: include: The positioning module locates the pixel rows where the edge pixels of the bright spot in the current frame laser image are farthest from the reference row in the vertical direction of the pixel row, so as to set the ROI of the next frame laser image; An acquisition module is configured to acquire a pixel row with a larger distance from a reference row in the pixel rows where the edge pixels at both ends are located, so as to set a row boundary of the ROI; A processing module forms another row boundary of the ROI by axially symmetricly taking the reference row as the row boundary, so that when an abnormal bright spot exists in the bright spot, the ROI located can accommodate a smaller laser spot in the next frame image approaching the focus; Wherein, the reference line is the pixel row in the image where the centroid position of the laser line formed in the focused state of the laser beam is rounded off; The bright spot includes a laser spot or an abnormal bright spot.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements a ROI positioning method based on laser autofocus according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the executable instructions to implement the ROI positioning method based on laser autofocus according to any one of claims 1 to 7.

Images