Laser auto-focusing based roi positioning method, system, medium and electronic device

Abstract

The application discloses a kind of ROI positioning method, system, medium and electronic equipment based on laser automatic focusing, the method comprises: positioning the edge pixel point of the farthest in the pixel row of the current frame laser image bright spot in the vertical direction of pixel row from reference row, to set the ROI of next frame laser image;The pixel row of the edge pixel point of two ends is obtained, and the pixel row with greater spacing from reference row is set to set one row boundary of ROI;In the application, the upper and lower boundaries of pixel row vertical direction are determined by the position of the spot in the current frame image, to set the row boundary of ROI based on the greater spacing from reference row in upper and lower boundaries, and form ROI with reference row to be symmetrical, real-time adjustment ROI size and position, to capture the spot in next frame image, ensure the continuity of effective laser spot image acquisition, improve the refresh rate of defocus amount calculated by spot centroid, realize high-speed real-time focusing.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to a ROI positioning method and system based on laser automatic focusing, a medium and an electronic device. BACKGROUND

[0002] In the automatic focusing device, a face array sensor is used to shoot the laser image reflected by the focusing object (wafer, screen, etc.), the distance between the current position and the focus position, i.e. the defocus amount, is obtained by calculating the difference between the centroid position of each frame of laser image and the reference centroid position, and then the Z-axis is controlled to move by a corresponding distance to realize real-time focusing. In order to realize high-speed focusing, the refresh rate of the defocus amount needs to be fast enough (10KHz order), and the frame rate of the general face scanning camera can only reach 400-500fps, which cannot meet the requirement of high refresh rate, and therefore the ROI function must be used.

[0003] However, in the actual focusing process, with the change of the defocus amount, the size and offset of the laser image also change constantly, and the fixed ROI cannot always shoot the complete laser image, which affects the calculation of the defocus amount.

[0004] Patent CN108024065A discloses a terminal shooting method, a terminal and a computer readable storage medium, which comprises: acquiring an initial shooting image of a shooting scene; identifying a shooting subject in the initial shooting image, and determining the position information of the shooting subject in the initial shooting image; adjusting the region of interest (ROI) in the initial shooting image according to the position information of the shooting subject; the percentage of the area occupied by the shooting subject in the adjusted ROI is greater than a first threshold value; and shooting the shooting scene by taking the adjusted ROI as a focusing area to obtain a final shooting image. In this way, the size of the ROI is adaptively adjusted according to the size of the shooting subject, and the accuracy of focusing on the shooting object is improved.

[0005] Patent CN115546316A discloses an industrial camera automatic ROI setting method, which relates to the technical field of industrial detection, and comprises the following steps: aligning the industrial camera lens to the upper surface of the conveying belt, collecting an image of the conveying belt with an arbitrary exposure time and without containing test objects as a calibration image, recording the row mean value, column mean value and exposure time of the calibration image as calibration values; after starting the conveying belt, the test objects on the conveying belt enter the field of view of the industrial camera, the industrial camera continuously collects images as test images at a certain frame rate, and records the row mean value, column mean value and exposure time corresponding to all test images; the row mean value and column mean value of the test images are subtracted from the row mean value and column mean value of the calibration image to obtain a row change curve and a column change curve; all inflection points of the row change curve and the column change curve are searched as the corner points of the ROI; and the image is cropped into multiple rectangular intervals according to all ROI corner points and merged into a large rectangle as the output ROI.

[0006] In the above patents, the ROI region cannot cover the target subject of the next frame image due to the existence of bright spots with mutations in the image or other environmental factors, thereby affecting the frame rate of image acquisition.

[0007] Therefore, the present application provides a ROI positioning method based on laser autofocus, which adjusts the size and offset of the ROI of the area array sensor frame by frame, ensures the shooting of complete laser images, realizes high refresh rate, and improves the focusing performance. SUMMARY

[0008] The present application aims to overcome the above problems existing in the prior art and provides a ROI positioning method, system, medium and electronic device based on laser autofocus.

[0009] To achieve the above technical purposes and effects, the present application realizes the following technical solutions:

[0010] A ROI positioning method based on laser autofocus, which adjusts the ROI dynamically to adapt and capture the laser spot, realizes real-time automatic focusing, comprising:

[0011] Locating the pixel row where the edge pixel points farthest from the reference row in the vertical direction of the pixel row of the bright spot in the current frame laser image to set the ROI of the next frame laser image;

[0012] Obtaining the pixel row with a larger distance from the reference row in the pixel row where the edge pixel points at both ends to set one row boundary of the ROI;

[0013] Symmetrically forming another row boundary of the ROI with the reference row as the axis to make the ROI positioned when there is an abnormal bright spot in the bright spot contain the smaller laser spot in the next frame image approaching the focus point;

[0014] Wherein, the reference row is the pixel row where the center position of the laser line formed by the laser beam in the focus state is rounded in the image;

[0015] The bright spot includes a laser spot or an abnormal bright spot.

[0016] Further, locating the pixel row where the edge pixel points farthest from the reference row in the vertical direction of the pixel row of the bright spot in the current frame laser image, comprising:

[0017] Using a first threshold to binarize the image, and searching the two regions above and below the reference row respectively, searching from the first row and the last row of the image to the reference row respectively, taking the first binarized image intersection line as one boundary of the bright spot, and determining the upper and lower boundaries of the bright spot based on the intersection line;

[0018] Wherein, the first threshold is obtained based on the background noise calibration.

[0019] Further, the first binarization image boundary is used as one of the boundaries of the bright spot, including:

[0020] If the boundary of the bright spot is not found, the row spacing between the first or last row of the image and the reference row is multiplied by the first preset coefficient, and the row obtained is used as one of the boundaries of the bright spot.

[0021] Further, the first binarization image boundary is used as one of the boundaries of the bright spot, including:

[0022] The image row mean curve is obtained, and the rows where the corresponding points on both sides of the highest peak vertex have a first row mean less than a second threshold value are located to obtain a row difference absolute value.

[0023] The row difference absolute value is multiplied by a second preset coefficient to obtain a bright spot row height.

[0024] The upper and lower boundaries of the bright spot are obtained based on the bright spot row height and the row where the highest peak vertex is located.

[0025] Further, the second threshold value is greater than the row mean corresponding to the second highest peak vertex and less than the row mean corresponding to the highest peak vertex.

[0026] Further, the two column boundaries of the ROI are the pixel columns in the image where the integer positions of the two endpoints of the laser line are located.

[0027] Further, the first preset coefficient is greater than 1 to reserve space for the change of the next frame of laser light spot.

[0028] An ROI positioning system based on laser automatic focusing, comprising:

[0029] A positioning module is configured to locate the pixel row where the edge pixel point farthest from the reference row in the vertical direction of the pixel row of the bright spot in the current frame of laser image, so as to set the ROI of the next frame of laser image.

[0030] An acquisition module is configured to acquire the pixel row with a larger spacing from the reference row among the pixel rows where the edge pixel points at both ends are located, so as to set one row boundary of the ROI.

[0031] A processing module is configured to symmetrically form another row boundary of the ROI with the row boundary around the reference row as the axis, so that when there is an abnormal bright spot in the bright spot, the ROI positioned can accommodate the smaller laser light spot in the next frame of image close to the focus point.

[0032] The reference row is the pixel row in the image where the integer position of the centroid of the laser line formed in the focusing state of the laser light beam is located.

[0033] The bright spot includes a laser spot or an abnormally bright spot.

[0034] A storage medium having stored thereon a computer program, the computer program being executed by a processor to implement a laser auto-focusing based ROI positioning method as described above.

[0035] An electronic device comprising:

[0036] A processor; and

[0037] A memory for storing executable instructions of the processor;

[0038] The processor is configured to execute the executable instructions to implement the laser auto-focusing based ROI positioning method as described above.

[0039] The present application has the following advantages:

[0040] (1) In the present application, the upper and lower boundaries of the pixel row vertical direction are determined based on the position of the bright spot in the current frame image, the row boundary of the ROI is set based on the boundary with larger spacing from the reference row in the upper and lower boundaries, the reference row is used to form a complete ROI region by symmetrically arranging the upper and lower boundaries, the size and position of the ROI are adjusted in real time to capture the laser spot in the next frame image, the continuity of the effective laser spot image acquisition is ensured, the refresh rate of the defocus amount calculated by the laser spot centroid is improved, and high-speed real-time focusing is achieved.

[0041] By positioning the upper and lower boundaries of the bright spot in the pixel row vertical direction in the current frame laser image, the ROI of the next frame image can be reasonably predicted by the current frame bright spot position and the addition of reserved space without consuming too much computing resource.

[0042] By setting one row boundary of the ROI based on the boundary with larger spacing from the reference row in the upper and lower boundaries, the position of the laser spot in the next frame image can be accurately predicted, the next frame laser spot image can be contained in the ROI, and the centroid of the effective laser spot can be calculated in each frame image to achieve high-speed refresh of the defocus amount calculation and real-time focusing.

[0043] By symmetrically setting the row boundary to form another row boundary of the ROI, the ROI is symmetrically distributed about the reference row, which can accurately predict the laser spot appearing near the reference row and overcome the bright spot mutation caused by environmental signals appearing at the edge of the image to ensure the continuity of the effective laser spot image.

[0044] (2) In the present application, the image is binarized based on the first threshold value to filter the influence of background noise on the recognition of the bright spot boundary, and the upper and lower boundaries of the bright spot are quickly and accurately recognized based on the intersection line of the binarized image, the position of the bright spot in the current frame image is adaptively scaled by the first preset coefficient to predict and accommodate the laser spot in the next frame image, the bright spot boundary is positioned by the intersection line of the binarized image to improve the accuracy, and the ROI boundary range is positioned to greatly improve the image data processing speed, maintain the high refresh rate of the laser spot centroid, and improve the focusing performance.

[0045] (3) In the present application, the laser spot image is mapped to the vicinity of the highest peak of the curve by obtaining the image row mean curve, and the peak width of the highest peak is positioned based on the second threshold value index of the row where the highest peak is located, the peak width is scaled by the second preset coefficient to estimate the bright spot row height, the missing of the one-side bright spot information caused by the asymmetry of the highest peak is overcome, and the estimated upper and lower boundaries of the bright spot are obtained based on the bright spot row height and the row where the highest peak vertex is located is symmetrically set, the second threshold value is set to be greater than the vertex of the second highest peak and less than the row mean value corresponding to the vertex of the highest peak, so as to eliminate the bright spot or background noise interference outside the laser spot image, the anti-interference ability is stronger, the refresh rate is more stable, and the performance is better. BRIEF DESCRIPTION OF DRAWINGS

[0046] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 is a flow chart of the ROI positioning method based on laser automatic focusing in the present application;

[0048] Figure 2 is a flow chart of the ROI positioning system based on laser automatic focusing in the present application;

[0049] Figure 3 is a row mean curve schematic diagram of embodiment 2 in the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] In laser autofocus, the defocus state of the objective lens is determined by the shape of the spot formed when a semi-elliptical laser beam is focused onto the surface of the object. The defocus amount is calculated by the centroid of the laser spot image reflected back to the image sensor. Based on the defocus state and the defocus amount, the distance between the objective lens and the object surface is adjusted to ensure that the objective lens focus is on the object surface, achieving real-time focusing. To achieve high-speed focusing, the defocus amount refresh rate must be fast enough (on the order of 10kHz). Typical area scan cameras with full-frame sensors can only achieve a frame rate of 400-500fps, which is insufficient for high refresh rates, necessitating the use of a ROI (Real Area of ​​Interest) function.

[0052] The existing autofocus ROI settings have the following problems:

[0053] 1) Using a full-frame sensor to capture laser images results in a low refresh rate, making it unsuitable for demanding focusing applications;

[0054] 2) Full-frame cameras are prone to capturing stray light, which affects the calculation of defocus.

[0055] 3) A fixed ROI cannot capture a complete laser image at long focal length, which affects focusing performance.

[0056] To solve the above problems, such as Figure 1 As shown, this invention first provides a ROI positioning method based on laser autofocus, which achieves real-time autofocus by dynamically adjusting the ROI to adapt to and capture the laser spot, including:

[0057] Locate the pixel row containing the edge pixels that are furthest from the reference row at both ends of the bright spot in the vertical direction of the current frame laser image, in order to set the ROI of the next frame laser image;

[0058] Obtain the pixel row with the larger distance from the reference row among the pixel rows containing the edge pixels at both ends, in order to set a row boundary of the ROI;

[0059] The ROI is formed by symmetrically defining the row boundary with the reference row, so that the ROI located when there is an abnormal bright spot in the bright spot can accommodate a smaller laser spot in the next frame image that is close to the focus.

[0060] Wherein, the reference line is the pixel row in the image after rounding the centroid position of the laser line formed under the focusing state of the laser beam;

[0061] The bright spots include laser spots or abnormal bright spots.

[0062] In the embodiment, the upper and lower boundaries of the bright spot in the pixel row vertical direction in the current frame laser image are located to set the ROI of the next frame laser image; the boundary with a larger distance from the reference row is obtained to set one row boundary of the ROI, so that the ROI completely contains the next frame laser spot image; the row boundary is symmetrically set to form another row boundary of the ROI, so that the ROI is symmetrically distributed about the reference row, overcoming the bright spot mutation on the image edge caused by the environmental signal.

[0063] In the embodiment, in the laser automatic focusing application scenario, the spot shape includes an upper half-ellipse or a lower half-ellipse, indicating that the focused object is in different defocusing directions, when the spot shape becomes a laser line, indicating that the object to be measured is in a focusing state, and during the focusing process, the change of the spot is often linearly gradual and concentrates near the laser line, and from the image, the change of the laser spot image concentrates near the pixel row where the laser line is, that is, the reference row, therefore, the setting of the ROI cannot be away from the reference row, so that the positioning of the ROI accurately predicts the position of the next frame laser spot image.

[0064] In the embodiment, the change of the laser spot image in the next frame is predicted based on the laser spot image in the current frame, which is used to set the corresponding ROI to reduce the image output data and the data processing amount, to improve the data processing speed, to ensure the refresh rate of the laser spot image to improve the centroid calculation speed, and to realize real-time automatic focusing.

[0065] In the embodiment, the change of the laser spot occurs in the pixel row vertical direction, and there is only a slight change in the spot width in the column direction, therefore, only the boundaries of the bright spot in the pixel row vertical direction need to be located to predict and limit the pixel row vertical direction position of the ROI in the next frame, the reason for this is that the spot shape changes are basically linearly gradual, rather than sudden, during the automatic focusing process, therefore, only the ROI in the next frame is obtained based on the position of the laser spot image in the current frame and a suitable coefficient, specifically, the coefficient can be obtained through experimental calibration.

[0066] In the embodiment, the pixel row where the centroid position of the laser line formed by the laser beam in the focusing state is taken as the reference row, if the solved centroid position is not an integer, it is rounded and the integer pixel row where it is located is taken as the reference row, which is set based on the characteristics of laser automatic focusing, the spot change always surrounds the laser line during the focusing process, and the defocusing direction is determined by judging the upper and lower half-ellipse directions, therefore, taking the pixel row where the laser line is as the reference row can maximize the accuracy of the positioning of the ROI position.

[0067] In the embodiment, for the auto-focusing process of the same focusing object, the laser spot shape in the next frame image is smaller than the laser spot shape in the current frame image, and the smaller laser spot in the next frame image is closer to the focus point, so the ROI of the next frame converges frame by frame with respect to the ROI, thereby reducing the hardware computing resources and improving the laser spot centroid calculation efficiency.

[0068] In the embodiment, after locating the upper and lower boundaries of the bright spot in the vertical direction of the pixel row, the end of the upper and lower boundaries far from the reference row is further obtained as one boundary of the ROI, and a certain space is reserved for the boundary to realize accurate prediction of the laser spot image of the next frame, thereby avoiding the case that the ROI cannot capture the laser spot image of the next frame.

[0069] In the embodiment, the ROI row boundary formed based on the bright spot boundary far from the reference row is symmetrical about the reference row, another row boundary of the ROI is formed, and the area formed between the two row boundaries is used as the ROI of the next frame image, thereby forming the ROI which is always symmetrical about the reference row to adapt to the upper half or lower half elliptical shape of the laser spot and meet the focusing state in different defocusing directions, and the case that the ROI area cannot frame the laser spot image is avoided.

[0070] In the embodiment, the reason for setting the bright spot boundary far from the reference row is that:

[0071] In laser auto-focusing, the environmental signals include the background signals between the two focusing objects when the focusing object moves out of the observation range and then enters again, or the influence of lens stray light, which will cause abnormal bright spots at the edge of the image, thereby affecting the identification and determination of the ROI area, so that the ROI is set at the edge of the image, and the laser spot cannot be contained in the next frame ROI image which converges frame by frame, so that the laser spot centroid cannot be determined, thereby affecting the auto-focusing effect.

[0072] In the embodiment, the bright spot includes the laser spot formed by focusing the laser beam on the focusing object and the abnormal bright spot, so when the abnormal bright spot far from the reference row appears compared with the laser spot, the boundary of the abnormal bright spot is used as the spot boundary to ensure that the laser spot of the next frame will not be lost in the located ROI, and the real-time tracking can be realized by the ROI which converges frame by frame.

[0073] Further, unlike the prior art, by identifying the boundary of the far bright spot and limiting the row position of the ROI with respect to the reference row, when abnormal bright spot mutations or environmental disturbances occur in the image, the ROI formed by the bright spot boundary and the ROI formed by the reference row symmetry have higher image output range, so the next frame of image is still contained in the ROI, thereby avoiding the influence of the bright spot mutation appearing at the edge of the image when the focused object moves out and then moves into the image again.

[0074] In the present application, the upper and lower boundaries of the pixel row in the vertical direction are determined by the bright spot position in the current frame image, the row boundary of the ROI is set based on the larger distance between the upper and lower boundaries and the reference row, and the ROI complete area is formed by the upper and lower symmetry of the reference row, so as to adjust the size and position of the ROI in real time, capture the laser spot in the next frame image, ensure the continuity of the effective laser spot image acquisition, improve the refresh rate of the defocus amount calculated by the laser spot centroid, and realize high-speed real-time focusing.

[0075] By positioning the upper and lower boundaries of the bright spot in the pixel row in the vertical direction in the current frame laser image, the ROI of the next frame image can be reasonably predicted by the bright spot position of the current frame and the addition of reserved space without consuming too much computing resource;

[0076] By setting one row boundary of the ROI by obtaining the boundary with larger distance from the reference row among the upper and lower boundaries, the position of the laser spot in the next frame image can be accurately predicted, so that the ROI can accommodate the laser spot image of the next frame, and the centroid of the effective laser spot can be effectively calculated to realize high-speed refresh of the defocus amount calculation and real-time focusing;

[0077] By symmetrically setting the row boundary to form another row boundary of the ROI, the ROI is symmetrically distributed with respect to the reference row, which can accurately predict the laser spot appearing near the reference row and overcome the bright spot mutation appearing at the edge of the image caused by environmental signals, thereby ensuring the continuity of the effective laser spot image.

[0078] In fact, due to the influence of unexpected situations, the positioned ROI region in the present application cannot guarantee to completely accommodate the laser spot in the next frame image, but based on the identification of the bright spot boundary including the abnormal bright spot and the laser spot and the symmetric setting of the ROI with respect to the reference row in the present application, the next frame ROI image at least contains part of the laser spot image, and the ROI region is gradually scaled by the laser spot image, so that the laser spot gradually converging is finally completely accommodated.

[0079] In order to accurately locate the position of the bright spot in the image, in some embodiments, the pixel row in which the edge pixel points farthest from the reference row in the vertical direction of the pixel row of the light spot in the current frame laser image is located is located, including:

[0080] The image is binarized using a first threshold, and the upper and lower regions of the reference row are searched respectively, and the first binarized image intersection line is taken as one of the boundaries of the bright spot, and the upper and lower boundaries of the bright spot are determined based on the intersection line.

[0081] The first threshold is obtained based on the background noise.

[0082] In this embodiment, the pixel values of each pixel point in the image are binarized to facilitate the distinction between the laser spot image and the background image in the image, to quickly identify the upper and lower boundaries of the bright spot in the vertical direction of the pixel row, and the image is divided into two regions above and below the reference row to search for the bright spot boundary respectively, to facilitate subsequent comparison of the distance between the boundary and the reference row and symmetrical processing. The binarized image can be distinguished by black and white regions, or by 1 and 0. The demarcation line of the black and white regions or 1 and 0 is considered as the boundary of the bright spot, so the pixel row in which the demarcation line is located can be located as the upper and lower boundaries of the bright spot.

[0083] In this embodiment, during the background noise calibration, the target object image is collected under the condition of no laser beam to count the pixel values, and the first threshold is obtained based on the pixel values, which is used to exclude the influence of background noise.

[0084] In some embodiments, the first binarized image intersection line is taken as one of the boundaries of the bright spot, including:

[0085] If the intersection line is not searched in the region, the row distance from the first row or the last row of the region to the reference row is multiplied by the first preset coefficient, and the resulting row is taken as one of the boundaries of the bright spot.

[0086] In this embodiment, if the intersection line of the binarized image cannot be identified in the upper and lower half regions of the reference row, it means that the boundary of the laser spot image is located outside the image, and the half region in which the boundary of the laser spot image is not identified is expanded by a multiple of the first preset coefficient to serve as one of the row boundaries of the ROI, so that the next frame image can completely accommodate the laser spot image.

[0087] In this embodiment, due to the characteristics of laser autofocus, the focusing object needs to be in the focusing state, i.e. from the telephoto state to the close-up state, and the height of the bright spot row is continuously reduced, so the region where the intersection line is searched directly takes the spot boundary as one of the row boundaries of the ROI, which can completely accommodate the next frame laser spot image.

[0088] In this embodiment, the first preset coefficient can be obtained by experiment calibration.

[0089] Embodiment 1

[0090] In this embodiment, the light spot boundary is quickly and accurately found through the binarized image, and the specific steps are as follows:

[0091] A1, initialize the row height as H0 and the column width as W0.

[0092] A2, collect the laser image I(h, w).

[0093] A3, use the first threshold TH1 to binarize the image to obtain the mask image M(h, w), and assign 1 to the value greater than the first threshold TH1, which represents the laser spot; assign 0 to the value less than the first threshold TH1, which represents the background.

[0094] A4, search from the 0th row, and stop searching under two conditions: (a) stop when a laser spot is encountered in the current row (the mask sum of the current row is greater than 0); (b) stop when a zero point is encountered in the current row H ref0 , which is the reference row. Record the row index as H1.

[0095] A5, determine whether H1=0 is true: if yes, it means that the upper end of the upper half of the current laser image has contacted the laser spot, and the adjusted height of the upper half is calculated as:

[0096]

[0097] where abs is the absolute value, r_expand represents the width expansion multiple, i.e. the first preset coefficient, which is a coefficient greater than 1, and a too large value will cause the image height to change sharply, and a too small value will cause the image height to be adjusted too late, for example, the value can be taken as about 1.2, or selected through experiment. Otherwise, it means that the upper half of the current laser image can contain the laser spot, and the height D1 is calculated as: ref0

[0098] A6, search from the H max -1th row upwards, and stop searching under two conditions: (a) stop when a laser spot is encountered in the current row (the mask sum of the current row is greater than 0); (b) stop when a zero point is encountered in the current row H ref0 . Record the row index as H2.

[0099] A7, determine whether H2=0 is true: if yes, it means that the lower end of the lower half of the current laser image has contacted the laser spot, and the adjusted height of the lower half is calculated as:

[0100]

[0101] ​Otherwise, it represents that the lower half of the current laser image can contain a laser spot, and the height D2=abs(H2-H ref0 )

[0102] A8, calculate the adjusted laser image half-height D=max(D1, D2).

[0103] Then the corresponding starting row: H start =H ref0 -D, the row height H=2D.

[0104] A9, according to the starting row and the row height, the ROI row boundary is symmetrically set with the reference row, and step A2 is entered to iteratively execute the above steps.

[0105] Specifically, the image sensor overflow judgment is performed before setting the final ROI region in the above steps, and when the image sensor range is exceeded, the ROI is set to the full image to obtain the next frame of laser spot image.

[0106] In the present application, the image is binarized based on the first threshold to filter the influence of background noise on spot boundary identification, and the upper and lower boundaries of the bright spot are quickly and accurately identified based on the binarized image boundary line. The position of the bright spot in the current frame image is adaptively scaled by the first preset coefficient to predict and accommodate the laser spot in the next frame image. The bright spot boundary is positioned by the binarized image boundary line to improve accuracy, and the ROI boundary range is positioned to greatly improve the image data processing speed, maintain the high refresh rate of the laser spot centroid, and improve the focusing performance.

[0107] In order to further reduce the influence of noise on the determination of the position of the spot, in some embodiments, the pixel row in which the edge pixel point farthest from the reference row in the vertical direction of the pixel row of the spot in the current frame laser image is located is positioned, comprising:

[0108] Obtain the image row mean curve, and respectively locate the rows in which the corresponding points of the first row mean less than the second threshold on both sides of the highest peak vertex to obtain the row difference absolute value;

[0109] Multiply the row difference absolute value by the second preset coefficient to obtain the bright spot row height.

[0110] Based on the bright spot row height, the upper and lower boundaries of the bright spot are symmetrically obtained based on the row in which the highest peak vertex is located.

[0111] In the present embodiment, the mean or total value of each row of pixel values is obtained based on the image pixel values to form a row mean curve, which is used to reflect the pixel value difference of each row in the image, wherein, for example, Figure 3As shown, there are several peaks in the row mean curve, the highest peak reflects the region where the laser spot image is located, and the remaining peaks represent abnormal situations such as background noise or sudden bright spots. By setting a second threshold to remove the noise effect in the row mean curve, the height of the bright spot in the vertical direction of the pixel row is determined by locating the peak width of the highest peak under the second threshold. Based on the asymmetry of the highest peak in the row mean curve, the bright spot is not symmetrical to the row where the highest peak is located, so the second preset coefficient is needed to expand the peak width to obtain the estimated bright spot row height to include the bright spot on one side of the asymmetric peak, preventing the loss of the bright spot. Then, the upper and lower boundaries of the bright spot are estimated based on the bright spot row height.

[0112] In order to highlight the difference between the laser spot and other abnormal bright spot noise in the image, in some embodiments, the second threshold is greater than the vertex of the second highest peak and less than the row mean value corresponding to the vertex of the highest peak.

[0113] In this embodiment, the second threshold is used to remove the influence of noise on the identification of the bright spot boundary, so that the peaks in the row mean curve other than the highest peak represent noise. Therefore, the second threshold is set to be greater than the row pixel mean value of the second highest peak and less than the highest peak, so as to exclude the positioning accuracy of the bright spot boundary by noise other than the laser spot image. In actual process, the second threshold can be calibrated by experiment or set by experience.

[0114] Embodiment 2

[0115] In this embodiment, the peak width is calculated based on the row mean curve to determine the height of the spot, thereby locating the upper and lower boundaries of the spot. The specific steps are as follows:

[0116] B1, initialize the row height as H0 and the column width as W0.

[0117] B2, acquire the laser image I(h,w).

[0118] B3, calculate the row mean curve:

[0119]

[0120] B4, set the second threshold, in this embodiment, the following method is used to obtain it in real time: obtain the maximum value V max of the row mean curve peak and the minimum value V min , where H peak is the row where the highest peak vertex is located.

[0121] and then obtain the peak width threshold:

[0122]

[0123] r_peak is a ratio with a value of 0-1, which can be adjusted (recommended value = 0.5).

[0124] B5, from the maximum value of the row H peak Search up and down two half figures respectively, find the first row mean on both sides is less than TH peak The corresponding row, the row index is H1 and H2 respectively.

[0125] B6, calculate the peak width D peak =H2-H1, calculate the height containing the laser spot: D=r_multi*D peak

[0126] r_multi is a second preset coefficient, the value is greater than 1, obtained by experiment.

[0127] B7, calculate the row index of both ends containing the laser spot:

[0128]

[0129] B8, calculate the distance from both end row index to the row where the zero point is located:

[0130]

[0131] B9, calculate the adjusted laser image half height D=max(D1,D2). Then the corresponding starting row: H start =H ref0 -D, row height H=2D.

[0132] B10, according to the starting row and the row height, the ROI row boundary is symmetrically set based on the reference row, and step B2 is entered to iteratively execute the above steps.

[0133] In the present application, by obtaining the image row mean curve, the laser spot image is mapped to the vicinity of the highest peak of the curve, and based on the second threshold index of the row where the highest peak is located, the peak width of the highest peak is located, and the peak width is scaled by the second preset coefficient to estimate the bright spot row height, which overcomes the asymmetry of the highest peak, and based on the bright spot row height, the upper and lower boundaries of the laser spot are obtained by symmetrically setting the row where the highest peak is located, and the second threshold is set to be greater than the second highest peak and less than the row mean corresponding to the highest peak, so as to eliminate the bright spot or background noise interference outside the laser spot image, and the anti-interference ability is stronger, the refresh rate is more stable, and the performance is better.

[0134] In order to further reduce the hardware operation resources and improve the refresh rate, in some embodiments, the ROI column width is set to a fixed length greater than the length of the laser line, and its column boundary is located by calibration.

[0135] In the embodiment, based on the shape characteristics of the laser spot in the auto-focusing process, there is a change in the row height in the vertical direction of the pixel row, and there is only a slight difference in the column width in the column direction, so the position of the laser line can be calibrated in advance, and the ROI column width is set to a fixed length greater than the length of the laser line to locate the two column boundaries of the ROI in the column direction.

[0136] In order to cope with the change of the next frame laser spot image, the first preset coefficient and the second preset coefficient are greater than 1, so as to reserve the space for the change of the next frame laser spot.

[0137] In the embodiment, based on the upper and lower boundaries of the spot positioned by the current frame laser spot image, the preset coefficient is multiplied to appropriately expand the ROI, so as to adapt to the change of the next frame spot shape, so as to accommodate the next frame laser spot image, therefore, the preset coefficient needs to be greater than 1, so as to expand the ROI by a preset multiple based on the spot boundary, and the specific coefficient setting can be obtained based on experimental calibration.

[0138] Based on the same inventive concept, as shown in Figure 2 The application further provides a ROI positioning system based on laser auto-focusing, comprising:

[0139] A positioning module is configured to position the pixel row in which the edge pixel points at the two ends of the bright spot in the current frame laser image are located in the vertical direction of the pixel row, so as to set the ROI of the next frame laser image;

[0140] An acquisition module is configured to acquire the pixel row with a larger distance from the reference row among the pixel rows in which the edge pixel points at the two ends are located, so as to set one row boundary of the ROI;

[0141] A processing module is configured to form another row boundary of the ROI by axisymmetrically setting the row boundary with the reference row as the axis, so that when there is an abnormal bright spot in the bright spot, the ROI positioned by the abnormal bright spot can accommodate the smaller laser spot in the next frame image close to the focus point;

[0142] The reference row is the pixel row in which the laser line centroid position formed under the focusing state of the laser beam is rounded.

[0143] The bright spot includes a laser spot or an abnormal bright spot.

[0144] Based on the same inventive concept, the application further provides a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the ROI positioning method based on laser auto-focusing.

[0145] Based on the same inventive concept, the application finally provides an electronic device, comprising:

[0146] a processor; and

[0147] a memory for storing executable instructions of the processor;

[0148] wherein the processor is configured to execute the executable instructions to implement the method for locating ROI based on laser auto-focusing as described above.

[0149] In the description of the present specification, the description referring to the terms "one embodiment", "an example", "a specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0150] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only illustrative of the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A method for ROI positioning based on laser auto-focusing, which realizes real-time auto-focusing by dynamically adjusting the ROI to adapt and capture the laser spot, characterized in that, The method comprises the following steps: locating the pixel row in which the edge pixel points farthest from the reference row in the vertical direction of the pixel row are located in the current frame laser image to set the ROI of the next frame laser image; acquiring the pixel row with a larger distance from the reference row among the pixel rows in which the edge pixel points at both ends are located to set one row boundary of the ROI; symmetrically forming another row boundary of the ROI with the reference row as the axis to make the ROI positioned when there is an abnormal bright spot in the bright spot contain the smaller laser spot in the next frame image approaching the focal point; wherein the reference row is the pixel row in which the center of the laser line formed under the focusing state of the laser beam is located after being rounded off in the image; the bright spot comprises a laser spot or an abnormal bright spot.

2. The method of claim 1, wherein, The method for locating the pixel row in which the edge pixel points farthest from the reference row in the vertical direction of the pixel row are located in the current frame laser image comprises the following steps: using a first threshold to binarize the image, and searching the upper and lower regions of the reference row respectively, searching from the first row and the last row of the image to the reference row respectively, taking the first binarized image intersection line as one boundary of the bright spot, and determining the upper and lower boundaries of the bright spot based on the intersection line; wherein the first threshold is acquired based on the background noise.

3. The method of claim 2, wherein, The method for taking the first binarized image intersection line as one boundary of the bright spot comprises the following steps: if no intersection line is searched in the region, multiplying the distance between the first row or the last row of the image in the region and the reference row by a first preset coefficient to obtain the row in which one boundary of the bright spot is located.

4. The method of claim 1, wherein, The method for locating the pixel row in which the edge pixel points farthest from the reference row in the vertical direction of the pixel row are located in the current frame laser image comprises the following steps: acquiring the image row mean curve, locating the rows in which the corresponding points of the first row mean less than a second threshold are located on both sides of the highest peak vertex to obtain the row difference absolute value; multiplying the row difference absolute value by a second preset coefficient to obtain the bright spot row height; based on the bright spot row height, the upper and lower boundaries of the bright spot are obtained by symmetrically taking the row in which the highest peak vertex is located.

5. The method of claim 4, wherein, The second threshold is greater than the row mean corresponding to the second highest peak vertex and less than the row mean corresponding to the highest peak vertex.

6. The method of claim 1-4, wherein, The two column boundaries of the ROI are the pixel columns in which the end points of the laser line are located after being rounded off in the image.

7. The method of claim 3, wherein the ROI is positioned by the laser auto-focusing based on the distance between the ROI and the laser focus point. The first preset coefficient is greater than 1 to reserve space for the change of the next frame laser spot.

8. A ROI positioning system based on laser auto-focusing, characterized in that, The method comprises the following steps: a positioning module for locating the pixel row in which the edge pixel points farthest from the reference row in the vertical direction of the pixel row are located in the current frame laser image to set the ROI of the next frame laser image; an acquisition module for acquiring the pixel row with a larger distance from the reference row among the pixel rows in which the edge pixel points at both ends are located to set one row boundary of the ROI; a processing module for symmetrically forming another row boundary of the ROI with the reference row as the axis to make the ROI positioned when there is an abnormal bright spot in the bright spot contain the smaller laser spot in the next frame image approaching the focal point; wherein the reference row is the pixel row in which the center of the laser line formed under the focusing state of the laser beam is located after being rounded off in the image; the bright spot comprises a laser spot or an abnormal bright spot.

9. A storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the laser auto-focusing based ROI positioning method according to any one of claims 1-7.

10. An electronic device, comprising: Comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement the laser auto-focusing based ROI positioning method according to any one of claims 1-7.

Images