A method for automatic ROI setting of industrial cameras

Abstract

This invention discloses an automatic ROI setting method for an industrial camera, relating to the technical field of industrial inspection. The method includes: aligning the industrial camera lens with the upper surface of a conveyor belt, acquiring an image of the conveyor belt with arbitrary exposure time, excluding the test object, as a calibration image; recording the row mean, column mean, and exposure time of the calibration image as calibration values; after starting the conveyor belt, the test object on the conveyor belt enters the field of view of the industrial camera, and the industrial camera continuously acquires images at a certain frame rate as test images, recording the row mean, column mean, and exposure time corresponding to all test images; subtracting the row mean and column mean of the test image from the row mean and column mean of the calibration image to obtain row variation curves and column variation curves, respectively; searching for all inflection points of the above row variation curves and column variation curves as corner points of the ROI; cropping the image into multiple rectangular intervals based on all ROI corner points and merging them into a large rectangle as the ROI output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial detection, in particular to an industrial camera automatic ROI setting method. BACKGROUND

[0002] With the development of visual technology, the resolution of images is higher and higher, and the performance requirements of industrial computers are also higher and higher. Various schemes are adopted by various visual companies to exclude invalid data and only keep valid data for detection. For example, ROI selection, Gaussian pyramid, downsampling and moving target tracking.

[0003] As a camera manufacturer, it is necessary to ensure that there is no loss of valid data. For example, the Gaussian pyramid and downsampling will cause loss of valid information. At the same time, the complex moving target tracking algorithm has too high requirements for hardware, and a fast and general ROI setting method with low calculation amount is needed. SUMMARY

[0004] The purpose of the present application is to provide an industrial camera automatic ROI setting method which is fast and general and can identify multiple ROIs.

[0005] The technical solution of the present application is: an industrial camera automatic ROI setting method, which comprises:

[0006] S1, aligning the lens of the industrial camera to the upper surface of the conveying belt, collecting a conveying belt image without test objects as a calibration image at an arbitrary exposure time, and recording the row mean value, column mean value and exposure time of the calibration image as calibration values;

[0007] S2, after starting the conveying belt, the test objects on the conveying belt enter the field of view of the industrial camera, the industrial camera continuously collects images as test images at a certain frame rate, and records the row mean value, column mean value and exposure time corresponding to all test images;

[0008] S3, subtracting the row mean value of the test image from the row mean value of the calibration image to obtain a row change curve, and subtracting the column mean value of the test image from the column mean value of the calibration image to obtain a column change curve;

[0009] S4, searching for all inflection points of the row change curve and the column change curve as the edge corner points of the ROI;

[0010] S5, cutting the image into multiple rectangular intervals according to all ROI edge corner points obtained in step S4, and finally merging all rectangular intervals into a large rectangle as the output ROI.

[0011] In any of the above technical solutions, further, according to the exposure time Exposure current of all test images, the row mean value RowMeanstd and column mean ColMean std performing gray correction to obtain new row mean RowMean std_new and new column mean ColMean std_new .

[0012] In any of the above technical solutions, further, in the gray correction, the new row mean RowMean std_new and the new column mean ColMean std_new are calculated according to the following formula:

[0013]

[0014]

[0015] where j is the number of rows and i is the number of columns.

[0016] In any of the above technical solutions, further, the step S3 includes: subtracting the row mean RowMean current of the to-be-tested image from the new row mean RowMean std_new of the calibration image corresponding thereto to obtain a row change curve RowMeanCurve[j] = abs(RowMean current [j]-RowMean std_new [j]), and subtracting the column mean ColMean current of the to-be-tested image from the new column mean ColMean std_new of the calibration image corresponding thereto to obtain a column change curve ColMeanCurve[i] = abs(ColMean current [i]-ColMean std_new [i]).

[0017] In any of the above technical solutions, further, the step of searching for all inflection points of the inflection point row change curve and the inflection point column change curve includes: using a local windowing determination method to sequentially traverse each point, the size of the window being set to 2n points, n being any integer, the width threshold being set to Thres wid , the inflection point threshold being set to Thres area , the area of the first n points being calculated as a left area Area left , the area of the last n points being calculated as a right area Area right , a point satisfying Area being searched for as a ROI starting point Point start , a point satisfying Area being searched for as a ROI ending point Point end , and Point end -Point start ≥ Threswid If yes, it is determined that the current ROI is valid, and otherwise, it is abandoned and the searching continues.

[0018] In any of the technical solutions above, further, the step S5 comprises: taking 2x corner points of x valid ROIs found by the row change curve as the horizontal coordinates, taking 2y corner points of y valid ROIs found by the column change curve as the vertical coordinates, cutting the image into xy rectangular intervals by the corner points, and finally merging all the rectangular intervals into a large rectangle as the ROI output.

[0019] The present application has the following advantages:

[0020] The technical solution in the present application adopts the way of calculating the row and column change curves, can simultaneously collect multiple ROI intervals, and in order to adapt to the output format of the industrial camera, merges the multiple ROI intervals into a rectangular ROI interval for output; the FPGA only records the row and column mean values of each frame of the image to be measured, occupies very small cache, and has low calculation amount. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and additional aspects of the present application will become apparent and easy to understand from the following description of the embodiments, given in conjunction with the accompanying drawings, in which:

[0022] Figure 1 is a schematic flow chart of an industrial camera automatic ROI setting method according to an embodiment of the present application;

[0023] Figure 2 is an image acquisition schematic diagram of an industrial camera automatic ROI setting method according to an embodiment of the present application;

[0024] Figure 3 is a row curve inflection point schematic diagram of an industrial camera automatic ROI setting method according to an embodiment of the present application;

[0025] Figure 4 is an inflection point finding flow chart of an industrial camera automatic ROI setting method according to an embodiment of the present application;

[0026] Figure 5 is an ROI cutting schematic diagram of an industrial camera automatic ROI setting method according to an embodiment of the present application;

[0027] Figure 6 is an ROI merging schematic diagram of an industrial camera automatic ROI setting method according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0029] In the following description, many specific details are set forth in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0030] like Figure 1 As shown, this embodiment provides a method for automatically setting the ROI for an industrial camera, the method including:

[0031] S1, such as Figure 2 As shown, the industrial camera lens is aimed at the upper surface of the conveyor belt, and an image of the conveyor belt with an arbitrary exposure time, excluding the test object, is acquired as a calibration image. The row mean (RowMean) of the calibration image is recorded. std ColMean std Exposure time Std As a calibration value.

[0032] Specifically, after acquiring the calibration image, the row mean (RowMean) is calculated using the industrial camera's FPGA. std ColMean std Exposure time Std Stored in the camera's memory.

[0033] S2. After the conveyor belt is started, the test object on the conveyor belt enters the field of view of the industrial camera. The industrial camera continuously acquires images at a certain frame rate as the test images and records the RowMean value corresponding to all test images. current ColMean current Exposure time current .

[0034] Specifically, the acquired images to be tested include images of the conveyor belt and the object to be tested. The FPGA will calculate the row average and column average of the current frame of the image to be tested in real time, so that the FPGA's buffer is only one row plus one column in length.

[0035] This embodiment provides a method for predicting the ROI in the next frame. When the detected object moves quickly, the user can write the predicted velocity of the detected object into the industrial camera. The industrial camera uses velocity vector calibration to calculate the affine transformation matrix M. Multiplying the coordinates of all ROI corner points obtained in the previous frame by M and connecting them gives the detection range for the current frame. This method saves computing power and speeds up processing. When the detected object moves slowly, the user does not write the predicted velocity of the detected object. The camera defaults to using the conventional image motion tracking method, expanding a certain range around the ROI area in the previous frame as the detection area.

[0036] S3. Based on the exposure time corresponding to all images to be tested, Exposure current The row mean of the calibration image is then calculated sequentially. std ColMean std The new row mean (RowMean) is obtained by performing grayscale correction. std_new And the new column mean ColMean std_new The row mean of the image under test is calculated. current The corresponding calibration image newline mean (RowMean) std_new Subtracting the two yields the row change curve RowMeanCurve[j] = abs(RowMean current [j]-RowMean std_new [j]), the column mean ColMean of the image to be tested current The corresponding calibration image new column mean ColMean std_new Subtracting the two yields the column curve ColMeanCurve[i] = abs(ColMean current [i]-ColMean std_new [i]), where j is the row number and i is the column number.

[0037] Specifically, in actual use, industrial cameras use different exposure parameters and exposure times due to the differences in grayscale values ​​of different detection targets. This can lead to inconsistencies between the real-time acquired background image and the calibrated background image, necessitating correction of the row and column mean values.

[0038]

[0039]

[0040] After correction, the background grayscale of the calibration image will be consistent with the background grayscale of the test image in that frame, ensuring the accuracy of subsequent operations.

[0041] S4. Search for all inflection points of the row and column change curves mentioned above as corner points of the ROI; use a local windowing method to iterate through each point sequentially, setting the window size to 2n points, where n is any integer, and setting the width threshold to Thres. wid The inflection point threshold is set to Thres. area Calculate the area of ​​the first n points as the left area (Area). left The area of ​​the n points after calculation is taken as the right area. right If we find that satisfy the following conditions: and The two points are defined as the starting point of the ROI, Point. start and ROI termination point end When Point end -Point start ≥Thres wid Determine if the current ROI is valid; otherwise, discard it and continue the search.

[0042] like Figure 3 As shown, in real-world scenarios, factors such as unstable light sources and reflections from the measured object result in significant noise in the calculated row and column variation curves; for example... Figure 4 As shown, at the starting point of ROI, Point start Then find the corresponding ROI termination point (Point). end The two are combined to determine whether the difference between them is greater than the width threshold Thres. wid .

[0043] S5. Use the 2x corner points of the x effective ROIs found by the row variation curve as the x-axis, and the 2y corner points of the y effective ROIs found by the column variation curve as the y-axis. The odd-numbered corner points are the starting points (Points) of the ROIs. start An even number of corner points are the ROI termination points corresponding to their preceding corner points. end The aforementioned paired corner points crop the image into xy rectangular intervals and extract them. Finally, all the extracted rectangular intervals are merged into a large rectangle as the ROI output.

[0044] like Figure 5 As shown, this example yields four x-coordinate corner points C1, C2, C3, and C4, and four y-coordinate corner points R1, R2, R3, and R4. C1, C3, R1, and R3 are the starting points of the ROI. start C2, C4, R2, and R4 are the ROI termination points. end C1 and C2, C3 and C4, R1 and R2, R3 and R4 are paired, and these eight points can be used to divide the region into four effective ROI intervals; for exampleFigure 6 As shown, the four effective ROI regions obtained in this embodiment are cropped and extracted, and then merged into a single image according to their original position order, which is used as the ROI of the current frame image to be tested.

[0045] In summary, this invention proposes an automatic ROI setting method for industrial cameras, comprising:

[0046] S1. Aim the industrial camera lens at the upper surface of the conveyor belt and acquire an image of the conveyor belt with an arbitrary exposure time that does not contain the test object as a calibration image. Record the row mean, column mean, and exposure time of the calibration image as calibration values.

[0047] S2. After the conveyor belt is started, the test object on the conveyor belt enters the field of view of the industrial camera. The industrial camera continuously acquires images as test images at a certain frame rate and records the row mean, column mean and exposure time of all test images.

[0048] S3. Subtract the row mean of the image under test from the row mean of the calibration image to obtain the row variation curve, and subtract the column mean of the image under test from the column mean of the calibration image to obtain the column variation curve.

[0049] S4. Search for all inflection points of the above row and column change curves as corner points of the ROI.

[0050] S5. Based on all the ROI corner points obtained in step S4, crop the image into multiple rectangular regions, and finally merge all the rectangular regions into one large rectangle as the ROI output.

[0051] The steps in this invention can be adjusted, combined, or deleted according to actual needs.

[0052] Although the invention has been disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and not intended to limit the application of the invention. The scope of protection of the invention is defined by the appended claims and may include various modifications, alterations, and equivalents made to the invention without departing from the scope and spirit of the invention.

Claims

1. An industrial camera automatic ROI setting method, characterized by, The method comprises: S1, aligning the industrial camera lens to the surface of the conveying belt, collecting an image of the conveying belt without the test object as a calibration image, recording the row mean value, column mean value and exposure time of the calibration image as calibration values; S2, after starting the conveying belt, the test object on the conveying belt enters the field of view of the industrial camera, the industrial camera continuously collects images as test images at a certain frame rate, and records the row mean value, column mean value and exposure time corresponding to all test images; S3, subtracting the row mean value of the test image from the row mean value of the calibration image to obtain a row change curve, and subtracting the column mean value of the test image from the column mean value of the calibration image to obtain a column change curve; S4, searching for all inflection points of the row change curve and the column change curve as the edge corner points of the ROI; S5, cropping the image into multiple rectangular intervals according to all ROI edge corner points obtained in step S4, and finally merging all rectangular intervals into a large rectangle as the ROI output.

2. The industrial camera automatic ROI setting method of claim 1, wherein, According to the exposure time Exposure corresponding to all the images to be tested current , the row mean RowMean std and the column mean ColMean std of the calibration image are sequentially subjected to gray correction to obtain new row mean RowMean std_new and new column mean ColMean std_new .

3. The industrial camera automatic ROI setting method of claim 2, wherein, The gray scale correction calculates a new row mean RowMean according to the following equation std_new and a new column mean ColMean according to the following equation std_new : Where j is the number of rows and i is the number of columns.

4. The industrial camera automatic ROI setting method of claim 1 or 3, wherein, The step S3 comprises: calculating a row mean RowMean current A new row mean RowMean std_new of the calibration image corresponding to the row mean RowMean current [j] of the test image, and a column mean ColMean std_new [j] of the test image, and a column mean ColMean current A new column mean ColMean std_new of the calibration image corresponding to the column mean ColMean current [i] of the test image, and a column mean ColMean std_new [i] of the test image.

5. The industrial camera automatic ROI setting method of claim 1, wherein, The step of searching all inflection points of the inflection row change curve and the inflection column change curve comprises: sequentially traversing each point using a local windowing determination method, the size of the window being set as 2n points, n being an arbitrary integer, the width threshold being set as Thres wid , the inflection point threshold being set as Thres area , the area of the first n points being calculated as a left area Area left , the area of the last n points being calculated as a right area Area right , a point satisfying being searched as a ROI starting point Point start , a point satisfying being searched as a ROI ending point Point end , whether Point end -Point start ≥Thres wid is satisfied, if satisfied, determining that the current ROI is valid, otherwise discarded and continue searching.

6. The industrial camera automatic ROI setting method of claim 1, wherein, The step S5 comprises: taking 2x edge corner points of x effective ROIs found by the row change curve as the horizontal coordinates, taking 2y edge corner points of y effective ROIs found by the column change curve as the vertical coordinates, cropping the image into xy rectangular intervals by the above edge corner points, and finally merging all rectangular intervals into a large rectangle as the ROI output.

Images