Image sensor splicing determination method, system, adjustment method and storage medium

Abstract

The present invention provides an image sensor splicing determination method, system, adjustment method and storage medium. The image sensor splicing determination method includes splicing at least two image sensors parallel to the detection surface, collecting the images to be spliced ​​in adjacent areas along the pixel row direction, calculating the weight of the black grayscale value and / or the white grayscale value of each pixel on the row image; the grayscale values ​​of the pixels on the pixel row on the row image are inconsistent; analyzing the weight difference of the black grayscale value or the white grayscale value of two adjacent pixels on the row image; calculating the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample; and determining whether there is a feature loss after the adjacent two images to be spliced ​​are spliced. The present invention can analyze the inclination angle of the image sensor, and determine whether there is a feature loss after the two collected images to be spliced ​​are spliced ​​according to the inclination angle, and perform installation adjustment to improve the image quality.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an image sensor splicing determination method, system, adjustment method and storage medium. Background Art

[0002] The image sensors in a multi-sensor camera or multiple cameras with one sensor collect images separately, and the collected images are stitched together to expand the scanning field of view of a single image sensor. The installation errors of each image sensor inside the camera and the installation errors of the camera cause the images of the expanded scanning field of view to have gaps and misalignments after stitching, which in turn causes the images at the stitching points to be unclear, seriously affecting the image quality.

[0003] The parallel arrangement of image sensors in multi-sensor cameras or multiple cameras can expand the field of view, improve the integration of equipment, and lower the threshold for customers to use. However, it is impossible to achieve absolute parallelism during the assembly and use of multi-sensors. There is an angle between two adjacent image sensors. As the number of cameras or image sensors arranged in parallel increases, the amount of image cropping after stitching the captured images is aggravated, the effective area of ​​the image is reduced, and even the images collected by two adjacent cameras or image sensors are missing features after stitching, which seriously reduces the image quality.

[0004] Chinese patent CN116156342B discloses a method for splicing multi-line array image sensors, a line array imaging system, a device and an apparatus. The scanned images of two adjacent line array image sensors of the multi-line array image sensor have overlapping areas, and the scanned images are not completely overlapped. The method comprises: correcting the scanned image A of any line array image sensor a to a deflected scanned image A' according to the deflection angle of the sensitive light through a delayed integral relationship, so that it is parallel to the scanned image B of another line array image sensor b; performing pixel deviation correction on the deflected scanned image A'; wherein the pixel deviation correction includes horizontal pixel deviation correction and vertical pixel deviation correction. In addition, the prior art does not disclose how to judge whether there is feature loss based on the installation rotation direction and installation angle, and adjust the installation angle based on whether there is feature loss, so as to ensure that the splicing features are not lost and the effective area is large under the expanded scanning field of view, and how to judge the tilt angle of the image sensor based on the grayscale value under the black and white stripes, and at the same time, it is impossible to exclude the interference of the ambient light source on the grayscale value during the image acquisition process of the image sensor, thereby resulting in inaccurate tilt angle judgment.

[0005] Therefore, the present invention proposes an image sensor stitching determination method, system, adjustment method and storage medium, which can be used to evaluate the installation quality of a camera or image sensor and make adjustments based on the installation angle to improve the quality of the stitched image. Summary of the invention

[0006] The purpose of the present invention is to overcome the above problems existing in the prior art. To achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0007] An image sensor splicing determination method splices at least two image sensors whose photosensitive surfaces are parallel to a detection surface, and collects images to be spliced ​​in adjacent areas of the detection surface along a pixel row direction to expand a scanning field of view, comprising:

[0008] Calculate the weight of the black grayscale value and / or the white grayscale value of each pixel on the row image; wherein the row image is an image with inconsistent grayscale values ​​along the pixel row extracted from the image to be spliced;

[0009] Analyze the weight difference of the black gray value or white gray value ratio of two adjacent pixels on the row image;

[0010] Calculate the tilt angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample;

[0011] Determine whether there is feature missing after two adjacent images to be stitched are stitched together;

[0012] The detection surface is coplanar with the upper surface of the standard sample, the standard sample is composed of black and white stripes of equal width, and the width of the black or white stripes is not less than one pixel.

[0013] An image sensor splicing determination system, comprising:

[0014] A weight proportion module extracts pixel row images with inconsistent grayscale values ​​along the pixel rows in the image to be stitched, and calculates the proportion weight of the black grayscale value and / or the white grayscale value in each pixel on the row image;

[0015] The tilt angle analysis module calculates the difference of the black grayscale value or the white grayscale value proportion weight of two adjacent pixel points along the pixel row direction in the same image to be stitched, which is used to analyze the tilt angle of each image sensor;

[0016] The splicing missing judgment module determines whether there is feature missing after the two adjacent images to be spliced ​​are spliced ​​based on the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample. It is used to determine whether the overlapping area of ​​the images captured by the two image sensors is sufficient.

[0017] A method for installing and adjusting an image sensor, comprising:

[0018] Calculating the relative tilt angle of two adjacent image sensors based on the tilt angle of the image to be stitched in the installation rotation direction of the two adjacent image sensors;

[0019] Comparing the relative tilt angle with a first angle threshold corresponding to two adjacent image sensors with the same installation rotation direction to determine whether the installation angle of the image sensor needs to be adjusted;

[0020] If the relative tilt angle is greater than a first angle threshold corresponding to two adjacent image sensors with the same installation rotation direction, the installation angle of at least one image sensor is adjusted; otherwise, no adjustment is required.

[0021] A computer-readable storage medium stores a computer program, which implements the above determination method when executed by a processor.

[0022] Beneficial effects of the present invention:

[0023] 1. The present invention performs difference analysis on the weighted proportions of black grayscale values ​​or white grayscale values ​​of two adjacent pixels within the allowable error range in the same image to be stitched, so as to obtain the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample, and determines whether there is feature loss after the two adjacent images to be stitched are stitched according to the inclination angle. The inclination degree of the image sensor can be obtained based on the black and white stripe standard sample, and then the quality of subsequent image stitching can be evaluated.

[0024] 2. The present invention calculates the average value of the weighted difference of the black gray value or white gray value ratio of two adjacent pixels in the same image to be spliced, thereby eliminating errors and improving the accuracy of the calculation of the tilt angle of the image to be spliced.

[0025] 3. The present invention constructs a splicing missing width calculation model to analyze the splicing missing width of two adjacent image sensors under different installation rotation directions and rotation angles through the inclination angle between the pixel row direction of the image sensor and the standard sample, the installation rotation direction and the deflection distance in the pixel row direction of the length of the black and white stripes in the image to be spliced. According to the splicing missing width value, it can accurately determine whether there is feature missing after the images to be spliced ​​collected by the two adjacent image sensors are spliced.

[0026] 4. The present invention uses a splicing missing width calculation model to analyze the rotation angle limitation conditions of two adjacent image sensors under different installation rotation directions when there is no splicing missing in the images to be spliced ​​acquired by two adjacent image sensors, so that when the rotation angle limitation conditions of the two adjacent image sensors are met, the images to be spliced ​​acquired by the two image sensors have no feature missing after splicing, and the installation rotation direction and angle of one of the image sensors can be installed and controlled.

[0027] 5. The present invention compares the installation angle of any one of the two adjacent image sensors with the first angle threshold corresponding to the two adjacent image sensors in the same rotation direction, performs an initial adjustment on the installation angle of the image sensor, and compares the relative tilt angle of the two adjacent image sensors with the first angle threshold corresponding to the two adjacent image sensors in the same rotation direction, selectively adjusts the installation angle of the image sensor to meet the second angle threshold, until the relative tilt angle is less than the first angle threshold of the same installation rotation direction, and realizes multiple step-by-step adjustments of the installation angles of the two image sensors, so that the angle between the pixel row direction of the image sensor and the length direction of the black and white stripes in the standard sample gradually approaches 0°, so that the utilization rate of the image after image stitching and rotation is guaranteed to be large while ensuring no missing stitching, and the field of view presented in the image is improved.

[0028] 6. The present invention can quantify the light intensity by analyzing the relative light intensity that needs to be adjusted in the current shooting environment and the light intensity when the grayscale value of the pixels in the white area is 255 and the grayscale value of the pixels in the black area is 0. By using the change in the grayscale value when the grayscale value changes from the current light intensity to the maximum or minimum light intensity under different light intensities, the grayscale value after compensation under the current light intensity is evaluated. The grayscale value affected by the light intensity in the environment can be adjusted to eliminate environmental interference, providing a reliable and accurate basis for calculating the weights of the white and black proportions of pixels in the grayscale value calculation formula. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0030] Figure 1 is a flow chart of the image sensor splicing determination method of the present invention;

[0031] Figure 2 is a schematic diagram of the projection of the image sensor in the present invention on the plane where the standard sample is located;

[0032] Figure 3 is a schematic diagram of the projection of the image sensor in the present invention on the plane where the standard sample is located;

[0033] Figure 4 is a schematic diagram of the projection of the image sensor in the present invention on the plane where the standard sample is located;

[0034] Figure 5 It is a schematic diagram of the splicing of two image sensors along the pixel row direction. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] like Figure 1 As shown, the present invention first provides an image sensor stitching determination method, comprising:

[0037] Splice at least two image sensors whose photosensitive surfaces are parallel to the detection surface, and collect the images to be stitched in adjacent areas of the detection surface along the pixel row direction to expand the scanning field of view;

[0038] When the line scan image sensors corresponding to the images to be stitched in adjacent areas along the pixel row direction have no angle relative to the length direction of the black and white stripes on the standard sample, there is an overlapping area in the images to be stitched captured by the two line scan image sensors, and the two images to be stitched do not completely overlap.

[0039] Calculate the weight of the black gray value and / or the white gray value of each pixel on the row image;

[0040] The row image is an image with inconsistent grayscale values ​​along the pixel row extracted from the image to be spliced;

[0041] Analyze the weight difference of the black gray value or white gray value ratio of two adjacent pixels on the row image;

[0042] Calculate the tilt angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample;

[0043] Determine whether there is feature missing after two adjacent images to be stitched are stitched together;

[0044] Based on the inclination angle between the pixel row direction of each line scan image sensor and the length direction of the black and white stripes on the standard sample, it is determined whether there is feature loss after the two adjacent images to be stitched are stitched together, and it is used to determine whether the overlapping area of ​​the images collected by the two line scan image sensors is sufficient;

[0045] Two line scan image sensors parallel to the standard sample capture images of the standard sample, and image stitching is performed on the captured images to be stitched. If there is no missing or omitted features in the stitched standard sample image, the overlapping area of ​​the images captured by the two line scan image sensors is sufficient. If there are some missing or omitted features in the stitched standard sample image, the overlapping area of ​​the images captured by the two line scan image sensors is insufficient.

[0046] The detection surface is coplanar with the upper surface of the standard sample, the standard sample is composed of black and white stripes of equal width, and the width of the black or white stripes is not less than one pixel.

[0047] In the present application, the image sensor may be a line scan image sensor or an area scan image sensor.

[0048] The angle between the pixel row direction of each line scan image sensor and the length direction of the black and white stripes on the standard sample is composed of the installation angle of the line scan image sensor mounted on the camera relative to the camera and the installation angle of the camera relative to the length direction of the black and white stripes on the standard sample.

[0049] The weight of the proportion of black grayscale value and white grayscale value in each pixel is calculated using the grayscale value calculation formula: f(x,y)=g1(x,y)×f1+g2(x,y)×f2, f1 represents the grayscale value of white, the grayscale value of white is 255, f2 represents the grayscale value of black, the grayscale value of black is 0, g1(x,y) and g2(x,y) represent the weight of the proportion of white grayscale value and the weight of the proportion of black grayscale value of the pixel with coordinates (x,y), respectively, and the sum of the weight of the proportion of white grayscale value and the weight of the proportion of black grayscale value of the same pixel is equal to 1.

[0050] Pixel rows in which the grayscale values ​​of each pixel on the pixel row or column in the image to be stitched are not all equal are extracted, and according to the grayscale values ​​corresponding to each pixel in the pixel row, a grayscale value calculation formula is used to derive the white grayscale value proportion weight Q(k) of each pixel point arranged sequentially in the direction of the pixel row or column in which the grayscale values ​​on the image to be stitched collected by the same image sensor are not all equal. Q(k) is expressed as the white grayscale value proportion weight of the kth pixel point in the direction of the pixel row or column in which the grayscale values ​​on the image to be stitched are not all equal, and the black grayscale value proportion weight is calculated based on the white grayscale value proportion weight.

[0051] Screen out at least two groups of adjacent pixels in the same image to be stitched, where the difference in the weight of the black grayscale value or white grayscale value ratio of the two adjacent pixels is within the allowable error range, analyze the difference in the weight of the black grayscale value or white grayscale value ratio of the two adjacent pixels, and calculate the inclination angle between the pixel row direction of the line scan image sensor and the length direction of the black and white stripes in the standard sample. Figure 2 As shown, the width of the black and white stripes in the standard sample is the same and is one pixel wide. Figure 3 and Figure 4 As shown in the figure, the width of the black and white stripes in the standard sample is the same and greater than the width of one pixel. Figure 4 As shown, when the grayscale values ​​on the pixel rows are the same, the proportion weights of the black grayscale value or the white grayscale value of each pixel in the row image are the same, and cannot be calculated based on the proportion weights of the black grayscale value or the white grayscale value.

[0052] In the same image to be stitched, if the error between the difference in weights of the black grayscale value or the white grayscale value proportion of two adjacent pixels on a pixel row and the difference in weights of the black grayscale value or the white grayscale value proportion of other two adjacent pixels on the pixel row is greater than a set threshold, then it indicates that the selected two adjacent pixels do not meet the requirements, so that there is a large error in using the difference in weights of the black grayscale value or the white grayscale value proportion of the selected two adjacent pixels to calculate the inclination angle between the pixel row direction and the length direction of the black and white stripes on the standard sample, wherein the set threshold is designed according to the calculation accuracy of the inclination angle between the pixel row direction and the length direction of the black and white stripes on the standard sample.

[0053] When the difference in the weights of the black grayscale values ​​or white grayscale values ​​between two adjacent pixels is equal to the difference in the weights of the black grayscale values ​​or white grayscale values ​​between other two adjacent pixels on the pixel row, it indicates that the selected two adjacent pixels meet the requirements. The difference in the weights of the black grayscale values ​​or white grayscale values ​​between the selected two adjacent pixels can be used to accurately calculate the inclination angle between the pixel row direction and the length direction of the black and white stripes on the standard sample.

[0054] Each line scan image sensor is based on the difference △Q of the white gray value proportion weight between two adjacent pixels in the pixel row direction in the collected image 行 , analyze the angle between the pixel row direction of the line scan image sensor and the length direction of the black and white stripes in the standard sample: β=arctan△Q 行 ;

[0055] Each line scan image sensor is based on the weight difference △Q of the white gray value ratio between two adjacent pixels in the pixel column direction in the collected image. 列 , analyze the angle between the pixel row direction of the line scan image sensor and the length direction of the black and white stripes in the standard sample: β=90°-arctan△Q 列 .

[0056] Similarly, the angle between the pixel row direction of the line scan image sensor and the length direction of the black and white stripes in the standard sample can also be analyzed based on the difference in the weight of the black grayscale value between two adjacent pixels in the pixel row or column direction in the captured image.

[0057] Through the above method, the angles between the pixel row directions of the two line scan image sensors and the length directions of the black and white stripes in the standard sample are analyzed respectively, that is, the angle between the pixel row direction of the line scan image sensor located on the left side of the expanded scanning field of view and the length direction of the black and white stripes in the standard sample is β1, and the angle between the pixel row direction of the line scan image sensor located on the right side of the expanded scanning field of view and the length direction of the black and white stripes in the standard sample is β2.

[0058] When the acquired image to be stitched is rotated clockwise with the upper left corner of the image to be stitched, the angle between the length direction of the black and white stripes in the image to be stitched captured by the line scan image sensor and the pixel row direction of the line scan image sensor is greater than 0; when the acquired image to be stitched is rotated counterclockwise with the upper left corner of the image to be stitched, the angle between the length direction of the black and white stripes in the image to be stitched captured by the line scan image sensor and the pixel row direction of the line scan image sensor is less than 0.

[0059] Among them, the standard installation position and standard installation angle of two adjacent line scan image sensors are set. Under the standard installation position and installation angle, the images to be spliced ​​collected by the two adjacent line scan image sensors with an expanded scanning field of view have overlapping areas. In order to facilitate the determination of the installation position and angle of the line scan image sensor, it is necessary to determine the installation position of the upper left corner of the line scan image sensor according to the width of the overlapping area of ​​the images to be spliced ​​collected by the two adjacent line scan image sensors with an expanded scanning field of view, that is, the standard installation position of the line scan image sensor. Under the standard installation angle, the angle between the length direction of the black and white stripes in the image to be spliced ​​collected by the line scan image sensor and the pixel row direction of the line scan image sensor is equal to 0. When the two line scan image sensors adopt the standard installation position and the standard installation angle, the overlapping area of ​​the images collected by the two line scan image sensors is sufficient, and the images to be spliced ​​collected by the two line scan image sensors have no feature loss after splicing.

[0060] Due to the influence of installation accuracy, the standard installation position and standard installation angle of the two-line scan image sensor cannot be guaranteed, resulting in deflection, translation, etc. of the image to be stitched acquired by the two-line scan image sensor. Example 1

[0061] The line scan image sensor can be selected from CCD sensor and CMOS sensor.

[0062] Line scan image sensor is an imaging device that uses a one-dimensional or multi-dimensional linear array to continuously scan along the direction of motion of the object to be detected. Images are acquired line by line through continuous single-line scanning. It is usually used in application scenarios that require high-precision and high-speed imaging.

[0063] Splice at least two image sensors whose photosensitive surfaces are parallel to the detection surface, and collect the images to be stitched in adjacent areas of the detection surface along the pixel row direction to expand the scanning field of view;

[0064] Calculate the weight of the black grayscale value and / or the white grayscale value of each pixel on the row image, wherein the row image is an image with inconsistent grayscale values ​​along the pixel row extracted from the image to be spliced;

[0065] Screen out at least two groups of adjacent two pixels in the same image to be stitched, the difference between the black grayscale value or the white grayscale value ratio weight of the adjacent two pixels being within the allowable error range, analyze the average value of the difference between the black grayscale value or the white grayscale value ratio weight of the adjacent two pixels, and calculate the inclination angle between the pixel row direction of the line scan image sensor and the length direction of the black and white stripes in the standard sample;

[0066] Based on the images to be stitched collected at the inclination angle between the pixel row direction of each line scan image sensor and the length direction of the black and white stripes on the standard sample, the stitching missing width of two adjacent images to be stitched is calculated to determine whether there is feature missing after the two adjacent images to be stitched are stitched.

[0067] The images to be stitched, which are acquired by two line-scan image sensors for expanding the scanning field of view, are stitched. Due to the installation error of the line-scan image sensors, features of the two images to be stitched may be lost after stitching. The stitching missing width of the two images to be stitched is analyzed according to the inclination angles β1 and β2 between the length direction of the black and white stripes on the images to be stitched, which are acquired by the two line-scan image sensors and the pixel row direction of the line-scan image sensors.

[0068] The calculation formula of splicing missing width D is: D=d1+d2-w

[0069] Among them, when D>0, it indicates that there is a missing part, if D≤0, it indicates that there is no missing part, d1 is expressed as the deflection distance in the pixel row direction of the line scan image sensor located on the left side of the stitching direction for collecting the length of the black and white stripes in the image to be stitched, d2 is expressed as the deflection distance in the pixel row direction of the line scan image sensor located on the right side of the stitching direction for collecting the length of the black and white stripes in the image to be stitched, w is expressed as the width of the overlapping area of ​​the two images to be stitched at the standard installation angle, w>0.

[0070] Among them, d1=(1-cosβ1)×b+sinβ1×c and d2=λsinβ2×c or d1=(1-cosβ1)×b and d2=λsinβ2×c, b is the length of the image to be stitched acquired by the line scan image sensor, c is the width of the image to be stitched acquired by the line scan image sensor, c is a pixel width that is a positive integer multiple of a pixel width, and λ represents whether the installation rotation directions of two adjacent line scan image sensors used to expand the scanning field of view are the same. When the installation rotation directions of two adjacent line scan image sensors used to expand the scanning field of view are opposite, λ takes the value of 1, and when the installation rotation directions of two adjacent line scan image sensors used to expand the scanning field of view are the same, λ takes the value of -1.

[0071] Specifically, when the installation rotation directions of two adjacent line scan image sensors for expanding the scanning field of view are both clockwise, λ is equal to -1, that is, D=(1-cosβ1)×b+sinβ1×c-λsinβ2×cw.

[0072] like Figure 5 As shown, Figure 5 The image shown in (a) and Figure 5 The images shown in (b) are the images stitched together by the line scan image sensor along the line scan direction, where the acquisition Figure 5 The installation rotation direction of the line scan image sensor of the stitched image shown in (a) is counterclockwise, and the acquisition Figure 5 The installation rotation direction of the line scan image sensor of the stitched image shown in (b) is clockwise.

[0073] When the installation rotation direction of the line scan image sensor on the left is clockwise and the installation rotation direction of the line scan image sensor on the right is counterclockwise, λ is equal to 1, that is, D=(1-cosβ1)×b+sinβ1×c+λsinβ2×cw.

[0074] When the installation rotation direction of two adjacent line scan image sensors for expanding the scanning field of view is counterclockwise, λ is equal to -1, that is, D=(1-cosβ1)×b-λsinβ2×cw.

[0075] When the installation rotation direction of the line scan image sensor on the left is counterclockwise and the installation rotation direction of the line scan image sensor on the right is clockwise, λ is equal to 1, that is, D=(1-cosβ1)×b+λsinβ2×cw.

[0076] The present invention also provides an image sensor splicing determination system, comprising:

[0077] A weight proportion module extracts pixel row images with inconsistent grayscale values ​​along the pixel rows in the image to be stitched, and calculates the proportion weight of the black grayscale value and / or the white grayscale value in each pixel on the row image;

[0078] The tilt angle analysis module calculates the difference of the black gray value or the white gray value proportion weight of two adjacent pixels along the pixel row direction in the same image to be stitched, and is used to analyze the tilt angle of each image sensor, wherein the difference of the black gray value or the white gray value proportion weight of at least two groups of the two adjacent pixels in the same image to be stitched is within the allowable error range;

[0079] The splicing missing judgment module determines whether there is feature missing after the two adjacent images to be spliced ​​are spliced ​​based on the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample. It is used to determine whether the overlapping area of ​​the images captured by the two image sensors is sufficient. Example 2

[0080] In this embodiment, the two line scan image sensors for expanding the scanning field of view may be located in the same camera or in different cameras.

[0081] To determine whether there is feature missing after two adjacent images to be stitched are stitched, a stitching missing model is used to analyze the installation rotation angle relationship between the two line scan image sensors used to expand the scanning field of view when there is no feature missing between the two adjacent images to be stitched.

[0082] Whether the images to be stitched, acquired by the two-line scan image sensor with an expanded scanning field of view, have missing features after being stitched is affected by the installation rotation direction and installation angle of the two-line scan image sensor.

[0083] When the installation rotation direction of the line scan image sensor on the left side is clockwise, if the installation rotation direction of the line scan image sensor on the right side is also clockwise, then the clockwise rotation angle range of the line scan image sensor on the left side increases as the clockwise rotation angle of the line scan image sensor on the right side increases; if the installation rotation direction of the line scan image sensor on the right side is counterclockwise, then the clockwise rotation angle range of the line scan image sensor on the left side decreases as the counterclockwise rotation angle of the line scan image sensor on the right side increases.

[0084] Therefore, when the installation direction of the line scan image sensor on the left is clockwise, the rotation angle defines the model: .

[0085] When the installation rotation direction of the line scan image sensor on the left side is counterclockwise, if the installation rotation direction of the line scan image sensor on the right side is clockwise, then the counterclockwise rotation angle range of the line scan image sensor on the left side decreases as the clockwise rotation angle of the line scan image sensor on the right side increases; if the installation rotation direction of the line scan image sensor on the right side is counterclockwise, then the counterclockwise rotation angle range of the line scan image sensor on the left side increases as the counterclockwise rotation angle of the line scan image sensor on the right side increases.

[0086] Therefore, when the installation rotation direction of the line scan image sensor on the left is counterclockwise, the rotation angle defines the model: .

[0087] Wherein, λ represents whether the installation rotation directions of the two adjacent image sensors used to expand the scanning field of view are the same. When the installation rotation directions of the two adjacent image sensors used to expand the scanning field of view are opposite, λ takes the value of 1. When the installation rotation directions of the two adjacent image sensors used to expand the scanning field of view are the same, λ takes the value of -1. b is the length of the image to be stitched acquired by the line scan image sensor, and c is the width of the image to be stitched acquired by the line scan image sensor.

[0088] According to the installation rotation direction of the two-line scan image sensors, the corresponding rotation angle limitation model is selected to adjust the angle value of the two-line scan image sensors in the installation rotation direction. The installation rotation direction and rotation angle of one of the line scan image sensors can be effectively adjusted based on the installation rotation direction and rotation angle of the other line scan image sensor, thereby avoiding the situation where the splicing missing width of the two-line scan image sensors used to expand the scanning field of view cannot obtain effective installation angle guidance due to the unlimited installation rotation direction and rotation angle of the two-line scan image sensors. Example 3

[0089] In this embodiment, the number of line scan image sensors for expanding the scanning field of view is no less than 2, and each line scan image sensor may be located in at least one camera.

[0090] When the number of line scan image sensors that expand the scanning field of view is greater than two, the inclination angle between the length direction of the black and white stripes on the image to be stitched captured by the two line scan image sensors and the pixel row direction of the line scan image sensors is calculated, and based on the images to be stitched captured at the inclination angle between the pixel row direction of each line scan image sensor and the standard sample, the stitching missing width of two adjacent images to be stitched is calculated, and then it is determined whether there is feature missing after the images to be stitched captured by multiple line scan image sensors are stitched.

[0091] When the number of line scan image sensors that expand the scanning field of view is greater than 2, the calculation formula for the splicing missing width group is:

[0092] D(n-1) represents the value of the stitching missing width between the images to be stitched captured by the n-1th line scan image sensor and the nth line scan image sensor, d(n-1) represents the deflection distance in the pixel row direction of the line scan image sensor of the length of the black and white stripes in the image to be stitched captured by the n-1th line scan image sensor located on the left side in the stitching direction of the two adjacent line scan image sensors, d2 represents the deflection distance in the pixel row direction of the line scan image sensor of the length of the black and white stripes in the image to be stitched captured by the nth line scan image sensor located on the right side in the stitching direction of the two adjacent line scan image sensors, w represents the width of the overlapping area of ​​the two images to be stitched at the standard installation angle, w>0, n is greater than 2.

[0093] When at least one value in D1~D(n-1) is greater than 0, it indicates that the images to be stitched acquired by several line scan image sensors with an expanded scanning field of view have feature missing after stitching. If all data in D1~D(n-1) are less than 0, it indicates that the images to be stitched acquired by two adjacent line scan image sensors have no feature missing, that is, the images to be stitched acquired by all line scan image sensors with an expanded scanning field of view have no feature missing after stitching.

[0094] In the calculation formula of the splicing missing width group, whether the installation rotation angle and the installation rotation direction of two adjacent line scan image sensors are the same directly determines the splicing missing width of the to-be-stitched images acquired by the two adjacent line scan image sensors. Example 4

[0095] The line scan image sensor used to expand the scanning field of view is parallel to the standard sample. When the distances between at least two adjacent line scan image sensors and the standard sample are different, image registration is performed on the overlapping areas in the images captured by the at least two adjacent line scan image sensors. After the image registration is completed, the images captured by the at least two adjacent line scan image sensors are spliced.

[0096] The relative tilt angle of two adjacent line scan image sensors projected on the plane where the standard sample is located directly affects whether the corresponding two line scan image sensors have stitching missing at the current tilt angle, and the installation angles of the two cameras or line scan image sensors with stitching missing are adjusted so that the images captured at the installation angle have no stitching missing after stitching.

[0097] Analyze the relative tilt angle of the two line scan image sensors projected on the plane where the standard sample is located. The relative tilt angle is:

[0098] △β=β1+λβ2.

[0099] Among them, λ represents whether the installation rotation directions of two adjacent line scan image sensors used to expand the scanning field of view are the same. When the installation rotation directions of two adjacent line scan image sensors used to expand the scanning field of view are opposite, λ takes the value of 1; when the installation rotation directions of two adjacent line scan image sensors used to expand the scanning field of view are the same, λ takes the value of -1.

[0100] According to the installation rotation direction of the two line scan image sensors, it is determined whether the relative tilt angle between the two line scan image sensors is greater than a first angle threshold set under the installation rotation direction.

[0101] If the installation rotation direction of the line scan image sensor located on the left is clockwise, the first angle threshold allowed under this rotation direction is set; if the installation rotation direction of the line scan image sensor located on the right is counterclockwise, the first angle threshold allowed under this rotation direction is set. When the relative inclination angle corresponding to the installation rotation directions of the two line scan image sensors is greater than the first angle threshold allowed under the installation rotation direction, at least adjust the installation angle of one line scan image sensor or camera relative to the other line scan image sensor in the pixel row direction to reduce the first angle threshold between the two line scan image sensors or cameras.

[0102] The installation adjustment is performed based on a first angle threshold corresponding to the installation rotation direction of two adjacent line scan image sensors. The present invention provides a specific installation adjustment method, comprising the following steps:

[0103] Comparing the relative tilt angle with a first angle threshold corresponding to two adjacent line scan image sensors installed in the same rotation direction to determine whether the installation angle of the line scan image sensor needs to be adjusted;

[0104] If the relative tilt angle is greater than the first angle threshold corresponding to two adjacent line scan image sensors with the same installation rotation direction, then the installation angle of at least one line scan image sensor needs to be adjusted; otherwise, no adjustment is required.

[0105] That is, according to the different installation rotation directions of two adjacent line scan image sensors, the corresponding first angle threshold is screened.

[0106] The installation rotation direction of two adjacent line scan image sensors includes four installation rotation directions, namely, the installation rotation direction of the line scan image sensor on the left is clockwise and the installation rotation direction of the line scan image sensor on the right is counterclockwise, the installation rotation direction of the line scan image sensor on the left is clockwise and the installation rotation direction of the line scan image sensor on the right is clockwise, the installation rotation direction of the line scan image sensor on the left is counterclockwise and the installation rotation direction of the line scan image sensor on the right is counterclockwise, and the installation rotation direction of the line scan image sensor on the left is counterclockwise and the installation rotation direction of the line scan image sensor on the right is clockwise.

[0107] When the installation rotation directions of two adjacent line scan image sensors are different, the first angle thresholds corresponding to the installation rotation directions of the two line scan image sensors are also different.

[0108] If the relative tilt angle of two adjacent line scan image sensors in the installation rotation direction is greater than the first angle threshold corresponding to two adjacent line scan image sensors with the same installation rotation direction, then at least one installation angle of the line scan image sensor needs to be adjusted, including:

[0109] Step S1, determining in sequence whether the absolute value of the installation angle of each of two adjacent line scan image sensors in the installation rotation direction is greater than the absolute value of the first angle threshold corresponding to two adjacent line scan image sensors in the same installation rotation direction;

[0110] Step S2: if the absolute value of the angle is greater than the first angle threshold corresponding to two adjacent line scan image sensors with the same installation rotation direction, adjusting the installation angle greater than the line scan image sensor so that the absolute value of the adjusted installation angle of the line scan image sensor is less than the absolute value of the second angle threshold corresponding to the installation rotation direction;

[0111] Step S3, extracting the adjusted installation angle of the image sensor, and determining whether the relative inclination angle of two adjacent line scan image sensors in the installation rotation direction after the installation adjustment is greater than a first angle threshold corresponding to two adjacent line scan image sensors in the same installation rotation direction;

[0112] Step S4: If the angle is greater than the first angle threshold corresponding to two adjacent line scan image sensors with the same installation rotation direction, adjust the installation angle of the line scan image sensor with the largest absolute value of the installation angle of the two adjacent line scan image sensors until the relative inclination angle of the two adjacent line scan image sensors is less than the first angle threshold corresponding to the two adjacent line scan image sensors with the same installation rotation direction.

[0113] The absolute value of the first angle threshold of two adjacent line scan image sensors installed in the same rotation direction is greater than the absolute value of the second angle threshold in the installation rotation direction.

[0114] By comparing the installation angle of any one of two adjacent line scan image sensors with the first angle threshold corresponding to the two adjacent image sensors in the same rotation direction, the installation angle of the line scan image sensor is initially adjusted, and the relative tilt angle of the two adjacent line scan image sensors is compared with the first angle threshold corresponding to the two adjacent image sensors in the same rotation direction, and the installation angle of the line scan image sensor is selectively adjusted to meet the second angle threshold until the relative tilt angle is less than the first angle threshold of the same installation rotation direction, thereby realizing multiple step-by-step adjustments of the installation angles of the two line scan image sensors, so that the angle between the pixel row direction of the line scan image sensor and the length direction of the black and white stripes in the standard sample gradually approaches 0°, so that the utilization rate of the image after image stitching and rotation is guaranteed to be large while satisfying the requirement of no missing stitching, thereby improving the field of view presented in the image. Example 5

[0115] The technical solutions of the above embodiments do not consider the influence of the light source environment on the grayscale value. In a standard light source environment, the grayscale value of white is 255 and the grayscale value of black is 0. In a non-standard light source environment, the grayscale value will change due to the influence of the light source environment.

[0116] If the light source environment changes, the grayscale value of each pixel will change according to the grayscale value calculation formula f(x,y)=g1(x,y)×f1+g2(x,y)×f2. The white proportion weight and black proportion weight of each pixel calculated according to the grayscale value calculation formula will deviate from the actual value, affecting the accuracy of the tilt angle calculation of the image sensor or camera. The influence of the light source environment on the grayscale value is now corrected.

[0117] The standard image sample consists of black stripes and white stripes of the same width, and the widths of the black stripes and the white stripes are both integer pixel widths greater than n, where n is a positive integer greater than 2.

[0118] Considering that the shooting light environment will affect the grayscale value, the image sensor collects images of the black stripe area and the white stripe area respectively, filters the maximum grayscale value or the minimum grayscale value in the row or column of pixels, and determines whether the maximum grayscale value is equal to 255 and whether the minimum grayscale value is equal to 0. If the maximum grayscale value is not equal to 255 or the minimum grayscale value is not equal to 0, the color corresponding to the pixel with the smallest grayscale value is black, and the color corresponding to the pixel with the largest grayscale value is white.

[0119] The grayscale value in the standard image changes with the brightness of the environment. The grayscale value of white, which is 255, gradually decreases as the brightness decays, while the grayscale value of black, which is 0, gradually increases as the brightness increases. Therefore, the grayscale value in the standard image presents different grayscale values ​​under different light intensities.

[0120] Since the width of the black / white stripes of the standard sample is greater than the width of 2 pixels, the line scan image sensor collects the standard sample image, and there is at least a maximum grayscale value and a minimum grayscale value. If the minimum grayscale value is not equal to 0 and the maximum grayscale value is not equal to 255, it means that the grayscale value of the pixel on the collected image of the standard sample is affected by the ambient light, and the grayscale value of the image captured by the ambient light is corrected.

[0121] If the influence of light intensity on pixel grayscale value is not eliminated, errors will occur in the calculation of the weights of black and white proportions in each pixel, which will in turn affect the calculation of the tilt angle of the line scan image sensor.

[0122] Before the step of calculating the weight of the proportion of black and / or white in each pixel, a method of correcting the grayscale value based on light intensity is added to eliminate the interference of light intensity on the grayscale value in the collected image and improve the accuracy of the weight of the proportion of black and white in the pixel.

[0123] The method for correcting each grayscale value based on light intensity (non-standard light intensity) includes the following steps:

[0124] Step W1, construct a relative light intensity model, and analyze the relative light intensity corresponding to the light intensity.

[0125] Different gray values ​​are selected at equal gray value intervals as gray values ​​to be detected, and the gray values ​​corresponding to each gray value under different relative light intensities are detected.

[0126] In the black and white stripes, the minimum light intensity μ1 when the gray value of the pixels in the white area is 255 min , Dangdang continues to increase the light intensity, making the light intensity greater than the minimum light intensity μ1 min When the gray value is 255, the white area is overexposed; when the gray value of the pixels in the black area is 0, the light intensity is the maximum light intensity μ2 max , μ1 min >μ2 max , when the light intensity continues to decrease, making the light intensity less than the maximum light intensity μ2 max The grayscale value in the black area is 0.

[0127] The light intensity is optimized to obtain a relative light intensity δ, the relative light intensity value range is between 0 and 1, and the relative light intensity is adjusted in sequence.

[0128] Relative light intensity model formula:

[0129] Wherein, μ represents the light intensity corresponding to the shooting environment, and δ represents the relative light intensity corresponding to the light intensity μ in the shooting environment.

[0130] Step W2: Determine the difference between the relative illumination intensity and the set relative illumination intensity threshold, and determine the first adjustment parameter ε and the second adjustment parameter ρ.

[0131] When the relative illumination intensity is lower than the set relative illumination intensity threshold, the gray value of the collected image becomes smaller.

[0132] ε=-log2μ, ;

[0133] When the relative illumination intensity is higher than the set relative illumination intensity threshold, the grayscale value of the collected image is larger. At this time, ε=exp(1-μ) and ρ=1.

[0134] The set relative illumination intensity threshold is the relative illumination intensity corresponding to the illumination intensity when the grayscale value range of the black area in the black and white stripes collected under the shooting environment is [0, A] and the grayscale value range of the white area is [255-A, 255], which is a positive integer of 0<A<40.

[0135] Step W3: Use the gamma transformation formula and combine it with step W2 to analyze the relative light intensity that needs to be adjusted in the current shooting environment:

[0136] Expressed as the relative light intensity after adjustment, δ 标准 It is expressed as the relative light intensity corresponding to the light intensity under the standard shooting environment.

[0137] Step W4: training the relative illumination intensity threshold and the relative illumination intensity required to be adjusted in the current environment to obtain the grayscale value after compensation under the current illumination intensity.

[0138] Gray value compensation model under relative light intensity:

[0139] in, It is expressed as the gray value of the pixel after compensation under the current light intensity. γ1 and γ2 are the set first gray compensation amount and the second gray compensation amount respectively. The first gray compensation amount is the gray value f0 under the light intensity μ transformed from the current light intensity μ to the minimum light intensity μ1. min The gray value increase during the process, the second gray compensation amount is the gray value f0 under the light intensity μ is transformed from the current light intensity μ to the maximum light intensity μ2 max The attenuation during the process, μ is the light intensity in the current shooting environment, and f0 is the grayscale value under the current light intensity μ.

[0140] By analyzing the relative light intensity that needs to be adjusted in the current shooting environment and the minimum light intensity when the grayscale value of the pixels in the white area is 255, and the light intensity when the grayscale value of the pixels in the black area is 0, the light intensity can be quantified. By using the change in grayscale value when the grayscale value changes from the current light intensity to the maximum or minimum light intensity under different light intensities, the grayscale value after compensation under the current light intensity is evaluated. The grayscale value affected by the light intensity in the environment can be adjusted to eliminate environmental interference, providing a reliable and accurate basis for calculating the weights of the white and black proportions of pixels in the grayscale value calculation formula.

[0141] Based on the same inventive concept, the present application also proposes a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for determining image stitching missing of a multi-network port linear array camera as described above is implemented.

[0142] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0143] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A method for determining image sensor splicing, splicing at least two image sensors whose photosensitive surfaces are parallel to the detection surface, collecting images to be spliced ​​in adjacent areas of the detection surface along the pixel row direction to expand the scanning field of view, characterized in that: include: Calculate the weight of the black grayscale value and / or white grayscale value of each pixel on the row image; wherein the row image is an image with inconsistent grayscale values ​​along the pixel row extracted from the image to be stitched, the weight of the white grayscale value = the grayscale value of the pixel / the grayscale value of white, and the weight of the black grayscale value = 1 - the weight of the white grayscale value; Screen out at least two groups of two adjacent pixel points in the same image to be stitched, the two adjacent pixel points whose difference in the weight of the black grayscale value or the white grayscale value ratio is within the allowable error range, and analyze the difference in the weight of the black grayscale value or the white grayscale value ratio of the two adjacent pixel points on the row image, wherein the difference in the weight of the black grayscale value or the white grayscale value ratio of the two adjacent pixel points is not equal to 0; According to the weight difference of the black gray value or the white gray value ratio of the two adjacent pixel points on the row image, the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample is calculated; Determine whether there is feature missing after two adjacent images to be stitched are stitched together; The detection surface is coplanar with the upper surface of the standard sample, the standard sample is composed of black and white stripes of equal width, and the width of the black or white stripes is not less than one pixel.

2. The image sensor splicing determination method according to claim 1, characterized in that: The gray value of the pixel point is composed of the weight of the proportion of the black gray value and the black gray value, and the weight of the proportion of the white gray value and the white gray value.

3. The image sensor splicing determination method according to claim 1, characterized in that: The inclination angle between the pixel row direction of the image sensor and the length direction of the black and white stripes on the standard sample is obtained by using the difference analysis of the black grayscale value or the white grayscale value proportion weights of the two adjacent pixel points in the pixel row direction, wherein the two adjacent pixel points used are in the same image to be stitched, and the difference in the black grayscale value or the white grayscale value proportion weights of at least two groups of the two adjacent pixel points is within the allowable error range.

4. The image sensor splicing determination method according to claim 3, characterized in that: Based on the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample, it is determined whether there is a feature loss after the two adjacent images to be spliced ​​are spliced, including: The relationship between the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample, the installation rotation direction of the image sensor, and the deflection distance of the length of the black and white stripes in the image to be stitched collected by the image sensor in the pixel row direction of the image sensor is extracted, and the stitching missing width after the stitching of two adjacent images to be stitched is analyzed. According to the value of the stitching missing width, it is determined whether there is feature missing after the stitching of two adjacent images to be stitched.

5. The image sensor splicing determination method according to claim 4, characterized in that: Based on the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample, it is determined that there is no feature loss after the two adjacent images to be stitched are stitched, including: The installation rotation direction of two adjacent image sensors and the length and width of the images to be stitched acquired by the image sensors are used to analyze the installation rotation angle relationship between the two adjacent image sensors used to expand the scanning field of view when there is no feature loss in the two adjacent images to be stitched.

6. The image sensor splicing determination method according to claim 1, characterized in that: Before calculating the weight of the black and / or white proportion in each pixel, a correction method for the gray value based on the light intensity is also included, specifically: Construct a relative light intensity model based on light intensity and analyze the relative light intensity corresponding to the light intensity; Determine the relative light intensity and the preset relative light intensity threshold to determine the first adjustment parameter and the second adjustment parameter; Analyzing the relative light intensity required to be adjusted in the current shooting environment based on the first adjustment parameter, the second adjustment parameter and the gamma transformation formula; The relative light intensity threshold and the relative light intensity required to be adjusted in the current environment are trained to obtain the grayscale value after compensation under the light intensity of the current environment.

7. An image sensor splicing determination system, characterized in that: include: A weight proportion module extracts pixel row images with inconsistent grayscale values ​​along the pixel rows in the image to be stitched, and calculates the proportion weight of the black grayscale value and / or the white grayscale value in each pixel on the row image; Among them, the weight of white gray value proportion = gray value of pixel point / gray value of white, and the weight of black gray value proportion = 1 - weight of white gray value proportion; The tilt angle analysis module selects at least two groups of adjacent two pixel points in the same image to be stitched, the difference of the black gray value or the white gray value ratio weight of the adjacent two pixel points is within the allowable error range, and calculates the difference of the black gray value or the white gray value ratio weight of the adjacent two pixel points along the pixel row direction, which is used to analyze the tilt angle of each image sensor, wherein the difference of the black gray value or the white gray value ratio weight of the adjacent two pixel points is not equal to 0; The splicing missing judgment module determines whether there is feature missing after the two adjacent images to be spliced ​​are spliced ​​based on the inclination angle between the pixel row direction of each image sensor and the length direction of the black and white stripes on the standard sample. It is used to determine whether the overlapping area of ​​the images captured by the two image sensors is sufficient.

8. A method for installing and adjusting an image sensor, characterized in that: The installation adjustment method is based on the tilt angle determined by the image sensor splicing determination method according to any one of claims 1 to 6, and includes: Calculating the relative tilt angle of two adjacent image sensors based on the tilt angle of the image to be stitched in the installation rotation direction of the two adjacent image sensors; Comparing the relative tilt angle with a first angle threshold corresponding to two adjacent image sensors installed in the same rotation direction to determine whether the installation angle of the image sensor needs to be adjusted; If the relative tilt angle is greater than a first angle threshold corresponding to two adjacent image sensors with the same installation rotation direction, the installation angle of at least one image sensor is adjusted; otherwise, no adjustment is required.

9. The image sensor installation and adjustment method according to claim 8, characterized in that: At least one installation angle of the image sensor is adjusted. Specifically, the adjustment method includes: Determining in sequence whether the absolute value of the installation angle of each image sensor in the installation rotation direction of two adjacent image sensors is greater than the absolute value of the first angle threshold corresponding to the two adjacent image sensors in the same installation rotation direction; If it is greater than the absolute value of the first angle threshold corresponding to two adjacent image sensors with the same installation rotation direction, then adjust the installation angle greater than the image sensor so that the absolute value of the adjusted installation angle is less than the absolute value of the second angle threshold corresponding to the installation rotation direction; Extracting the adjusted installation angle of the image sensor, and determining whether the relative inclination angle of two adjacent image sensors after the installation adjustment in the installation rotation direction is greater than a first angle threshold corresponding to two adjacent image sensors in the same installation rotation direction; If the relative tilt angle is greater than the first angle threshold corresponding to two adjacent image sensors with the same installation rotation direction, adjust the installation angle of the image sensor with the largest absolute value of the installation angle of the two adjacent image sensors until the relative tilt angle of the two adjacent image sensors is less than the first angle threshold corresponding to the two adjacent image sensors with the same installation rotation direction; The absolute value of the first angle threshold of two adjacent image sensors installed in the same rotation direction is greater than the absolute value of the second angle threshold in the installation rotation direction.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an image sensor stitching determination method according to any one of claims 1 to 6 is implemented.

Images