A method, system, device, and medium for calibration of adjacent image sensors

Abstract

The application discloses a kind of methods, systems, equipment and media for the calibration of adjacent image sensors, related to the field of machine vision.The calibration method comprises: controlling adjacent image sensors to scan a linear calibration plate along the longitudinal direction synchronously, obtaining a calibration image; according to the number of horizontal pixel points in the calibration image and the total number of pixel columns in the image sensor, dividing the scanning image and the connecting line of different image sensors in the calibration image; calculating the average longitudinal distance of the intersection point of adjacent straight lines and connecting lines as the longitudinal offset of adjacent image sensors; calculate the average value of the included angle of adjacent straight lines as the included angle of adjacent image sensors; calculate the pixel distance of the nearest vertical line segment on both sides of the connecting line in the calibration image, and the difference between the pixel interval of adjacent vertical line segments in the calibration image, as the lateral offset of adjacent image sensors. Thus, the application can more accurately obtain the positional relationship of adjacent linear array sensors, reduce the calibration error.

Description

Technical Field

[0001] This invention belongs to the field of machine vision, and particularly relates to a calibration method, system, device and medium for adjacent image sensors. Background Technology

[0002] In the field of machine vision, to meet the requirements of large field-of-view imaging, it is often necessary to stitch multiple image sensors together. Taking CIS cameras as an example, the stitching methods include a straight line stitching of multiple linear array sensors arranged in a horizontal direction and a triangular stitching of multiple linear array sensors.

[0003] During the stitching process of the aforementioned multiple image sensors, due to factors such as mechanical installation errors, component processing precision, and changes in ambient temperature, problems such as positional offset (horizontal offset and vertical offset) and angular tilt often occur between adjacent sensors.

[0004] To correct the aforementioned issues of positional offset and angular tilt, existing technologies generally use alternating light and dark stripes to calibrate the physical quantities of positional offset and angular tilt. However, during the calibration process, there is a problem of blurred recognition at the boundaries of the alternating light and dark stripes, resulting in slight errors in the calibrated positional offset and angular tilt.

[0005] Therefore, a calibration scheme is urgently needed to improve the accuracy of recognizing the positional offset and angular tilt of adjacent image sensors. Summary of the Invention

[0006] This application proposes a calibration method, system, device, and medium for adjacent image sensors to improve the accuracy of position offset and angle tilt recognition of adjacent image sensors.

[0007] To achieve the above objectives, this application proposes the following technical solutions: In a first aspect of this application, a calibration method for adjacent image sensors is provided, comprising: Adjacent image sensors are controlled to synchronously scan a linear calibration plate along the longitudinal direction to obtain a calibration image; wherein, the linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments; Based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor, the scan images and connecting lines of different image sensors in the calibration image are divided; where horizontal refers to the connection direction of pixels in the same row, and the connecting line is the boundary line between adjacent scan images; The average longitudinal distance between the intersections of adjacent straight lines and the connecting line is calculated as the longitudinal offset of adjacent image sensors; the average angle between adjacent straight lines is calculated as the angle between adjacent image sensors; the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image is calculated, and the difference between this pixel distance and the pixel interval of adjacent vertical line segments in the calibration image is calculated as the lateral offset of adjacent image sensors; wherein, the adjacent straight lines are two straight lines of the same horizontal line segment on a straight calibration plate in different scanned images; the interval between the vertical line segments is higher than the maximum lateral offset of adjacent image sensors caused by installation errors.

[0008] Optionally, after dividing the scanned images of different image sensors and connecting lines in the calibration image based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor, if there are multiple vertical line segments on both sides of the connecting line, the average distance between adjacent vertical line segments on both sides is calculated as the vertical line segment interval; wherein, there is no connecting line between adjacent vertical line segments.

[0009] Optionally, the linear calibration plate is divided into multiple regions, each region having multiple parallel vertical line segments and at least one horizontal line segment; wherein the line widths of the vertical and horizontal line segments are different in different regions to adapt to the calibration process of adjacent image sensors at different image resolutions.

[0010] Optionally, the linear calibration plate is provided with calibration objects at its four corners to locate the area of ​​the linear calibration plate in the calibration image.

[0011] Optionally, the vertical and horizontal line segments in the linear calibration plate do not intersect.

[0012] Optionally, after obtaining the longitudinal offset, lateral offset, and included angle of adjacent image sensors, the calibration method further includes: The deviation between adjacent image sensors in physical position is determined based on the longitudinal offset, lateral offset, and included angle. The physical position of the other image sensor, excluding the reference sensor, is adjusted to correct the deviation. The reference sensor refers to the image sensor that serves as the position reference for adjacent image sensors during the offset calibration process.

[0013] Optionally, after obtaining the longitudinal offset, lateral offset, and included angle of adjacent image sensors, the calibration method further includes: Using the scanned image of the reference sensor as a reference, the horizontal pixel position of another scanned image is adjusted according to the horizontal offset, and interpolation algorithm is used for interpolation supplementation; the vertical offset of each pixel column is calculated according to the vertical offset, the included angle and the horizontal pixel coordinates of the other scanned image, and the pixel position of the vertical pixels of the other scanned image is adjusted according to the offset, and interpolation algorithm is used for interpolation supplementation. Here, the reference sensor refers to the image sensor that serves as the position reference during the offset calibration process for adjacent image sensors.

[0014] In a second aspect of this application, a calibration system for adjacent image sensors is provided, comprising: The image acquisition module is used to control adjacent image sensors to synchronously scan a linear calibration plate along the longitudinal direction to obtain a calibration image; wherein, the linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments; The image segmentation module is used to segment the scanned images and connecting lines of different image sensors in the calibration image based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor; where horizontal refers to the connection direction of pixels in the same row, and connecting lines are the boundary lines between adjacent scanned images; The offset calculation module is used to calculate the average longitudinal distance between the intersection points of adjacent straight lines and connecting lines, as the longitudinal offset of adjacent image sensors; calculate the average angle between adjacent straight lines, as the angle between adjacent image sensors; calculate the difference between the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image and the pixel interval between adjacent vertical line segments in the calibration image, as the lateral offset of adjacent image sensors; wherein, the adjacent straight lines are two straight lines of the same horizontal line segment of the straight line calibration plate in different scan images; the interval between the vertical line segments is higher than the maximum lateral offset of adjacent image sensors caused by installation errors.

[0015] In a third aspect of this application, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the calibration method described in any of the first aspects.

[0016] In a fourth aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the calibration method described in any of the first aspects.

[0017] The beneficial effects of this application are as follows: This application provides a calibration method for adjacent image sensors, including: Adjacent image sensors are controlled to synchronously scan a linear calibration plate along the longitudinal direction to obtain a calibration image; wherein, the linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments; Based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor, the scan images and connecting lines of different image sensors in the calibration image are divided; where horizontal refers to the connection direction of pixels in the same row, and the connecting line is the boundary line between adjacent scan images; The average longitudinal distance between the intersections of adjacent straight lines and the connecting line is calculated as the longitudinal offset of adjacent image sensors; the average angle between adjacent straight lines is calculated as the angle between adjacent image sensors; the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image is calculated, and the difference between this pixel distance and the pixel interval of adjacent vertical line segments in the calibration image is calculated as the lateral offset of adjacent image sensors; wherein, the adjacent straight lines are two straight lines of the same horizontal line segment on a straight calibration plate in different scanned images; the interval between the vertical line segments is higher than the maximum lateral offset of adjacent image sensors caused by installation errors.

[0018] Based on the above processing, this application uses a linear calibration plate as the calibration carrier. Compared with traditional checkerboard or alternating light and dark stripe patterns as calibration plates, the edge features of the line segments are easier to extract, and it is less affected by the seams between adjacent image sensors. This allows for more accurate acquisition of the positional relationship (tilt angle, offset) between adjacent linear array sensors, reducing calibration errors and improving subsequent correction effects. Furthermore, this application does not rely on laser calibration devices or high-precision guide rails, reducing equipment costs and simplifying the calibration process. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of a single-line image sensor provided in this application; Figure 2 This is a schematic diagram of the structure of a triangular image sensor provided in this application; Figure 3 This is a flowchart illustrating a calibration method provided in this application; Figure 4 This is a schematic diagram of a linear calibration plate provided in this application; Figure 5 This is a schematic diagram of another linear calibration plate provided in this application; Figure 6 This is a schematic diagram of a linear calibration plate with multiple line widths provided in this application; Figure 7 This is a schematic diagram of another linear calibration plate provided in this application; Figure 8 This application provides a calibration image generated by adjacent image sensors; Figure 9 This is a structural diagram of a calibration system provided in this application; Figure 10 This is a structural diagram of an electronic device provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0021] To meet the requirements of large field-of-view imaging, multi-line array sensors are often stitched together in practical applications for image acquisition. During the stitching process, factors such as mechanical installation errors, component manufacturing precision, and environmental temperature changes can easily lead to positional shifts and angular tilts between adjacent image sensors. This results in defects in the stitched image, such as blurred seams, pixel misalignment, and imaging distortion, severely affecting image quality and the accuracy of subsequent detection and recognition. Taking a CIS camera as an example, stitching methods include a linear arrangement of multiple linear array sensors arranged horizontally in a straight line and a triangular arrangement of multiple linear array sensors in a triangular distribution, as shown below. Figure 1 , 2 As shown.

[0022] To correct the aforementioned issues of positional offset and angular tilt, existing technologies generally use alternating light and dark stripes to define the physical quantities of positional offset and angular tilt. However, the boundaries between the light and dark stripes can be blurry, leading to slight errors in the calibrated positional offset and angular tilt, thus reducing calibration accuracy.

[0023] To address the aforementioned problems, this application provides a calibration method for adjacent image sensors, such as... Figure 3 As shown, the method includes the following steps: S1. Control adjacent image sensors to synchronously scan the linear calibration plate along the longitudinal direction to obtain a calibration image. The linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments.

[0024] S2. Based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor, divide the scanned images and connecting lines of different image sensors in the calibration image. The horizontal direction refers to the connection direction of pixels in the same row, and the connecting line is the boundary line between adjacent scanned images, with vertical line segments on both sides of the connecting line.

[0025] S3. Calculate the average longitudinal distance between the intersection points of adjacent straight lines and connecting lines, as the longitudinal offset of adjacent image sensors; calculate the average angle between adjacent straight lines, as the angle between adjacent image sensors; calculate the difference between the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image and the pixel interval between adjacent vertical line segments in the calibration image, as the lateral offset of adjacent image sensors.

[0026] The adjacent straight lines are two straight lines on the same horizontal line segment of the linear calibration plate in different scanned images; the vertical line segment interval is higher than the maximum lateral offset caused by installation error of the adjacent image sensor.

[0027] Based on the above processing, this application uses a linear calibration plate as the calibration carrier. Compared with traditional checkerboard or alternating light and dark striped calibration plates, its line segment edge features are easier to extract, and it is less affected by the seams between adjacent image sensors. This allows for more accurate acquisition of the positional relationship (tilt angle, offset) between adjacent linear array sensors, reducing calibration errors and improving subsequent correction effects. Furthermore, this application does not rely on laser calibration devices or high-precision guide rails, reducing equipment costs and simplifying the calibration process.

[0028] Regarding step S1, the image sensor provided in this application is typically a linear array sensor, such as a CIS (Contact Image Sensor Camera). The stitching method for adjacent image sensors includes a straight line or a triangular pattern. Figure 1 , 2 As shown, there are errors in the lateral deviation, longitudinal deviation, and tilt angle between adjacent image sensors.

[0029] In this application, one of two adjacent image sensors is selected as the reference sensor, whose scanned image, pixel coordinate system, and imaging plane serve as the calibration reference. The other image sensor is the sensor to be calibrated. The lateral offset, longitudinal offset, and included angle between adjacent image sensors are all determined based on the position of the reference sensor.

[0030] There are no rigid requirements for selecting the reference sensor; it needs to be determined based on the image sensor splicing method, the actual application scenario, and the hardware installation foundation. For example, if one of the adjacent sensors is a reference component during mechanical installation (such as the sensor fixed on the left in a straight line splicing or the sensor fixed in the center in a triangular splicing), that sensor can be directly selected as the reference sensor to ensure that the calibration deviation is consistent with the reference dimension of the mechanical installation deviation, which facilitates subsequent physical position correction.

[0031] For linear stitching, the left (or top) sensor is typically selected as the reference sensor. The lateral offset is positively calculated based on the pixel row direction of the reference sensor, and the vertical offset is positively calculated based on the pixel column direction of the reference sensor. For triangular stitching: the center sensor is selected as the reference sensor, and the other two sensors are used as sensors to be calibrated. The deviation relative to the center reference sensor is calculated for each sensor.

[0032] Before performing step S1, this application can set up a calibration device in advance. The linear calibration plate is fixed on the horizontal displacement stage, and the adjacent image sensor is fixed on the vertical displacement stage to ensure that the shooting direction of the image sensor is perpendicular to the surface of the linear calibration plate. The adjustment device is used to make the camera lens roughly aligned with the center of the linear calibration plate to ensure clear imaging and no obvious distortion during the calibration process.

[0033] For step S1, the linear calibration plate contains multiple parallel vertical line segments and several horizontal line segments. The vertical line segments are spaced evenly, and the vertical and horizontal line segments are perpendicular to each other within the calibration plate. There is at least one horizontal line segment and at least two vertical line segments.

[0034] like Figure 4 As shown, the linear calibration plate provided in this application has six parallel and evenly spaced vertical line segments, which are, from left to right, l v1 l v2 l v3 l v4 l v5 and l v6 There are 3 horizontal line segments, namely l h1 l h2 and l h3 If there are multiple horizontal line segments in the linear calibration plate, these horizontal line segments are parallel to each other, but it is not necessary to limit the spacing between adjacent horizontal line segments to be consistent. It is only necessary to ensure that the spacing between adjacent horizontal line segments is higher than the vertical offset of adjacent image sensors.

[0035] like Figure 5 As shown, the linear calibration plate provided in this application has a minimum of two vertical line segments and a minimum of one horizontal line segment. Before executing the calibration method of this application, the distance between the two vertical line segments needs to be obtained in advance as a known interval. Simultaneously, during the calibration process, it is necessary to ensure that the connecting lines of adjacent image sensors are between two parallel vertical line segments. Therefore, based on the difference between the interval between the two vertical line segments in the calibration image and the known interval, the lateral offset of the adjacent image sensors can be calculated.

[0036] For linear calibration boards, this application employs regularly arranged vertical and horizontal line segments. The width and spacing of each segment are rationally designed based on the pixel resolution of the image sensor and the splicing gap to ensure that each line is clearly identifiable after imaging. This application can adjust the line width and spacing of the line segments in the linear calibration board based on the splicing accuracy requirements of adjacent image sensors, so that the lateral offset of adjacent image sensors is lower than the spacing of adjacent vertical line segments, and the maximum longitudinal offset of adjacent image sensors is lower than the spacing of adjacent horizontal line segments.

[0037] In some embodiments, calibration objects are provided at the four corners of the linear calibration plate provided in this application, thereby locating the image area of ​​the linear calibration plate in the calibration image. The calibration objects in this application can be black rectangles, black triangles, black circles, etc. Figure 4 As shown, the calibration objects at the four corners of the linear calibration board are four black rectangles. For the calibration image, the calibration objects at the four corners of the calibration board can be quickly located using an edge recognition algorithm. Then, the image region composed of the four calibration objects is taken as the region of interest within the calibration image, which is the calibration board image. Based on the aforementioned processing, during the calibration process, the region containing only the linear calibration board can be quickly located, avoiding the influence of other regions on the offset calculation results and effectively ensuring the accuracy of the calibration results.

[0038] Before scanning the linear calibration plate, the linear calibration plate is driven to move horizontally by a horizontal displacement stage. The horizontal position of the linear calibration plate is controlled so that the middle position of the calibration plate is aligned with the vicinity of the seam of the adjacent linear array sensor. This ensures that any image sensor can scan a sufficient number of vertical line segments, thereby improving the accuracy of feature point extraction and distance analysis results in subsequent steps.

[0039] The scanning process of the linear calibration plate includes: controlling the adjacent image sensors to move at a constant speed in the vertical direction via a vertical displacement stage, while simultaneously activating the image sensors to perform scanning and imaging, acquiring image data containing the complete features of the linear calibration plate. During the scanning process, the moving speed of the vertical displacement stage is controlled to match the scanning frequency of the camera to avoid image blurring or pixel stretching / compression distortion, ensuring that every straight line in the acquired image is clear and has complete edges.

[0040] In practical applications, for multi-sensor stitching systems with more than two image sensors, a linear calibration plate is used to cover the field of view of each pair of adjacent image sensors. The image sensors are then controlled to scan the linear calibration plate synchronously, acquiring images sequentially. For example, the first scan covers image sensors numbered 1 and 2, the second scan covers image sensors numbered 2 and 3, and so on.

[0041] In some embodiments, the extraction of vertical and horizontal line segments from a calibration image can be performed in the following manner: Method 1: First, perform erosion followed by dilation on the calibration image to eliminate minor noise and smooth object boundaries. Then, use a Gaussian filter to improve the signal-to-noise ratio. Next, use the Canny operator to perform edge detection on the obtained image, extracting the contours corresponding to each line segment. Finally, use the Hough transform method to extract line segments from the contours, obtaining the line segments in the calibration image.

[0042] Method 2: Based on the pixel value difference between vertical and horizontal line segments and the image background, a line center extraction algorithm is used to extract the center point of each line segment's width. By connecting the center points of segments that are aligned with the same line, each line segment is extracted, and its linear equation is obtained. The least squares line fitting method can be used to fit a straight line to the center points of each line segment, thus obtaining the linear equation of each line segment. The coefficients of each linear equation can be saved to the processor. For example, a stripe center extraction method based on the Hessian matrix can be used to extract the sub-pixel coordinates of the center points of each line segment's width in the calibrated image. The obtained center points are then connected and fitted to form a center line, which is used as the extracted current line segment.

[0043] In addition, line center extraction algorithms can be categorized into gray-scale centroid method, threshold method, extreme value method, binary morphological thinning method, Steger algorithm, and curve fitting method, etc., without specific limitations here.

[0044] Considering that the number of pixels corresponding to the same line width varies at different image resolutions, this may affect the accuracy of line width center point extraction. For example, when the line width is large and the corresponding number of pixels is high, errors in line center point extraction are more likely to occur, reducing the accuracy of line segment extraction. Conversely, when the line width is small and the corresponding number of pixels is low, it is more susceptible to image noise, which also reduces the accuracy of line segment extraction.

[0045] In some embodiments, the linear calibration plate provided in this application can be divided into multiple regions, each region having multiple parallel vertical line segments and at least one horizontal line segment; wherein the line widths of the vertical and horizontal line segments are different in different regions to accommodate the calibration process of adjacent image sensors at different image resolutions. Figure 6 As shown, the linear calibration plate is divided into three regions from top to bottom. Each region contains 15 parallel, equally spaced vertical line segments and 6 parallel horizontal line segments. The line widths of the line segments vary in different regions. In the actual calibration process, for the three regions in the calibration image, the region with the best line segment extraction accuracy can be selected to calculate the offset of adjacent image sensors.

[0046] In some embodiments, for the linear calibration plate provided in this application, the vertical line segments and the horizontal line segments do not intersect. Therefore, based on the aforementioned processing, interference between the vertical and horizontal line segments during line segment extraction can be effectively avoided, thereby improving the accuracy of the calibration results.

[0047] The calibration scheme provided in this application can also be applied to straight-line calibration plates where vertical and horizontal line segments intersect, such as... Figure 7 As shown. For the aforementioned type of linear calibration plate, since there are intersection points between the vertical and horizontal line segments, errors are easily made in extracting the center point of the line width at the intersection points, affecting the line segment extraction results. Therefore, in practical work, for linear calibration plates where the vertical and horizontal line segments intersect, the intersection points between the vertical and horizontal line segments can be directly ignored during the line width center point extraction process.

[0048] In this application, the vertical direction corresponds to the direction of the vertical line segment in the linear calibration plate, and the horizontal direction is perpendicular to the vertical direction and parallel to the horizontal line segment in the linear calibration plate. The vertical direction corresponds to the pixel column direction of the image generated by the image sensor, and the horizontal direction corresponds to the pixel row direction in the image. In the solution provided in this application, image data generated by adjacent image sensors are connected according to the horizontal position of the image sensors for pixels in the same row. That is, in this application, the calibration image is formed by sequentially connecting pixels in the same pixel row from adjacent image sensors.

[0049] For step S2, after obtaining the calibration images generated by adjacent image sensors, this application can divide the calibration image into connecting lines for adjacent image sensors by counting the same number of pixel columns horizontally in the calibration image based on the total number of pixel columns of the left image sensor. Specifically, in the calibration image, the image to the left of the connecting line corresponds to the scanned image of the left image sensor, and the image to the right of the connecting line corresponds to the scanned image of the right image sensor. In this application, the connecting line represents the boundary line between images generated by adjacent image sensors, corresponding to the straight line formed by the connection points of each row of pixels in the adjacent image sensors, and perpendicular to the pixel columns of the calibration image. The connecting line can also be called a seam; if the seam has a certain width, the connecting line corresponds to the center line of the seam along the longitudinal direction.

[0050] For step S3, in the generated calibration image, the coordinates of all intersection points of adjacent straight lines and connecting lines are calculated. Then, the average longitudinal coordinate difference (i.e., longitudinal distance) between the two intersection points of adjacent straight lines is calculated as the longitudinal offset of the adjacent image sensors. In this application, both the longitudinal offset and the lateral offset represent the number of pixel deviations along the longitudinal and lateral directions in the calibration image generated by the adjacent image sensors.

[0051] exist Figure 8 In the calibration image shown, adjacent lines include the left side l. h1 and the l on the righth1 Two line segments, the left side l h2 and the l on the right h2 Two line segments, and the left side l h3 and the l on the right h3 Two line segments. For the difference in the vertical coordinates, let l... h1 For example, the left and right sides l h1 The longitudinal distance between the intersection of the line segment and the connecting line. For Figure 8 The calibration image shown uses the average longitudinal coordinate difference between three adjacent straight lines as the longitudinal offset of adjacent image sensors. Similarly, if there is only one horizontal line segment in the linear calibration plate, the longitudinal coordinate difference of a single adjacent straight line is directly calculated, which is the longitudinal offset of adjacent image sensors.

[0052] The angle between adjacent lines can be calculated by taking the average slope of the adjacent lines; this angle corresponds to the angle between adjacent image sensors. For Figure 8 The calibration image shown uses the average angle between three adjacent straight lines as the angle between adjacent image sensors. h1 For example, the left and right sides l h1 By extending the line segment, the included angle between adjacent lines can be obtained. Alternatively, the included angle between adjacent lines can be calculated directly from their equations. If there is only one horizontal line segment in the linear calibration plate, the included angle between a single adjacent line can be calculated directly, which is the included angle between adjacent image sensors.

[0053] For step S3, during the calculation of the lateral offset, the interval between vertical line segments can be calculated based on the pre-marked intervals in the linear calibration plate and the magnification of the image sensor. Taking a CIS camera as an example, with a magnification of 1, the interval between adjacent vertical line segments in the linear calibration plate is the pre-marked interval.

[0054] In one implementation, the average horizontal spacing between adjacent vertical line segments in each scanned image can be calculated in the calibration image and used as the vertical line segment spacing.

[0055] by Figure 8 Taking the calibration image shown as an example, the vertical segments on both sides of the connecting line are lv3 and lv4, and the distance between the vertical segments on both sides of the connecting line is the interval between lv3 and lv4.

[0056] In some embodiments, if there are multiple vertical line segments on both sides of the connecting line, the average distance between adjacent vertical line segments on both sides is calculated as the interval between the vertical line segments. Wherein, there is no connecting line between the adjacent vertical line segments.

[0057] like Figure 8As shown, the average distances between lv1 and lv2, lv2 and lv3, lv3 and lv4, and lv5 and lv6 in the calibration image can be calculated as the vertical line segment intervals. Furthermore, the lateral offset of adjacent image sensors can be obtained by calculating the difference between the distance between lv3 and lv4 and the aforementioned average distance. It can be understood that if the difference is positive, it indicates that adjacent image sensors have overlapping areas in the lateral direction, and the absolute value of the difference corresponds to the length of the overlapping area. If the difference is negative, it indicates that adjacent image sensors have separated areas in the lateral direction, and the absolute value of the difference corresponds to the length of the separated area.

[0058] Since the image sensor scans the vertical linear calibration plate along the longitudinal direction, and the scanning scheme is consistent with the direction of the vertical line segment, even in the scanned image formed by the tilted image sensor, the vertical line segment still corresponds to the pixels in the same column. That is, the vertical line segment will not be tilted in the calibration image.

[0059] In some embodiments, after obtaining the final longitudinal offset, lateral offset, and included angle between adjacent image sensors, this application may further include the following: The deviation between adjacent image sensors in physical position is determined based on the longitudinal offset, lateral offset, and included angle. The physical position of the other image sensor, excluding the reference sensor, is adjusted to correct the deviation. The reference sensor refers to the image sensor that serves as the position reference for adjacent image sensors during the offset calibration process.

[0060] Considering the limited accuracy of physical position adjustment, in one implementation, after obtaining the longitudinal offset, lateral offset, and included angle of adjacent image sensors, this application performs geometric transformation correction on the scanned images corresponding to the image sensors to correct the positional relationship between adjacent image sensors, including the following: Step S4: Using the scanned image of the reference sensor as a reference, adjust the horizontal pixel position of the other scanned image according to the horizontal offset, and use an interpolation algorithm to perform interpolation supplementation; calculate the vertical offset of each pixel column according to the vertical offset, the included angle and the horizontal pixel coordinates of the other scanned image, adjust the vertical pixel position of the other scanned image according to the offset, and use an interpolation algorithm to perform interpolation supplementation.

[0061] For step S4, taking a CIS linear array sensor as an example, when adjacent sensors are connected in a straight line, the lateral offset of adjacent linear array sensors is corrected using a sub-pixel interpolation algorithm. Since a small tilt angle of the linear array sensor has little impact on the horizontal direction (i.e., lateral), accurate correction can be achieved using a uniform ratio of sub-pixel interpolation. By supplementing or adjusting the pixel positions in the horizontal direction through interpolation, the lateral error between adjacent image sensors is eliminated, ensuring pixel alignment in the horizontal direction. The sub-pixel interpolation can employ bilinear interpolation or cubic interpolation algorithms, balancing correction accuracy and computational efficiency.

[0062] When adjacent sensors are connected in a triangular pattern, considering that the two image sensors overlap in the horizontal direction, the integer part of the horizontal offset is used as the benchmark, and the scanning area on either side of the connecting line is truncated. The decimal part of the horizontal offset is used as the weighting coefficient, and the new pixel value is calculated with reference to the horizontal weighted average algorithm. The specific process can be referred to the sub-pixel alignment technique in Chinese patent CN118134751A.

[0063] Before interpolating and supplementing the vertical pixels of another scanned image, the calibration method provided in this application also includes the following: Calculate the subpixel interpolation ratio for different pixel columns based on the included angle and the lateral pixel coordinates of another scanned image.

[0064] To address the longitudinal deviation between adjacent image sensors, parallelism correction, planar pixel difference correction, and sub-pixel alignment techniques disclosed in Chinese Patent CN118134751A can be referenced. Specifically, using the left image sensor as a reference, this application can calculate the longitudinal error caused by the angle θ in the left scanned image as the product of the horizontal pixel coordinates and the angle θ, based on the included angle θ and the horizontal pixel coordinates of the corresponding scanned image of the right image sensor. Furthermore, by adding the aforementioned longitudinal error caused by the angle θ to the longitudinal offset obtained in step S1, the final longitudinal offset of different pixel columns in the right scanned image can be obtained.

[0065] Similarly, referring to the technical means provided in Chinese Patent CN118134751A, for the final vertical offset of different pixel columns in the right-side scanned image, for integer multiple pixel differences, the coordinate values ​​of pixels in the right-side scanned image can be corrected by using the delay integral relationship.

[0066] For sub-pixel differences, a weighted average algorithm is used to calculate the pixel value of each pixel to be aligned vertically. Based on the pixel values ​​of the actual pixels above and below it. and ,use Corrections were made. Among them... To correct the multiplier, It should be noted here that the vertical differences between sensors may be significant, requiring a large buffer to store a sufficient number of pixel rows to achieve the desired result.

[0067] In summary, considering the significant impact of the included angle on the longitudinal deviation of the linear array sensor tilt, a sub-pixel interpolation algorithm with different proportions is applied to each pixel during the longitudinal deviation correction process. Based on the tilt angle, lateral pixel coordinates, and longitudinal offset obtained from calibration, the vertical offset of each column of pixels is calculated, and a corresponding sub-pixel interpolation ratio is assigned to different columns of pixels. By adjusting the vertical pixel position through differentiated interpolation, the vertical imaging distortion caused by the tilt of the linear array sensor is corrected, ensuring the alignment of the longitudinal imaging boundaries of adjacent linear array sensors.

[0068] In the technical solution provided in this application, the vertical subpixel interpolation coefficients can also be dynamically adjusted according to the aspect ratio of the acquired image. By acquiring the width and height parameters of the image in real time, calculating the image aspect ratio, establishing a correlation model between the interpolation coefficients and the image aspect ratio, and automatically adjusting the interpolation ratio of each pixel, the correction effect can be adapted to images with different resolutions and aspect ratios, further improving the correction accuracy and avoiding correction deviations caused by changes in image size.

[0069] In addition, in this application, physical position correction (adjusting the installation position of the image sensor) can be performed first. If the physical correction accuracy is insufficient, image geometric transformation correction can then be performed.

[0070] In some embodiments, this application also provides a calibration system for adjacent image sensors, such as... Figure 9 As shown, the calibration system includes: The image acquisition module 901 is used to control adjacent image sensors to synchronously scan a linear calibration plate along the longitudinal direction to obtain a calibration image. The linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments.

[0071] The image segmentation module 902 is used to segment the scanned images and connecting lines of different image sensors in the calibration image based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor. The horizontal direction refers to the connection direction of pixels in the same row, and the connecting lines are the boundaries between adjacent scanned images.

[0072] The offset calculation module 903 is used to calculate the average longitudinal distance between the intersection points of adjacent straight lines and connecting lines, as the longitudinal offset of adjacent image sensors; calculate the average angle between adjacent straight lines, as the angle between adjacent image sensors; and calculate the difference between the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image and the pixel interval of adjacent vertical line segments in the calibration image, as the lateral offset of adjacent image sensors. The adjacent straight lines are two straight lines of the same horizontal line segment on a linear calibration plate in different scanned images; the interval between the vertical line segments is higher than the maximum lateral offset of adjacent image sensors caused by installation errors.

[0073] This application also provides an electronic device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004. Memory 1003 is used to store computer programs; The processor 1001, when executing the program stored in the memory 1003, implements any of the above calibration methods.

[0074] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0075] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0076] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0077] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0078] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements any of the calibration method steps described above.

[0079] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the calibration method steps in the above embodiments.

[0080] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A calibration method for adjacent image sensors, characterized in that, include: Adjacent image sensors are controlled to synchronously scan a linear calibration plate along the longitudinal direction to obtain a calibration image; wherein, the linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments; Based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor, the scan images and connecting lines of different image sensors in the calibration image are divided; where horizontal refers to the connection direction of pixels in the same row, and the connecting line is the boundary line between adjacent scan images; The average longitudinal distance between the intersections of adjacent straight lines and the connecting line is calculated as the longitudinal offset of adjacent image sensors; the average angle between adjacent straight lines is calculated as the angle between adjacent image sensors; the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image is calculated, and the difference between this pixel distance and the pixel interval of adjacent vertical line segments in the calibration image is calculated as the lateral offset of adjacent image sensors; wherein, the adjacent straight lines are two straight lines of the same horizontal line segment on a straight calibration plate in different scanned images; the interval between the vertical line segments is higher than the maximum lateral offset of adjacent image sensors caused by installation errors.

2. The calibration method according to claim 1, characterized in that, After dividing the scanned images of different image sensors and connecting lines in the calibration image based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor, if there are multiple vertical line segments on both sides of the connecting line, the average distance between adjacent vertical line segments on both sides is calculated as the vertical line segment interval; wherein, there is no connecting line between adjacent vertical line segments.

3. The calibration method according to claim 1, characterized in that, The linear calibration plate is divided into multiple regions, each region having multiple parallel vertical line segments and at least one horizontal line segment; the line widths of the vertical and horizontal line segments are different in different regions to adapt to the calibration process of adjacent image sensors at different image resolutions.

4. The calibration method according to claim 1, characterized in that, The linear calibration plate has calibration objects at its four corners, which are used to locate the area of ​​the linear calibration plate in the calibration image.

5. The calibration method according to claim 1, characterized in that, The vertical and horizontal line segments in the linear calibration plate do not intersect.

6. The calibration method according to claim 1, characterized in that, After obtaining the vertical offset, horizontal offset, and included angle of adjacent image sensors, the calibration method further includes: The deviation between adjacent image sensors in physical position is determined based on the longitudinal offset, lateral offset, and included angle. The physical position of the other image sensor, excluding the reference sensor, is adjusted to correct the deviation. The reference sensor refers to the image sensor that serves as the position reference for adjacent image sensors during the offset calibration process.

7. The calibration method according to claim 1, characterized in that, After obtaining the vertical offset, horizontal offset, and included angle of adjacent image sensors, the calibration method further includes: Using the scanned image of the reference sensor as a reference, the horizontal pixel position of another scanned image is adjusted according to the horizontal offset, and interpolation algorithm is used for interpolation supplementation; the vertical offset of each pixel column is calculated according to the vertical offset, the included angle and the horizontal pixel coordinates of the other scanned image, and the pixel position of the vertical pixels of the other scanned image is adjusted according to the offset, and interpolation algorithm is used for interpolation supplementation. Here, the reference sensor refers to the image sensor that serves as the position reference during the offset calibration process for adjacent image sensors.

8. A calibration system for adjacent image sensors, characterized in that, include: The image acquisition module is used to control adjacent image sensors to synchronously scan a linear calibration plate along the longitudinal direction to obtain a calibration image; wherein, the linear calibration plate contains multiple parallel vertical line segments and at least one horizontal line segment; the vertical line segments are spaced evenly; the vertical line segments are perpendicular to the horizontal line segment; the longitudinal direction is the extension direction of the vertical line segments; The image segmentation module is used to segment the scanned images and connecting lines of different image sensors in the calibration image based on the number of horizontal pixels in the calibration image and the total number of pixel columns in the image sensor; where horizontal refers to the connection direction of pixels in the same row, and connecting lines are the boundary lines between adjacent scanned images; The offset calculation module is used to calculate the average longitudinal distance between the intersection points of adjacent straight lines and connecting lines, as the longitudinal offset of adjacent image sensors; calculate the average angle between adjacent straight lines, as the angle between adjacent image sensors; calculate the difference between the pixel distance between the nearest vertical line segments on both sides of the connecting line in the calibration image and the pixel interval between adjacent vertical line segments in the calibration image, as the lateral offset of adjacent image sensors; wherein, the adjacent straight lines are two straight lines of the same horizontal line segment of the straight line calibration plate in different scan images; the interval between the vertical line segments is higher than the maximum lateral offset of adjacent image sensors caused by installation errors.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the calibration method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, implements the calibration method according to any one of claims 1-7.

Images