A line array camera image reconstruction method, system and medium

Abstract

The application discloses a kind of line array camera image reconstruction method, system and medium.The method comprises: triggering two line array sensors simultaneously;Two line array sensors are selected in the offset of pixel arrangement direction, and the offset contains not less than two integer pixel offsets, the correlation coefficient between row images is calculated, and the actual position offset in the pixel arrangement direction is fitted based on integer pixel offset and the corresponding correlation coefficient;The position offset of each row image in the scanning motion direction under different exposure times is analyzed;Intermediate resolution image is constructed, and based on the mapping relationship between intermediate resolution image and high-resolution grid, high-resolution image is reconstructed along the scanning motion direction equidistant distribution.The application obtains super-resolution image by reconstructing the row image collected by line array camera, reduces the cost and implementation difficulty of super-resolution image, effectively solves the speed fluctuation problem of line array camera along the scanning motion direction.

Description

Technical Field

[0001] This invention belongs to the field of machine vision, and particularly relates to a method, system and medium for image reconstruction using a line scan camera. Background Technology

[0002] Linear array sensors are widely used in industrial inspection, printing monitoring, and continuous material surface defect detection due to their high scanning frequency and high resolution. To improve the spatial resolution of linear array sensors, the industry often uses subpixel offset technology. This involves hardware mounting to create a 0.5-pixel misalignment between two linear array sensors in the x-direction (pixel arrangement direction). This allows for the acquisition of two images with subpixel phase differences in a single scan, which are then merged using an image fusion algorithm, effectively doubling the resolution in the x-direction.

[0003] However, existing image fusion technologies for offset linear array sensors all rely on the assumption that the sensor moves at a constant speed in the y-direction (scanning motion direction). In practical industrial applications, due to mechanical vibrations, acceleration / deceleration, or dynamic changes in the speed of the object being measured, the sensor often exhibits variable-speed scanning in the y-direction. When two sensors simultaneously trigger image acquisition, using traditional fixed-grid-based fusion algorithms will result in severe stretching, compression, or misalignment distortion of the image in the y-direction. This not only fails to achieve high-resolution reconstruction but may even lead to distorted and unusable fused images.

[0004] Therefore, in order to improve image quality, the present invention provides a method, system and medium for image reconstruction using a line scan camera. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned problems in the prior art and to provide a method, system and medium for image reconstruction of a line scan camera.

[0006] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution: A method for image reconstruction using a linear scan camera, wherein the linear scan camera consists of two linear sensors, each having only one row of pixels and offset by sub-pixels along the scanning motion direction, including: Simultaneously trigger two linear array sensors to acquire images, obtain the scanning speed along the scanning motion direction in real time, and analyze the position of the acquired image in the scanning motion direction; Select the offset of two linear array sensors in the pixel arrangement direction, so that it contains no less than two integer pixel offsets. Analyze the correlation coefficient between the row images acquired by the two linear array sensors under the integer pixel offset. According to the magnitude of the correlation coefficient, select no less than three smallest integer pixel offsets that are close to the actual offset. Based on the integer pixel offset and the corresponding correlation coefficient, fit the actual position offset of the two linear array sensors in the pixel arrangement direction. Based on the scanning position of the linear array sensor under different exposure times, the positional offset of each row of images acquired by each linear array sensor under different exposure times in the scanning motion direction is analyzed. Based on the actual position offset of the two linear array sensors in the pixel arrangement direction, an intermediate resolution image with enhanced resolution in the pixel arrangement direction is constructed. The position offset of each row of images acquired by each linear array sensor at different exposure times in the scanning motion direction is analyzed to determine the sub-pixel position offset of two adjacent rows of images in the scanning motion direction. The mapping relationship between the intermediate resolution image and the high-resolution grid is used to reconstruct a high-resolution image that is evenly distributed along the scanning motion direction.

[0007] Furthermore, based on the initial scanning position of the line scan camera and the scanning speed of the line scan camera along the scanning motion direction at each exposure, the position of the line image acquired at each exposure in the scanning motion direction is analyzed and obtained.

[0008] Furthermore, the actual offset of the two linear array sensors in the pixel arrangement direction is analyzed, and the method includes: Two integer pixel offsets are set, and the correlation coefficient between row images acquired by two linear array sensors under the same integer pixel offset and the same exposure is analyzed. Obtain the correlation coefficients corresponding to two integer pixel offsets, namely the first correlation coefficient and the second correlation coefficient. Determine the magnitude of the first correlation coefficient and the second correlation coefficient. If the first correlation coefficient is greater than the second correlation coefficient, the actual offset of the two linear sensors in the pixel arrangement direction is close to the integer pixel offset corresponding to the first correlation coefficient. Conversely, if the first correlation coefficient is less than the second correlation coefficient, the actual offset of the two linear sensors in the pixel arrangement direction is close to the integer pixel offset corresponding to the second correlation coefficient. Based on the relationship between the first and second correlation coefficients, the smallest integer pixels that are close to the actual offset are selected, and several of these smallest integer pixel offsets are extracted to form several integer pixel offsets. Using these integer pixel offsets and their corresponding correlation coefficients, an offset parabola is fitted, and the actual offset is obtained by solving the equation. The minimum integer pixel offset used for filtering is not equal to the two set integer pixel offsets.

[0009] Furthermore, the scanning positions of the same linear array sensor under different exposure times are extracted, and the positional offset of the same linear array sensor along the scanning motion direction under different exposure times is analyzed based on the scanning positions of the linear array sensor under different exposure times.

[0010] Furthermore, by using the scanning positions of different linear array sensors under different exposure times, and combining the positional offsets in the y-direction of the line images acquired by the two linear array sensors under the same exposure, the combined positional offsets of the two linear array sensors constituting the linear array camera under different exposure times are obtained.

[0011] Furthermore, based on the position offset of the row image under one exposure, the sub-pixel position offset between the row images under the exposure is analyzed. The sub-pixel position offset is linearly weighted and fused with the gray values ​​of two adjacent integer pixel positions in the same column of the high-resolution grid to establish a mapping relationship between the measured gray values ​​of each pixel position collected under the exposure in the intermediate resolution grid and the gray values ​​of two adjacent integer pixel positions in the same column of the high-resolution grid.

[0012] Furthermore, the positions of two adjacent pixels in the row images acquired by the two linear array sensors under the same exposure are extracted. Based on the actual position offset of the two linear array sensors in the x-direction, linear interpolation is used to analyze the gray values ​​corresponding to the odd-numbered pixel positions in the intermediate resolution grid. The number of pixels in the intermediate resolution grid in the x-direction is twice the number of pixels in the linear array sensor in the x-direction.

[0013] Furthermore, a method for brightness correction of a line image based on the scanning speed at different exposure times along the scanning motion direction includes: A calibration object is selected, which is a white board with a uniform and diffuse reflective surface. Under the same lighting conditions, scan the whiteboard at different scanning speeds to obtain at least one line of image; The average gray value of the line image is calculated. The ratio between the average gray value of the line image at different scanning speeds and the average gray value at a preset standard speed is used to obtain the variable speed brightness change rate, thereby obtaining the brightness change with scanning speed at different scanning speeds. The corrected brightness data for different scanning speeds is obtained by multiplying the line image at different scanning speeds with the variable brightness change rate at that scanning speed.

[0014] This invention provides an image reconstruction system for a line scan camera. The line scan camera consists of two line scan sensors, each with only one row of pixels and offset by sub-pixels along the scanning motion direction. The scanning position acquisition module is used to simultaneously trigger two linear array sensors to acquire images, obtain the scanning speed along the scanning motion direction in real time, and analyze the position of the acquired line image in the scanning motion direction. The lateral offset analysis module selects two linear array sensors whose offset in the pixel arrangement direction includes no less than two integer pixel offsets, analyzes the correlation coefficient between the row images acquired by the two linear array sensors under the integer pixel offset, and selects no less than three smallest integer pixel offsets close to the actual offset and their corresponding correlation coefficients according to the magnitude of the correlation coefficient, and fits the actual position offset of the two linear array sensors in the pixel arrangement direction. The interline offset analysis module analyzes the position offset of each line image acquired by each linear array sensor under different exposure times in the scanning motion direction, based on the scanning position of the linear array sensor under different exposure times. The image reconstruction module constructs an intermediate resolution image with enhanced pixel arrangement direction resolution based on the actual position offset of the two linear array sensors in the pixel arrangement direction. It also analyzes the sub-pixel position offset of adjacent rows of images in the scanning motion direction by using the position offset of each row of images acquired by each linear array sensor at different exposure times. Finally, it reconstructs a high-resolution image with equal spacing along the scanning motion direction by using the mapping relationship between the intermediate resolution image and the high-resolution grid.

[0015] A computer-readable storage medium includes a computer program that, when executed by a processor, implements the reconstruction method as described in any of the preceding claims.

[0016] The beneficial effects of this invention are: The image reconstruction method for a line scan camera provided by this invention designs sub-pixel offsets in the pixel arrangement direction of two line scan sensors and combines them with variable-speed scanning in the scanning motion direction to obtain the position of the image at different exposure times, determine the sub-pixel offsets in the scanning motion direction, and reconstruct the image acquired by the line scan camera based on the sub-pixel offsets in the pixel arrangement direction and the scanning motion direction to obtain a super-resolution image. This reduces the cost and implementation difficulty of super-resolution images, forms a high-precision and highly robust super-resolution image, and solves the speed fluctuation problem of the line scan camera along the scanning motion direction.

[0017] This invention simultaneously triggers two linear array sensors to acquire images, ensuring that the y-position of the row images acquired by the two linear array sensors under the same exposure remains consistent. It also acquires the scanning speed under different exposures, and analyzes the position corresponding to the row images acquired by the linear array camera based on a fixed row frequency. This allows for accurate acquisition of the actual position coordinates corresponding to each row image, thus laying the foundation for subsequent calculation of the offset of the linear array sensors in the y-direction.

[0018] This invention sets the offset of two linear array sensors in the x-direction to an integer pixel offset of not less than two, and calculates the correlation coefficient between the row images acquired by the two linear array sensors at the two integer pixel offsets. By judging the magnitude of the correlation coefficient, the integer pixel displacement that the actual offset of the two linear array sensors is close to is determined. In order to fit the actual offset of the two linear array sensors in the x-direction based on an integer pixel displacement of not less than three and the corresponding correlation, the error caused by the installation process can be eliminated, and the sub-pixel offset in the x-direction can be accurately obtained. Compared with traditional calibration methods, this invention greatly reduces calibration costs and operational complexity. The sub-pixel calibration accuracy is achieved based on the fitted curve, providing accurate offset data for subsequent super-resolution reconstruction in the x-direction.

[0019] This invention reconstructs the sub-pixel offset in the x-direction to obtain an intermediate resolution image with increased resolution in the x-direction. By analyzing the positions of the same or different linear array sensors under different exposures, the position offset in the scanning motion direction is determined. To address the non-uniform sampling problem caused by changes in scanning speed, a mapping relationship between the intermediate resolution grid and the high resolution grid is established, enabling the mapping of measured gray values ​​from non-uniform sampling onto equally spaced high resolution grids. In this process, a coupled position offset is introduced to improve the accuracy of the position offset in the y-direction, thereby obtaining a high resolution image with equally spaced distribution in the scanning motion direction and improving the quality of the reconstructed image.

[0020] This invention calibrates the variable brightness change rate at different scanning speeds, enabling brightness correction of line images acquired at different scanning speeds. This eliminates the difference in grayscale values ​​between different line images acquired by the same linear array sensor due to different speeds, thus improving the stability of scanning quality. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of the fusion method in this invention; Figure 2 These are the scanning positions under different exposure coefficients in this invention; Figure 3 This is a schematic diagram of the pixel positions of the two linear array sensors in the pixel arrangement direction in this invention; Figure 4 This is a flowchart of the actual offset analysis method in the x-direction of this invention; Figure 5 This is a block diagram of the fusion system structure in this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The linear array super-resolution camera employs two linear array sensors arranged side-by-side along the pixel arrangement direction (X-direction). Optical path splitting ensures that the images acquired by the two sensors are offset by 0.5 pixels in the X-direction. Both linear array sensors are either CCD or CMOS sensors. The two linear array sensors scan along the scanning motion direction (Y-direction). Movement control in the Y-direction allows for the determination of the displacement deviation along the Y-direction between adjacent rows of images acquired by the same linear array sensor.

[0024] like Figure 1 As shown, this embodiment first provides an image reconstruction method for a line scan camera. The line scan camera consists of two line scan sensors with only one row of pixels and offset by sub-pixels along the scanning motion direction. The method includes the following steps: A line scan camera consists of two line scan sensors, namely sensor A and sensor B. Both sensor A and sensor B have only one row of photosensitive units, that is, only one row of pixels. The two line scan sensors are offset by 0.5 pixels in the pixel arrangement direction (X direction) and overlap in the scanning motion direction.

[0025] The line scan camera triggers two line scan sensors to expose simultaneously at a fixed line frequency f. Each time it is triggered, each of the two line scan sensors outputs an image line. The time interval Δt between two adjacent triggers is fixed. When the moving speed of the two line scan sensors along the scanning motion direction (Y direction) remains constant, the offset of the scanning position of the same line scan sensor along the scanning motion direction is fixed under two adjacent triggers.

[0026] Step 1: Simultaneously trigger two linear array sensors to acquire images, obtain the scanning speed along the scanning motion direction in real time, and analyze the position of the acquired line images in the scanning motion direction. Establish the time tk corresponding to the kth exposure of the camera, k=0,1,2,3,...h, tk=k*Δt, the time interval between two adjacent exposures is Δt=1 / f, and the scanning speed along the scanning motion direction is vk=v(tk) at each exposure time, forming the scanning speed set {v0,v1,v2,...,h}, where h represents the total number of exposures.

[0027] Since the two linear array sensors are triggered synchronously, the row data collected by the two linear array sensors at the same trigger moment have exactly the same physical position coordinates in the scanning motion direction (Y direction), with an offset of only 0.5 pixels in the x direction.

[0028] Assume the initial scanning position D of the linear scan camera Y0 Based on the initial scanning position of the line scan camera and the scanning speed of the line scan camera along the scanning motion direction under each exposure, the position of the line image acquired under each exposure number in the scanning motion direction is obtained by analysis.

[0029] like Figure 2 As shown, the y-positions corresponding to the camera scan under each exposure are: .

[0030] By using the above method, at the initial scanning position, combined with the scanning speed of the line scan camera along the scanning motion direction at each exposure number, the y-coordinates corresponding to different rows of images acquired by the camera can be analyzed, so as to obtain accurate position coordinates.

[0031] Step 2: Set the offset of the two linear array sensors in the pixel arrangement direction to no less than two integer pixel offsets. Analyze the correlation coefficient between the row images acquired by the two linear array sensors under the integer pixel offset. Based on the magnitude of the correlation coefficient, select no less than three smallest integer pixel offsets that are close to the actual offset. Based on the integer pixel offset and the corresponding correlation coefficient, fit the actual offset of the two linear array sensors in the pixel arrangement direction. Since the two linear array sensors are offset by 0.5 pixels in the x-direction and the two sensors have the same scanning speed, the offset of the two linear array sensors in the x-direction remains constant at 0.5 pixels under the same exposure.

[0032] In this embodiment, the offset of the two linear array sensors in the x-direction is fixed at 0.5 pixels. Due to limitations in chip manufacturing processes and costs, it is impossible to guarantee that the offset of the images acquired by the two linear array sensors in the x-direction is precisely controlled within 0.5 pixels. By calibrating the offset of the two linear array sensors in the x-direction, the offset Δx of the two linear array sensors in the x-direction can be determined. Figure 3 As shown.

[0033] The offset of the two linear array sensors in the x-direction may not be maintained at a fixed 0.5 pixels due to the influence of processes such as installation. Based on this, the row images acquired by the two linear array sensors under the same exposure are obtained.

[0034] Since the two linear array sensors are offset by sub-pixels in the x-direction, a set integer pixel offset d is selected. xFor cases where the pixel value is 0 or 1, analyze the correlation between the row images acquired by the two linear array sensors. Establish the correlation between the row images acquired by the two linear array sensors under a set offset. Alternatively, an integer pixel offset d can be set. x For the cases of -1, 0, and 1 pixel, the negative sign is used to indicate the direction of the offset.

[0035] The actual offset of the two linear array sensors in the pixel arrangement direction (x direction) is analyzed, such as... Figure 4 As shown, the method is as follows: Step 21: Set two integer pixel offsets d x， Analyze the correlation between row images acquired by two linear array sensors under the same integer pixel offset and the same exposure. Here, two integer pixel offsets d are set. x The two integer pixel offsets d selected in this embodiment are either 0 pixels and 1 pixel, or a combination of -1 pixel and 0 pixels. x They are 0 pixels and 1 pixel respectively.

[0036] Correlation model between row images acquired by two linear array sensors under the same exposure:

[0037] Among them, W(d) x This indicates that the two linear array sensors are offset by d in the x-direction. x The correlation coefficient, This represents the grayscale value of sensor A at the x-th pixel. This indicates that sensor B is at the x+d level. x grayscale value at each pixel and These represent the average grayscale values ​​of the row images acquired by sensor A and sensor B, respectively.

[0038] Step 22: Obtain the correlation coefficients corresponding to the two set integer pixel offsets, namely the first correlation coefficient and the second correlation coefficient. Determine the magnitude of the first correlation coefficient W(0) and the second correlation coefficient W(1). If W(0) > W(1), the actual offset of the two linear sensors in the pixel arrangement direction is close to the integer pixel offset corresponding to the first correlation coefficient W(0). Conversely, the actual offset of the two linear sensors in the pixel arrangement direction is close to the integer pixel offset corresponding to the second correlation coefficient W(1). Step 23: Based on the relationship between the first correlation coefficient W(0) and the second correlation coefficient W(1), select the smallest integer pixel that is close to the actual offset, extract several smallest integer pixel offsets that are close to the actual offset, form several integer pixel offsets, use several integer pixel offsets and their corresponding correlation coefficients to fit the offset parabola, and solve to obtain the actual offset, which is the sub-pixel position offset.

[0039] The minimum integer pixel offset used for filtering is not equal to the two set integer pixel offsets.

[0040] When W(0) > W(1), the extracted smallest integer pixel offsets close to the actual offset are -1, 0, and 1, respectively; when W(0) < W(1), the extracted smallest integer pixel offsets close to the actual offset are 0, 1, and 2, respectively. By selecting the three smallest integer pixel offsets close to the actual offset and their corresponding correlation coefficients, a parabola is fitted. By solving the parabola, the pixel offset corresponding to the largest correlation coefficient is obtained. The pixel offset corresponding to the largest correlation coefficient is the actual position offset D of the two linear array sensors in the x-direction. X .

[0041] Step 3: Based on the scanning position of the linear array sensor under different exposures, analyze the positional offset between the rows of images acquired by different linear array sensors under different exposure times; At different exposure times, the scanning speed vk of the camera along the scanning motion direction changes, and the amount of change in the scanned position of adjacent pixel rows is different at different exposure times.

[0042] Extracting the scanning position of the linear array sensor under different exposure times Based on the scanning position of the linear array sensor under different exposure times, we analyze the positional offset of the same linear array sensor under different exposure times and the combined positional offset of the two linear array sensors that make up the linear array camera under different exposure times.

[0043] Based on the scanning position of the same linear array sensor under different exposure times, the positional offset between two adjacent exposures can be obtained. ,in, , and These represent the scanning positions under the k-th and k+1-th exposures, respectively. i can be selected as A or B, which are used to represent the two linear array sensors.

[0044] Based on step 2 above, it can be determined that the two linear array sensors have a fixed offset D in the x-direction. XAt the same time, the fixed offset of the two linear array sensors in the x-direction will not cause the positional offset in the y-direction. However, if the two linear array sensors are affected by the direction of motion or the installation angle of the linear array sensors, there may be a small rotation angle between the two linear array sensors.

[0045] When there is a small rotation angle between the two linear array sensors, the two installed linear array sensors are selected to capture images of the uniform grid calibration plate. The y-coordinates of the same feature position in the calibration plate are extracted from the images captured by the two linear array sensors. Based on the y-coordinates of the same feature position in the two linear array sensors, the difference is calculated to obtain the coupling displacement offset ΔXY in the y direction.

[0046] In addition, the coupling position offset ΔXY in the y-direction can also be determined based on the relative tilt angle between the two linear array sensors, using the relative tilt angle and the offset D of the two linear array sensors in the x-direction. X The product of the two can be used to obtain the coupled position offset in the y direction caused by the position offset in the x direction.

[0047] The combined positional offset of any two linear array sensors in the y-direction caused by the positional offset of two linear array sensors in the x-direction is calculated by adding the deviation between the scanning positions of the two linear array sensors in the y-direction under different exposures.

[0048] In this embodiment, two single-line array sensors are selected, but multiple single-line sensors can also be used. By selecting one single-line array sensor as the standard linear array sensor, the positional offset of other linear array sensors relative to the standard linear array sensor in the x-direction is analyzed. Based on the coupling positional offset in the y-direction caused by the positional offset of other linear array sensors relative to the standard linear array sensor in the x-direction, this offset is added to the deviation between the scanning positions of the linear array sensor and the standard linear array sensor in the y-direction under different exposure times to obtain the comprehensive positional offset of the linear array sensor and the standard linear array sensor in the y-direction. Thus, by using the above method, the comprehensive positional offset of other linear array sensors and the standard linear array sensor in the y-direction can be obtained.

[0049] Combined positional offset of the two linear array sensors in the y-direction under different exposure cycles: , This represents the combined positional offset in the y-direction between the row images acquired during the k-th exposure of linear array sensor A and the f-th exposure of linear array sensor B. This represents the position of the row image acquired during the k-th exposure of the linear array sensor A in the y-direction. Let f represent the position of the row image acquired by the linear array sensor B in the y-direction during the f-th exposure, where f ≠ k.

[0050] For the sub-pixel position offsets of two linear array sensors A and B in the y direction under different exposure times: ,and , The value indicates rounding down. The comprehensive position offset ΔXY is used to reflect the position offset in the y-direction of the row images acquired by the two linear array sensors under the same exposure.

[0051] For the sub-pixel position offset in the Y direction of the same linear array sensor under different exposure times: .

[0052] Step 4: Based on the sub-pixel position offset of the two linear array sensors in the pixel arrangement direction, construct an intermediate resolution image with enhanced resolution in the pixel arrangement direction. Then, using the position offset of each row of images acquired by each linear array sensor at different exposure times in the scanning motion direction, analyze the sub-pixel position offset of two adjacent rows of images in the scanning motion direction, and reconstruct a high-resolution image that is evenly distributed along the scanning motion direction.

[0053] Since the two linear array sensors are offset by a sub-pixel distance in the x-direction (the offset between the two linear array sensors in the x-direction may be equal to 0.5 pixels, or it may be greater or less than 0.5 pixels), the sampling interval of the intermediate resolution grid in the x-direction is set to 0.5 pixels. The row image data of the same y position collected by the two linear array sensors are sequentially placed in the same row image of the high-resolution grid, thereby improving the resolution in the x-direction. That is, the resolution of the intermediate resolution grid in the x-direction is twice that of the row image collected by the linear array sensor.

[0054] The positions of two adjacent pixels in the row images acquired by two linear array sensors under the same exposure are extracted. Based on the actual position offset of the two linear array sensors in the x-direction, linear interpolation is used to analyze the gray values ​​corresponding to the odd-numbered pixel positions in the intermediate resolution grid. The number of pixels in the intermediate resolution grid in the x-direction is twice the number of pixels in the linear array sensor in the x-direction.

[0055] When the two linear array sensors are offset by 0.5 pixels in the x-direction, G k (2x) = Y A,k (x),G k (2x+1)=Y B,k (x), k represents the row image data of the k-th row, where x takes values ​​of 0, 1, 2, ..., N-1, and N represents the number of pixels acquired by linear array sensors A and B along the x-direction. G is used. k (2x) and Gk Alternating between (2x+1) can obtain the gray values ​​of each position in the same row of the image, that is, to construct the intermediate resolution image of the super-resolution in the x direction. The intermediate resolution grid is represented by G(s,k), where s represents the horizontal coordinate and the value range of s is [0,2N-1], and k represents the vertical coordinate.

[0056] When the two linear array sensors are offset by more than 0.5 pixels in the x-direction, the grayscale value corresponding to the even-numbered pixel position in the intermediate resolution grid is: G k (2x) = Y A,k (x), k represents the row image data of the kth row, x takes values ​​of 0, 1, 2, ..., N-1, and N represents the number of pixels acquired by linear array sensors A and B along the x direction.

[0057] For the gray values ​​corresponding to odd-numbered pixel positions in the intermediate resolution grid: If 0 < D x <0.5, G k (2x+1)=(0.5-D) x Y A,k (x+1) / (1-D x )+0.5×Y B,k (x) / (1-D x） ; If 0.5≤D x <1, G k (2x+1)=(D) x -0.5)Y A,k (x) / D x +0.5Y B,k (x) / D x .

[0058] Y A,k (x+1) represents the grayscale value corresponding to the (x+1)th pixel position in the row image acquired by linear array sensor A during the k-th exposure, Y B,k (x) represents the grayscale value corresponding to the x-th pixel position in the row image acquired by linear array sensor B during the k-th exposure, G k (2x+1) represents the gray value corresponding to the 2x+1th pixel position in the row image corresponding to the kth exposure in the intermediate resolution image.

[0059] The sampling interval of the high-resolution grid in the y-direction is affected by the change in scanning speed, resulting in different positional offsets between the row images under adjacent exposures. That is, the sampling points along the scanning motion direction are non-uniform sampling points.

[0060] Based on the position offset of the row image under one exposure, the sub-pixel position offset between the row images under the exposure is analyzed. The sub-pixel position offset is linearly weighted and fused with the gray values ​​of two adjacent integer pixel positions in the same column of the high-resolution grid to establish the mapping relationship between the measured gray values ​​of each pixel position collected under the exposure in the intermediate resolution grid and the gray values ​​of two adjacent integer pixel positions in the same column of the high-resolution grid.

[0061] For each row in a high-resolution grid, the grayscale value at each position in each row can be obtained by using the offset relationship between two linear array sensors in the x-direction.

[0062] The row images at different y positions acquired under different exposure times are processed to obtain the acquired values ​​at each position in the high-resolution grid.

[0063] Based on the high-resolution image constructed in the x-direction, super-resolution reconstruction is performed in the y-direction by combining the row images and their positions acquired by the two linear array sensors with the pixel data acquired at the same position on the linear array sensor along the scanning motion direction.

[0064] Establish a mapping model between intermediate resolution and high resolution meshes: G(s,k) = (1-αk)×H(s,mk) + αk×H(s,mk+1), mk represents the integer index of the k-th row sampling point in the high-resolution grid along the y-direction, p represents the pixel size in the low-resolution image, and αk represents the sub-pixel offset of the k-th row sampling point in the y-direction (scanning motion direction). The weights of two adjacent y pixels in a high-resolution image are determined by H(s,mk) and H(s,mk+1), which represent the gray values ​​at positions (s,mk) and (s,mk+1) in the high-resolution grid, respectively. Positions (s,mk) and (s,mk+1) are adjacent coordinate positions along the y direction. G(s,k) represents the gray value at position (s,k) in the intermediate resolution grid.

[0065] Based on the mapping relationship model between the intermediate resolution grid and the high resolution grid, a mapping relationship model for each linear array sensor on each x column is established. By solving the mapping relationship model, a high resolution image in each x column direction is obtained. Multiple high resolution images in x columns are used to form an initial high resolution image H of the whole frame with equal spacing in the y direction.

[0066] The initial high-resolution image of the entire frame is reconstructed to obtain a reconstructed high-resolution image, thereby improving the quality of the high-resolution image. The reconstruction method can be iterative back-projection or regularized least squares. When using the iterative back-projection method, by setting an initial high-resolution image, a mapping relationship model between the intermediate resolution grid and the high-resolution grid is used to predict the predicted row image data of each row of the initial high-resolution image generation linear array sensor under adjacent exposures. The gray value of each position in the predicted row image data of each row is compared with the gray value of the same position in the row image data collected by the corresponding linear array sensor to obtain the gray value difference of each position in the row image. Based on the gray value difference of each position in the row image, the above mapping relationship model is used to obtain the gray value compensation amount of the corresponding row image in the high-resolution grid. Finally, the gray value compensation amount of each row image is used to compensate the gray value of the corresponding row image in the initially set high-resolution image to complete one correction, thereby obtaining the corrected high-resolution image.

[0067] In this process, the correction iteration conditions for the high-resolution image can be determined by comparing the gray values ​​at each position in the predicted row image determined by the above mapping relationship model with the gray values ​​at the same position in the row image acquired by the corresponding linear array sensor. If the average gray value difference of the row image is less than a set gray value difference, the correction of the high-resolution image is stopped. Similarly, the number of correction iterations can be set. If the number of correction iterations reaches the set number of correction iterations, the correction of the high-resolution image is stopped. This embodiment does not limit other correction iteration conditions for the high-resolution image.

[0068] By setting correction iteration conditions, the computational load and the quality of the reconstructed image can be balanced, ensuring that the deviation between the actual low-resolution image and the low-resolution image corresponding to the final high-resolution image continuously decreases until convergence, significantly improving the quality of the final super-resolution reconstructed image. It also provides researchers with the adaptive capability to adjust parameters according to actual accuracy requirements, and has the characteristics of strong robustness and high practical value.

[0069] In this embodiment, for each column of y image in the x direction, a regularized least squares method can be selected based on the established mapping relationship model to generate a high-resolution image, thereby achieving high resolution for the entire frame image.

[0070] In this embodiment, because the camera's line frequency f is fixed and the scanning speed of each line is different, the actual spatial position corresponding to each line of image data is non-uniform in the y-direction. For scanning the same pixel line, as the scanning speed increases, the scanning time for each line decreases, resulting in grayscale differences in the same uniform pixel line target acquired at different speeds. To improve the fusion accuracy, it is necessary to correct for the response differences caused by changes in scanning speed.

[0071] The method for brightness correction of line images based on scanning speed at different exposure times along the scanning motion direction includes: A calibration object is selected, which is a white board with a uniform and diffuse reflective surface. Under the same lighting conditions, scan the whiteboard at different scanning speeds to obtain at least one line of image; The average gray value of the line image is calculated. The ratio between the average gray value of the line image at different scanning speeds and the average gray value at a preset standard speed is used to obtain the variable speed brightness change rate, thereby obtaining the brightness change with scanning speed at different scanning speeds. The corrected brightness data for different scanning speeds is obtained by multiplying the line image at different scanning speeds with the variable brightness change rate at that scanning speed, so as to eliminate the problem of differences in grayscale values ​​of the line image caused by changes in scanning speed.

[0072] In the above steps, the combined positional offset of the two linear array sensors in the y-direction under different exposure times is determined based on the scanning speed. Affected by mechanical vibration, speed measurement noise, or calibration error, there will be a small error between the combined positional offset of the two linear array sensors in the y-direction calculated in step 3 and the actual image offset. The deviation between the theoretical combined positional offset of the two linear array sensors in the y-direction and the actual positional offset of the image is less than the set deviation threshold β, which is less than 0.5 pixels.

[0073] In this embodiment, the combined positional offset between the row images acquired by the two linear array sensors under different exposures is used. The composite position offset is used to translate the row image acquired by one of the linear array sensors. The pixel values ​​corresponding to the translated row image and the row image acquired by the other linear array sensor are extracted. The pixel error value of the row image is obtained based on the sum of the squares of the pixel changes corresponding to each pixel. In this process, the maximum position deviation ΔD is selected based on the theoretical composite position offset. Y1 and minimum positional deviation △D Y2 Within the range, the minimum pixel error value is used to determine the minimum overall positional deviation △Ymin.

[0074] The maximum position deviation is equal to the sum of the theoretical comprehensive position offset and the set deviation threshold, and the minimum position deviation is equal to the theoretical comprehensive position offset minus the set deviation threshold.

[0075] To improve the accuracy of the deviation between the theoretical and actual composite position offsets, a quadratic curve is fitted based on the minimum composite position offset and the pixel error values ​​of its neighboring points. The first derivative of the quadratic curve is then taken to obtain the abscissa corresponding to the vertex of the curve. This abscissa represents the actual relative displacement between adjacent rows of images. The actual relative displacement is used instead of the theoretical position offset to reduce error interference. The neighboring points of the minimum composite position offset ΔYmin are ΔYmin-β and ΔYmin+β, respectively.

[0076] The actual relative displacement determined by the actual image is used to correct the combined position offset of the two linear array sensors in the y direction, so as to obtain the actual relative displacement between adjacent rows of images.

[0077] This step analyzes the theoretically calculated comprehensive positional offset to obtain the actual relative displacement, thereby obtaining the fine positional offset between each row of images, which improves the accuracy of the data and provides reliable data support for subsequent image fusion.

[0078] This application uses a fixed exposure time to obtain the y-position corresponding to the camera scan at each exposure count. To ensure that the exposure amount corresponding to different scanning speeds at different exposure counts along the scanning motion direction of the line scan camera, this embodiment can also adjust the line frequency f of the line scan camera to meet the exposure requirements at different speeds. That is, the line frequency f changes with the scanning speed vi, ti=p / vi, where p represents the physical width of one pixel, and vi represents the scanning speed corresponding to the i-th exposure. By adjusting the exposure time for each exposure according to the scanning speed at each exposure count, the exposure amount of the line scan camera is made consistent, ensuring the image quality of each exposure.

[0079] Based on the same inventive concept, such as Figure 5 As shown, the present invention also provides an image reconstruction system for a line scan camera. The line scan camera consists of two line scan sensors, each with only one row of pixels and offset by sub-pixels along the scanning motion direction. The system includes: The scanning position acquisition module is used to simultaneously trigger two linear array sensors to acquire images, obtain the scanning speed along the scanning motion direction in real time, and analyze the position of the acquired line image in the scanning motion direction. The lateral offset analysis module selects the offset of two linear array sensors in the pixel arrangement direction, making it include no less than two integer pixel offsets. It analyzes the correlation coefficient between the row images acquired by the two linear array sensors under the integer pixel offset. Based on the magnitude of the correlation coefficient, it selects no less than three smallest integer pixel offsets close to the actual offset and their corresponding correlation coefficients, and fits the actual position offset of the two linear array sensors in the pixel arrangement direction. The interline offset analysis module analyzes the position offset of each line image acquired by each linear array sensor under different exposure times in the scanning motion direction, based on the scanning position of the linear array sensor under different exposure times. The image reconstruction module constructs an intermediate resolution image with enhanced pixel arrangement direction resolution based on the actual position offset of the two linear array sensors in the pixel arrangement direction. It also analyzes the sub-pixel position offset of adjacent rows of images in the scanning motion direction by using the position offset of each row of images acquired by each linear array sensor at different exposure times. Finally, it reconstructs a high-resolution image with equal spacing along the scanning motion direction by using the mapping relationship between the intermediate resolution image and the high-resolution grid.

[0080] For detailed operation methods and principles of each module in the image reconstruction system, please refer to the image reconstruction method described above, and they will not be repeated here.

[0081] The present invention also provides a computer-readable storage medium including a computer program that, when executed by a processor, implements the above-described fusion method.

[0082] In practical applications, a computer-readable storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0083] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0084] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0085] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0086] In the description of this specification, references to terms such as "an embodiment," "example," and "specific example" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for image reconstruction using a linear scan camera, wherein the linear scan camera comprises two linear sensors, each having only one row of pixels and offset by sub-pixels along the scanning motion direction, characterized in that... include: Simultaneously trigger two linear array sensors to acquire images, obtain the scanning speed along the scanning motion direction in real time, and analyze the position of the acquired line image in the scanning motion direction; The offset of two linear array sensors in the pixel arrangement direction includes no less than two integer pixel offsets. The correlation coefficient between the row images acquired by the two linear array sensors under the integer pixel offset is analyzed. According to the magnitude of the correlation coefficient, no less than three smallest integer pixel offsets close to the actual offset are selected. Based on the integer pixel offset and the corresponding correlation coefficient, the actual position offset of the two linear array sensors in the pixel arrangement direction is fitted. Based on the scanning position of the linear array sensor under different exposure times, the positional offset of each row of images acquired by each linear array sensor under different exposure times in the scanning motion direction is analyzed. Based on the actual position offset of the two linear array sensors in the pixel arrangement direction, an intermediate resolution image with enhanced resolution in the pixel arrangement direction is constructed. The position offset of each row of images acquired by each linear array sensor at different exposure times in the scanning motion direction is analyzed to determine the sub-pixel position offset of two adjacent rows of images in the scanning motion direction. The mapping relationship between the intermediate resolution image and the high-resolution grid is used to reconstruct a high-resolution image that is evenly distributed along the scanning motion direction.

2. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, Based on the initial scanning position of the line scan camera and the scanning speed of the line scan camera along the scanning motion direction at each exposure, the position of the line image acquired at each exposure in the scanning motion direction is obtained by analysis.

3. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, The method used to analyze the actual offset of the two linear array sensors in the pixel arrangement direction is as follows: Two integer pixel offsets are set, and the correlation coefficient between row images acquired by two linear array sensors under the same integer pixel offset and the same exposure is analyzed. Obtain the correlation coefficients corresponding to two integer pixel offsets, namely the first correlation coefficient and the second correlation coefficient. Determine the magnitude of the first correlation coefficient and the second correlation coefficient. If the first correlation coefficient is greater than the second correlation coefficient, the actual offset of the two linear sensors in the pixel arrangement direction is close to the integer pixel offset corresponding to the first correlation coefficient. Conversely, if the first correlation coefficient is less than the second correlation coefficient, the actual offset of the two linear sensors in the pixel arrangement direction is close to the integer pixel offset corresponding to the second correlation coefficient. Based on the relationship between the first and second correlation coefficients, the smallest integer pixels that are close to the actual offset are selected, and several of these smallest integer pixel offsets are extracted to form several integer pixel offsets. Using these integer pixel offsets and their corresponding correlation coefficients, an offset parabola is fitted, and the actual offset is obtained by solving the equation. The minimum integer pixel offset used for filtering is not equal to the two set integer pixel offsets.

4. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, Extract the scanning position of the same linear array sensor under different exposure times, and analyze the positional offset of the same linear array sensor along the scanning motion direction under different exposure times based on the scanning position of the linear array sensor under different exposure times.

5. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, By using the scanning positions of different linear array sensors at different exposure times, and combining the positional offsets in the y-direction of the line images acquired by the two linear array sensors under the same exposure, the combined positional offsets of the two linear array sensors constituting the linear array camera at different exposure times are obtained.

6. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, Based on the position offset of the row image under one exposure, the sub-pixel position offset between the row images under the exposure is analyzed. The sub-pixel position offset is linearly weighted and fused with the gray values ​​of two adjacent integer pixel positions in the same column of the high-resolution grid to establish the mapping relationship between the measured gray values ​​of each pixel position collected under the exposure in the intermediate resolution grid and the gray values ​​of two adjacent integer pixel positions in the same column of the high-resolution grid.

7. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, The positions of two adjacent pixels in the row images acquired by two linear array sensors under the same exposure are extracted. Based on the actual position offset of the two linear array sensors in the x direction, linear interpolation is used to analyze the gray values ​​corresponding to the odd-numbered pixel positions in the intermediate resolution grid. The number of pixels in the intermediate resolution grid in the x direction is twice the number of pixels in the linear array sensor.

8. The image reconstruction method for a linear scan camera according to claim 1, characterized in that, Brightness correction of line images is performed based on scanning speeds at different exposure times along the scanning motion direction. The methods employed include: A calibration object is selected, which is a white board with a uniform and diffuse reflective surface. Under the same lighting conditions, scan the whiteboard at different scanning speeds to obtain at least one line of image; The average gray value of the line image is calculated. The ratio of the average gray value of the line image at different scanning speeds to the average gray value at a preset standard speed is used to obtain the variable speed brightness change rate, thereby obtaining the brightness change with scanning speed at different scanning speeds. The corrected brightness data for different scanning speeds is obtained by multiplying the line image at different scanning speeds with the variable brightness change rate at that scanning speed.

9. An image reconstruction system for a linear scan camera, wherein the linear scan camera comprises two linear scan sensors, each having only one row of pixels and offset by sub-pixels along the scanning motion direction, characterized in that, include: The scanning position acquisition module is used to simultaneously trigger two linear array sensors to acquire images, obtain the scanning speed along the scanning motion direction in real time, and analyze the position of the acquired line image in the scanning motion direction. The lateral offset analysis module selects the offset of two linear array sensors in the pixel arrangement direction, making it include no less than two integer pixel offsets. It analyzes the correlation coefficient between the row images acquired by the two linear array sensors under the integer pixel offset. Based on the magnitude of the correlation coefficient, it selects no less than three smallest integer pixel offsets close to the actual offset and their corresponding correlation coefficients, and fits the actual position offset of the two linear array sensors in the pixel arrangement direction. The interline offset analysis module analyzes the position offset of each line image acquired by each linear array sensor under different exposure times in the scanning motion direction, based on the scanning position of the linear array sensor under different exposure times. The image reconstruction module constructs an intermediate resolution image with enhanced pixel arrangement direction resolution based on the actual position offset of the two linear array sensors in the pixel arrangement direction. It also analyzes the sub-pixel position offset of adjacent rows of images in the scanning motion direction by using the position offset of each row of images acquired by each linear array sensor at different exposure times. Finally, it reconstructs a high-resolution image with equal spacing along the scanning motion direction by using the mapping relationship between the intermediate resolution image and the high-resolution grid.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the reconstruction method as described in any one of claims 1-8.

Images