An adaptive line array camera image capturing control method, system and medium

Abstract

The application discloses a kind of self-adapting linear array camera sampling control method, system and medium.The sampling control method includes: extracting the speed data of continuous acquisition, predicting the speed data of next time;Based on the speed deviation between the speed data of predicted next time and the speed data of current time and the pulse width of current time, the compensation amount of the pulse width of next time is analyzed;The position deviation amount between the actual interval distance corresponding to adjacent two line images after adjusting pulse width and the theoretical interval distance corresponding to adjacent two line images is analyzed, and the pulse width correction amount is calculated.The application actively predicts the speed, effectively overcomes the sampling deviation caused by the adjustment lag due to speed change, so that the stable sub-pixel offset amount between adjacent two line images is maintained, and the offset amount of line image along pixel row direction collected by two linear array sensors is the same, which improves the imaging stability and measurement accuracy of linear array camera.

Description

Technical Field

[0001] This invention belongs to the field of machine vision, and particularly relates to an adaptive linear scan camera image acquisition control method, system and medium. Background Technology

[0002] Line scan cameras, as imaging devices that use line scan image sensors as photosensitive elements, are widely used in industrial inspection, printed material quality inspection, and surface defect identification. Unlike area scan cameras, line scan cameras acquire only one line of image data per exposure. Through the relative movement between the object under test and the camera, multiple lines of images are continuously acquired and stitched together to form a complete two-dimensional image. To ensure that the acquired images are not distorted in the scanning direction, it is necessary to ensure that the lateral accuracy (along the pixel row direction) is equal to the longitudinal accuracy (along the scanning movement direction).

[0003] In applications requiring high imaging resolution, super-resolution imaging technology using line-scan cameras is widely adopted. This technology precisely arranges two linear sensor chips side-by-side and uses a beam-splitting optical path to guide incident light onto the two sensors, resulting in a fixed sub-pixel offset in the pixel row direction of the images acquired by the two sensors. For example, two linear CCDs can be offset by half a pixel in the pixel row direction, and a high-resolution image can be synthesized by subsequent interleaving and reconstructing of the image data. The two sensors can be either CCD or CMOS sensors.

[0004] However, a significant technical challenge arises in practical applications: the line scan camera's line acquisition is controlled by an external trigger pulse. When the trigger pulse width is fixed, the camera's line acquisition frequency (line frequency) remains constant. However, due to the uneven speed of the conveyor belt and factors such as the weight, shape, and friction of different products, the speed of the object under test relative to the line scan camera along the scanning direction varies. When the object's speed relative to the line scan camera changes while the trigger pulse width (i.e., the line acquisition frequency) remains constant, the actual physical distance between adjacent lines of images changes with the speed. When the speed increases, the sampling interval increases, compressing the image; when the speed decreases, the sampling interval decreases, stretching the image.

[0005] Uneven spacing between adjacent image rows caused by speed variations affects the fusion of images acquired by two linear array sensors, leading to a decrease or even failure of super-resolution reconstruction and impacting image accuracy. Therefore, to address the imaging distortion problem caused by speed variations in linear array cameras, image acquisition control of the linear array camera is necessary. This would solve problems such as the inability to maintain stable sub-pixel offsets between rows of images acquired by two linear array sensors and the mismatch between the offsets of the two linear array sensors along the pixel row direction. There is an urgent need to provide an adaptive linear array camera image acquisition control method, system, and medium. Summary of the Invention

[0006] The purpose of this invention is to overcome the above-mentioned problems in the prior art and to provide an adaptive linear scan camera image acquisition control method, system and medium.

[0007] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution: An adaptive linear scan camera image acquisition control method includes: Continuously acquire velocity data of the object under test relative to the linear scan camera along the scanning direction; Extract continuously collected velocity data, construct a curve of velocity change over time, predict the velocity change trend, and predict the velocity data at the next moment. Based on the speed deviation between the predicted speed data of the next moment and the speed data of the current moment, and the pulse width of the current moment, the compensation amount of the pulse width of the next moment is analyzed. On the basis of the pulse width of the current moment, the compensation amount of the pulse width is superimposed to obtain the pulse width corresponding to the predicted speed data of the next moment. The positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images is analyzed. Based on the ratio between the positional deviation and the predicted velocity data of the next moment, the pulse width correction amount corresponding to the positional deviation is determined, and the pulse width of the predicted next moment is corrected to obtain the pulse width of the next moment. The theoretical interval distance is the interval distance between two adjacent rows of images when the speed of the object under test relative to the linear array camera along the scanning direction is uniform and the pulse width is fixed.

[0008] Furthermore, the acquisition frequency of velocity data of the object under test relative to the linear scan camera along the scanning movement direction is not lower than the line sampling frequency of the linear scan camera.

[0009] Furthermore, based on the ratio of the reciprocal of the error variance predicted by the historical velocity data in each sliding window at the current moment to the sum of the reciprocals of the error variance predicted by the historical velocity data in all sliding windows, the weight value of each sliding window at the current moment is obtained.

[0010] Furthermore, the method for predicting the velocity data at the next moment includes: based on the error variance of the historical velocity data within each sliding window and the weight value of each sliding window, weighting and fusing the velocity data predicted by the corresponding sliding window at the next moment to obtain the predicted velocity at the next moment.

[0011] Furthermore, before obtaining the weight values ​​of each sliding window at the current moment, the method further includes: extracting the error variance predicted by the historical speed data in a sliding window at the previous moment, and analyzing the square of the speed difference between the speed data predicted by the historical speed data in the same sliding window at the current moment and the actual speed data, and analyzing the error variance predicted by the historical speed data in each sliding window at the current moment.

[0012] Furthermore, based on the speed deviation between the predicted speed data for the next moment and the speed data for the current moment, and combined with the pulse width at the current moment, the compensation amount for the pulse width at the next moment is analyzed.

[0013] Furthermore, based on the pulse width at the current moment, the compensation amount for the pulse width at the next moment is increased to obtain the pulse width at the next moment. When the pulse width at the next moment exceeds the exposure time range allowed by the line scan camera, the minimum value of the distance between the pulse width at the next moment and the upper and lower limits of the exposure time range is selected as the actual pulse width at the next moment.

[0014] Furthermore, it is determined whether the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images is greater than the set allowable deviation threshold. If it is greater than the set allowable deviation threshold, the pulse width correction amount corresponding to the positional deviation amount is analyzed to correct the pulse width of the predicted next moment. Otherwise, the pulse width of the predicted next moment is not corrected.

[0015] Based on the same inventive concept, this invention discloses an adaptive linear scan camera image acquisition control system, comprising: The velocity acquisition module is used to continuously acquire velocity data of the object under test relative to the linear scan camera along the scanning movement direction. The analysis and prediction module is used to extract continuously collected speed data, construct a curve of speed changing over time, predict the speed change trend, and predict the speed data at the next moment. The pulse width compensation module analyzes the compensation amount for the pulse width at the next moment based on the speed deviation between the predicted speed data at the next moment and the speed data at the current moment, as well as the pulse width at the current moment. It increases the compensation amount for the pulse width based on the pulse width at the current moment to obtain the pulse width corresponding to the next moment. The adjustment and correction module is used to analyze the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images. Based on the ratio between the positional deviation and the predicted velocity data of the next moment, the pulse width correction amount corresponding to the positional deviation is determined, and the pulse width of the predicted next moment is corrected to obtain the pulse width of the next moment.

[0016] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the control method described in any of the preceding claims.

[0017] The beneficial effects of this invention are: The adaptive line scan camera image acquisition control method provided by this invention constructs a complete closed-loop control system of "velocity prediction - pulse compensation - feedback correction" through the coordination of steps 1 to 4. This fundamentally solves the technical problem of image distortion caused by fluctuations in the moving speed of the object under test relative to the line scan camera. By introducing velocity trend prediction, control is shifted from passive response to active adjustment, effectively overcoming sampling deviations caused by adjustment lag due to velocity changes. By employing pulse width compensation and pulse width correction analysis, a stable sub-pixel offset is maintained between adjacent rows of images acquired by the same line scan sensor, and this offset is the same as the offset of the two line scan sensors along the pixel row direction under the same trigger. This comprehensively improves the imaging stability and measurement accuracy of the line scan camera in complex industrial scenarios, providing reliable data support for high-resolution image reconstruction.

[0018] This invention achieves high-precision prediction of speed change trends by weighted fusion of predicted movement speeds from different sliding windows, providing reliable data support for subsequent pulse width compensation. It can analyze the pulse width compensation amount for the next moment based on the predicted movement speed at the current speedometer pulse width, enabling accurate prediction of speed data even when speed fluctuates drastically or at high frequencies. This lays the foundation for proactive pulse width adjustment and fundamentally solves the adjustment lag problem caused by "adjusting pulse width based on the detected speed."

[0019] This invention accurately calculates the compensation amount for the pulse width at the next moment by combining the velocity deviation between the predicted velocity data at the next moment and the velocity data at the current moment with the pulse width at the current moment. This achieves a matching relationship between the adjusted pulse width and the predicted velocity data at the next moment, facilitating accurate prediction and adjustment of the pulse width at the next moment. This reduces the instability between sub-pixel offsets between adjacent rows of images. Furthermore, by combining the positional deviation between the actual interval distance and the theoretical interval distance between the two rows of images, the predicted pulse width at the next moment is corrected. By employing a combination of pulse width compensation and correction techniques, the actual interval distance between the two rows of images is ensured to remain stable within the offset of the two linear array sensors along the pixel row direction, achieving closed-loop correction. This allows the linear array camera to operate stably even when the moving speed of the camera relative to the object under test varies, providing a reliable guarantee for the stitching quality and fusion accuracy of the row images acquired by the linear array camera.

[0020] This invention constructs four LR images by controlling the interval between two adjacent rows of images along the scanning movement direction to be equal to the offset of the two linear array sensors along the pixel row direction. By constructing the mapping relationship between each pixel position in the high-resolution grid and each LR image, and by weighted fusion of the equivalent gray value estimates of each LR image at the same position in the high-resolution grid, a high-resolution image is generated. Based on the accuracy and consistency of the offset relationship between the four LR images, the super-resolution reconstruction can still obtain an artifact-free HR image in scenes with drastic speed fluctuations. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of the calibration method in this invention; Figure 2 This is a schematic diagram showing the distribution of the interval distance between adjacent rows of images before and after compensation and correction in this invention; Figure 3 This is a block diagram of the calibration system structure in this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The linear array super-resolution camera employs two linear array sensor chips, precisely positioned side-by-side. Optical beam splitting ensures a fixed offset along the pixel row direction in the images acquired by the two sensor chips. The two sensor chips are either CCD or CMOS sensors. Scanning occurs along the scanning direction. Due to the varying velocity of the object relative to the camera along this direction, the distance between adjacent rows of images differs even with a fixed trigger pulse width.

[0024] When the object under test moves at a constant or gradually changing speed, the current speed of the object can be obtained in real time by a speed sensor. Based on this, the trigger pulse width of the linear scan camera can be calculated and adjusted so that the pulse width is dynamically adjusted with the speed, which can meet the basic imaging requirements. However, in actual industrial scenarios, the speed of the object under test often fluctuates in various ways: the impact of the start and stop of the conveyor belt, the periodic speed pulsation of the mechanical transmission system, and the instantaneous acceleration and deceleration caused by the uneven mass distribution of the object itself. When the speed changes drastically or frequently, the lag in adjusting the pulse width based on the speed change will cause the following problems: (1) At the moment of speed change, the adjustment of the trigger pulse width has not been completed, and the actual spatial interval of the sampling deviates from the theoretical value, causing the position deviation of the image in that row; (2) For a pixel offset imaging system using a dual linear array sensor, the sampling interval deviation at the moment of speed change will destroy the precise matching of the fixed sub-pixel offset of the two linear array sensors along the pixel row direction, resulting in artifacts or resolution reduction in the sub-pixel reconstructed image; (3) In the scenario of high-frequency speed fluctuation, the lag adjustment method may cause different degrees of deviation in each row of sampling, and the cumulative effect seriously affects the stitching quality and measurement accuracy of the whole image.

[0025] Existing technologies, such as using shaft encoder output pulses as trigger signals for linear scan cameras, can achieve a certain degree of responsiveness in terms of acquisition frequency and speed. However, shaft encoders are also a type of real-time response control; their output pulse frequency reflects the current speed, not the speed at a future moment. Therefore, they still cannot solve the problem of instantaneous lag in speed changes.

[0026] like Figure 1 As shown, this embodiment discloses an adaptive linear scan camera image acquisition control method, including: Step 1: Continuously acquire velocity data of the object under test relative to the linear scan camera along the scanning direction; The speed detection module continuously acquires the real-time speed of the object under test or the linear scan camera along the scanning direction. The speed detection module includes, but is not limited to, one or more of the following: speed sensor, laser velocimeter, and speed detection device based on machine vision.

[0027] The acquisition frequency of velocity data of the object under test relative to the line scan camera along the scanning direction is not lower than the line sampling frequency of the line scan camera, so as to ensure that the temporal resolution of the acquired velocity data is sufficient to support subsequent trend prediction.

[0028] Since line scan cameras scan line by line, each exposure requires an independent trigger pulse width. When the sampling frequency of the acquired velocity data is not less than the line sampling frequency of the line scan camera, the velocity detection module obtains velocity data at least once during the current line scan. When the sampling frequency of the acquired velocity data is less than the line sampling frequency of the line scan camera, there is at least one interval scan line between the scan lines corresponding to two adjacent velocity samples, resulting in some scan lines having no detected velocity data, thus making it impossible to acquire the velocity data corresponding to each scan line.

[0029] By continuously acquiring real-time velocity data of the object under test relative to the line scan camera along the scanning direction, the continuous velocity data is fitted in order to predict the velocity data at the next moment. If the sampling frequency of the acquired velocity data is less than the line sampling frequency of the line scan camera, the line scan camera has acquired multiple lines of images during the two velocity sampling processes, resulting in severe distortion of the fitted velocity curve and making it impossible to obtain the high-frequency jitter between the two velocity samplings.

[0030] Step 2: Extract continuously collected velocity data, construct a velocity-time curve, predict the velocity change trend, and obtain the velocity data for the next moment. Extract the previous N (N≥3) consecutively collected historical velocity data at the current moment, construct a velocity-time curve based on the historical velocity data, and obtain the velocity data for the next moment based on the velocity-time curve to achieve velocity prediction.

[0031] In a short period of time, the velocity of the object under test relative to the linear scan camera along the scanning direction changes linearly with time. Using the N consecutive historical velocity data collected at the current moment, a linear model of velocity versus time is constructed. The least squares method is used to solve the linear model of velocity versus time. Based on the solved linear model, the velocity data at the next moment is predicted.

[0032] In the above embodiments, the velocity data of the next moment is predicted based on the previous N consecutively collected historical velocity data of the current moment, so as to improve the accuracy of the velocity data prediction for the next moment.

[0033] This embodiment also discloses a velocity prediction method. The method involves extracting the N consecutively collected historical velocity data points from the current moment, calculating the acceleration at the current moment based on the collected historical velocity data, and determining whether the absolute value of the acceleration is greater than a set acceleration threshold. If it is greater than the set acceleration threshold, it indicates that the velocity is in a state of significant change; if it is not greater than the set acceleration threshold, it indicates that the velocity is in a stable state. The set acceleration threshold is used as an indicator to measure the trend of velocity change and is set according to actual needs.

[0034] Select sliding windows of different lengths before the current time, calculate the error variance of historical velocity data collected within each sliding window of different lengths, and determine the weights corresponding to k sliding windows of different lengths based on the relationship between the error variance and a set first threshold and a second threshold. The sum of the products of the weight of each sliding window and the velocity predicted by that sliding window at the next time moment is used to obtain the comprehensive predicted velocity for the next time moment. Preferably, the number of sliding windows is equal to 3, and the lengths of the k sliding windows are m1, m2, and m3, respectively, where 1 < m1 < m2 < m3.

[0035] The first threshold is less than the second threshold. These two thresholds are used to divide the error variance. Based on the relationship between the error variance and the first and second thresholds, the weights corresponding to sliding windows of different lengths are determined. When the error variance within the k-th sliding window is greater than the second threshold, the weight c corresponding to that sliding window is... k Let c be the first weight value. When the error variance within the k-th sliding window is greater than the first threshold and not greater than the second threshold, then the weight c corresponding to that sliding window is... k The second weight value is determined when the error variance within the k-th sliding window is not greater than the first threshold. k The third weight value is the first weight value, which is greater than the second weight value, which is greater than the third weight value, and which is less than the first weight value.

[0036] Furthermore, based on the relationship between the error variance within each sliding window and the error variance across all sliding windows, the weight c corresponding to each sliding window is determined. k .

[0037] Specifically, the weight value of each sliding window at the current moment is obtained by taking the ratio of the reciprocal of the error variance predicted by the historical velocity data within each sliding window at the current moment to the sum of the reciprocals of the error variances predicted by all sliding windows.

[0038] The weights of each sliding window are as follows: E k (t) represents the error variance predicted by the k-th sliding window at the current time t, where k = 1, 2, 3. E k (t) = βE k (t-1)+(1-β)(v k (t)-v k (t)') 2 E k (t-1) represents the error variance predicted by the k-th sliding window at time t-1, where t ≥ m1+1, v k(t)' represents the velocity data predicted by the k-th sliding window based on the historical velocity data within the sliding window before the current time t. β represents the coefficient, which takes the value of 0.95 or 0.85. The current time is set as time t, and N consecutive historical velocity data before the current time t are selected. Since the length of the selected consecutive historical velocity data is different, the sliding window of different length is determined.

[0039] Where, when m1+1≤t<m2+1, E k (t-1) = (1-β)(v k (t-1)-v k (t-1)') 2 At this time, only the error variance predicted by the first sliding window at time t-1 can be obtained; when m2+1≤t<m3+1, only the error variance predicted by the first and second sliding windows at time t-1 can be obtained; when t≥m3+1, the error variance predicted by each sliding window at time t-1 can be obtained.

[0040] This embodiment does not limit the above method of calculating the weight of the sliding window. The weight of each sliding window can be obtained by averaging the number of sliding windows.

[0041] Before obtaining the weight values ​​of each sliding window at the current moment, the method further includes: extracting the error variance predicted by the historical speed data in a sliding window at the previous moment, and the square of the speed difference between the speed data predicted by the historical speed data in the same sliding window at the current moment and the actual speed data, and analyzing the error variance predicted by the historical speed data in each sliding window at the current moment.

[0042] Based on the error variance of historical velocity data within each sliding window and the weight values ​​of each sliding window, the predicted velocity v for the next moment is obtained by weighted fusion of the weight values ​​of each sliding window and the velocity data predicted by the corresponding sliding window. c k This indicates the weight of each sliding window. This represents the velocity predicted at the next moment using the k-th sliding window. Based on the historical velocity data within the k-th sliding window, a linear model of velocity versus time is used to predict the velocity data for the next moment.

[0043] The value of β is set based on experience. When the absolute value of acceleration is greater than the set acceleration threshold, the variance corresponding to the prediction error responds quickly, and the variance corresponding to the prediction error at the previous moment accounts for a small proportion. β is set to 0.85. When the absolute value of acceleration is not greater than the set acceleration threshold, β is set to 0.95. At this time, the variance corresponding to the prediction error is in a stable state.

[0044] In this embodiment, the speed at the current moment and the speed change rate at the current moment can also be used to predict the speed at the next moment. The speed change rate at the current moment is determined by the ratio of the difference between the speed at the current moment and the speed at the previous moment to the interval between two adjacent moments. Based on the speed change rate at the current moment and the speed data at the current moment, the speed data at the next moment is predicted.

[0045] Using the above method, based on historical speed data and speed changes, the speed data for the next moment can be predicted, thus predicting the speed change trend and providing reliable data support for adjusting the pulse width based on the predicted speed in the later stage.

[0046] Step 3: Based on the speed deviation between the predicted speed data of the next moment and the speed data of the current moment, and the pulse width of the current moment, analyze the compensation amount of the pulse width of the next moment. On the basis of the pulse width of the current moment, add the compensation amount of the pulse width to obtain the pulse width corresponding to the predicted speed data of the next moment.

[0047] When the predicted velocity data for the next moment is greater than the velocity data for the current moment, it indicates that the distance the object under test moves relative to the line scan camera will increase in the next moment. To ensure that the distance between adjacent samples along the scanning direction remains constant, the trigger pulse width needs to be shortened, thus reducing the time interval between adjacent samples to compensate for the increased interval between the two sampling positions caused by the increased velocity. Conversely, when the predicted velocity data for the next moment is not greater than the velocity data for the current moment, it indicates that the distance the object under test moves relative to the line scan camera will decrease in the next moment. To ensure that the distance between adjacent samples along the scanning direction remains constant, the trigger pulse width needs to be increased, thus increasing the time interval between adjacent samples to compensate for the decreased interval between the two sampling positions caused by the decreased velocity.

[0048] Based on the predicted velocity data for the next moment, as well as the velocity data and pulse width at the current moment, the compensation amount for the pulse width at the next moment is analyzed, and the compensation amount formula used is as follows: Where v represents the predicted velocity data for the next moment, and T cur The pulse width at the current moment, v curThis represents the speed data at the current moment.

[0049] Preferably, the pulse width at the current moment is increased by a compensation amount for the pulse width at the next moment to obtain the pulse width at the next moment. When the pulse width at the next moment exceeds the exposure time range allowed by the line scan camera, the minimum distance between the pulse width at the next moment and the upper and lower limits of the exposure time range is selected as the actual pulse width at the next moment.

[0050] Predicted pulse width T at the next moment comp Calculation formula: T comp =△T+T cur , among which, T min <T comp <T max T min and T max These represent the lower and upper limits of the exposure time allowed by a line scan camera, respectively.

[0051] If the predicted pulse width T of the next moment comp The exposure time T is less than the lower limit allowed by the line scan camera. min Then, the difference between the current pulse width and the lower limit of the exposure time allowed by the linear scan camera is used as the compensation amount for the pulse width at the next moment, and the compensation amount for the predicted pulse width at the next moment is updated accordingly; if the predicted pulse width T at the next moment... comp The exposure time T is greater than the upper limit allowed by the line scan camera. max The difference between the upper limit of the exposure time allowed by the line scan camera and the pulse width at the current moment is used as the compensation amount for the pulse width at the next moment, so as to update the compensation amount for the predicted pulse width at the next moment.

[0052] Based on the current velocity, the current pulse width, and the predicted velocity for the next moment, a compensation amount for the pulse width at the next moment is predicted. The predicted pulse width for the next moment is obtained by summing the compensation amounts for the current and next pulse widths. The predicted pulse width for the next moment is compared with the allowable exposure time range of the line scan camera to limit and update the compensation amount, resolving the conflict between the predicted pulse width and the physical limits of the line scan camera's hardware, and avoiding the loss of line data acquired by the line scan camera.

[0053] Step 4: Analyze the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images. Based on the ratio of the positional deviation to the predicted velocity data of the next moment, determine the pulse width correction amount corresponding to the positional deviation amount, correct the predicted pulse width of the next moment, and obtain the pulse width of the next moment to ensure that the actual interval distance between two adjacent row images acquired tends to the theoretical interval distance.

[0054] The theoretical interval between two adjacent rows of images is the interval between two adjacent rows of images when the speed of the object under test relative to the linear array camera along the scanning direction is uniform and the pulse width is fixed.

[0055] like Figure 2 As shown, without compensation and correction, the actual spacing between rows of images is inconsistent; after compensation and correction, the actual spacing between rows of images approaches the theoretical spacing, which is a fixed distance.

[0056] Based on the predicted pulse width, the line scan camera is adjusted to acquire the adjusted row image at the current moment. The positional deviation between the two row images is calculated using the grayscale values ​​of the current row image and the previous row image. If the positional deviation is less than a set allowable deviation threshold, no correction is needed for the predicted pulse width at the next moment. If it exceeds the threshold, the difference between the positional deviation and the threshold is calculated to obtain the correction value for the current row image's position. Based on this correction value, the correction value for the predicted pulse width at the next moment is analyzed, and the predicted pulse width at the next moment is corrected using this correction value. By correcting the pulse width, the actual interval between the two adjusted row images approaches the theoretical interval, ensuring a fixed sampling interval between adjacent rows that does not change with the scanning speed of the object relative to the line scan camera.

[0057] In this embodiment, a set allowable deviation threshold is used to reduce the frequency of pulse width correction. The system determines whether the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images exceeds the set allowable deviation threshold. If it exceeds the threshold, the pulse width correction amount corresponding to the positional deviation is analyzed, and the predicted pulse width for the next moment is corrected; if it does not exceed the threshold, the predicted pulse width for the next moment is not corrected.

[0058] The set allowable deviation threshold is obtained based on experience and is used to measure the degree of deviation between the actual interval distance and the theoretical interval distance.

[0059] The correction value for the pulse width at the predicted next moment is the position deviation divided by the velocity data at the predicted next moment.

[0060] Based on the above, the pulse width of the predicted next moment is corrected to correct the actual interval distance between the two line images, so that the actual interval distance between the two line images approaches the theoretical interval distance, and the sampling interval in the scanning movement direction is fixed.

[0061] The above steps allow the actual distance between two adjacent rows of images to approximate the theoretical distance of a fixed value. When performing super-resolution synthesis with a linear array camera, the camera includes two linear array sensors. Both sensors are triggered simultaneously to acquire images, ensuring that the continuous rows of images acquired by the two sensors are processed to obtain a high-resolution image.

[0062] Based on steps 1 to 4 above, this invention also discloses an adaptive line scan camera image processing method. The image processing method includes steps 1 to 4 above, and adds the calculation of the offset of the two line scan sensors constituting the line scan camera along the pixel row direction, and based on the theoretical interval distance between adjacent row images acquired by the same line scan sensor and the offset of the two line scan sensors along the pixel row direction, the row images continuously acquired by the two line scan sensors are stitched together to obtain the LR image corresponding to each line scan sensor. The adjacent frame LR images corresponding to the two line scan sensors are extracted and fused to obtain a high-resolution image.

[0063] The offset of adjacent LR images corresponding to the same linear array sensor along the scanning movement direction is the theoretical interval distance. Under the same trigger, the offset of the LR images generated by the two linear array sensors along the pixel row direction is a fixed value, which is affected by the installation position and installation process of the two linear array sensors.

[0064] To improve image resolution, it is necessary to ensure that the distance between two adjacent rows of images acquired by the same linear array sensor along the scanning movement direction is equal to the offset of the two linear array sensors after the linear array camera is installed along the pixel row direction.

[0065] Based on this, the present invention provides an image processing method for an adaptive line scan camera, which, in addition to steps 1 to 4 above, further includes: Step 5: Analyze the correlation coefficient between the row images acquired by the two linear array sensors at the same time, filter the offset corresponding to the largest correlation coefficient, extract the neighborhood integer offset of the offset corresponding to the largest correlation coefficient, and use an interpolation model to calculate the offset of the two linear array sensors along the pixel row direction.

[0066] The calculation of the offset of the two linear array sensors along the pixel row direction requires that, within a set offset range, the correlation coefficient between the row images acquired by the two linear array sensors corresponding to each offset is calculated, and the correlation coefficient between the row images acquired by the two linear array sensors at each offset is compared. The offset of the two linear array sensors along the pixel row direction is then obtained through calculation.

[0067] The offset d x The range is [-3, -2, -1, 0, 1, 2, 3]. The offset of the two linear array sensors along the pixel row direction is within this offset range. By setting this offset range, the actual offset of the two linear array sensors along the pixel row direction is determined.

[0068] At the same time, the correlation coefficient between the row images acquired by the two linear array sensors at each offset within the offset range is calculated. The offset corresponding to the largest correlation coefficient is selected. The neighboring integer offsets corresponding to the offset corresponding to the largest correlation coefficient are selected from the offset range. The interpolation model is constructed using the offset corresponding to the largest correlation coefficient and the correlation coefficients corresponding to the neighboring integer offsets to obtain the actual positional deviation of the two linear array sensors in the pixel row direction.

[0069] Within this offset range, the correlation coefficient between the row images acquired by the two linear array sensors at the same time is analyzed at each offset. The calculation formula used is as follows: N represents the pixel value of the preset selection window in the acquired row image, f1(x) represents the gray value corresponding to the x-th pixel in the row image acquired by a linear array sensor, and f2(x+d) represents the gray value of the x-th pixel in the row image acquired by a linear array sensor. x () represents the x+dth row in the image acquired by another linear array sensor. x The grayscale value corresponding to each pixel. and These represent the average grayscale values ​​of pixels within a preset selection window in the row images acquired by the two linear array sensors.

[0070] By selecting the maximum correlation coefficient W(d) between the row images acquired by the two linear array sensors at each offset within this offset range, x ) max The corresponding offset d S And extract the offset d S Neighborhood integer offset d S-1 and d S +1 corresponds to the correlation coefficient.

[0071] Interpolation model D x : , where W(d) s )=W(d x ) max .

[0072] By using the offset corresponding to the maximum value of the selected correlation coefficient and the correlation coefficient of the neighboring integer offsets of that horizontal offset, the offset of the two linear array sensors along the pixel row direction (x direction) is calculated using an interpolation model.

[0073] Based on the offset of the row images acquired by the two linear array sensors along the pixel row direction, multiple consecutive row images acquired by the same linear array sensor are processed to obtain the LR image of each linear array sensor.

[0074] Step 6: Based on the offset Dx along the pixel row direction of the LR images acquired by the two linear array sensors and the interval distance Dy between two adjacent row images along the scanning movement direction, extract the neighborhood integer pixels of the corresponding pixel positions in the adjacent frame LR images of the two linear array sensors respectively.

[0075] To improve the accuracy of super-resolution images from a linear array camera, the offset Dx of the LR images acquired by the two linear array sensors along the pixel row direction must be equal to the distance Dy between two adjacent row images along the scanning movement direction.

[0076] After pulse adjustment, the actual interval between two adjacent rows of images along the scanning movement direction approaches the theoretical interval distance Dy. In this embodiment, the theoretical interval distance is used as the interval between two adjacent rows of images along the scanning movement direction.

[0077] Assume that images 1 and 3 are adjacent LR images acquired by the same sensor chip, and images 2 and 4 are adjacent LR images acquired by another sensor chip. Images 1 and 2 are images acquired simultaneously, and images 3 and 4 are images acquired simultaneously. The positional deviations of the four LR images relative to image 1 are (dx1,dy1), (dx2,dy2), (dx3,dy3), and (dx4,dy4), respectively, which correspond to (0,0), (Dx,0), (0,Dy), and (Dx,Dx).

[0078] Select one of the LR images (optionally the first LR image). The pixel position HR(x,y) of each pixel position in the high-resolution grid is (x / 2−dxf,y / 2−dyf) in the LR image. Round down the pixel position (x / 2−dxf,y / 2−dyf) to obtain the center integer pixel position (m,n), where f=1,2,3,4.

[0079] Extract the positions of the four neighboring integer pixels of the pixel position (x / 2−dxf, y / 2−dyf), which are the pixel positions (m,n), (m+1,n), (m,n+1) and (m+1,n+1).

[0080] Based on the distance between the positions of four neighboring integer pixels and the pixel positions HR(x, y) in the high-resolution grid mapped to the pixel positions (x / 2−dxf, y / 2−dyf) in the LR image, the weight coefficients of the four neighboring integer pixels are analyzed as Wf1, Wf2, Wf3 and Wf4, and Wf1+Wf2+Wf3+Wf4=1.

[0081] The formula for calculating the weight coefficients of the four neighboring integer pixels is as follows:

[0082] in, This represents the pixel distance (less than one pixel) in the x-direction from the left neighboring integer pixels in the LR image to the pixel location mapped to it. This represents the pixel distance (less than one pixel) in the y-direction from the integer neighboring pixels above the pixel location mapped to the LR image.

[0083] Similarly, following the above method, the positions of the four neighboring integer pixels of the pixel position HR(x,y) in the high-resolution grid and the corresponding weight coefficients Wfi,i=1,2,3,4,f=1,2,3,4 are extracted from the pixel position in other LR images.

[0084] The actual gray values ​​of four neighboring integer pixels in the first LR image, the second LR image, the third LR image, and the fourth LR image are extracted respectively. Combined with the weight values ​​corresponding to the four neighboring integer pixels, the equivalent gray value of the neighboring integer pixels of the LR image mapped to the pixel position HR(x,y) of the HR image is calculated respectively.

[0085] The actual gray values ​​of four neighboring integer pixels are multiplied by their corresponding weight values ​​and then added together to obtain the equivalent gray value of the neighboring integer pixels of an LR image mapped to the pixel position HR(x,y) of the HR image. The equivalent gray values ​​of the four LR images mapped to the pixel position HR(x,y) of the HR image are then weighted and fused to obtain the gray value of the pixel position HR(x,y) of the HR image, thus obtaining the HR image.

[0086] Taking the calculation of the equivalent grayscale estimate of the pixel position (x / 2−dxf,y / 2−dyf) at the pixel position HR(x,y) in the HR image as an example, HR1(x,y) = W11×f(m,n) + W12×S(m+1,n) + W13×S(m,n+1) + W14×S(m+1,n+1), where f(m,n) represents the grayscale value. Similarly, the equivalent grayscale estimates of the pixel position (x / 2−dxf,y / 2−dyf) at the pixel position HR(x,y) in the second, third, and fourth LR images can be calculated, which are HR2(x,y), HR3(x,y), and HR4(x,y), respectively.

[0087] Step 7: Based on the gray values ​​of neighboring integer pixels, analyze the equivalent gray value estimates mapped to the pixel positions on the HR image, and fuse the equivalent gray value estimates of adjacent LR images to obtain the gray value of each pixel position on the HR image, so as to generate the HR image.

[0088] The equivalent grayscale estimates at pixel location HR(x,y) of the four LR images mapped to the HR image are weighted and fused to obtain the grayscale value of the pixel location coordinate HR(x,y) of the HR image. The fusion calculation formula is as follows: β1, β2, β3 and β4 represent the overall weight coefficients of each LR image, and β1+β2+β3+β4=1, with each value being 0.25.

[0089] By analyzing the offset Dx along the pixel row direction of the LR images acquired by two linear array sensors and the interval distance Dy between two adjacent row images along the scanning movement direction, the position of each location in the high-resolution image can be determined in each LR image. Then, the images are fused to obtain the grayscale value of each pixel position in the HR image.

[0090] Based on the same inventive concept, such as Figure 3 As shown, the present invention also discloses an adaptive linear scan camera image acquisition and control system, the system comprising: The velocity acquisition module is used to continuously acquire velocity data of the object under test relative to the linear scan camera along the scanning movement direction. The analysis and prediction module is used to extract continuously collected speed data, construct a curve of speed changing over time, predict the speed change trend, and predict the speed data at the next moment. The pulse width compensation module analyzes the compensation amount for the pulse width at the next moment based on the speed deviation between the predicted speed data at the next moment and the speed data at the current moment, as well as the pulse width at the current moment. It increases the compensation amount for the pulse width based on the pulse width at the current moment to obtain the pulse width corresponding to the next moment. The adjustment and correction module is used to analyze the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images. Based on the ratio between the positional deviation and the predicted velocity data of the next moment, the pulse width correction amount corresponding to the positional deviation is determined, and the pulse width of the predicted next moment is corrected to obtain the pulse width of the next moment.

[0091] For detailed operating methods and principles of each module in the image acquisition and control system, please refer to the image acquisition and control method described above, which will not be repeated here.

[0092] The present invention also provides a computer-readable storage medium including a computer program, which, when executed by a processor, implements the above-described image acquisition control method.

[0093] In practical applications, a computer-readable storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0094] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0095] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, or any suitable combination thereof.

[0096] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via an Internet connection using an Internet service provider).

[0097] In the description of this specification, references to terms such as "an embodiment," "example," and "specific example" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0098] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. An adaptive linear scan camera image acquisition control method, characterized in that, include: Continuously acquire velocity data of the object under test relative to the linear scan camera along the scanning direction; Extract continuously collected velocity data, construct a curve of velocity change over time, predict the velocity change trend, and predict the velocity data at the next moment. Based on the speed deviation between the predicted speed data of the next moment and the speed data of the current moment, and the pulse width of the current moment, the compensation amount of the pulse width of the next moment is analyzed. On the basis of the pulse width of the current moment, the compensation amount of the pulse width is superimposed to obtain the pulse width corresponding to the predicted speed data of the next moment. The positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images is analyzed. Based on the ratio between the positional deviation and the predicted velocity data of the next moment, the pulse width correction amount corresponding to the positional deviation is determined, and the pulse width of the predicted next moment is corrected to obtain the pulse width of the next moment. The theoretical interval distance is the interval distance between two adjacent rows of images when the speed of the object under test relative to the linear array camera along the scanning direction is uniform and the pulse width is fixed.

2. The adaptive linear scan camera image acquisition control method according to claim 1, characterized in that, The acquisition frequency of velocity data of the object under test relative to the line scan camera along the scanning direction is not lower than the line sampling frequency of the line scan camera.

3. The adaptive linear scan camera image acquisition control method according to claim 1, characterized in that, The weight value of each sliding window at the current moment is obtained by taking the ratio of the reciprocal of the error variance predicted by the historical velocity data in each sliding window at the current moment to the sum of the reciprocals of the error variance predicted by the historical velocity data in all sliding windows.

4. The adaptive linear scan camera image acquisition control method according to claim 3, characterized in that, The method for predicting velocity data at the next moment includes: based on the error variance of historical velocity data within each sliding window and the weight value of each sliding window, weighting and fusing the velocity data predicted by the corresponding sliding window at the next moment to obtain the predicted velocity at the next moment.

5. The adaptive linear scan camera image acquisition control method according to claim 3, characterized in that, Before obtaining the weight values ​​of each sliding window at the current moment, the method further includes: extracting the error variance predicted by the historical speed data in a sliding window at the previous moment, and the square of the speed difference between the speed data predicted by the historical speed data in the same sliding window at the current moment and the actual speed data, and analyzing the error variance predicted by the historical speed data in each sliding window at the current moment.

6. The adaptive linear scan camera image acquisition control method according to claim 1, characterized in that, Based on the speed deviation between the predicted speed data for the next moment and the speed data for the current moment, and combined with the pulse width at the current moment, the compensation amount for the pulse width at the next moment is analyzed.

7. The adaptive linear scan camera image acquisition control method according to claim 6, characterized in that, Based on the pulse width at the current moment, the compensation amount for the pulse width at the next moment is increased to obtain the pulse width at the next moment. When the pulse width at the next moment exceeds the exposure time range allowed by the line scan camera, the minimum value of the distance between the pulse width at the next moment and the upper and lower limits of the exposure time range is selected as the actual pulse width at the next moment.

8. The adaptive linear scan camera image acquisition control method according to claim 1, characterized in that, The system determines whether the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images is greater than the set allowable deviation threshold. If it is greater than the set allowable deviation threshold, the system analyzes the pulse width correction amount corresponding to the positional deviation amount to correct the pulse width of the predicted next moment. Otherwise, the system does not correct the pulse width of the predicted next moment.

9. An adaptive linear scan camera image acquisition control system, applied to the adaptive linear scan camera image acquisition control method according to any one of claims 1-8, characterized in that, include: The velocity acquisition module is used to continuously acquire velocity data of the object under test relative to the linear scan camera along the scanning movement direction. The analysis and prediction module is used to extract continuously collected speed data, construct a curve of speed changing over time, predict the speed change trend, and predict the speed data at the next moment. The pulse width compensation module analyzes the compensation amount for the pulse width at the next moment based on the speed deviation between the predicted speed data at the next moment and the speed data at the current moment, as well as the pulse width at the current moment. It increases the compensation amount for the pulse width based on the pulse width at the current moment to obtain the pulse width corresponding to the next moment. The adjustment and correction module is used to analyze the positional deviation between the actual interval distance between two adjacent row images acquired after pulse width adjustment and the theoretical interval distance between two adjacent row images. Based on the ratio between the positional deviation and the predicted velocity data of the next moment, the pulse width correction amount corresponding to the positional deviation is determined, and the pulse width of the predicted next moment is corrected to obtain the pulse width of the next moment.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the control method as described in any one of claims 1-8.

Images