Space-time step size recombination and image restoration method and system for time-sharing imaging of line-scan digital camera

Abstract

The time-space step size recombination and image restoration method for time-sharing imaging of the linear array camera comprises the following steps: S1, an image acquisition and parameter analysis step: acquiring a synthetic image data stream, and analyzing configuration parameters; s2, a step of establishing a differential line offset model: determining the physical position of each photosensitive channel relative to a reference channel according to the topology type of the sensor, and calculating the total offset S {target} of a target in combination with the time-sharing number K; meanwhile, according to the camera internal space correction state identifier F, estimating a preset offset S {internal} executed by the camera internal ISP; s3, a differential compensation amount calculation step: calculating a differential line offset compensation amount DeltaS required to be executed by the software end, wherein the calculation formula is DeltaS = S {target}-S {inter}; s4, an image reverse repairing step: establishing a data buffer area in a memory of the image, and performing longitudinal translation on lagged channel data in the synthesized image according to DeltaS; and S5, a time sequence demultiplexing step: carrying out row-level interlaced extraction of the mode K, and reconstructing K luminosity stereo images or multispectral images of independent light sources.

Description

Technical Field

[0001] This invention relates to the technical field of image restoration in machine vision, and in particular to a method and system for spatiotemporal step reconstruction and image restoration using time-division imaging with a line scan camera. Background Technology

[0002] Multi-source time-division imaging (photometric stereo) technology acquires the normal vector or multispectral information of an object's surface using a single linear array camera by rapidly switching light sources.

[0003] In existing technologies, processing time-division images from color line scan cameras faces significant challenges:

[0004] 1. Diversity of sensor structures: Cameras on the market come in various structures such as dual-line (RG-BG) and triple-line (Tri-linear), with different physical row spacing, resulting in different ways of image misalignment.

[0005] 2. Interference from internal camera processing: Most commercial color cameras have "Spatial Correction" preprocessing mode enabled by default. This function can align colors during normal shooting, but in time-series shooting mode, the camera will incorrectly force the merging of line data taken at different times (i.e., under different light sources), thus destroying the integrity of the time-series data.

[0006] 3. Limitations of existing solutions: Most current image splitting algorithms only work on original images with "correction off". If an engineer on site accidentally turns on the camera's correction function, the existing algorithms will fail, resulting in defects such as "purple-green stripes" in the image, which cannot be repaired by simple geometric transformations. Summary of the Invention

[0007] The purpose of this invention is to overcome the above-mentioned defects in the prior art and provide a spatiotemporal step reconstruction and image restoration method and system for time-division imaging of a line array camera. The system and method utilize a general differential line offset calculation model, which can adaptively and compatiblely support black and white, two-line, and three-line cameras, and can effectively restore images with internal spatial correction enabled, thereby achieving low-cost, highly versatile, and clear multispectral time-division imaging.

[0008] To achieve the above objectives, the present invention is accomplished through the following two aspects:

[0009] In a first aspect, the present invention provides a spatiotemporal step-size reconstruction and image restoration method for time-division imaging with a linear array camera, comprising the following steps:

[0010] S1: Image acquisition and parameter parsing steps: The image processing device acquires the synthetic image data stream transmitted by the line scan camera through the acquisition card and parses the configuration parameters of the imaging system. The configuration parameters include: time-division number K, sensor topology type, physical row spacing D between each channel of the sensor, and camera internal space correction status flag F; the camera internal space correction status flag F is used to indicate whether the camera internal ISP has enabled the row alignment preprocessing mode for static scenes.

[0011] S2: Differential line offset model establishment steps: The image processing device determines the physical position of each photosensitive channel relative to the reference channel according to the sensor topology type, and calculates the total target offset S_{target} required to completely eliminate temporal parallax theoretically, in conjunction with the time division quantity K; at the same time, it estimates the preset offset S_{internal} that has been executed by the camera's internal ISP according to the camera's internal spatial correction status flag F.

[0012] S3: Differential compensation calculation steps: The image processing device calculates the differential line offset compensation DeltaS that needs to be executed by the software. The calculation formula is: DeltaS=S_{target}-S_{internal}.

[0013] S4: Image reverse restoration step: The image processing device establishes a data buffer in its memory, and performs vertical translation on the hysteresis channel data in the synthesized image according to the calculated differential line offset compensation amount DeltaS; when DeltaS>0, an upward translation operation is performed, and when DeltaS<0, a downward translation operation is performed, thereby obtaining the restored image data in the spatiotemporal matching state.

[0014] S5: Temporal demultiplexing step: The image processing device performs row-level interlaced extraction on the repaired image data in the manner of module K, and reconstructs K photometric stereo images or multispectral images of independent light sources.

[0015] Preferably, in step S1,

[0016] Image processing equipment can be industrial PCs, personal computers, or embedded computing platforms;

[0017] An imaging system refers to a machine vision inspection system consisting of a line scan camera, a lens, multiple sets of rapidly switchable light sources, and an image acquisition card.

[0018] Preferably, in step S1,

[0019] The method for obtaining the synthetic image data stream is as follows:

[0020] It receives raw or RGB-Packed data transmitted from line scan cameras via CameraLink, CoaXPress, or GigE interfaces and stores it in the memory buffer of the image processing device.

[0021] Preferably, in step S1,

[0022] The sensor topology can be any of the following:

[0023] Single-line / black and white sensor: its physical line spacing D=0;

[0024] Dual-line color sensor: contains two photosensitive lines, with a physical line spacing D between the first and second photosensitive lines;

[0025] Three-line color sensor: contains three independent photosensitive lines, and there is usually a physical line spacing D between each pair of red (R), green (G), and blue (B) lines.

[0026] Preferably, in step S2,

[0027] The specific method for calculating the total target offset S_{target} and the preset offset S_{internal} is as follows:

[0028] S_{target}=D×K; where D is the number of physical rows of the current channel relative to the base channel, and K is the number of time-sharing segments;

[0029] S_{internal}=D×alpha; where alpha is the internal correction factor. When the camera's internal space correction status flag F is on, alpha=1; when the internal space correction status flag F is off, alpha=0.

[0030] Preferably, in step S3...

[0031] The specific calculation logic for the differential line offset compensation DeltaS further includes a Bayer phase correction term delta, as shown in the following formula:

[0032] For the i-th color channel: DeltaS_i = (D_i × K) - (D_i × alpha) + delta;

[0033] in:

[0034] D_i is the physical row number of the i-th color channel relative to the reference channel, where the color channel refers to the physically separated R, G, or B photosensitive rows on the sensor;

[0035] delta is a fine-tuning correction term used to compensate for phase deviations in Bayer interpolation of dual-line cameras or sub-pixel line deviations caused by mechanical mounting.

[0036] Preferably, the method supports reverse repair for the special case of "camera enabling internal space correction F", i.e., when alpha=1:

[0037] When it is detected that the linear scan camera has been misaligned based on the assumption of a static scene, and the misalignment is assumed to be a displacement of D rows, while time-division imaging actually requires a displacement of D×K rows, step S3 automatically calculates the positive additional compensation amount DeltaS=D×KD.

[0038] When the alignment inside the line scan camera is detected to cause data misalignment, step S4 implements logical rollback or appending of the data by moving the memory address pointer.

[0039] Preferably, in step S5...

[0040] The specific steps of time-series demultiplexing are:

[0041] Construct K target image containers, which are pre-allocated memory blocks for storing images of independent light sources;

[0042] Traverse the repaired image data and obtain the row index Row_{idx} of the current row;

[0043] Calculate the target index Target_{idx} = Row_{idx} pmod K;

[0044] The current row of data is copied to the corresponding position in the Target_{idx}th target image container, thereby separating the illumination information at different times.

[0045] Secondly, an image processing system applied to the spatiotemporal step reconstruction and image restoration method for time-division imaging of a linear scan camera as described in the first aspect, comprising a user interaction module and a logic control module, used for:

[0046] Dynamically receive user input instructions and select "whether the input image includes camera in-camera correction";

[0047] When the user selects to include internal correction, the system automatically locks the number of time divisions of the dual-line camera to K=2 and forces the application of compensation logic with alpha=1 to prevent aliasing errors caused by internal correction in the high-frequency time division mode of the dual-line Bayer interpolation structure.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] 1. This invention features a differential line offset model: Addressing the "temporal data reconstruction" problem in the specific scenario of multi-source time-division imaging with a line scan camera, this invention uses a "differential line offset" calculated based on the sensor's physical line spacing, the number of time-division operations, and the camera's internal ISP state. This "differential line offset" algorithm calculates the difference between the "theoretically required total target offset to completely eliminate temporal parallax" and the "preset offset already executed by the camera's internal ISP," only filling in the intermediate difference. This makes the repair method and system highly compatible with different sensor topologies.

[0050] 2. The reverse repair logic of this invention for the internal ISP preprocessing mode:

[0051] In particular, the core of this solution is the reverse repair logic for the case where spatial correction is enabled inside the camera (i.e., the aforementioned alpha=1). This is a "data rollback" mechanism designed to be compatible with the internal preprocessing of the hardware. When the camera enables spatial correction, the output RGB image is actually disordered in time. This invention uses the formula DeltaS=D×KD to accurately calculate the offset that still needs to be added or rolled back, and successfully restores these data that are "incorrectly aligned" by the hardware into a usable time-division image.

[0052] 3. This invention features Bayer phase adaptive correction: For dual-line camera RGB-Packed format, this invention introduces a Bayer phase correction term delta in differential line offset compensation, which solves the color separation problem caused by the inversion of red and green channel line attributes under even-number time division.

[0053] 4. In summary, this invention discloses a spatiotemporal step reconstruction and image restoration method and system for time-division imaging with a line scan camera. Addressing the image misalignment, color separation, and artifacts caused by differences in sensor physical row spacing, number of time divisions, and camera internal preprocessing modes (i.e., spatial correction states) in multi-source time-division imaging modes of line scan cameras, this invention proposes a "universal differential row offset model." This method first obtains the sensor topology type, number of time divisions K, and camera internal correction state of the image; then, it establishes a unified offset calculation model to calculate the difference between the theoretically required temporal offset for each spectral channel and the spatial offset already executed within the camera; using this difference, it performs reverse row translation restoration on RGB-Packed or grayscale image data; finally, it performs temporal interlaced extraction. This invention is adaptively compatible with black-and-white, dual-line, and triple-line cameras and can effectively restore images with internal spatial correction enabled, achieving low-cost, highly versatile multispectral time-division imaging. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram illustrating the specific operational logic of a spatiotemporal step reconstruction and image restoration method for time-division imaging of a line array camera, as provided in Embodiment 2 of the present invention, applied to a three-line camera. Detailed Implementation

[0056] The technical solution of this embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiment is one embodiment of the present invention, and not all embodiments thereof. Based on this embodiment of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Example 1:

[0058] Embodiment 1 of the present invention provides a spatiotemporal step-size reconstruction and image inpainting method for time-division imaging with a linear array camera, comprising the following steps:

[0059] S1: Image Acquisition and Parameter Parsing Steps: The image processing device acquires the synthetic image data stream transmitted by the line scan camera through the acquisition card and parses the configuration parameters of the imaging system. The configuration parameters include: time-division number K (i.e., the number of light source switching cycles), sensor topology type, physical row spacing D between each sensor channel, and camera internal spatial correction status flag F; the camera internal spatial correction status flag F is used to indicate whether the camera's internal ISP has enabled the row alignment preprocessing mode for static scenes.

[0060] In step S1,

[0061] (1). The image processing equipment is an industrial control computer, a personal computer, or an embedded computing platform;

[0062] (2). The imaging system refers to a machine vision inspection system consisting of a line scan camera, a lens, multiple sets of rapidly switchable light sources, and an image acquisition card;

[0063] (3). The method for obtaining the synthesized image data stream is as follows:

[0064] Receive raw or RGB-Packed data transmitted from line scan cameras via CameraLink, CoaXPress, or GigE interfaces, and store it in the memory buffer of the image processing device.

[0065] (4). The sensor topology type is any of the following:

[0066] Single-line / black and white sensor: its physical line spacing D=0;

[0067] Dual-line color sensor: contains two photosensitive lines, with a physical line spacing D between the first and second photosensitive lines;

[0068] Three-line color sensor: contains three independent photosensitive lines, and there is usually a physical line spacing D between each pair of red (R), green (G), and blue (B) lines;

[0069] S2: Differential line offset model establishment steps: The image processing device determines the physical position of each photosensitive channel relative to the reference channel according to the sensor topology type, and calculates the total target offset S_{target} required to completely eliminate temporal parallax theoretically, in conjunction with the time division quantity K; at the same time, it estimates the preset offset S_{internal} that has been executed by the camera's internal ISP according to the camera's internal spatial correction status flag F.

[0070] In step S2,

[0071] (1). The specific method for calculating the total target offset S_{target} and the preset offset S_{internal} is as follows:

[0072] S_{target}=D×K; where D is the number of physical rows of the current channel relative to the base channel, and K is the number of time-sharing segments;

[0073] S_{internal}=D×alpha; where alpha is the internal correction factor. When the camera's internal space correction status flag F is enabled, alpha=1; when the internal space correction status flag F is disabled, alpha=0.

[0074] (2). The method in (1) above supports reverse repair for the special case of "camera enabling internal space correction F", i.e., when alpha=1:

[0075] When it is detected that the linear scan camera has been misaligned based on the assumption of a static scene, and the misalignment is assumed to be a displacement of D rows, while time-division imaging actually requires a displacement of D×K rows, the following S3 step automatically calculates the positive additional compensation amount DeltaS=D×KD.

[0076] When the alignment inside the line scan camera is detected to cause data misalignment, the following S4 step implements logical rollback or appending of data by moving the memory address pointer;

[0077] S3: Differential Compensation Calculation Step: The image processing device calculates the differential line offset compensation DeltaS that needs to be performed by the software. The calculation formula is: DeltaS=S_{target}-S_{internal}; This step aims to calculate how many more lines need to be moved to align with the time axis, or how many more lines need to be rolled back to eliminate preprocessing errors inside the camera.

[0078] In step S3,

[0079] (1). The specific calculation logic of the differential line offset compensation amount DeltaS further includes the Bayer phase correction term delta, as shown in the following formula:

[0080] For the i-th color channel: DeltaS_i = (D_i × K) - (D_i × alpha) + delta

[0081] in:

[0082] D_i is the physical row number of the i-th color channel relative to the reference channel, where the color channel refers to the physically separated R, G, or B photosensitive rows on the sensor;

[0083] delta is a fine-tuning correction term used to compensate for phase deviation or sub-pixel line deviation caused by mechanical mounting in dual-line cameras during Bayer interpolation.

[0084] S4: Image reverse restoration step: The image processing device establishes a data buffer in its memory, and performs vertical translation on the hysteresis channel data in the synthesized image according to the calculated differential line offset compensation amount DeltaS; when DeltaS>0, an upward translation operation is performed, and when DeltaS<0, a downward translation operation is performed, thereby obtaining the restored image data in the spatiotemporal matching state.

[0085] S5: Temporal demultiplexing step: The image processing device performs row-level interlaced extraction of the repaired image data using ModK to reconstruct K photometric stereo images or multispectral images of independent light sources;

[0086] In step S5,

[0087] (1). Construct K target image containers, wherein the target image containers are pre-allocated memory blocks for storing images of independent light sources;

[0088] Traverse the repaired image data and obtain the row index Row_{idx} of the current row;

[0089] Calculate the target index Target_{idx} = Row_{idx} pmod K;

[0090] The current row of data is copied to the corresponding position in the Target_{idx}th target image container, thereby separating the illumination information at different times (i.e. reconstructing K photometric stereo images or multispectral images of independent light sources).

[0091] Furthermore, Embodiment 1 of the present invention also provides an image processing system that applies a spatiotemporal step reconstruction and image restoration method for time-division imaging using a linear array camera. This system includes a user interaction module and a logic control module, used to: dynamically receive user input instructions and select whether the input image includes camera internal correction; when the user selects to include internal correction, the system automatically locks the time-division number K=2 for the dual-line camera and forces the application of compensation logic with alpha=1 to prevent aliasing errors caused by internal correction in the high-frequency time-division mode of the dual-line Bayer interpolation structure.

[0092] Furthermore, the following two points are stated in Embodiment 1 of the present invention:

[0093] (1). In step S2, the photosensitive channel is a general term for the photosensitive row of the sensor topology type. The photosensitive channel refers to each photosensitive row of the sensor topology type.

[0094] The reference channel in step S2 refers to the reference photosensitive channel that does not require offset processing and is used as a zero point.

[0095] In step S2, the current channel refers to the photosensitive channel being processed in the specific method for calculating the total target offset S_{target} and the preset offset S_{internal} in step S2.

[0096] The color channel in step S3 refers to the photosensitive channel being processed in the calculation of the differential line offset compensation amount DeltaS in step S3, which further includes the Bayer phase correction term delta.

[0097] (2). In steps S1-S4, the symbol K is actually a unified constant that corresponds to the number of light sources K in the time-division linear scan. K light sources correspond to K time-division quantities, K images, and also the modulus K in interlaced extraction. Therefore, it is a positive integer constant.

[0098] The advantages of the spatiotemporal step reconstruction and image restoration method for time-division imaging with a line scan camera in Embodiment 1 of the present invention are as follows: The present invention discloses a spatiotemporal step reconstruction and image restoration method and system for time-division imaging with a line scan camera. Addressing the problems of image misalignment, color separation, and artifacts caused by differences in sensor physical row spacing, number of time divisions, and camera internal preprocessing modes (i.e., spatial correction states) in multi-source time-division imaging mode of a line scan camera, the present invention proposes a "universal differential row offset model." This method first obtains the sensor topology type, number of time divisions K, and camera internal correction state of the image; then establishes a unified offset calculation model to calculate the difference between the theoretically required temporal offset for each spectral channel and the spatial offset already executed within the camera; uses this difference to perform reverse row translation restoration on RGB-Packed or grayscale image data; and finally performs temporal interlaced extraction. The present invention is adaptively compatible with black and white, dual-line, and triple-line cameras, and can effectively restore images with internal spatial correction enabled, achieving low-cost and highly versatile multispectral time-division imaging.

[0099] Example 2:

[0100] Embodiment 2 of the present invention demonstrates the application of the spatiotemporal step reconstruction and image restoration method of the linear array camera time-division imaging of Embodiment 1 of the present invention to a three-line camera (RGB), in a 4-time division (K=4) mode, with the camera having internal spatial correction enabled (alpha=1) and physical line spacing D=1.

[0101] Please refer to the specific operational logic. Figure 1 The specific execution steps are as follows:

[0102] S1: Image acquisition and parameter parsing: The image processing device parses the configuration parameters, confirms that the time-division quantity K=4, the sensor is a three-line RGB structure, the physical line spacing D=1, and the camera internal space correction status flag F is enabled (i.e., α=1).

[0103] S2: Differential line offset model establishment: Determine the R channel as the reference channel (physical position 0), and the G and B channels as the photosensitive channels to be processed (physical positions 1 and 2 respectively).

[0104] For the G channel: calculate the total target offset Starget = 1 × 4 = 4 rows; estimate the preset offset Sinternal = 1 × 1 = 1 row.

[0105] For channel B: Calculate the total target offset Starget = 2 × 4 = 8 rows; estimate the preset offset Sinternal = 2 × 1 = 2 rows.

[0106] S3: Calculation of differential compensation amount:

[0107] G-channel compensation amount: DeltaSG = 4 − 1 = 3 rows.

[0108] B-channel compensation amount: DeltaSB = 8 − 2 = 6 rows.

[0109] The calculation results all showed positive values, indicating that additional compensation is needed.

[0110] S4: Image Reverse Repair: In the memory buffer, the software shifts the G channel data up 3 rows and the B channel data up 6 rows. This operation eliminates the erroneous preprocessing performed by the camera internally based on a static scene (K=1).

[0111] S5: Temporal demultiplexing: Modulo 4 (Mod 4) interlacing extraction is performed on the repaired image.

[0112] Technical effect: The mixed pixels that were originally caused by the camera's internal merging error were separated again, restoring the correct time correspondence and outputting 4 images of independent light sources with clear edges and no color edges.

[0113] Example 3:

[0114] Embodiment 3 of the present invention demonstrates the application of the spatiotemporal step reconstruction and image restoration method of the linear array camera time-division imaging of Embodiment 1 of the present invention to a dual-line camera (RG-BG). In the 2-time division (K=2) mode, the camera has its internal spatial correction turned off (alpha=0), the physical line spacing D=1, and there is a Bayer phase deviation delta=-1.

[0115] The specific execution steps are as follows:

[0116] S1 Image Acquisition and Parameter Analysis: Analyze the configuration parameters, confirm K=2, the sensor is a dual-line structure, and the physical row spacing D=1.

[0117] The camera calibration status indicator F is off (i.e., α=0).

[0118] S2 Differential Row Offset Model Establishment: Determine the first photosensitive row (e.g., RG row) as the reference channel, and the second photosensitive row (e.g., BG row) as the photosensitive channel to be processed.

[0119] For the second photosensitive line: Calculate the total target offset Starget.

[0120] =1×2=2 rows; Estimated preset offset Sinternal=1×0=0 rows.

[0121] S3 Differential Compensation Calculation:

[0122] Introduce the Bayer phase correction term delta=−1.

[0123] Second photosensitive line compensation amount:

[0124] DeltaS = (1×2)−0 + (−1) = 1 row.

[0125] S4 Image Reverse Repair: The software shifts the data of the second photosensitive row upwards by one row.

[0126] S5 temporal demultiplexing: Modulo 2 (Mod2) interlacing extraction is performed on the repaired image.

[0127] Technical effects: It not only corrects the temporal parallax caused by physical spacing, but also corrects the phase problem of the Bayer array through the delta term, preventing color interpolation errors in even-number time divisions and achieving accurate splitting.

[0128] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for spatiotemporal step-size reconstruction and image inpainting of linear array cameras using time-division imaging, characterized in that, Includes the following steps: S1: Image acquisition and parameter parsing steps: The image processing device acquires the synthetic image data stream transmitted by the line scan camera through the acquisition card and parses the configuration parameters of the imaging system. The configuration parameters include: time-division number K, sensor topology type, physical row spacing D between each channel of the sensor, and camera internal space correction status flag F; the camera internal space correction status flag F is used to indicate whether the camera internal ISP has enabled the row alignment preprocessing mode for static scenes. S2: Differential line offset model establishment steps: The image processing device determines the physical position of each photosensitive channel relative to the reference channel according to the sensor topology type, and calculates the total target offset S_{target} required to completely eliminate temporal parallax theoretically, in conjunction with the time division quantity K; at the same time, it estimates the preset offset S_{internal} that has been executed by the camera's internal ISP according to the camera's internal spatial correction status flag F. S3: Differential compensation calculation steps: The image processing device calculates the differential line offset compensation DeltaS that needs to be executed by the software. The calculation formula is: DeltaS=S_{target}-S_{internal}. S4: Image reverse restoration step: The image processing device establishes a data buffer in its memory, and performs vertical translation on the hysteresis channel data in the synthesized image according to the calculated differential line offset compensation amount DeltaS; when DeltaS>0, an upward translation operation is performed, and when DeltaS<0, a downward translation operation is performed, thereby obtaining the restored image data in the spatiotemporal matching state. S5: Temporal demultiplexing step: The image processing device performs row-level interlaced extraction on the repaired image data in the manner of module K, and reconstructs K photometric stereo images or multispectral images of independent light sources.

2. The spatiotemporal step reconstruction and image inpainting method for time-division imaging with a linear array camera according to claim 1, characterized in that, In step S1 Image processing equipment can be industrial PCs, personal computers, or embedded computing platforms; An imaging system refers to a machine vision inspection system consisting of a line scan camera, a lens, multiple sets of rapidly switchable light sources, and an image acquisition card.

3. The spatiotemporal step reconstruction and image inpainting method for time-division imaging with a linear array camera according to claim 1, characterized in that, In step S1 The method for obtaining the synthetic image data stream is as follows: It receives raw or RGB-Packed data transmitted from line scan cameras via CameraLink, CoaXPress, or GigE interfaces and stores it in the memory buffer of the image processing device.

4. The spatiotemporal step reconstruction and image inpainting method for time-division imaging with a linear array camera according to claim 1, characterized in that, In step S1 The sensor topology can be any of the following: Single-line / black and white sensor: its physical line spacing D=0; Dual-line color sensor: contains two photosensitive lines, with a physical line spacing D between the first and second photosensitive lines; Three-line color sensor: contains three independent photosensitive lines, and there is usually a physical line spacing D between each pair of red (R), green (G), and blue (B) lines.

5. The spatiotemporal step reconstruction and image inpainting method for time-division imaging with a linear array camera according to claim 4, characterized in that, In step S2 The specific method for calculating the total target offset S_{target} and the preset offset S_{internal} is as follows: S_{target}=D×K; where D is the number of physical rows of the current channel relative to the base channel, and K is the number of time-sharing segments; S_{internal}=D×alpha; where alpha is the internal correction factor. When the camera's internal space correction status flag F is on, alpha=1; when the internal space correction status flag F is off, alpha=0.

6. The spatiotemporal step reconstruction and image inpainting method for time-division imaging with a linear array camera according to claim 5, characterized in that, In step S3 The specific calculation logic for the differential line offset compensation DeltaS further includes a Bayer phase correction term delta, as shown in the following formula: For the i-th color channel: DeltaS_i = (D_i × K) - (D_i × alpha) + delta; in: D_i is the physical row number of the i-th color channel relative to the reference channel, where the color channel refers to the physically separated R, G, or B photosensitive rows on the sensor; delta is a fine-tuning correction term used to compensate for phase deviations in Bayer interpolation of dual-line cameras or sub-pixel line deviations caused by mechanical mounting.

7. The spatiotemporal step reconstruction and image inpainting method for time-division imaging of a linear array camera according to claim 5, characterized in that, The method supports reverse repair for the special case of "camera enabling internal space correction F", i.e., when alpha=1: When it is detected that the linear scan camera has been misaligned based on the assumption of a static scene, and the misalignment is assumed to be a displacement of D rows, while time-division imaging actually requires a displacement of D×K rows, step S3 automatically calculates the positive additional compensation amount DeltaS=D×KD. When the alignment inside the line scan camera is detected to cause data misalignment, step S4 implements logical rollback or appending of the data by moving the memory address pointer.

8. The spatiotemporal step reconstruction and image inpainting method for time-division imaging with a linear array camera according to claim 1, characterized in that, In step S5 The specific steps of time-series demultiplexing are: Construct K target image containers, which are pre-allocated memory blocks for storing images of independent light sources; Traverse the repaired image data and obtain the row index Row_{idx} of the current row; Calculate the target index Target_{idx} = Row_{idx} pmod K; The current row of data is copied to the corresponding position in the Target_{idx}th target image container, thereby separating the illumination information at different times.

9. An image processing system applied to the spatiotemporal step reconstruction and image restoration method for time-division imaging of a linear array camera as described in any one of claims 1-8, characterized in that, Includes a user interaction module and a logic control module, used for: Dynamically receive user input instructions and select "whether the input image includes camera in-camera correction"; When the user selects to include internal correction, the system automatically locks the number of time divisions of the dual-line camera to K=2 and forces the application of compensation logic with alpha=1 to prevent aliasing errors caused by internal correction in the high-frequency time division mode of the dual-line Bayer interpolation structure.

Images