Image inpainting method and device, electronic equipment and storage medium

Abstract

This invention discloses an image restoration method, apparatus, electronic device, and storage medium, relating to the field of image processing technology. The method includes: acquiring image data and image data status information; storing the image data and image data status information in a first storage area; determining the row data to be detected in the image data based on the image data status information; storing the row data to be detected in a second storage area; performing a preset detection on the row data to be detected and determining abnormal image data based on the detection results; temporarily replacing the abnormal image data with preset data; generating target data based on a preset model and relevant data of the abnormal image data; updating the image data in the first storage area based on the target data; and obtaining and outputting the restored image data. This invention solves the problems that image restoration easily leads to poor image frame continuity, loss of entire frames, and long image recovery times, which can easily cause display stuttering or algorithm malfunctions.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to image restoration methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Most existing image transmission systems adopt a real-time streaming architecture. Image data is output from the image sensor, processed by the image processing unit, buffered through DDR (Double Data Rate) DRAM, and finally uploaded to the host or display device. During high-speed image transmission, instability of the image sensor can easily cause image anomalies, such as missing pixels, abnormal row data, or buffer congestion.

[0003] Currently, image repair methods for image anomalies typically involve discarding the current frame and waiting for the next frame to resynchronize, resetting the entire sending module, or stopping data output while waiting for the buffer to recover. However, these methods often result in poor image frame continuity, loss of entire frames, and long image recovery times, which can lead to display stuttering or algorithm malfunctions. Summary of the Invention

[0004] This invention provides an image restoration method, apparatus, electronic device, and storage medium to solve the problems that image restoration easily leads to poor image frame continuity, loss of the entire frame, long image recovery time, and easy display lag or algorithm abnormalities.

[0005] In a first aspect, the present invention provides an image restoration method, the method comprising: Obtain image data and corresponding image data status information, and store the image data and image data status information in the first storage area; Based on the image data status information, determine the row data to be detected in the image data and store the row data to be detected in the second storage area; Pre-set detection is performed on the data to be detected, and abnormal image data is determined based on the detection results; During the process of outputting image data in the first storage area, abnormal image data is temporarily replaced by preset data and output in the second storage area according to the preset model and the relevant data of the abnormal image data. The relevant data is included in the image data. The image data in the first storage area is updated based on the target data in the second storage area to obtain the repaired image data, and then the repaired image data is output.

[0006] The image restoration method provided in this embodiment can maintain the continuity of image frame structure during the image restoration process, avoiding the loss of entire frames or system reset failures. This method improves image restoration accuracy and image output stability by establishing a second storage area and combining it with a preset model to compensate and restore abnormal image data. Furthermore, the restoration process does not require interruption of the overall image transmission link, effectively reducing the risk of display stuttering, image tearing, and subsequent algorithm failures in high-speed image systems. It solves the problems that image restoration easily leads to poor image frame continuity, loss of entire frames, long image restoration times, and is prone to display stuttering or algorithm malfunctions.

[0007] In one optional implementation, storing the row data to be detected in a second storage area includes: The pixel coordinates and row position information of the row data to be detected are determined based on the image data status information. If there is already stored data in the second storage area, the data to be deleted is identified from the already stored data and then deleted. The storage capacity of the second storage area is determined based on the changes in the image data. The data of the row to be detected, pixel coordinate information, and row position information are stored in the second storage area.

[0008] In this embodiment, the second storage area can adaptively adjust its capacity, automatically clear old row data, and synchronously store row data and coordinate information. This balances cache resource usage with repair reference data, adapts to different scene scenarios, and provides sufficient reference data for image anomaly recovery.

[0009] In one optional implementation, a preset detection is performed on the row data to be detected, and abnormal image data is determined based on the detection results, including: Determine the number of pixels, cache status information, transmission status information, and image row marker information of the row data to be detected; Based on the number of pixels, cache status information, transmission status information, and image row marker information, integrity checks are performed on the row data to be detected to obtain the detection results. Based on the detection results, image rows that do not meet the preset conditions are identified and treated as abnormal image data. These image rows are included in the data of rows to be detected. Determine the frame number, row number, and first data length corresponding to the abnormal image data, and generate the abnormal location information corresponding to the abnormal image data based on the frame number, row number, and first data length.

[0010] In this embodiment, integrity detection is carried out from multiple dimensions, including pixels, cache, transmission, and line markers. Various image transmission faults are identified from multiple dimensions, abnormal image lines are accurately determined, and abnormal location information carrying frame, line number, and length is generated to provide accurate positioning basis for subsequent image repair.

[0011] In one optional implementation, the abnormal image data is temporarily replaced and output according to preset data, including: Determine the second data length of the abnormal image data; When the second data length is less than or equal to the first preset threshold, the first position of the missing pixel in the abnormal image data is obtained, and an abnormal marker of the abnormal image data is established in the second storage area according to the first position. When the second data length is greater than the first preset threshold, the second position of the missing pixel in the abnormal image data and the positional relationship between the abnormal image row and the adjacent image row in the abnormal image data are obtained, and an abnormal marker of the abnormal image data is established in the second storage area according to the second position and the positional relationship. Write the abnormal location information into the second storage area; In the first storage area, a state to be recovered is established for the image area corresponding to the abnormal image data, and the data output of the image area is paused; The image region data is temporarily replaced and output based on preset data.

[0012] In this embodiment, anomalies are distinguished by their size based on the second data length of the abnormal image data, and the abnormal image data is marked differently to store complete anomaly information for easy repair. Output to only the local abnormal area is paused while the overall link continues to transmit, with temporary data padding to ensure frame structure integrity and avoid problems such as whole-frame failure, display stuttering, and module synchronization issues.

[0013] In one optional implementation, in the second storage area, target data corresponding to the abnormal image data is generated based on a preset model and related data of the abnormal image data, including: Determine the third data length of the abnormal image data; When the length of the third data is less than or equal to the second preset threshold, the neighboring pixel data of the abnormal image data and the spatial distribution relationship between the abnormal image data and the neighboring pixel data are obtained from the data stored in the second storage area, wherein the neighboring pixel data and the spatial distribution relationship are included in the relevant data. The target data is obtained by inputting the neighborhood pixel data and spatial distribution relationship into the preset model; If the length of the third data is greater than the second preset threshold, the adjacent image row data of the abnormal image data and the historical area data of the area where the abnormal image data is located are obtained from the data stored in the first storage area, wherein the adjacent image row data and the historical area data are included in the relevant data; The target data is obtained by inputting adjacent image row data and historical region data into a preset model.

[0014] In this implementation, reference data is retrieved hierarchically based on the length of the abnormal region. For small-area anomalies, neighboring pixels and spatial relationships are extracted from the cache and input into the model for prediction and repair. For large-area anomalies, adjacent valid rows and frames from the external cache are read, and the image is reconstructed based on spatiotemporal correlation. The same recovery model is reused, and only the reference data source is switched to adapt to various loss scenarios. The two-level cache is updated synchronously, which significantly improves the image repair accuracy under different fault conditions.

[0015] In one alternative implementation, the method further includes: In the event that the target data corresponding to the abnormal image data cannot be generated, the first padding data is generated by a first preset method, wherein the first padding data has the same data length as the target data; The image data in the first storage area is updated according to the first filling data to obtain the repaired image data, and the repaired image data is output.

[0016] In this embodiment, the entire frame of the image is not directly discarded. Instead, local recovery errors are allowed while ensuring the integrity of the image frame structure, so as to improve the stability and continuous operation capability of the high-speed image link under abnormal conditions.

[0017] In one optional implementation, the output restored image data includes: Determine whether the repaired image data meets the preset output requirements; If the repaired image data meets the preset output requirements, the repaired image data will be output to the display device or the subsequent image processing device. If the repaired image data does not meet the preset output requirements, reference data corresponding to the repaired image data is generated according to the preset recovery strategy, and the repaired image data is adjusted according to the reference data to obtain the first intermediate data. If the first intermediate data meets the preset output requirements, the first intermediate data will be output to the display device or the subsequent image processing device. If the first intermediate data does not meet the preset output requirements, the second fill data is generated by the second preset method, the first intermediate data is adjusted according to the second fill data to obtain the second intermediate data, and the second intermediate data is output to the display device or the subsequent image processing device.

[0018] In a second aspect, the present invention provides an image restoration apparatus, the apparatus comprising: The information acquisition module is used to acquire image data and image data status information corresponding to the image data, and store the image data and image data status information in the first storage area; The data determination module is used to determine the row data to be detected in the image data based on the image data status information, and store the row data to be detected in the second storage area; The data detection module is used to perform preset detection on the data to be detected and to identify abnormal image data based on the detection results; The data repair module is used to temporarily replace abnormal image data with preset data during the process of outputting image data in the first storage area, and to generate target data corresponding to the abnormal image data in the second storage area according to the preset model and the relevant data of the abnormal image data, wherein the relevant data is included in the image data; The data output module is used to update the image data in the first storage area according to the target data in the second storage area, obtain the repaired image data, and output the repaired image data.

[0019] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the image restoration method described in the first aspect or any corresponding embodiment thereof.

[0020] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the image restoration method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic flowchart of a first embodiment of an image restoration method according to the present invention; Figure 2 This is a schematic diagram of the structure of an image fault-tolerant recovery system based on frame structure continuity according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a second process for an image restoration method according to an embodiment of the present invention; Figure 4 This is a structural block diagram of an image restoration apparatus according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0025] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0026] With the development of high-speed image acquisition systems, interface-based image transmission solutions have been widely applied in industrial vision, autonomous driving, edge computing, and embedded vision systems. Existing image transmission systems transmit images from image sensors to host or display devices via image links. In a typical image link, image data is output from the image sensor, processed by the image processing unit, buffered via DDR, and finally uploaded to the host or display device. Most existing image transmission systems employ a real-time streaming architecture. During high-speed image transmission, sensor instability can easily cause interruptions in image row data, pixel loss, or abnormal frame structure. This can further lead to issues such as AXI (Advanced eXtensible Interface) bus backpressure, FIFO (First In First Out) underload, ISP (Image Signal Processing) processing delays, or link jitter on the transmission link. To address these problems, it is necessary to process abnormal data in the image.

[0027] Traditional image processing methods typically employ the following approaches: discarding the current frame and waiting for resynchronization with the next frame; performing a complete reset of the sending module; and stopping data output while waiting for buffer recovery. However, images processed using these methods often suffer from poor frame continuity, are prone to frame loss, have long system recovery times, and are susceptible to display stuttering or algorithm malfunctions in high real-time scenarios. Downstream ISPs or vision modules may also fail due to frame structure corruption. Especially in high-resolution, high-frame-rate image scenarios, traditional methods struggle to balance image integrity with link stability.

[0028] Based on the above, this invention provides an image restoration method. During high-speed image transmission, a dynamic buffer window and an anomaly coordinate marking mechanism are established to detect the integrity of image rows in real time. When missing image pixels or anomalies in row structure are detected, a learning model is used to combine neighborhood data in the current buffer window and reference row data from the external buffer to compensate and restore the abnormal region. This method maintains the continuity of the image frame structure, ensuring the complete output of the entire frame structure while allowing for local image restoration errors, thereby avoiding problems such as half-line output, frame structure destruction, and link jamming. Simultaneously, this method, through the linkage of the dynamic buffer window and the learning model, realizes an adaptive image restoration mechanism based on historical image data and neighborhood correlation, improving the anomaly tolerance and system stability of the high-speed image link in complex scenarios.

[0029] According to an embodiment of the present invention, an image restoration method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This embodiment provides an image restoration method that can be used in the aforementioned computer system. Figure 1 This is a flowchart of an image restoration method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain image data and image data status information corresponding to the image data, and store the image data and image data status information in the first storage area.

[0031] Specifically, the image inpainting method in this embodiment is implemented by an image fault-tolerant recovery system based on frame structure continuity, hereinafter referred to as the system, such as... Figure 2As shown, the system includes an image processing unit, a cache unit, an external cache control unit, a learning model, and an external cache. The cache unit and the learning model together constitute the image anomaly recovery structure. Image data is input from the image sensor into the FPGA (Field Programmable Gate Array), processed by the image processing unit, and then written to the external cache via the cache control unit. During image transmission, the system establishes a dynamic cache window in the cache unit and detects the integrity of image rows. When an image data anomaly is detected, the system marks the location of the abnormal pixels and, in conjunction with the learning model, compensates for and recovers the abnormal region, thereby maintaining the continuity of the image frame structure. The learning model here is the preset model referred to below.

[0032] The FPGA acquires image data. Image data output from the image sensor is input into the FPGA via an image interface. The image data enters the image processing unit in frame, line, and pixel order, and is output to the buffer control unit after basic image processing. Basic image processing includes, for example, processing by the ISP image preprocessing unit according to system requirements before outputting to the buffer control unit. The ISP is a general image processing algorithm, including BLC (Black Level Correction), LSC (Lens Shading Correction), white balance, etc., with functions added or removed according to user needs.

[0033] During the process of inputting image data into the FPGA via the image interface, the system synchronously establishes the image data status information for the current image data. This image data status information is image frame status information, including: current frame number, current row number, current row pixel position, and current image row length. This image data status information is used for subsequent image integrity detection and anomaly recovery processing.

[0034] The cache control unit stores the image data and image data status information into the first storage area based on the current image resolution and image row information. The first storage area may be, for example, the corresponding storage area in the external cache.

[0035] Step S102: Determine the row data to be detected in the image data according to the image data status information, and store the row data to be detected in the second storage area.

[0036] Specifically, the system establishes a cache unit within the FPGA as a second storage area. The size of the second storage area can be configured by the user or dynamically adjusted by the trained model based on changes in the image data.

[0037] Based on the image data status information, newly received image row data are sequentially used as row data to be detected. The row data to be detected is stored in the second storage area. In addition, while storing the row data to be detected, the second storage area also records the pixel coordinate information and row position information corresponding to the row data to be detected, thereby providing a reference data source for subsequent abnormal pixel recovery.

[0038] Step S103: Perform preset detection on the data to be detected and determine abnormal image data based on the detection results.

[0039] Specifically, preset detection methods include integrity checks, such as current row pixel count detection, row end marker detection, cache status detection, and data transmission status detection. These preset checks are performed on the rows of data to be inspected, yielding the results. Based on the results, it is determined whether each row of data contains anomalies, thus identifying anomalous image data. Anomalies include: current row pixel count less than the standard value, interrupted image data, abnormal row end position, or missing consecutive pixels.

[0040] In step S104, during the process of outputting image data in the first storage area, the abnormal image data is temporarily replaced and output according to preset data, and in the second storage area, the target data corresponding to the abnormal image data is generated according to the preset model and the relevant data of the abnormal image data, wherein the relevant data is included in the image data.

[0041] Specifically, during the output of image data in the first storage area, the output of abnormal image data is paused, and the abnormal image data is temporarily replaced by preset data, such as historical valid data, neighborhood reference data, default padding values, or virtual pixel data. This ensures the integrity of image line length and frame structure, avoiding problems such as display stuttering, link interruption, or synchronization failure of subsequent modules.

[0042] Retrieve relevant data from abnormal image data, such as neighboring pixel data of abnormal image data, valid image row data before and after, and historical region data of the current frame.

[0043] The preset model can be implemented using existing image restoration models, prediction algorithms, or other image compensation algorithms; there are no restrictions on which one is preferred. The preset model is only one possible method for restoring abnormal regions, and its specific network structure and training process are not limited. In the second storage area, the preset model generates target data corresponding to the abnormal image data based on relevant data from the abnormal image data.

[0044] Step S105: Update the image data in the first storage area according to the target data in the second storage area to obtain the repaired image data, and output the repaired image data.

[0045] Specifically, the target data in the second storage area is rewritten into the first storage area to update the image data in the first storage area, resulting in repaired image data. The cache control unit rereads the repaired image data according to the image output timing sequence and outputs it to the display device or subsequent image processing device.

[0046] The above methods are applicable to industrial vision, edge artificial intelligence, embedded vision processing, and high-speed image acquisition.

[0047] The image restoration method provided in this embodiment acquires image data and image data status information, and stores the image data and image data status information in a first storage area; determines the row data to be detected in the image data based on the image data status information, and stores the row data to be detected in a second storage area; performs preset detection on the row data to be detected, and determines abnormal image data based on the detection results; temporarily replaces the abnormal image data with preset data, and generates target data based on a preset model and relevant data of the abnormal image data; updates the image data in the first storage area based on the target data, and obtains and outputs the restored image data. This method can maintain the continuity of the image frame structure during image restoration, avoiding the loss of entire frames or system reset failures. By establishing a second storage area and combining it with a preset model to compensate and restore abnormal image data, this method improves the accuracy of image restoration and the stability of image output. Furthermore, it does not require interruption of the overall image transmission link during the restoration process, effectively reducing the risk of display stuttering, image tearing, and subsequent algorithm failures in high-speed image systems. It solves the problems that image restoration easily leads to poor image frame continuity, loss of entire frames, and long image restoration times, which can easily cause display stuttering or algorithm abnormalities.

[0048] This embodiment provides an image restoration method that can be used in the aforementioned computer system. Figure 3 This is a flowchart of an image restoration method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain image data and corresponding image data status information, and store the image data and image data status information in the first storage area. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0049] Step S302: Determine the row data to be detected in the image data according to the image data status information, and store the row data to be detected in the second storage area.

[0050] Specifically, step S302 includes: Step S3021: Determine the pixel coordinate information and row position information of the row data to be detected based on the image data status information.

[0051] Step S3022: If there is already stored data in the second storage area, determine the data to be deleted from the already stored data and delete the data to be deleted. The storage capacity of the second storage area is determined according to the changes in the image data.

[0052] Step S3023: Store the row data to be detected, pixel coordinate information, and row position information into the second storage area.

[0053] Specifically, the system establishes a cache unit within the FPGA as a second storage area. This second storage area is used for dynamic caching of image row data.

[0054] The second storage area employs a sliding window structure. The window size of the second storage area can be configured by the user or dynamically adjusted by a cache adjustment model based on changes in image data. For example, the cache adjustment model analyzes features such as pixel differences between consecutive image frames, edge changes, or the frequency of historical anomalies. When image content changes little and the link operates stably, the cache adjustment model reduces the cache window size, retaining only the current row and a small amount of reference row data to reduce cache resource consumption. When image content changes significantly, there are many moving targets, or the frequency of anomalies increases, the cache adjustment model increases the cache window size, retaining more preceding and following reference rows to increase the amount of reference information available for anomaly region recovery. Through this method, a dynamic balance is achieved between cache resource consumption and anomaly recovery capabilities.

[0055] The second storage area records the pixel coordinates and row position information of the row data to be detected while storing the row data to be detected, thus providing a reference data source for subsequent abnormal pixel recovery. Therefore, the pixel coordinates and row position information of the row data to be detected are determined based on the image data status information.

[0056] The second storage area stores at least the current row of data and several reference rows before and after it. During image transmission, the cache window is automatically updated each time a new row of image data is received, and the oldest cached data row is moved out of the window area. If there is already stored data in the second storage area, the data to be deleted is identified from the stored data and then deleted. The data to be deleted is, for example, the oldest cached data row, i.e., the earliest cached image data row.

[0057] The data of the row to be detected, pixel coordinate information, and row position information are stored in the second storage area.

[0058] In this embodiment, the second storage area can adaptively adjust its capacity, automatically clear old row data, and synchronously store row data and coordinate information. This balances cache resource usage with repair reference data, adapts to different scene scenarios, and provides sufficient reference data for image anomaly recovery.

[0059] Step S303: Perform preset detection on the data to be detected and determine abnormal image data based on the detection results.

[0060] Specifically, step S303 includes: Step A1: Determine the number of pixels, cache status information, transmission status information, and image row marker information of the row data to be detected.

[0061] Step A2: Based on the number of pixels, cache status information, transmission status information, and image row marker information, perform integrity checks on the row data to be detected to obtain the detection results.

[0062] Step A3: Based on the detection results, determine the image rows that do not meet the preset conditions, and treat the image rows that do not meet the preset conditions as abnormal image data. The image rows that do not meet the preset conditions are included in the data of rows to be detected.

[0063] Step A4: Determine the frame number, row number, and first data length corresponding to the abnormal image data, and generate the abnormal location information corresponding to the abnormal image data based on the frame number, row number, and first data length.

[0064] Specifically, preset detections include integrity detection, such as current row pixel count detection, row end marker detection, cache status detection, and data transmission status detection.

[0065] Based on the number of pixels in the row data to be detected, cache status information, transmission status information, and image row marker information, the system performs integrity checks on the row data to be detected during image data transmission and obtains the detection results. For example, the system performs current row pixel count detection, cache status detection, and data transmission status detection on the row data to be detected.

[0066] The current row pixel count detection includes: the system compares the actual number of pixels received in the row data to be detected with the standard number of pixels corresponding to the preset resolution. When the system detects that the current row pixel count is less than the standard value, the image data is interrupted, the row end position is abnormal, or there are consecutive missing pixels, the system determines that the row data to be detected is abnormal.

[0067] Cache status detection includes real-time monitoring of data occupancy, read / write pointer status, remaining cache depth, and cache update status in cache units. When the remaining cache data is detected to be below the corresponding threshold, the cache occupancy exceeds the preset upper limit, or the cache has not been updated for a long time, the system determines that the data in the row to be detected is abnormal and triggers the abnormal marking process.

[0068] Data transmission status detection includes monitoring the handshake status, line synchronization timing, frame synchronization timing, and data continuity of the image data stream. When the system detects that the bus backpressure duration exceeds the corresponding threshold, the line synchronization interval is abnormal, the frame synchronization period is abnormal, or the image data continuity is disrupted, it determines that there is an anomaly in the line data to be detected and records the corresponding anomaly location.

[0069] Preset conditions include, for example: the number of pixels in an image row equals a pre-set value; the remaining cache data for the corresponding image row is greater than or equal to a corresponding threshold; the cache occupancy rate for the corresponding image row does not exceed a preset upper limit; and the bus backpressure duration for the corresponding image row does not exceed a corresponding threshold. Image rows that do not meet the preset conditions are considered abnormal image data.

[0070] Simultaneously, the system marks the coordinates of the area where the abnormal image data is located, determines and records the frame number, row number, and first data length corresponding to the abnormal image data, thereby establishing the abnormal location information corresponding to the abnormal image data and providing a location basis for subsequent image recovery. The first data length specifically refers to the data length of the abnormal image data.

[0071] In this embodiment, integrity detection is carried out from multiple dimensions, including pixels, cache, transmission, and line markers. Various image transmission faults are identified from multiple dimensions, abnormal image lines are accurately determined, and abnormal location information carrying frame, line number, and length is generated to provide accurate positioning basis for subsequent image repair.

[0072] In step S304, during the process of outputting image data in the first storage area, the abnormal image data is temporarily replaced and output according to preset data, and in the second storage area, the target data corresponding to the abnormal image data is generated according to the preset model and the relevant data of the abnormal image data, wherein the relevant data is included in the image data.

[0073] Specifically, step S304 includes: Step B1: Determine the second data length of the abnormal image data.

[0074] Step B2: If the second data length is less than or equal to the first preset threshold, obtain the first position of the missing pixel in the abnormal image data, and establish an abnormal marker for the abnormal image data in the second storage area based on the first position.

[0075] Step B3: If the second data length is greater than the first preset threshold, obtain the second position of the missing pixel in the abnormal image data, the positional relationship between the abnormal image row and the adjacent image row in the abnormal image data, and establish an abnormal marker of the abnormal image data in the second storage area according to the second position and positional relationship.

[0076] Step B4: Write the abnormal location information into the second storage area.

[0077] Step B5: In the first storage area, establish a state to be recovered for the image region corresponding to the abnormal image data, and pause the data output of the image region.

[0078] Step B6: Temporarily replace the data in the image region with preset data and output the replacement data.

[0079] Specifically, the second data length of the abnormal image data is determined. The area containing the abnormal image data can be referred to as the abnormal region. In this embodiment, the abnormal region is determined to be large or small based on its length. The length of the abnormal region is the number of consecutive missing pixels or the number of pixels covered by the abnormal region. The length of the abnormal region is equal to the second data length of the abnormal image data.

[0080] When the length of an abnormal region is less than a first preset threshold, the system classifies it as a small-area abnormal region; when the length of an abnormal region is greater than or equal to the first preset threshold, the system classifies it as a large-area abnormal region. The first preset threshold can be, for example, 32 pixels, 48 ​​pixels, or other numbers of pixels. This threshold can be configured by the user or dynamically set based on image resolution, cache resource size, or historical anomaly statistics. For example, at a 1920×1080 resolution, a region is classified as small-area abnormal region when the number of consecutive missing pixels is less than 32 pixels, and as large-area abnormal region when the number of consecutive missing pixels is greater than or equal to 32 pixels. The determination of the size of the abnormal region can be completed by the cache control unit. Based on the anomaly location information and abnormal region length information output by the anomaly detection module, the cache control unit classifies the anomaly level and outputs the corresponding recovery strategy selection result to the subsequent recovery module.

[0081] When the system detects an image anomaly, the cache control unit establishes an anomaly marker for the current cache area based on the anomaly coordinate information and writes the anomaly location information into the second storage area. The anomaly location information includes, for example, the frame number of the anomaly area, the line number of the anomaly area, the length of the anomaly area, the anomaly type, and the anomaly marker status. The anomaly type can include pixel loss, line data interruption, line length anomaly, or cache anomaly; the anomaly marker status indicates whether the corresponding anomaly area has been recovered. The cache control unit writes the above anomaly location information into the cache unit so that the subsequent recovery module can quickly locate the anomaly area and perform compensation and recovery operations based on the anomaly location information.

[0082] If the length of the second data is less than or equal to the first preset threshold, it indicates that the abnormal region is small, and the system only records the location of the missing pixels. Therefore, the first location of the missing pixels in the abnormal image data is obtained, and an abnormal marker for the abnormal image data is established in the second storage area based on the first location.

[0083] If the second data length exceeds the first preset threshold, it indicates a large abnormal region. The system records the location of the missing pixels and, simultaneously, the positional relationship between the current abnormal row and its corresponding preceding and following reference rows. It obtains the second location of the missing pixels in the abnormal image data, the positional relationship between the abnormal image row and adjacent image rows, and establishes anomaly markers for the abnormal image data in the second storage area based on the second location and positional relationships.

[0084] After marking the anomaly, the system establishes a pending recovery state for the image region corresponding to the abnormal image data in the second storage area and suspends data output for the current abnormal region, i.e., suspends data output for the image region. It should be noted that suspending data output for the current abnormal region does not suspend data transmission of the entire image link, but rather suspends the direct output of the original data corresponding to the abnormal region. Simultaneously, it maintains the overall image frame structure without interruption, thus avoiding the problem in traditional solutions where a single line of anomaly causes the entire frame to fail.

[0085] Before the abnormal image data repair is completed, the system maintains continuous operation of the image output link and temporarily replaces the data in the image region with preset data. Preset data includes, for example, historical valid data, neighborhood reference data, default padding values, or virtual pixel data. This ensures the integrity of the image line length and frame structure, avoiding problems such as display stuttering, link interruption, or synchronization failure of subsequent modules.

[0086] In this embodiment, anomalies are distinguished by their size based on the second data length of the abnormal image data, and the abnormal image data is marked differently to store complete anomaly information for easy repair. Output to only the local abnormal area is paused while the overall link continues to transmit, with temporary data padding to ensure frame structure integrity and avoid problems such as whole-frame failure, display stuttering, and module synchronization issues.

[0087] In some optional implementations, step S304 above further includes: Step C1: Determine the third data length of the abnormal image data.

[0088] Step C2: When the length of the third data is less than or equal to the second preset threshold, obtain the neighboring pixel data of the abnormal image data and the spatial distribution relationship between the abnormal image data and the neighboring pixel data from the data stored in the second storage area, wherein the neighboring pixel data and the spatial distribution relationship are included in the relevant data.

[0089] Step C3: Input the neighborhood pixel data and spatial distribution relationship into the preset model to obtain the target data.

[0090] Step C4: If the length of the third data is greater than the second preset threshold, obtain the adjacent image row data of the abnormal image data and the historical area data of the area where the abnormal image data is located from the data stored in the first storage area, wherein the adjacent image row data and the historical area data are included in the related data.

[0091] Step C5: Input adjacent image row data and historical region data into the preset model to obtain target data.

[0092] Specifically, the third data length of the abnormal image data is determined. The region containing the abnormal image data can be referred to as the abnormal region. In this embodiment, the abnormal region is determined to be large or small based on its length. The length of the abnormal region is equal to the second data length of the abnormal image data.

[0093] The preset model can be implemented using existing image restoration models, prediction algorithms, or other image compensation algorithms; there are no restrictions on which one to use. The preset model is only one option for restoring abnormal regions, and its specific network structure and training process should be set according to actual needs. The second preset threshold can be, for example, 32 pixels, 48 ​​pixels, or other numbers of pixels. The second preset threshold can be configured by the user or dynamically set based on image resolution, cache resource size, or historical anomaly statistics.

[0094] If the length of the third data point is less than or equal to the second preset threshold, the abnormal region is determined to be small. The preset model directly calls the neighboring pixel data of the region near the abnormal image data and combines it with the data of the corresponding regions of the adjacent rows before and after to perform pixel prediction.

[0095] The system retrieves neighboring pixel data and the spatial distribution relationship between the abnormal image data and its neighboring pixels from the data stored in the second storage area. This neighboring pixel data and spatial distribution relationship are then input into a preset model. The preset model generates predicted values ​​for missing pixels based on these values ​​and the spatial distribution relationship, using these predicted values ​​as target data. The target data is then rewritten into the corresponding image regions in both the second and first cache areas. This method enables image data compensation in cases of localized pixel loss, thereby reducing the impact of abnormal image regions on the overall image quality.

[0096] If the length of the third data exceeds the second preset threshold, the abnormal region is determined to be large, for example, the abnormal image data contains a large number of missing pixels or continuous abnormal regions. The preset model actively calls the preceding and following reference row data in the external cache as the basis for recovery. The cache control unit retrieves the adjacent image row data of the abnormal image data and the historical region data of the region where the abnormal image data is located from the data stored in the first storage area based on the coordinates of the abnormal row. The adjacent image row data consists of the valid image row data before and after the abnormal region.

[0097] Adjacent image row data and historical region data are input into a preset model. Based on this data and the spatiotemporal correlation between multiple reference regions, the preset model recalculates and reconstructs the image of the abnormal region to obtain the target data. After recovery, the system updates the corresponding image data in the second buffer region and simultaneously updates the abnormal region content in the first buffer region, thus achieving the recovery processing of large-area image abnormal regions. Historical region data includes, for example, the historical region data of the current frame. Adjacent image row data includes, for example, data from image rows adjacent to the abnormal region that are not marked as abnormal.

[0098] The process of acquiring adjacent image row data is as follows: When an abnormal region is detected, the cache control unit generates a data read request based on the frame number, row number, and pixel position information corresponding to the abnormal region, and queries the abnormal marker information in the cache unit. For the selection of reference image rows, the system prioritizes selecting adjacent image rows that are not marked as abnormal as valid image row data; when an abnormality exists in an adjacent image row, the system continues to expand the search range forward or backward until a valid reference image row that meets the conditions is found.

[0099] The current frame historical region data consists of image region data within the current image frame that has been successfully received and has not been marked as abnormal. Examples include: image regions that were normally received before the abnormal region, historical image regions saved in the buffer window, and image region data without any abnormal markers. The process for obtaining historical region data is as follows: the buffer control unit generates a corresponding storage address based on the location of the abnormal region, reads valid image line data and the current frame historical region data from the external buffer, and sends them to the recovery module as reference data for abnormal region recovery.

[0100] It should be noted that the preset model used in steps C3 and C5 can be the same recovery model. The main difference between the two steps lies in the range of input reference data. When the abnormal region is small, the recovery model mainly uses neighboring pixel data and adjacent reference row data in the cache window for recovery; when the abnormal region is large, in addition to using the data in the cache window, the recovery model further reads valid reference row data in the external cache and historical region data of the current frame as the basis for recovery. That is, the recovery model itself can remain consistent, only selecting reference data of different scales and sources based on the abnormality level, thereby improving the recovery effect under different abnormal scenarios.

[0101] In this implementation, reference data is retrieved hierarchically based on the length of the abnormal region. For small-area anomalies, neighboring pixels and spatial relationships are extracted from the cache and input into the model for prediction and repair. For large-area anomalies, adjacent valid rows and frames from the external cache are read, and the image is reconstructed based on spatiotemporal correlation. The same recovery model is reused, and only the reference data source is switched to adapt to various loss scenarios. The two-level cache is updated synchronously, which significantly improves the image repair accuracy under different fault conditions.

[0102] In some alternative implementations, the method further includes: Step D1: In the case that the target data corresponding to the abnormal image data cannot be generated, the first padding data is generated by the first preset method, wherein the first padding data has the same data length as the target data.

[0103] Step D2: Update the image data in the first storage area according to the first filling data to obtain the repaired image data, and output the repaired image data.

[0104] Specifically, the system maintains the continuity of the image frame structure throughout the image restoration process.

[0105] If the target data corresponding to the abnormal image data cannot be generated, it indicates that the abnormal area cannot be fully recovered. The system generates first padding data using a first preset method, and the first padding data has the same data length as the target data. The image data in the first storage area is updated according to the first padding data to obtain the repaired image data, ensuring that the current image row meets the preset row length requirement, thereby avoiding problems such as half-line output, frame structure interruption, or link deadlock.

[0106] The first preset method includes, for example, filling in blank pixels, padding row data, or inserting virtual pixels.

[0107] Row padding refers to the process where, when the actual number of pixels received in the current image row is less than the preset row length, the system generates compensation data based on the number of missing pixels to bring the current image row to the preset length requirement. For example, the system can use methods such as filling with fixed pixel values ​​or filling with the average value of neighboring pixels to complete the row length padding.

[0108] Virtual pixel insertion refers to generating compensation pixels based on valid pixel data before and after the anomalous region, or corresponding region data in a reference image row, and then inserting these compensation pixels into the anomalous region. For example, the system can perform interpolation calculations based on valid pixels on both sides of the anomalous region, or generate predicted pixels based on corresponding positions in the reference rows before and after, thereby forming a virtual pixel sequence and inserting it into the missing region.

[0109] By padding rows and inserting virtual pixels, the system can maintain the integrity of image row length and frame structure during anomaly recovery, avoiding problems such as half-line output, image synchronization failure, or link interruption. Finally, the repaired image data is output.

[0110] In this embodiment, the entire frame of the image is not directly discarded. Instead, local recovery errors are allowed while ensuring the integrity of the image frame structure, so as to improve the stability and continuous operation capability of the high-speed image link under abnormal conditions.

[0111] Step S305: Update the image data in the first storage area according to the target data in the second storage area to obtain the repaired image data, and output the repaired image data.

[0112] Specifically, step S305 includes: Step E1: Determine whether the repaired image data meets the preset output requirements.

[0113] Step E2: If the repaired image data meets the preset output requirements, the repaired image data is output to the display device or the subsequent image processing device.

[0114] Step E3: If the repaired image data does not meet the preset output requirements, generate reference data corresponding to the repaired image data according to the preset recovery strategy, and adjust the repaired image data according to the reference data to obtain the first intermediate data.

[0115] Step E4: If the first intermediate data meets the preset output requirements, the first intermediate data is output to the display device or the subsequent image processing device.

[0116] Step E5: If the first intermediate data does not meet the preset output requirements, the second fill data is generated through the second preset method. The first intermediate data is adjusted according to the second fill data to obtain the second intermediate data, and the second intermediate data is output to the display device or the subsequent image processing device.

[0117] Specifically, the repaired image data is rewritten to the external cache area, i.e., the second storage area. The cache control unit rereads the recovered image data according to the image output timing and outputs it to the display device or subsequent image processing device. For example, the subsequent image processing device is an ISP image preprocessing unit.

[0118] During image output, the system again determines whether the repaired image data meets the preset output requirements. For example, the system re-verifies the line length, image synchronization status, and effective image area of ​​the repaired image data. Preset output requirements include: the data length of the repaired image data meets the requirements; the repaired image data has frame structure integrity; and the repaired image data meets frame synchronization and output timing requirements.

[0119] If the repaired image data meets the preset output requirements, the repaired image data is output to the display device or subsequent image processing device. In this way, the system can maintain complete image output even in the event of local image anomalies, thereby improving the overall stability of the image chain.

[0120] If the repaired image data does not meet the preset output requirements, the system can re-execute the abnormal recovery process (steps C1 to C5 above), or perform further compensation processing on the abnormal area according to the preset recovery strategy.

[0121] Preset restoration strategies include prioritizing neighborhood restoration and reference row restoration for image inpainting. Reference data corresponding to the restored image data is generated based on the preset restoration strategy. The restored image data is then adjusted according to the reference data to obtain the first intermediate data.

[0122] If the first intermediate data meets the above-mentioned preset output requirements, the first intermediate data will be output to the display device or the subsequent image processing device.

[0123] The second preset method includes, for example, virtual pixel insertion, row data padding, reference row copying, or default pixel filling. If the first intermediate data does not meet the above preset output requirements, the system further uses the second preset method to generate second padding data. The first intermediate data is then adjusted based on the second padding data to obtain the second intermediate data, ensuring the integrity of the image row length and frame structure of the second intermediate data. The second intermediate data is then output to a display device or a subsequent image processing device.

[0124] The goal of this embodiment is to maintain continuous output of image frame structure. Therefore, even if the abnormal area cannot fully recover the original image content, the system still ensures that the output image meets the preset line length, frame synchronization and output timing requirements, thereby avoiding image link interruption, display stuttering or loss of synchronization of subsequent processing modules.

[0125] Steps S301 to S305 described above can maintain the continuity of the image frame structure in situations where local pixel loss, row data anomalies, or buffer congestion occur in the image link, avoiding the problem of whole-frame image loss or system reset caused by local anomalies in traditional solutions. By establishing a dynamic buffer window and combining it with a preset model to compensate and restore abnormal areas, the system can adaptively repair abnormal image areas based on neighboring pixels and preceding and following reference row data, thereby improving image restoration accuracy and image output stability. In addition, this application does not require interruption of the overall image transmission link during image restoration, which can effectively reduce the risk of display stuttering, image tearing, and subsequent algorithm failure in high-speed image systems, improve the system's continuous operation capability and robustness in high-resolution, high-frame-rate scenarios, and is suitable for application scenarios such as industrial vision, edge artificial intelligence, embedded vision processing, and high-speed image acquisition.

[0126] The image restoration method provided in this embodiment can maintain the continuity of image frame structure during the image restoration process, avoiding the loss of entire frames or system reset failures. This method improves image restoration accuracy and image output stability by establishing a second storage area and combining it with a preset model to compensate and restore abnormal image data. Furthermore, the restoration process does not require interruption of the overall image transmission link, effectively reducing the risk of display stuttering, image tearing, and subsequent algorithm failures in high-speed image systems. It solves the problems that image restoration easily leads to poor image frame continuity, loss of entire frames, long image restoration times, and is prone to display stuttering or algorithm malfunctions.

[0127] This embodiment also provides an image restoration apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0128] This embodiment provides an image restoration device, such as... Figure 4 As shown, it includes: The information acquisition module 401 is used to acquire image data and image data status information corresponding to the image data, and store the image data and image data status information in the first storage area; The data determination module 402 is used to determine the row data to be detected in the image data according to the image data status information, and store the row data to be detected in the second storage area; The data detection module 403 is used to perform preset detection on the data to be detected and to determine abnormal image data based on the detection results; The data repair module 404 is used to temporarily replace the abnormal image data according to preset data during the process of outputting image data in the first storage area, and generate target data corresponding to the abnormal image data in the second storage area according to the preset model and the relevant data of the abnormal image data, wherein the relevant data is included in the image data; The data output module 405 is used to update the image data in the first storage area according to the target data in the second storage area, obtain the repaired image data, and output the repaired image data.

[0129] In some alternative implementations, the data determination module 402 includes: The first determining unit is used to determine the pixel coordinate information and row position information of the row data to be detected based on the image data status information; The second determining unit is used to determine the data to be deleted from the stored data when there is stored data in the second storage area, and to delete the data to be deleted, wherein the storage capacity of the second storage area is determined according to the changes in the image data. The storage unit is used to store the row data to be detected, pixel coordinate information, and row position information into the second storage area.

[0130] In some alternative implementations, the data detection module 403 includes: The third determining unit is used to determine the number of pixels, cache status information, transmission status information and image row marker information of the row data to be detected; The detection unit is used to perform integrity detection on the data of the row to be detected based on the number of pixels, cache status information, transmission status information and image row marker information, and obtain the detection result. The setting unit is used to determine the image rows that do not meet the preset conditions based on the detection results, and to treat the image rows that do not meet the preset conditions as abnormal image data, wherein the image rows that do not meet the preset conditions are included in the data of rows to be detected; The generation unit is used to determine the frame number, row number, and first data length corresponding to the abnormal image data, and generate abnormal location information corresponding to the abnormal image data based on the frame number, row number, and first data length.

[0131] In some alternative implementations, the data repair module 404 includes: The fourth determining unit is used to determine the second data length of the abnormal image data; The first establishment unit is used to obtain the first position of the missing pixel in the abnormal image data when the second data length is less than or equal to the first preset threshold, and to establish an abnormal marker of the abnormal image data in the second storage area according to the first position. The first acquisition unit is used to acquire, when the second data length is greater than the first preset threshold, the second position of the missing pixel in the abnormal image data, the positional relationship between the abnormal image row and the adjacent image row in the abnormal image data, and to establish an abnormal marker of the abnormal image data in the second storage area according to the second position and the positional relationship. The write unit is used to write the abnormal location information into the second storage area; The second establishment unit is used to establish a state to be recovered for the image region corresponding to the abnormal image data in the first storage area, and to pause the data output of the image region. The first output unit is used to temporarily replace the data of the image area with preset data.

[0132] In some alternative implementations, the data repair module 404 includes: The fifth determining unit is used to determine the third data length of the abnormal image data; The second acquisition unit is used to acquire, in the case that the third data length is less than or equal to the second preset threshold, the neighboring pixel data of the abnormal image data and the spatial distribution relationship between the abnormal image data and the neighboring pixel data from the data stored in the second storage area, wherein the neighboring pixel data and the spatial distribution relationship are included in the relevant data. The first input unit is used to input neighborhood pixel data and spatial distribution relationships into a preset model to obtain target data; The third acquisition unit is used to acquire adjacent image row data of the abnormal image data and historical region data of the region where the abnormal image data is located from the data stored in the first storage area when the third data length is greater than the second preset threshold. The adjacent image row data and historical region data are included in the relevant data. The second input unit is used to input adjacent image row data and historical region data into a preset model to obtain target data.

[0133] In some alternative embodiments, the device further includes: The generation module is used to generate first padding data in a first preset manner when it is impossible to generate target data corresponding to abnormal image data, wherein the first padding data has the same data length as the target data; The output module is used to update the image data in the first storage area according to the first filling data, obtain the repaired image data, and output the repaired image data.

[0134] In some alternative implementations, the data output module 405 includes: The judgment unit is used to determine whether the repaired image data meets the preset output requirements; The second output unit is used to output the repaired image data to a display device or a subsequent image processing device if the repaired image data meets the preset output requirements. The adjustment unit is used to generate reference data corresponding to the repaired image data according to the preset recovery strategy if the repaired image data does not meet the preset output requirements, and adjust the repaired image data according to the reference data to obtain the first intermediate data. The third output unit is used to output the first intermediate data to a display device or a subsequent image processing device when the first intermediate data meets the preset output requirements. The fourth output unit is used to generate second fill data in a second preset manner when the first intermediate data does not meet the preset output requirements, adjust the first intermediate data according to the second fill data to obtain the second intermediate data, and output the second intermediate data to the display device or the subsequent image processing device.

[0135] The image restoration apparatus provided in this embodiment of the invention can execute the image restoration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0136] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0137] The following is a detailed reference. Figure 5The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0138] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0139] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the image restoration method of the embodiments of the present invention.

[0140] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0141] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the image restoration method shown in the above embodiments is implemented.

[0142] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0143] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An image restoration method, characterized in that, The method includes: Acquire image data and image data status information corresponding to the image data, and store the image data and image data status information in a first storage area; Based on the image data status information, determine the row data to be detected in the image data, and store the row data to be detected in the second storage area; The data to be detected is subjected to a preset detection, and abnormal image data is determined based on the detection results; During the process of outputting image data in the first storage area, the abnormal image data is temporarily replaced and output according to preset data, and in the second storage area, target data corresponding to the abnormal image data is generated according to the preset model and the relevant data of the abnormal image data, wherein the relevant data is included in the image data; The image data in the first storage area is updated according to the target data in the second storage area to obtain the repaired image data, and the repaired image data is output.

2. The method according to claim 1, characterized in that, The step of storing the row data to be detected in the second storage area includes: The pixel coordinates and row position information of the row data to be detected are determined based on the image data status information. If there is already stored data in the second storage area, data to be deleted is determined from the already stored data and the data to be deleted is deleted, wherein the storage capacity of the second storage area is determined according to the changes in the image data; The data of the row to be detected, the pixel coordinate information, and the row position information are stored in the second storage area.

3. The method according to claim 1, characterized in that, The step of performing a preset detection on the data to be detected and determining abnormal image data based on the detection results includes: Determine the number of pixels, cache status information, transmission status information, and image row marker information of the row data to be detected; Based on the number of pixels, the cache status information, the transmission status information, and the image row marker information, integrity detection is performed on the row data to be detected to obtain the detection result. Based on the detection results, image rows that do not meet the preset conditions are identified and are included in the data of the abnormal images. The frame number, row number, and first data length corresponding to the abnormal image data are determined, and the abnormal location information corresponding to the abnormal image data is generated based on the frame number, the row number, and the first data length.

4. The method according to claim 3, characterized in that, The step of temporarily replacing the abnormal image data according to preset data includes: Determine the second data length of the abnormal image data; When the length of the second data is less than or equal to the first preset threshold, the first position of the missing pixel in the abnormal image data is obtained, and an abnormal marker of the abnormal image data is established in the second storage area according to the first position. When the length of the second data is greater than the first preset threshold, the second position of the missing pixel in the abnormal image data and the positional relationship between the abnormal image row and the adjacent image row in the abnormal image data are obtained, and an abnormal marker of the abnormal image data is established in the second storage area according to the second position and the positional relationship. Write the abnormal location information into the second storage area; In the first storage area, a state to be recovered is established for the image area corresponding to the abnormal image data, and the data output of the image area is paused; The data in the image region is temporarily replaced and output according to the preset data.

5. The method according to claim 1, characterized in that, The step of generating target data corresponding to the abnormal image data in the second storage area based on a preset model and related data of the abnormal image data includes: Determine the third data length of the abnormal image data; When the length of the third data is less than or equal to the second preset threshold, the neighboring pixel data of the abnormal image data and the spatial distribution relationship between the abnormal image data and the neighboring pixel data are obtained from the data stored in the second storage area, wherein the neighboring pixel data and the spatial distribution relationship are included in the related data; The target data is obtained by inputting the neighborhood pixel data and the spatial distribution relationship into the preset model; When the length of the third data is greater than the second preset threshold, the adjacent image row data of the abnormal image data and the historical region data of the region where the abnormal image data is located are obtained from the data stored in the first storage area, wherein the adjacent image row data and the historical region data are included in the related data; The adjacent image row data and the historical region data are input into the preset model to obtain the target data.

6. The method according to claim 1, characterized in that, The method further includes: In the event that the target data corresponding to the abnormal image data cannot be generated, first padding data is generated by a first preset method, wherein the first padding data has the same data length as the target data; The image data in the first storage area is updated according to the first filling data to obtain the repaired image data, and the repaired image data is output.

7. The method according to claim 1, characterized in that, The output of the repaired image data includes: Determine whether the repaired image data meets the preset output requirements; If the repaired image data meets the preset output requirements, the repaired image data will be output to a display device or a subsequent image processing device. If the repaired image data does not meet the preset output requirements, reference data corresponding to the repaired image data is generated according to the preset recovery strategy, and the repaired image data is adjusted according to the reference data to obtain the first intermediate data. If the first intermediate data meets the preset output requirements, the first intermediate data will be output to a display device or a subsequent image processing device. If the first intermediate data does not meet the preset output requirements, second fill data is generated through a second preset method, the first intermediate data is adjusted according to the second fill data to obtain the second intermediate data, and the second intermediate data is output to the display device or the subsequent image processing device.

8. An image restoration device, characterized in that, The device includes: An information acquisition module is used to acquire image data and image data status information corresponding to the image data, and store the image data and the image data status information in a first storage area; The data determination module is used to determine the row data to be detected in the image data according to the image data status information, and store the row data to be detected in the second storage area; The data detection module is used to perform preset detection on the data to be detected and to determine abnormal image data based on the detection results; The data repair module is used to temporarily replace the abnormal image data according to preset data during the process of outputting image data in the first storage area, and to generate target data corresponding to the abnormal image data in the second storage area according to the preset model and the relevant data of the abnormal image data, wherein the relevant data is included in the image data; The data output module is used to update the image data in the first storage area according to the target data in the second storage area, to obtain the repaired image data, and to output the repaired image data.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the image restoration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the image restoration method according to any one of claims 1 to 7.

Images