A method, system, apparatus, and storage medium for image display fault tolerance

Abstract

The application provides a method, system, device and storage medium for image display fault tolerance, the method comprising: in response to memory failure, receiving original data input in a first-in first-out queue; selecting a mode of interpolation calculation according to calculation accuracy, and determining to-be-processed data in the original data input that needs to be subjected to interpolation calculation and replacement; judging whether the to-be-processed data is the first data in the first-in first-out queue; and in response to the to-be-processed data not being the first data in the first-in first-out queue, performing interpolation calculation and replacement on the to-be-processed data according to the mode of interpolation calculation. The application can replace failed data with an interpolation calculation result of related data under the condition that part of memory units fails, form approximation and fuzzification processing of corresponding image pixel points with small resource loss, and effectively reduce the influence of partial failure of a memory device.

Description

Technical Field

[0001] This invention relates to the field of chip design, and more specifically, to a method, system, apparatus, and storage medium for image display fault tolerance. Background Technology

[0002] With the development of technologies such as computer vision and artificial intelligence image recognition, there are increasingly more scenarios requiring the transmission and processing of images (including videos and pictures), and the corresponding data volume is also increasing, posing a challenge to the reliability of chips that support image display functions. The image display function, as commonly referred to, does not involve image data acquisition, optimization, compression / decompression, etc.; it refers to sending received data to the display device according to pre-set standard requirements (often requiring some data format conversion). Because the image display function is relatively independent and executes in parallel with the preceding image acquisition and processing operations, the data sent to the image display function chip (or chip module) is first stored in a specific FIFO (First-In-First-Out) queue within the chip.

[0003] All storage structures are ultimately implemented using standard storage devices (SRAM (Static Random Access Memory), FLASH, etc.). These storage devices are inherently susceptible to damage or failure, especially under frequent, high-speed access operations. Furthermore, storage device failures are often unpredictable; the specific timing and sequence of failures of different storage cells manufactured using the same process and used in the same storage structure are unpredictable. Clearly, in a FIFO structure, if some storage cells fail, the corresponding image display will often exhibit regular display errors, such as a persistently distorted image. For human visual recognition, a small distorted area might not cause serious problems, but for machine recognition, it could lead to drastically different results.

[0004] Currently, in the mass production of chips, various methods are used to enhance the failure detection and testing of memory devices during the design phase and the manufacturing process to screen out defective chips. However, memory device failures that occur during use after the chips leave the factory can generally only be mitigated by adding redundant spare memory devices to reduce the risk of needing to replace the chip. Additionally, image displays often support multiple resolutions; in the event of an error, the use of memory devices can be reduced by manually or automatically lowering the resolution.

[0005] Adding redundant backup storage devices presents two problems: first, it increases chip cost; second, the increased number of memory devices integrated into a single chip inherently introduces a higher probability of errors in the manufacturing process. More importantly, due to the unpredictable nature of memory device failures, backup storage devices are unlikely to provide targeted fault tolerance. Simply reducing resolution, on the other hand, offers little flexibility and is unlikely to solve the problem efficiently. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method, system, computer device, and computer-readable storage medium for image display fault tolerance. This invention can replace the failed data with the interpolation calculation results of its related data when some memory units fail, with less resource consumption, to form the approximation and blurring of the corresponding image pixels. This effectively reduces the impact of partial memory device failure and avoids the situation where chips or even the entire subsystem have to be replaced due to small-scale memory device failures, thereby improving the fault tolerance and reliability of chips and subsystems.

[0007] Based on the above objectives, one aspect of the present invention provides a method for image display fault tolerance, comprising the following steps: in response to a memory failure, receiving raw data input from a first-in-first-out (FIFO) queue; selecting an interpolation calculation mode according to the calculation precision, and determining the data to be processed in the input raw data that needs to be interpolated and replaced; determining whether the data to be processed is the first data in the FIFO queue; and in response to the data to be processed not being the first data in the FIFO queue, performing interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode.

[0008] In some implementations, the step of interpolating and replacing the data to be processed according to the interpolation calculation pattern includes: in response to adopting the nearest neighbor interpolation method pattern, selecting the previous data of the data to be processed as the value of the data to be processed.

[0009] In some implementations, the method further includes setting the data to be processed to a preset value in response to the data to be processed being the first data in the first-in-first-out queue.

[0010] In some implementations, the method further includes: determining the number of cache registers based on the pattern of the difference calculation, and setting the bit width of the cache registers to be consistent with the bit width of the first-in-first-out queue.

[0011] In another aspect of this invention, a system for image display fault tolerance is provided, comprising: a receiving module configured to receive raw data input from a first-in-first-out (FIFO) queue in response to a memory failure; a selection module configured to select an interpolation calculation mode based on calculation precision and determine the data to be processed in the input raw data that requires interpolation calculation and replacement; a judging module configured to judge whether the data to be processed is the first data in the FIFO queue; and an execution module configured to perform interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode in response to the data to be processed not being the first data in the FIFO queue.

[0012] In some implementations, the execution module is configured to: select the previous data of the data to be processed as the value of the data to be processed in response to adopting the nearest neighbor interpolation mode.

[0013] In some implementations, the system further includes a setting module configured to: set the data to be processed to a preset value in response to the data to be processed being the first data in the first-in-first-out queue.

[0014] In some implementations, the system further includes a second setting module configured to: determine the number of cache registers according to the pattern of the difference calculation, and set the bit width of the cache registers to be consistent with the bit width of the first-in-first-out queue.

[0015] In another aspect of the present invention, a computer device is provided, comprising: at least one processor; and a memory storing computer instructions executable on the processor, the instructions, when executed by the processor, implementing the steps of the method described above.

[0016] In another aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method steps.

[0017] The present invention has the following beneficial technical effects: in the event of partial memory cell failure, it can replace the failed data with the interpolation calculation results of its related data with a small amount of resource consumption, forming an approximation and blurring of the corresponding image pixels, effectively reducing the impact of partial memory device failure, avoiding the situation where a chip or even the entire subsystem has to be replaced due to a small range of memory device failures, and improving the fault tolerance and reliability of the chip and subsystem. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an embodiment of the image display fault-tolerant method provided by the present invention;

[0020] Figure 2 A schematic diagram of an embodiment of the image display fault-tolerant system provided by the present invention;

[0021] Figure 3 A schematic diagram of the hardware structure of an embodiment of the computer device for image display fault tolerance provided by the present invention;

[0022] Figure 4 This is a schematic diagram of an embodiment of a computer storage medium for image display fault tolerance provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0024] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0025] In a first aspect, an embodiment of a method for image display fault tolerance is proposed. Figure 1 The diagram shown is a schematic representation of an embodiment of the image display fault-tolerant method provided by the present invention. Figure 1 As shown, the embodiments of the present invention include the following steps:

[0026] S1. In response to a memory failure, receive the raw data input from the first-in-first-out queue;

[0027] S2. Select the interpolation calculation mode according to the calculation accuracy, and determine the data to be processed in the input original data that needs to be interpolated and replaced;

[0028] S3. Determine whether the data to be processed is the first data in the first-in-first-out queue; and

[0029] S4. In response to the fact that the data to be processed is not the first data in the first-in-first-out queue, perform interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode.

[0030] FIFO (First-In-First-Out) is a fundamental structure in digital systems and is widely used in design. Compared to conventional storage structures, it provides data access in a fixed order, eliminates the need for address encoding and decoding, and is simpler and faster, making it ideal for temporary storage of serial data streams.

[0031] In image data storage and transmission formats, multi-dimensional attributes of a single pixel are typically used as units. For example, RGB format uses three components: Red, Green, and Blue, while YUV uses three components: Y (luminance), U, and V (U and V represent chrominance). The values ​​of these three components represent all the information of that pixel to be expressed on the display device. The temporary storage of these values ​​in the image display module follows different rules depending on the design. Some use all the information of a single pixel as the unit, temporarily storing it in a single FIFO (packed format). Others use multiple FIFOs to temporarily store different component information separately (planar format). Currently, planar or semi-planar formats are more commonly used (in YUV, Y has a separate FIFO, while UV shares a FIFO for cross-storage), temporarily storing each component of a single pixel as the unit. Regardless of the method, there must be a fixed order rule for pixel arrangement (because a two-dimensional image is expressed using a one-dimensional vector).

[0032] This invention addresses the needs of graphic display, particularly machine recognition, by adding a set of configurable fault-tolerant units to the FIFO used for temporary image data storage. When image data errors occur, a portion of the FIFO data can be replaced with the interpolation calculation results of its related data, resulting in approximation and blurring of the corresponding image pixels. This reduces the impact of storage device failures in the absence of large-scale storage device failures.

[0033] There is no unified standard for the storage rules and order of temporary data in the image display module, and this invention does not limit this either. Simplifying the unit values ​​in the FIFO can simplify subsequent interpolation calculations, making the entire function more efficient. Therefore, the following implementation descriptions are based on a planar format, and changing the format and storage order rules will not fundamentally affect the implementation of the method of this invention. When the fault-tolerant unit is not selected, it does not affect data processing and transmission; when selected, all data will first enter the fault-tolerant unit.

[0034] In response to a memory failure, the system receives raw data from a first-in-first-out (FIFO) queue. It selects the interpolation calculation mode based on the required precision and identifies the data in the raw input that needs interpolation and replacement.

[0035] The fault-tolerant unit includes a configuration register, which comprises two parts: a mode register, used to control different interpolation calculation modes, and a selection register, used to select the data in the FIFO data that needs to be interpolated and replaced. The configuration register is read-only within the module, meaning it can only be written to externally. Interpolation is an important method for approximating discrete functions. It allows us to estimate the approximate value of a function at other points based on its values ​​at a finite number of points. Simply put, interpolation can be used to estimate missing or lost data based on neighboring data. The most common interpolation algorithms include nearest neighbor interpolation, bilinear interpolation (calculating a linear average using the four nearest pixels), and bicubic interpolation (using a weighted average of the sixteen nearest pixels), etc. In this embodiment, the simplest nearest neighbor interpolation mode is used, so a mode register is not required. If more interpolation modes are needed in practice, more register bits can be selected during the design phase. An M*N bit selection register is used, where the Xth bit is 1, indicating that the Xth data in the FIFO output needs to be interpolated and replaced.

[0036] Determine whether the data to be processed is the first data in the first-in-first-out queue. If the data to be processed is not the first data in the first-in-first-out queue, perform interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode.

[0037] In some implementations, the step of interpolating and replacing the data to be processed according to the interpolation calculation pattern includes: in response to adopting the nearest neighbor interpolation method pattern, selecting the previous data of the data to be processed as the value of the data to be processed.

[0038] Interpolation is a relatively mature technology, and various calculation modes exist depending on the requirements. Generally, using more preceding and following data leads to more accurate interpolation results. However, for the scenario addressed by this invention—where a small number of pixels are lost (due to the failure of the corresponding storage device)—a particularly precise result is not required. Furthermore, more preceding and following data and more calculations result in greater time and resource consumption. Therefore, overly complex calculation methods are not recommended, although the method of this invention can be implemented with sufficient register and computing resources. This embodiment of the invention uses the simplest nearest neighbor interpolation method, directly selecting the previous data as the value of the current data.

[0039] In some implementations, the method further includes: in response to the data to be processed being the first data in the first-in-first-out queue, setting the data to be processed to a preset value. If the first data in the FIFO is lost, no calculation is needed; a fixed value is directly taken. This is because, as the outermost pixel, its importance is inherently low, and generally it is not worthwhile to modify the calculation logic separately for it.

[0040] It's important to note that this simplified calculation method doesn't even require any interpolation logic to perform any calculations. However, when faced with consecutive pixel loss, the interpolated image data will suffer significant distortion. For stronger fault tolerance, more complex interpolation calculations and more registers must be introduced during the design phase, and the interpolation logic will need additional computational functions. This is a trade-off issue in chip design, and this invention does not impose any limitations on it.

[0041] In some implementations, the method further includes: determining the number of cache registers based on the pattern of the difference calculation, and setting the bit width of the cache registers to be consistent with the bit width of the first-in-first-out queue.

[0042] The fault-tolerant unit includes a data buffer register group. This group is used to buffer preceding data for interpolation calculations. Obviously, the bit width of this register group must match the bit width of a single data item in the FIFO (typically, image data components range from 0 to 255, i.e., 8 bits wide). The number of data buffer registers needs to be consistent with the interpolation calculation mode. In this embodiment, the interpolation calculation only requires the preceding data item, so only one data buffer register is needed. If more complex interpolation calculations are required in practice, more registers can be selected during the design. Clearly, this approach will delay the overall data output by one clock cycle. However, because it operates in a pipelined manner and utilizes the high-speed internal clock of the chip, the impact of two clock cycles on the overall image display is negligible (for example, with a common 500MHz clock frequency, the corresponding two clock cycles are 2 nanoseconds, a frequency far exceeding the approximately 100Hz refresh rate of image display devices).

[0043] The embodiments of the present invention can replace the failed data with the interpolation calculation results of its related data when some memory cells fail, with less resource consumption, to form the approximation and blurring of the corresponding image pixels. This effectively reduces the impact of partial failure of memory devices and avoids the situation where a chip or even the entire subsystem has to be replaced due to a small range of memory device failures, thereby improving the fault tolerance and reliability of the chip and subsystem.

[0044] It should be particularly noted that the steps in the various embodiments of the above-described method for image display fault tolerance can be interchanged, substituted, added, or deleted. Therefore, these reasonable permutations and combinations of the method for image display fault tolerance should also fall within the protection scope of this invention, and the protection scope of this invention should not be limited to the embodiments.

[0045] Based on the above objectives, a second aspect of the present invention provides a system for image display fault tolerance. For example... Figure 2 As shown, system 200 includes the following modules: a receiving module configured to receive raw data input from a first-in-first-out (FIFO) queue in response to a memory failure; a selection module configured to select an interpolation calculation mode based on calculation precision and determine the data to be processed in the input raw data that needs to be interpolated and replaced; a judging module configured to judge whether the data to be processed is the first data in the FIFO queue; and an execution module configured to perform interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode in response to the data to be processed not being the first data in the FIFO queue.

[0046] In some implementations, the execution module is configured to: select the previous data of the data to be processed as the value of the data to be processed in response to adopting the nearest neighbor interpolation mode.

[0047] In some implementations, the system further includes a setting module configured to: set the data to be processed to a preset value in response to the data to be processed being the first data in the first-in-first-out queue.

[0048] In some implementations, the system further includes a second setting module configured to: determine the number of cache registers according to the pattern of the difference calculation, and set the bit width of the cache registers to be consistent with the bit width of the first-in-first-out queue.

[0049] Based on the above objectives, a third aspect of the present invention provides a computer device, comprising: at least one processor; and a memory storing computer instructions executable by the processor to perform the following steps: S1, in response to a memory failure, receiving raw data input from a first-in-first-out queue; S2, selecting an interpolation calculation mode according to the calculation precision, and determining the data to be processed in the input raw data that needs to be interpolated and replaced;

[0050] S3. Determine whether the data to be processed is the first data in the first-in-first-out queue; and S4. In response that the data to be processed is not the first data in the first-in-first-out queue, perform interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode.

[0051] In some implementations, the step of interpolating and replacing the data to be processed according to the interpolation calculation pattern includes: in response to adopting the nearest neighbor interpolation method pattern, selecting the previous data of the data to be processed as the value of the data to be processed.

[0052] In some implementations, the method further includes setting the data to be processed to a preset value in response to the data to be processed being the first data in the first-in-first-out queue.

[0053] In some implementations, the method further includes: determining the number of cache registers based on the pattern of the difference calculation, and setting the bit width of the cache registers to be consistent with the bit width of the first-in-first-out queue.

[0054] like Figure 3 The diagram shown is a hardware structure schematic of an embodiment of the computer device for image display fault tolerance provided by the present invention.

[0055] For example Figure 3 Taking the device shown as an example, the device includes a processor 301 and a memory 302.

[0056] Processor 301 and memory 302 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0057] The memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the image display fault-tolerant method in the embodiments of this application. The processor 301 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 302, thereby implementing the image display fault-tolerant method.

[0058] Memory 302 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the method for image display fault tolerance, etc. Furthermore, memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 302 may optionally include memory remotely located relative to processor 301, and these remote memories may be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0059] One or more computer instructions 303 corresponding to the image display fault tolerance method are stored in memory 302. When executed by processor 301, the image display fault tolerance method in any of the above method embodiments is executed.

[0060] Any embodiment of the computer device that performs the above-described method for image display fault tolerance can achieve the same or similar effects as any of the aforementioned method embodiments.

[0061] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs a method for image display fault tolerance.

[0062] like Figure 4 The diagram shown is a schematic representation of an embodiment of the computer storage medium for image display fault tolerance provided by the present invention. Figure 4 Taking the computer storage medium shown as an example, the computer-readable storage medium 401 stores a computer program 402 that, when executed by a processor, performs the above method.

[0063] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for the image display fault-tolerant method can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0064] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0065] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0066] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0067] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0068] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for image display fault tolerance, characterized in that, Includes the following steps: In response to a memory failure, it receives the raw data from the first-in-first-out queue. The interpolation calculation mode is selected based on the calculation accuracy, and the data to be processed in the input raw data that needs to be interpolated and replaced is determined. Determine whether the data to be processed is the first data in the first-in-first-out queue; and In response to the fact that the data to be processed is not the first data in the first-in-first-out queue, the data to be processed is interpolated and replaced according to the interpolation calculation mode.

2. The method according to claim 1, characterized in that, The step of interpolating and replacing the data to be processed according to the interpolation calculation pattern includes: In response to the use of the nearest neighbor interpolation method, the previous data of the data to be processed is selected as the value of the data to be processed.

3. The method according to claim 1, characterized in that, The method also includes: In response to the fact that the data to be processed is the first data in the first-in-first-out queue, the data to be processed is set to a preset value.

4. The method according to claim 1, characterized in that, The method also includes: The number of cache registers is determined according to the interpolation calculation mode, and the bit width of the cache registers is set to be consistent with the bit width of the first-in-first-out queue.

5. A system for image display fault tolerance, characterized in that, include: A receiving module configured to receive raw data input from a first-in-first-out queue in response to a memory failure; The selection module is configured to select the interpolation calculation mode based on the calculation accuracy, and to determine the data to be processed in the input raw data that needs to be interpolated and replaced; The judgment module is configured to determine whether the data to be processed is the first data in the first-in-first-out queue; as well as The execution module is configured to perform interpolation calculation and replacement on the data to be processed according to the interpolation calculation mode in response to the data to be processed not being the first data in the first-in-first-out queue.

6. The system according to claim 5, characterized in that, The execution module is configured to: In response to the use of the nearest neighbor interpolation method, the previous data of the data to be processed is selected as the value of the data to be processed.

7. The system according to claim 5, characterized in that, The system also includes a settings module for configuration: In response to the fact that the data to be processed is the first data in the first-in-first-out queue, the data to be processed is set to a preset value.

8. The system according to claim 5, characterized in that, The system also includes a second settings module, configured for: The number of cache registers is determined according to the interpolation calculation mode, and the bit width of the cache registers is set to be consistent with the bit width of the first-in-first-out queue.

9. A computer device, characterized in that, include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method according to any one of claims 1-4.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images