Protection device and method of image data processing module, electronic device and medium

Abstract

The embodiment of the present disclosure discloses a protection device and method of an image data processing module, an electronic device and a medium, wherein the device comprises: an input monitoring module connected with an input end of the image data processing module, used for monitoring the input of the image data processing module and obtaining an input monitoring result; an output monitoring module connected with an output end of the image data processing module, used for monitoring the output of the image data processing module and obtaining an output monitoring result; and a monitoring result processing module connected with the input monitoring module and the output monitoring module respectively, used for determining the working state of the image data processing module based on the input monitoring result and the output monitoring result. The embodiment of the present disclosure can realize effective monitoring of the working state of the image data processing module, without redundant hardware setting for the image data processing module, thereby effectively reducing hardware and power consumption on the basis of ensuring the functional safety of the image data processing module.

Description

Technical Field

[0001] This disclosure relates to image processing technology, and in particular to a protection device and method for an image data processing module, electronic equipment, and medium. Background Technology

[0002] In fields such as autonomous driving, where the functional safety of image data processing modules is paramount, two sets of hardware logic for the image data processing modules are typically configured with hardware redundancy to cross-verify the hardware functionality. However, hardware redundancy results in significant hardware overhead and power consumption. Summary of the Invention

[0003] To address the aforementioned technical problems of high hardware redundancy and power consumption, this disclosure is proposed. Embodiments of this disclosure provide a protection device and method for an image data processing module, an electronic device, and a medium.

[0004] According to one aspect of the present disclosure, a protection device for an image data processing module is provided, comprising: an input monitoring module connected to the input terminal of the image data processing module, for monitoring the input of the image data processing module and obtaining input monitoring results; an output monitoring module connected to the output terminal of the image data processing module, for monitoring the output of the image data processing module and obtaining output monitoring results; and a monitoring result processing module connected to both the input monitoring module and the output monitoring module, for determining the operating state of the image data processing module based on the input monitoring results and the output monitoring results.

[0005] According to another aspect of the present disclosure, a method for protecting an image data processing module is provided, comprising: monitoring the input of the image data processing module to obtain input monitoring results; monitoring the output of the image data processing module to obtain output monitoring results; and determining the working state of the image data processing module based on the input monitoring results and the output monitoring results.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for executing the protection method of the image data processing module described in any of the above embodiments of the present disclosure; or, the storage medium is used to store data that needs to be stored by at least one hardware logic circuit of the protection device of the image data processing module described in any of the above embodiments of the present disclosure, so that the hardware logic circuit can perform the corresponding function when it is working.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the protection method for the image data processing module described in any of the above embodiments of the present disclosure; or, the electronic device comprising a protection device for the image data processing module described in any of the above embodiments; wherein at least one module of the protection device for the image data processing module is implemented by hardware logic circuitry; or, at least one unit of at least one module is implemented by hardware logic circuitry; or, at least one subunit of at least one unit is implemented by hardware logic circuitry.

[0008] Based on the protection device and method, electronic device and medium of the image data processing module provided in the above embodiments of this disclosure, by monitoring the input and output of the image data processing module, the working state of the image data processing module is determined according to the monitoring results, thereby realizing effective monitoring of the working state of the image data processing module. If an error occurs in the working state, a corresponding error alarm signal can be output. There is no need to set up redundant hardware for the image data processing module, thereby effectively reducing hardware overhead and power consumption while ensuring the functional safety of the image data processing module.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the protection device for the image data processing module provided in this disclosure;

[0012] Figure 2 This is a schematic diagram of the structure of a protection device for an image data processing module provided in an exemplary embodiment of the present disclosure;

[0013] Figure 3 This is a schematic diagram of the structure of the input monitoring module 21 provided in an exemplary embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram of the structure of the output monitoring module 22 provided in an exemplary embodiment of this disclosure;

[0015] Figure 5 This is a schematic diagram of the structure of the monitoring result processing module 23 provided in an exemplary embodiment of this disclosure;

[0016] Figure 6 This is a schematic diagram of the structure of a protection device for an image data processing module provided in another exemplary embodiment of this disclosure;

[0017] Figure 7 This is a schematic diagram of the structure of the input signal component monitoring unit 211 provided in an exemplary embodiment of this disclosure;

[0018] Figure 8 This is a schematic diagram of the structure of the output signal component monitoring unit 221 provided in an exemplary embodiment of this disclosure;

[0019] Figure 9 This is a schematic diagram of the structure of the input control signal monitoring unit 212 provided in an exemplary embodiment of this disclosure;

[0020] Figure 10 This is a schematic diagram of the structure of the output control signal monitoring unit 222 provided in an exemplary embodiment of this disclosure;

[0021] Figure 11 This is a schematic diagram of the structure of a protection device for an image data processing module provided in another exemplary embodiment of the present disclosure;

[0022] Figure 12 This is a schematic flowchart of a protection method for an image data processing module provided in an exemplary embodiment of this disclosure;

[0023] Figure 13 This is a flowchart illustrating a method for protecting an image data processing module provided in another exemplary embodiment of this disclosure;

[0024] Figure 14 This is a flowchart illustrating step 401 provided in an exemplary embodiment of this disclosure;

[0025] Figure 15 This is a flowchart illustrating step 501 provided in an exemplary embodiment of this disclosure;

[0026] Figure 16 This is a schematic diagram of the structure of one application embodiment of the electronic device disclosed herein;

[0027] Figure 17 This is a schematic diagram of another application embodiment of the electronic device disclosed herein. Detailed Implementation

[0028] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0029] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0030] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0031] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0032] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0033] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0034] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0035] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0036] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0037] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0038] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0039] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0040] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0041] This disclosure outlines

[0042] In the process of developing this disclosure, the inventors discovered that in fields such as autonomous driving, where the functional safety of image data processing modules is highly critical, two sets of hardware logic for the image data processing modules are typically configured using hardware redundancy to mutually verify each other, in order to ensure the hardware functional safety of the image data processing modules. However, hardware redundancy results in significant hardware overhead and high power consumption.

[0043] Exemplary Overview

[0044] Figure 1This is an exemplary application scenario of the protection device for the image data processing module provided in this disclosure. The protection device for the image data processing module is simply referred to as the protection device. The protected image data processing module is a hardware logic circuit or chip used to implement certain image data processing functions, such as in the field of autonomous driving, where it performs image data processing in an in-vehicle computing platform. In this example, the image data processing module is taken as an image scaling module, used to perform scaling transformations on images. Scaling transformation refers to the transformation operation of reducing or enlarging an image. Using the protection device of this disclosure, the input and output of the image data processing module can be monitored, and the working status of the image data processing module can be determined based on the monitoring results, achieving effective monitoring of the working status of the image data processing module. If an error occurs in the working status, a corresponding error alarm signal can be output. There is no need for redundant hardware settings for the image data processing module. The hardware overhead and power consumption of the protection device of this disclosure are far less than those of redundant image data processing modules, thereby effectively reducing hardware overhead and power consumption while ensuring the functional safety of the image data processing module.

[0045] Of course, the protection device disclosed herein is not only applicable to image scaling modules, but can also be applied to any image data processing module where the output image exhibits eigenvalue invariance relative to the input image. This includes modules used for image rotation transformations (pixel values ​​remain unchanged, but pixel positions rotate), translation transformations (pixel values ​​remain unchanged, but pixel positions move parallel to each other), mirror transformations, and offset transformations (irregular movements). Eigenvalue invariance refers to the property that, relative to the input image, the proportion of pixels with different brightness levels remains unchanged across the entire or a portion of the image, as well as the proportion of pixels with different chromaticity levels. For example, in image scaling, when the input image is enlarged or reduced, although the scale changes, the proportion of pixels with different brightness levels remains constant. Based on eigenvalue invariance, the input and output are monitored to protect the image data processing module and ensure the correctness of image data processing. Therefore, the protection device disclosed herein can be applied to the protection of image data processing modules in any field and any scenario exhibiting this eigenvalue invariance; no specific field or scenario is limited by this disclosure.

[0046] Exemplary device

[0047] Figure 2 This is a schematic diagram of the structure of a protection device for an image data processing module provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, in-vehicle computing platforms. Figure 2 As shown, the device includes: an input monitoring module 21, an output monitoring module 22, and a monitoring result processing module 23.

[0048] The input monitoring module 21 is connected to the input end of the image data processing module and is used to monitor the input of the image data processing module and obtain the input monitoring results.

[0049] The output monitoring module 22 is connected to the output end of the image data processing module and is used to monitor the output of the image data processing module and obtain the output monitoring results.

[0050] The monitoring result processing module 23 is connected to the input monitoring module 21 and the output monitoring module 22 respectively, and is used to determine the working status of the image data processing module based on the input monitoring results and the output monitoring results.

[0051] The image data processing module is any image processing module with invariant eigenvalues. The regions where eigenvalues ​​remain unchanged are pre-determined, and the monitoring rules for the protection device are configured accordingly, monitoring both input and output. The image data processing module's operating status can include both correct and incorrect states. A correct status indicates normal operation; otherwise, an error has occurred, requiring appropriate measures. This can include outputting an error alarm signal and reporting it to the CPU for processing, the specifics of which can be configured according to actual needs.

[0052] In an optional example, the input to the image data processing module may include input image data, and correspondingly, the output may include output image data. Accordingly, the input monitoring module 21 can monitor the input image data to obtain input monitoring results, while the output monitoring module 22 monitors the output image data to obtain corresponding output monitoring results. The monitoring result processing module 23 then determines the operating status of the image data processing module based on the input and output monitoring results.

[0053] In an optional example, the input of the image data processing module may include an input control signal, and the corresponding output may include an output control signal. Then, the input monitoring module 21 and the output monitoring module 22 monitor the input control signal and the output control signal respectively to obtain the input monitoring result and the output monitoring result.

[0054] In an optional example, the input of the image data processing module includes input image data and input control signals, and the corresponding output includes output image data and output control signals. Then, the input monitoring module 21 monitors the input image data and input control signals to obtain input monitoring results, and the output monitoring module 22 monitors the output image data and output control signals to obtain output monitoring results.

[0055] Input control signals control the input of input image data, and may include information such as the start-of-frame signal, end-of-frame signal, and line valid signal. Output control signals control the output of output image data, and may include information such as the start-of-frame signal, end-of-frame signal, and line valid signal.

[0056] The protection device for the image data processing module provided in this embodiment monitors the input and output of the image data processing module, determines the working status of the image data processing module based on the monitoring results, and achieves effective monitoring of the working status of the image data processing module. If an error occurs in the working status, a corresponding error alarm signal can be output. There is no need to set up redundant hardware for the image data processing module, thereby effectively reducing hardware overhead and power consumption while ensuring the functional safety of the image data processing module.

[0057] Figure 3 This is a schematic diagram of the structure of the input monitoring module 21 provided in an exemplary embodiment of this disclosure.

[0058] In an optional example, the input monitoring module 21 includes: an input signal component monitoring unit 211, which is used to sample the component signals of the input image data of the image data processing module according to the input control signal of the image data processing module, obtain the first pixel value under each component, and obtain the number of first pixel values ​​included in each pixel range under each component according to the pixel range to which the first pixel value under each component belongs, as the input monitoring result.

[0059] The input control signal controls the input of the input image data. This may include information such as the start-of-frame signal, end-of-frame signal, and line valid signal. The start and end of an input image frame can be determined based on the input control signal, thus enabling the sampling of component signals for each frame. A component refers to an image path. For example, if the input image data is in YUV format, it includes three components: Y, U, and V. Similarly, if the input image data is in RGB format, it includes three components: R, G, and B. The specific image data format can be set according to actual needs, and this disclosure does not limit it. A pixel value refers to the pixel feature element value, which is assigned when the image is digitized. It represents the brightness and chromaticity information of a pixel in the image. For example, in an 8-bit width, the pixel value ranges from 0 to 255, with different levels representing different brightness and chromaticity. This disclosure divides the pixel value range (e.g., 0-255) into multiple pixel ranges. The specific number of divisions can be set according to actual needs, such as dividing the 0-255 range into 16 levels (i.e., 16 pixel ranges), 32 levels, etc., and this disclosure does not limit this. After acquiring the first pixel value under each component in the input image data, each first pixel value can be classified into the corresponding pixel range. For example, if a first pixel value is 28, and a total of 16 pixel ranges are divided, including 0-15 (level 1 range), 16-31 (level 2 range), 32-47, ..., 239-255 (level 16 range), then this first pixel value belongs to the second level range 16-31. Based on this, the pixel range of each first pixel value under each component can be determined, and then the number of first pixel values ​​included in each pixel range under each component can be counted to obtain the number of first pixel values ​​included in each pixel range under each component.

[0060] The input signal component monitoring unit 211 disclosed herein can be implemented using any implementable logic circuit or software; this embodiment is not limited thereto. To ensure real-time data processing, it can be implemented using hardware logic circuits.

[0061] This disclosure divides the input image data into multiple pixel ranges, categorizes the first pixel values ​​of each component obtained from sampling into each pixel range, and counts them. This allows for the calculation of the proportion of the first pixel values ​​in different pixel ranges. The same calculation is performed on the output image data. Based on the eigenvalue invariance of image transformation, the proportion of pixels in the same pixel range under the same component of the output image data and the input image data should be the same or nearly the same. Based on this, it can be determined whether the image data processing module is processing the input image correctly. If the proportion of the output image data differs significantly from that of the input image, it indicates that an error has occurred in the image data processing module, and an error alarm signal can be issued.

[0062] Figure 4This is a schematic diagram of the structure of the output monitoring module 22 provided in an exemplary embodiment of this disclosure.

[0063] In an optional example, the output monitoring module 22 includes: an output signal component monitoring unit 221, which is used to sample the component signals of the output image data of the image data processing module according to the output control signal of the image data processing module, obtain the second pixel value under each component, and obtain the number of second pixel values ​​included in each pixel range under each component according to the pixel range to which the second pixel value under each component belongs, as the output monitoring result.

[0064] Among them, the output control signal is the signal that controls the output of the image data, which may include relevant information such as the frame start signal, frame end signal, and line valid signal of the output image data.

[0065] The working principle of the output signal component monitoring unit 221 is similar to that of the input signal component monitoring unit 211, and will not be described in detail here.

[0066] The output signal component monitoring unit 221 of this disclosure can be implemented using any feasible logic circuit or software, and this embodiment is not limited thereto. To ensure the real-time performance of data processing, it can be implemented using hardware logic circuits.

[0067] It's important to clarify that monitoring the input and output image data focuses on regions where the feature values ​​remain unchanged after image data processing. Specifically, if the feature values ​​of the entire image region remain constant, the first sampled pixel value is the pixel value of the entire input image region, and the second sampled pixel value is the pixel value of the entire output image region. However, for transformations such as translation or offset transformations that may cause changes in the feature values ​​of some image regions—for example, translation shifts part of the input image out of the current space and moves a new part into the current space—some regions in the output image may not have corresponding pixels in the input image, and vice versa. In such cases, the sampling area needs to be set according to the specific function of the image data processing. The first and second sampled pixel values ​​are the regions in the input and output images where the feature values ​​remain unchanged, respectively—that is, regions where only the pixel position changes, but the pixel value remains the same. For example, in a translation transformation, if the left region A in the input image is transformed into the right region B in the output image, then the first sampled pixel value is the pixel value of the left region A in the input image, and the second sampled pixel value is the pixel value of the right region B in the output image. The specific sampling area can be set according to actual needs, and this disclosure does not impose any restrictions.

[0068] Figure 5 This is a schematic diagram of the structure of the monitoring result processing module 23 provided in an exemplary embodiment of this disclosure.

[0069] In an optional example, the input monitoring result includes the number of first pixel values ​​included in each pixel range under each component corresponding to the input image data; the output monitoring result includes the number of second pixel values ​​included in each pixel range under each component corresponding to the output image data; the monitoring result processing module 23 includes: a first processing unit 231, a first comparison unit 232, and a second processing unit 233.

[0070] The first processing unit 231 is used to normalize the number of second pixel values ​​included in each pixel range under each component based on the scaling ratio of the output image data relative to the input image data, and obtain the normalized number of second pixel values ​​included in each pixel range under each component; the first comparison unit 232 is used to compare the number of normalized second pixel values ​​included in each pixel range under each component with the number of corresponding first pixel values, and obtain a comparison result; the second processing unit 233 is used to determine whether an error has occurred in the image data processing module based on the comparison result, and if so, output an error alarm signal.

[0071] Since the output image data is obtained by transforming the input image data, in order to determine whether there is an anomaly by comparing the number of first pixel values ​​and the number of second pixel values ​​in each pixel range under each component, the total number of second pixel values ​​needs to be normalized to be the same as the total number of first pixel values. For example, if the input image is 100*100 (10,000 pixels), and the image data processing module reduces the input image to 50*50 (2,500 pixels), then the number of second pixel values ​​included in each pixel range under each component needs to be multiplied by 4 to complete the normalization. Then, the number of normalized second pixel values ​​is compared with the number of corresponding first pixel values ​​to obtain the comparison result.

[0072] Of course, you can also compare the proportion of the first pixel value in each pixel range under each component with the proportion of the second pixel value, in which case there is no need for normalization. The specific method can be set according to actual needs.

[0073] The judgment rule for determining whether the image data processing module has made an error based on the comparison results can be set according to actual needs. For example, a quantity difference threshold can be set. When the difference between the number of normalized second pixel values ​​and the number of corresponding first pixel values ​​in a certain pixel range under a certain component exceeds the quantity difference threshold, an error is determined to have occurred.

[0074] Optionally, the first processing unit 231, the first comparison unit 232, and the second processing unit 233 in this example can all be implemented by hardware logic circuits or software. In order to ensure real-time performance, this disclosure adopts hardware logic circuit implementation.

[0075] In one optional example, Figure 6 This is a schematic diagram of the structure of a protection device for an image data processing module provided in another exemplary embodiment of the present disclosure. In this example, the device of the present disclosure further includes: a data storage module 24. The data storage module 24 is connected to the input monitoring module and the first comparison unit respectively, and is used to store the input monitoring results obtained by the input monitoring module 21; the first comparison unit 232 is used to read the number of each first pixel value from the data storage module 24 in response to the read trigger signal of the first processing unit 231, and compare it with the number of each normalized second pixel value sent by the first processing unit 231 to obtain a comparison result.

[0076] The data storage module 24 can be implemented in any feasible way, such as using a FIFO (First-In, First-Out) memory. The FIFO memory stores the number of the first pixel values ​​for each pixel range under each component. The values ​​are stored sequentially, in a first-in, first-out manner, and cyclically. Subsequent modules need to issue read request signals to read the stored statistical value information sequentially. The specific principle will not be elaborated further.

[0077] Optionally, the counting of the number of first pixel values ​​can also be achieved using a histogram counting chip, which can be set according to actual needs.

[0078] This disclosure normalizes the number of second pixel values ​​in each pixel range under each component, thereby allowing the determination of whether an error has occurred by comparing the number of first pixel values ​​in the same pixel range under the same component with the number of normalized second pixel values, which facilitates the implementation of hardware logic.

[0079] In one optional example, Figure 7 This is a schematic diagram of the structure of an input signal component monitoring unit 211 provided in an exemplary embodiment of the present disclosure. In this example, the input signal component monitoring unit 211 includes: a first sampling subunit 2111, a first judgment logic subunit 2112, a first counting subunit 2113, and a first writing subunit 2114.

[0080] The first sampling subunit 2111 is used to sample the first pixel value under each component of the input image data; the first judgment logic subunit 2112 is connected to the first sampling subunit 2111 and is used to determine the pixel range to which the first pixel value under each component belongs; the first counting subunit 2113 is connected to the first judgment logic subunit 2112 and is used to count the first pixel values ​​contained in each pixel range under each component to obtain the number of first pixel values ​​contained in each pixel range under each component; the first writing subunit 2114 is connected to the first counting subunit 2113 and is used to write the number of first pixel values ​​contained in each pixel range under each component into the data storage module 24.

[0081] The first sampling subunit 2111 samples the first pixel value of each component from the input image data according to a preset sampling rule. The sampling rule can be set according to the data format of the input image data to be processed, and will not be elaborated further. Each first pixel value is sent to the first judgment logic subunit 2112, which determines the pixel range to which the first pixel value belongs. The first counting subunit 2113 counts the number of first pixel values ​​in different pixel ranges under different components, and the counting result is written to the data storage module 24 by the first writing subunit 2114.

[0082] In this example, any one of the first sampling subunit 2111, the first judgment logic subunit 2112, the first counting subunit 2113, and the first writing subunit 2114 can be implemented in hardware or software. To ensure real-time performance, this disclosure uses hardware logic circuits for implementation. For example, the first judgment logic subunit 2112 can determine the pixel range to which the first pixel value belongs through a combination of multiple comparators, and the first counting subunit 2113 can be implemented using a counter. The specific implementation can be set according to actual needs.

[0083] In one optional example, Figure 8 This is a schematic diagram of the structure of an output signal component monitoring unit 221 provided in an exemplary embodiment of the present disclosure. In this example, the output signal component monitoring unit 221 includes: a second sampling subunit 2211, a second judgment logic subunit 2212, a second counting subunit 2213, and a first output subunit 2214.

[0084] The second sampling subunit 2211 is used to sample the second pixel value under each component of the output image data; the second judgment logic subunit 2212, connected to the second sampling subunit 2211, is used to determine the pixel range to which the second pixel value under each component belongs; the second counting subunit 2213, connected to the second judgment logic subunit 2212, is used to count the second pixel values ​​contained in each pixel range under each component to obtain the number of second pixel values ​​contained in each pixel range under each component; the first output subunit 2214, connected to the second counting subunit 2213 and the monitoring result processing module 23, is used to output the number of second pixel values ​​contained in each pixel range under each component to the monitoring result processing module 23; the monitoring result processing module 23 is used to determine the working state of the image data processing module based on the input monitoring result and the number of second pixel values ​​contained in each pixel range under each component.

[0085] The working principle of each subunit of the output signal component monitoring unit 221 in this example is similar to that of the input signal component monitoring unit 211 described above, and will not be repeated here.

[0086] Similarly, any sub-unit in this example can be implemented in either hardware or software. To ensure real-time performance, it can be implemented using hardware logic circuits.

[0087] In an optional example, the input monitoring module 21 further includes an input control signal monitoring unit 212, which monitors whether the input control signal of the image data processing module is correct, and if not, outputs a corresponding error alarm signal.

[0088] The input control signals include information such as the frame start signal, frame end signal, and row valid signal of the input image data. By monitoring the input control signals, the data frame interval time, the number of rows and columns of each frame of the input image can be detected, thereby determining whether there are errors in the frame interval time, the number of rows and columns, and an error alarm signal can be output when an error occurs.

[0089] This disclosure further improves the monitoring effect and ensures the correct operation of the image data processing module by monitoring the input control signals.

[0090] In one optional example, Figure 9 This is a schematic diagram of the structure of an input control signal monitoring unit 212 provided in an exemplary embodiment of the present disclosure. In this example, the input control signals include a first frame start signal, a first frame end signal, and a first row valid signal of the input image data; the input control signal monitoring unit 212 includes: a first frame counting subunit 2121, a first row counting subunit 2122, a first column counting subunit 2123, a first comparison logic subunit 2124, a second comparison logic subunit 2125, and a third comparison logic subunit 2126.

[0091] The first frame counting subunit 2121 is used to record the first frame interval time of the input image data according to the first frame start signal and the first frame end signal of the input control signal; the first row counting subunit 2122 is used to record the number of pixels in each row of each frame of the input image data according to the first frame start signal, the first frame end signal and the first row valid signal of the input control signal; the first column counting subunit 2123 is used to record the number of rows in each frame of the input image data according to the first frame start signal, the first frame end signal and the first row valid signal of the input control signal; the first comparison logic subunit 2124 is connected to the first frame counting subunit 2121. The first comparison logic subunit 2125, connected to the first row counting subunit 2122, is used to compare the first frame interval time with the first preset interval time, and output a corresponding error alarm signal based on the relationship between the duration of the first frame interval time and the first preset interval time; the second comparison logic subunit 2126, connected to the first column counting subunit 2123, is used to output a corresponding error alarm signal based on the relationship between the number of pixels in each row of each frame of the input image data and the first preset number of rows threshold.

[0092] The first frame start signal indicates the start of an input image frame, the first frame end signal indicates the end of an input image frame, and the first row valid signal identifies a row of data in that frame. Pulling down the first row valid signal indicates the end of a row. Based on this, the frame interval time of the input image data (called the first frame interval time) can be timed. Based on the first frame start signal, the first frame end signal, and the first row valid signal, the number of rows and columns of pixels in an image frame (i.e., the number of pixels per row) can be counted. Specifically, the first row valid signal identifies the content of a row, and counting the pixel values ​​in that row determines the number of pixels in each row. Pulling up (rising edge) the first row valid signal indicates the start of a row, and pulling down (falling edge) indicates the end of a row. Thus, the number of rows in the image frame can be counted based on the pull-up and pull-down of the first row valid signal. The specific principle of triggering the counting with different signals will not be elaborated further. A frame loss error is determined when the interval between any first frames exceeds a first preset interval. An image column error is determined when the number of pixels in any row of any frame is not equal to a first preset threshold. An image row error is determined when the number of rows in any frame is not equal to a first preset threshold. Any error will generate a corresponding error alarm signal so that relevant personnel can take appropriate measures in a timely manner.

[0093] The first frame counting subunit 2121 resets the count of the frame interval time at the beginning of each frame, the first row counting subunit 2122 resets the count of the number of pixels in each row at the end of each row, and the first column counting subunit 2123 resets the count of the number of rows in each frame at the end of each frame.

[0094] In this example, any one of the following subunits—first frame counting subunit 2121, first row counting subunit 2122, first column counting subunit 2123, first comparison logic subunit 2124, second comparison logic subunit 2125, and third comparison logic subunit 2126—can be implemented in hardware or software. To ensure real-time performance, hardware logic circuits can be used.

[0095] This disclosure determines whether an error has occurred by counting and comparing the frame interval time, the number of rows and columns per frame of the input image data, thereby further effectively monitoring the correctness of the image data processing module, promptly detecting errors, and taking corresponding measures.

[0096] In an optional example, the output monitoring module 22 further includes an output control signal monitoring unit 222, which monitors whether the output control signal of the image data processing module is correct, and if not, outputs a corresponding error alarm signal.

[0097] The specific working principle of the output control signal monitoring unit 222 is similar to that of the input control signal monitoring unit 212, and will not be described in detail here.

[0098] In one optional example, Figure 10 This is a schematic diagram of the structure of the output control signal monitoring unit 222 provided in an exemplary embodiment of the present disclosure. In this example, the output control signals include a second frame start signal, a second frame end signal, and a second row valid signal for output image data; the output control signal monitoring unit 222 includes: a second frame counting subunit 2221, a second row counting subunit 2222, a second column counting subunit 2223, a fourth comparison logic subunit 2224, a fifth comparison logic subunit 2225, and a sixth comparison logic subunit 2226.

[0099] The second frame counting subunit 2221 is used to record the second frame interval time of the output image data according to the second frame start signal and the second frame end signal of the output control signal; the second row counting subunit 2222 is used to record the number of pixels in each row of each frame of the output image data according to the second frame start signal, the second frame end signal and the second row valid signal of the output control signal; the second column counting subunit 2223 is used to record the number of rows in each frame of the output image data according to the second frame start signal, the second frame end signal and the second row valid signal of the output control signal; the fourth comparison logic subunit 2224 is connected to the second frame counting subunit 2221. The second frame interval time is compared with the second preset interval time, and the corresponding error alarm signal is output based on the relationship between the duration of the second frame interval time and the second preset interval time; the fifth comparison logic subunit 2225, connected to the second row counting subunit 2222, is used to output the corresponding error alarm signal based on the relationship between the number of pixels in each row of each frame of the output image data and the second preset number threshold; the sixth comparison logic subunit 2226, connected to the second column counting subunit 2223, is used to output the corresponding error alarm signal based on the relationship between the number of rows in each frame of the output image data and the second preset number of rows threshold.

[0100] In this example, the specific operation of each subunit of the output control signal monitoring unit 222 is similar to that of the aforementioned input control signal monitoring unit 212, and will not be described in detail here.

[0101] Any sub-unit in this example can be implemented by hardware or software. To ensure real-time performance, hardware logic circuits can be used.

[0102] This disclosure determines whether an error has occurred by counting and comparing the frame interval time, the number of rows and columns per frame of the output image data, thereby further effectively monitoring the correctness of the image data processing module, promptly detecting errors, and taking corresponding measures.

[0103] In one optional example, Figure 11This is a schematic diagram of the structure of a protection device for an image data processing module provided in another exemplary embodiment of the present disclosure. In this example, the input signal component monitoring unit 211 samples the component signals of the input image data of the image data processing module according to the input control signal of the image data processing module, obtains the first pixel value under each component, and obtains the number of first pixel values ​​included in each pixel range under each component according to the pixel range to which the first pixel value under each component belongs, and stores it in the data storage module 24 as the input monitoring result; the output signal component monitoring unit 221 samples the component signals of the output image data of the image data processing module according to the output control signal of the image data processing module, obtains the second pixel value under each component, and obtains the number of second pixel values ​​included in each pixel range under each component according to the pixel range to which the second pixel value under each component belongs, and outputs it to the first processing unit as the output monitoring result. 231. The first processing unit 231 normalizes the number of second pixel values ​​included in each pixel range under each component based on the scaling ratio of the output image data relative to the input image data, obtains the normalized number of second pixel values ​​included in each pixel range under each component, and outputs it to the first comparison unit 232, and sends a read trigger signal to the first comparison unit 232; the first comparison unit 232 responds to the read trigger signal of the first processing unit 231, reads the number of each first pixel value from the data storage module 24, compares it with the number of each normalized second pixel value sent by the first processing unit 231, obtains the comparison result, and outputs it to the second processing unit 233; the second processing unit 233 determines whether an error has occurred in the image data processing module based on the comparison result, and if so, outputs an error alarm signal. The input control signal monitoring unit 212 monitors whether the input control signal of the image data processing module is correct, and if not, outputs the corresponding error alarm signal. The output control signal monitoring unit 222 monitors whether the output control signal of the image data processing module is correct, and if not, outputs the corresponding error alarm signal.

[0104] In an optional example, to ensure the real-time performance of the protection device, all modules, units under each module, and sub-units under each unit of the protection device disclosed herein are implemented using hardware logic circuits. This effectively reduces hardware and power consumption while ensuring the functional safety of the image data processing module, and guarantees the real-time performance of error alarms. In applications, this avoids serious consequences caused by untimely error alarms. For example, in the field of autonomous driving, it allows for real-time and effective monitoring of the image data processing module of the onboard computing platform, timely detection of errors, and prevention of danger, thereby effectively improving safety. Pure hardware implementation requires hardware initialization, i.e., initializing the pixel range, the sampling area of ​​the input image, the sampling area of ​​the output image, various thresholds, and other relevant information. After initialization, it can run automatically without software intervention.

[0105] Exemplary methods

[0106] Figure 12 This is a flowchart illustrating a protection method for an image data processing module provided in an exemplary embodiment of this disclosure. This method can be implemented using the apparatus of any of the above embodiments. Figure 12 As shown, the method disclosed herein may include the following steps:

[0107] Step 301: Monitor the input of the image data processing module and obtain the input monitoring results.

[0108] Step 302: Monitor the output of the image data processing module and obtain the output monitoring results.

[0109] Step 303: Determine the working status of the image data processing module based on the input monitoring results and the output monitoring results.

[0110] The specific operation of each step of the method disclosed herein has been described in detail in the foregoing device embodiments, and will not be repeated here.

[0111] Figure 13 This is a flowchart illustrating a method for protecting an image data processing module provided in another exemplary embodiment of this disclosure.

[0112] In an optional example, the input to the monitoring image data processing module in step 301, obtaining the input monitoring results, includes:

[0113] Step 3011: Sample the component signals of the input image data of the image data processing module according to the input control signal of the image data processing module to obtain the first pixel value of each component.

[0114] Step 3012: Based on the pixel range to which the first pixel value under each component belongs, obtain the number of first pixel values ​​included in each pixel range under each component, and use this as the input monitoring result.

[0115] In an optional example, the output of the monitoring image data processing module in step 302 obtains the output monitoring results, including:

[0116] Step 3021: Sample the component signals of the output image data of the image data processing module according to the output control signal of the image data processing module to obtain the second pixel value of each component.

[0117] Step 3022: Based on the pixel range to which the second pixel value belongs under each component, obtain the number of second pixel values ​​included in each pixel range under each component, and use this as the output monitoring result.

[0118] In an optional example, the input monitoring result includes the number of first pixel values ​​included in each pixel range under each component of the input image data; the output monitoring result includes the number of second pixel values ​​included in each pixel range under each component of the output image data; correspondingly, step 303, based on the input monitoring result and the output monitoring result, determines the working state of the image data processing module, including:

[0119] Step 3031: Based on the scaling ratio of the output image data relative to the input image data, normalize the number of second pixel values ​​included in each pixel range under each component to obtain the normalized number of second pixel values ​​included in each pixel range under each component.

[0120] Step 3032: Compare the number of normalized second pixel values ​​included in each pixel range under each component with the number of corresponding first pixel values ​​to obtain the comparison result.

[0121] Step 3033: Determine whether an error has occurred in the image data processing module based on the comparison result. If so, output an error alarm signal.

[0122] In an optional example, after step 301, the method of this disclosure further includes: storing the input monitoring results to a data storage module.

[0123] Accordingly, step 3032 includes: reading the number of each first pixel value from the data storage module, comparing it with the number of each normalized second pixel value, and obtaining a comparison result.

[0124] In an optional example, step 3012, which obtains the number of first pixel values ​​included in each pixel range under each component based on the pixel range to which the first pixel value under each component belongs, and uses it as input monitoring results, includes: determining the pixel range to which the first pixel value under each component belongs; counting the first pixel values ​​included in each pixel range under each component to obtain the number of first pixel values ​​included in each pixel range under each component, and using it as input monitoring results.

[0125] In an optional example, step 3022, which obtains the number of second pixel values ​​included in each pixel range under each component based on the pixel range to which the second pixel value under each component belongs, includes: determining the pixel range to which the second pixel value under each component belongs; counting the second pixel values ​​included in each pixel range under each component to obtain the number of second pixel values ​​included in each pixel range under each component; step 303 is specifically used to determine the working status of the image data processing module based on the input monitoring results and the number of second pixel values ​​included in each pixel range under each component.

[0126] In one optional example, the methods disclosed herein also include:

[0127] Step 401: Check if the input control signal of the monitoring image data processing module is correct. If not, output the corresponding error alarm signal.

[0128] Steps 401 and 301 are not in any particular order.

[0129] In one optional example, Figure 14 This is a flowchart illustrating step 401 provided in an exemplary embodiment of the present disclosure. In this example, the input control signals include a first frame start signal, a first frame end signal, and a first line valid signal for the input image data; step 401 includes:

[0130] Step 4011: Record the first frame interval time of the input image data according to the first frame start signal and the first frame end signal of the input control signal.

[0131] Step 4012: Based on the first frame start signal, the first frame end signal, and the first line valid signal of the input control signal, record the number of pixels in each line of each frame of the input image data.

[0132] Step 4013: Record the number of rows in each frame of the input image data according to the first frame start signal, the first frame end signal and the first line valid signal of the input control signal.

[0133] Steps 4011-4013 are not in any particular order.

[0134] Step 4014: Compare the first frame interval time with the first preset interval time, and output the corresponding error alarm signal based on the duration relationship between the first frame interval time and the first preset interval time.

[0135] Step 4015: Based on the relationship between the number of pixels in each row of each frame of the input image data and the first preset number threshold, output the corresponding error alarm signal.

[0136] Step 4016: Based on the relationship between the number of rows in each frame of the input image data and the first preset row number threshold, output the corresponding error alarm signal.

[0137] Steps 4014-4016 are not in any particular order.

[0138] In one optional example, the methods disclosed herein also include:

[0139] Step 501: Check if the output control signal of the monitoring image data processing module is correct. If not, output the corresponding error alarm signal.

[0140] Steps 501 and 302 are not in any particular order.

[0141] In one optional example, Figure 15 This is a flowchart illustrating step 501 provided in an exemplary embodiment of the present disclosure. In this example, the output control signals include a second frame start signal, a second frame end signal, and a second line valid signal for outputting image data; step 501 includes:

[0142] Step 5011: Record the second frame interval time of the output image data according to the second frame start signal and the second frame end signal of the output control signal.

[0143] Step 5012: Based on the second frame start signal, second frame end signal and second line valid signal of the output control signal, record the number of pixels in each line of each frame of the output image data.

[0144] Step 5013: Based on the second frame start signal, second frame end signal and second line valid signal of the output control signal, record the number of rows in each frame of the output image data.

[0145] Step 5014: Compare the second frame interval time with the second preset interval time, and output the corresponding error alarm signal based on the duration relationship between the second frame interval time and the second preset interval time.

[0146] Step 5015: Based on the relationship between the number of pixels in each row of each frame of the output image data and the second preset number threshold, output the corresponding error alarm signal.

[0147] Step 5016: Based on the relationship between the number of rows in each frame of the output image data and the second preset row number threshold, output the corresponding error alarm signal.

[0148] The protection method for any image data processing module provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, the protection method for any image data processing module provided in this disclosure can be executed by a processor, such as by a processor executing the protection method for any image data processing module mentioned in this disclosure by calling corresponding instructions stored in memory. Alternatively, the protection method for any image data processing module provided in this disclosure can be executed by a hardware logic circuit device. Further details will not be elaborated below.

[0149] Exemplary electronic devices

[0150] This disclosure also provides an electronic device, including: a memory for storing computer programs;

[0151] A processor is configured to execute a computer program stored in the memory, and when the computer program is executed, to implement the protection method of the image data processing module described in any of the above embodiments of this disclosure.

[0152] Alternatively, the electronic device may include a protection device for the image data processing module as provided in any of the above embodiments, to implement the protection method for the image data processing module described in any of the above embodiments of this disclosure.

[0153] Wherein, at least one module of the protection device of the image data processing module is implemented by hardware logic circuit; or, at least one unit of at least one module is implemented by hardware logic circuit; or, at least one sub-unit of at least one unit is implemented by hardware logic circuit.

[0154] To improve real-time performance, each module, each unit under each module, and each sub-unit under each unit in the device are implemented through hardware logic circuits, meaning the entire device is a hardware logic circuit.

[0155] Figure 16 This is a schematic diagram of an application embodiment of the electronic device disclosed herein. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0156] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0157] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0158] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0159] For example, the input device 13 can be the microphone or microphone array described above, used to capture the input signal of the sound source; the input device 13 can also be an image acquisition sensor, such as a camera, used to acquire image data.

[0160] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0161] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0162] Of course, for the sake of simplicity, Figure 16 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0163] Figure 17 This is a schematic diagram of another application embodiment of the electronic device disclosed herein. In this example, the electronic device includes a protection device for the image data processing module as provided in any of the above embodiments or examples, and may also include other related devices, which will not be described in detail here. At least one module in the protection device for the image data processing module is implemented by hardware logic circuitry; or, at least one unit in at least one module is implemented by hardware logic circuitry; or, at least one subunit in at least one unit is implemented by hardware logic circuitry.

[0164] For example, in order to improve real-time performance, all modules in the protection device of the image data processing module are implemented through hardware logic circuits.

[0165] Exemplary computer program products and computer-readable storage media

[0166] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0167] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0168] Furthermore, embodiments of this disclosure may also be computer-readable storage media having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0169] The storage medium disclosed herein may also store data that needs to be stored by at least one hardware logic circuit of the protection device for the image data processing module provided in the "Exemplary Device" section of this specification, so that the hardware logic circuit can perform the corresponding function during operation. For example, the storage medium may be a register of the hardware logic circuit, storing initialization configuration data, or storing data that needs to be stored during operation, without specific limitations. The at least one hardware logic circuit may be a hardware logic circuit of a module in the device, a hardware logic circuit of a unit under a module, or a hardware logic circuit of a sub-unit under a unit, without specific limitations.

[0170] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0171] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0173] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0174] The apparatus and methods of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0175] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0176] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0177] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A protection device for an image data processing module, the image data processing module being used to implement image data processing functions, the image data processing module having eigenvalue invariance, the protection device comprising: An input monitoring module is connected to the input end of the image data processing module. The input monitoring module includes an input signal component monitoring unit, which is used to sample the component signals of the input image data of the image data processing module according to the input control signal of the image data processing module and the sampling area set in advance according to the image data processing function, to obtain the first pixel value under each component, and to obtain the number of first pixel values ​​included in each pixel range under each component according to the pixel range to which the first pixel value under each component belongs, as the input monitoring result. The sampling area is a region where the feature value is unchanged, and the input monitoring result includes the number of first pixel values ​​included in each pixel range under each component corresponding to the region where the feature value is unchanged in the input image data. An output monitoring module is connected to the output end of the image data processing module. The output monitoring module includes an output signal component monitoring unit, which is used to sample the component signals of the output image data of the image data processing module according to the output control signal of the image data processing module and the sampling area, to obtain the second pixel value under each component, and to obtain the number of second pixel values ​​included in each pixel range under each component according to the pixel range to which the second pixel value under each component belongs, as the output monitoring result. The output monitoring result includes the number of second pixel values ​​included in each pixel range under each component corresponding to the region in the output image data where the feature value remains unchanged. The monitoring result processing module is connected to the input monitoring module and the output monitoring module respectively, and is used to determine the working status of the image data processing module based on the input monitoring results and the output monitoring results; The monitoring result processing module includes: The first processing unit is configured to normalize the number of second pixel values ​​included in each pixel range under each component based on the scaling ratio of the output image data relative to the input image data, so as to obtain the normalized number of second pixel values ​​included in each pixel range under each component. The first comparison unit is used to compare the number of normalized second pixel values ​​included in each pixel range under each component with the number of corresponding first pixel values ​​to obtain a comparison result; The second processing unit is used to determine the working status of the image data processing module based on the comparison result.

2. The apparatus according to claim 1, wherein, The input signal component monitoring unit includes: The first sampling subunit is used to sample the first pixel value under each component of the input image data; The first judgment logic subunit is connected to the first sampling subunit and is used to determine the pixel range to which the first pixel value under each component belongs; The first counting subunit, connected to the first judgment logic subunit, is used to count the first pixel values ​​contained in each pixel range under each component, and obtain the number of first pixel values ​​contained in each pixel range under each component. The first writing subunit, connected to the first counting subunit, is used to write the number of first pixel values ​​included in each pixel range under each component into the data storage module.

3. The apparatus according to claim 1, wherein, The input monitoring module also includes: The input control signal monitoring unit is used to monitor whether the input control signal of the image data processing module is correct. If not, it outputs the corresponding error alarm signal.

4. The apparatus according to claim 3, wherein, The input control signal includes a first frame start signal, a first frame end signal, and a first line valid signal of the input image data; the input control signal monitoring unit includes: The first frame counting subunit is used to record the first frame interval time of the input image data according to the first frame start signal and the first frame end signal of the input control signal; The first row counting subunit is used to record the number of pixels in each row of each frame of the input image data according to the first frame start signal, the first frame end signal and the first row valid signal of the input control signal. The first column counting subunit is used to record the number of rows in each frame of the input image data according to the first frame start signal, the first frame end signal and the first row valid signal of the input control signal. The first comparison logic subunit is connected to the first frame counting subunit and is used to compare the first frame interval time with the first preset interval time, and output a corresponding error alarm signal based on the duration relationship between the first frame interval time and the first preset interval time. The second comparison logic subunit, connected to the first row counting subunit, is used to output a corresponding error alarm signal based on the relationship between the number of pixels in each row of each frame of the input image data and a first preset number threshold. The third comparison logic subunit, connected to the first column counting subunit, is used to output a corresponding error alarm signal based on the relationship between the number of rows in each frame of the input image data and a first preset row number threshold.

5. The apparatus according to claim 1, wherein, The output signal component monitoring unit includes: The second sampling subunit is used to sample the second pixel value under each component of the output image data; The second judgment logic subunit is connected to the second sampling subunit and is used to determine the pixel range to which the second pixel value under each component belongs; The second counting subunit, connected to the second judgment logic subunit, is used to count the second pixel values ​​contained in each pixel range under each component, and obtain the number of second pixel values ​​contained in each pixel range under each component. The first output subunit is connected to the second counting subunit and the monitoring result processing module, and is used to output the number of second pixel values ​​included in each pixel range under each component to the monitoring result processing module; The monitoring result processing module is used to determine the working status of the image data processing module based on the input monitoring result and the number of second pixel values ​​included in each pixel range under each component.

6. The apparatus according to claim 5, wherein, The output monitoring module also includes: The output control signal monitoring unit is used to monitor whether the output control signal of the image data processing module is correct. If not, it outputs the corresponding error alarm signal.

7. The apparatus according to claim 1, wherein, The second processing unit is specifically used to determine whether the image data processing module has encountered an error based on the comparison result. If so, it outputs an error alarm signal.

8. A method for protecting an image data processing module, the image data processing module being used to implement image data processing functions, the image data processing module having eigenvalue invariance, the method comprising: The process of monitoring the input of the image data processing module and obtaining input monitoring results includes: sampling the component signals of the input image data of the image data processing module according to the input control signal of the image data processing module and the sampling area pre-set according to the image data processing function, obtaining the first pixel value under each component, and obtaining the number of first pixel values ​​included in each pixel range under each component according to the pixel range to which the first pixel value under each component belongs, as the input monitoring result. The sampling area is a region where the feature value remains unchanged, and the input monitoring result includes the number of first pixel values ​​included in each pixel range under each component corresponding to the region where the feature value remains unchanged in the input image data. Monitoring the output of the image data processing module to obtain an output monitoring result includes: sampling the component signals of the output image data of the image data processing module according to the output control signal of the image data processing module and the sampling area, obtaining the second pixel value under each component, and obtaining the number of second pixel values ​​included in each pixel range under each component according to the pixel range to which the second pixel value under each component belongs, as the output monitoring result, wherein the output monitoring result includes the number of second pixel values ​​included in each pixel range under each component corresponding to the region in the output image data where the feature value remains unchanged; Based on the input monitoring results and the output monitoring results, the working status of the image data processing module is determined; Determining the operating status of the image data processing module based on the input monitoring results and the output monitoring results includes: Based on the scaling ratio of the output image data relative to the input image data, the number of second pixel values ​​included in each pixel range under each component is normalized to obtain the normalized number of second pixel values ​​included in each pixel range under each component. The number of normalized second pixel values ​​included in each pixel range under each component is compared with the number of corresponding first pixel values ​​to obtain a comparison result; Based on the comparison results, the working status of the image data processing module is determined.

9. A computer-readable storage medium storing a computer program for executing the protection method of the image data processing module according to claim 8; or, The storage medium is used to store data that needs to be stored by at least one hardware logic circuit of the protection device of the image data processing module as described in any one of claims 1-7, so that the hardware logic circuit can perform the corresponding function when it is working.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the protection method of the image data processing module according to claim 8; or, The electronic device includes a protection device for the image data processing module as described in any one of claims 1-7; Wherein, at least one module of the protection device of the image data processing module is implemented through hardware logic circuitry; or, At least one unit in at least one module is implemented through hardware logic circuitry; or, At least one sub-unit of at least one unit is implemented by hardware logic circuitry.

Images