Image processing method, device, electronic device and readable storage medium

Abstract

The image processing method, device, electronic device and readable storage medium provided by the embodiments of the present invention are as follows: after the FPGA and the MCU perform a handshake connection, the FPGA generates first verification information according to the first generation method, and the MCU generates second verification information according to the second generation method and sends it to the FPGA. The FPGA determines whether the second verification information is the same as the first verification information; if they are the same and the update conditions are not met, the image data to be processed sent by the SOC is continuously received and processed using the first processing method. After determining that the update conditions are met, the FPGA / MCU updates the first / second generation method respectively and generates the first / second verification information again. In this way, the FPGA must work in conjunction with a legitimate MCU to operate normally, effectively protecting the technical secrets in the FPGA code from being cracked or stolen.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and in particular to an image processing method, device, electronic equipment and readable storage medium. Background Art

[0002] Field Programmable Gate Array (FPGA) devices are a type of semi-custom circuit within application-specific integrated circuits (ASICs) and are widely used in fields such as communications, image processing, and artificial intelligence. Currently, mainstream FPGA chips use an external memory architecture, where the FPGA code is stored and loaded from the external memory each time the FPGA is powered on.

[0003] Trade secrets play a crucial role in product market competitiveness, and therefore protecting the trade secrets involved in products has always been a key focus in the field of electronic information technology. For audio-visual devices like televisions and smartphones, image quality processing algorithms are a core trade secret. Due to the operating characteristics of FPGAs, image processing in audio-visual devices is typically implemented using FPGAs. However, for FPGA chips with external memory, since the code for the image quality processing algorithm is stored in the external memory, crackers can directly read the code from the external memory and copy it for pirated use. Summary of the Invention

[0004] Embodiments of the present invention provide an image processing method, device, electronic device, and readable storage medium, which are used to protect the working principle of FPGA from being misused.

[0005] In a first aspect, an embodiment of the present invention provides an image processing method applied to a field programmable gate array (FPGA), comprising:

[0006] After performing a handshake connection with the microcontroller unit MCU, generating first verification information according to a first generation method, and receiving second verification information sent by the MCU;

[0007] determining whether the second verification information is the same as the first verification information;

[0008] If the judgment result is the same and the update condition is not met, then continue to receive the image data to be processed sent by the system-on-chip (SOC), and process the image data to be processed using the first processing method;

[0009] After determining that the update condition is met, updating the first generation method, and returning to the step of generating the first verification information according to the first generation method;

[0010] The first generation method used after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection.

[0011] Optionally, the method further includes:

[0012] Sending the judgment result to the MCU;

[0013] In response to a reset instruction sent by the MCU, the working state is reset, and a handshake connection with the MCU is reestablished; wherein the reset instruction is an instruction sent by the MCU when it determines that the working state of the FPGA is abnormal, and the number of consecutive determinations of the abnormal working state of the FPGA is less than a threshold value;

[0014] If it is determined that the first processing method cannot be used for image processing, and the number of times it is continuously determined that the first processing method cannot be used for image processing is equal to a threshold, then ignoring the image data to be processed sent by the SOC; or continuously receiving the image data to be processed sent by the SOC, and processing the image data to be processed using the second processing method;

[0015] Wherein, it is determined that the first processing method cannot be used for image processing after one of the following situations occurs:

[0016] The judgment result is different;

[0017] The second verification information is not received within a first set time period after the first verification information is generated;

[0018] The handshake connection with the MCU fails;

[0019] The MCU determines that the FPGA operating state is abnormal after one of the following situations occurs:

[0020] The received judgment result is different;

[0021] The determination result is not received within a second set time period after the second verification information is sent;

[0022] The handshake connection with the FPGA failed.

[0023] Optionally, the method further includes:

[0024] Receiving a clock synchronization signal sent by the MCU at every preset time interval;

[0025] The clock signal thereof is corrected according to the clock synchronization signal.

[0026] In a second aspect, based on the same inventive concept, an embodiment of the present invention further provides an image processing method applied to an MCU, wherein the MCU includes a memory, the memory of the MCU is set to a read-protected state, and the MCU shields a debugging function;

[0027] The method comprises:

[0028] After establishing a handshake connection with the FPGA, second verification information is generated according to a second generation method and sent to the FPGA; the FPGA determines whether to use the first processing method to process the image data to be processed sent by the SOC based on a determination result of whether the second verification information is identical to the first verification information generated by the FPGA itself; wherein the first second generation method adopted after establishing a handshake connection with the MCU is determined based on initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection;

[0029] When the update condition is met, the second generation method is updated, and the process returns to the step of generating the second verification information according to the second generation method.

[0030] Optionally, the method further includes:

[0031] Determining that the working state of the FPGA is abnormal, and the number of consecutive determinations that the working state of the FPGA is abnormal is less than a threshold, resetting the working state of the MCU and sending a reset instruction to the FPGA to reset the working state of the FPGA;

[0032] Reconnect with the FPGA by handshake;

[0033] The FPGA operating state is determined to be abnormal when one of the following situations occurs:

[0034] The judgment result received from the FPGA is different;

[0035] The determination result is not received within a second set time period after the second verification information is sent;

[0036] The handshake connection with the FPGA failed.

[0037] Optionally, the method further includes:

[0038] At preset time intervals, a clock synchronization signal is sent to the FPGA, so that the FPGA corrects its own clock signal according to the clock synchronization signal.

[0039] In a third aspect, based on the same inventive concept, an embodiment of the present invention further provides a data processing device, including:

[0040] A receiving unit, configured to generate first verification information according to a first generation method after performing a handshake connection with the MCU, and receive second verification information sent by the MCU;

[0041] a verification unit, configured to determine whether the second verification information is the same as the first verification information;

[0042] a first processing unit, configured to, if the judgment result is the same and the update condition is not met, continue to receive the image data to be processed sent by the SOC, and process the image data to be processed using a first processing method;

[0043] an updating unit, configured to update the first generation method after determining that the update condition is satisfied, and return to the step of generating the first verification information according to the first generation method;

[0044] The first generation method used after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection.

[0045] In a fourth aspect, based on the same inventive concept, an embodiment of the present invention further provides a data processing device, wherein the memory of the image processing device is set to a read-protected state, and the image processing device shields a debugging function;

[0046] The device comprises:

[0047] A verification unit is configured to generate second verification information according to a second generation method and send the second verification information to the FPGA after a handshake connection is established with the FPGA; the FPGA is configured to determine whether to use the first processing method to process the image data to be processed sent by the SOC based on a determination result of whether the second verification information is identical to the first verification information generated by the FPGA itself; wherein the first second generation method used after a handshake connection with the MCU is determined based on initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during a handshake connection;

[0048] The updating unit is configured to update the second generation method and return to the step of generating the second verification information according to the second generation method when the update condition is met.

[0049] In a fifth aspect, based on the same inventive concept, an embodiment of the present invention further provides an electronic device, comprising: an MCU, an FPGA, an external memory, and a SOC;

[0050] Wherein, the MCU includes a memory;

[0051] The MCU is connected to the FPGA via a first bus, the FPGA is connected to the external memory via a second bus, and the SOC is connected to the FPGA;

[0052] The SOC is used to continuously send image data to be processed to the FPGA, the FPGA is used to implement the data processing method as described in the first aspect, and the MCU is used to implement the data processing method as described in the second aspect.

[0053] In a sixth aspect, based on the same inventive concept, an embodiment of the present invention further provides a readable storage medium, wherein the readable storage medium stores FPGA code, and the FPGA code is used to implement the data processing method as described in the first aspect;

[0054] Alternatively, the readable storage medium stores a computer program, and the computer program is used to implement the data processing method as described in the second aspect.

[0055] The beneficial effects of the present invention are as follows:

[0056] The image processing method, device, electronic device, and readable storage medium provided by the embodiments of the present invention utilize an FPGA and an MCU to generate and compare verification information according to the same rules. During operation, the MCU and FPGA continuously update the method for generating verification information, ensuring that the device must work in conjunction with a legitimate MCU for normal operation. Furthermore, by setting the MCU's memory to read-protected mode and disabling the MCU's debugging function, the method for generating verification information between the FPGA and MCU is prevented from being disclosed, effectively protecting the technical secrets contained in the FPGA code from being cracked or misused. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of an application scenario of an electronic device provided by an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0059] Figure 3 A second structural diagram of an electronic device provided by an embodiment of the present invention;

[0060] Figure 4 A third structural diagram of an electronic device provided by an embodiment of the present invention;

[0061] Figure 5 A fourth structural diagram of an electronic device provided in an embodiment of the present invention;

[0062] Figure 6One of the flow charts of the image processing method implemented by the electronic device provided in an embodiment of the present invention;

[0063] Figure 7 A second flowchart of the image processing method implemented by the electronic device provided in an embodiment of the present invention;

[0064] Figure 8 Flowchart 3 of the image processing method implemented by the electronic device provided in an embodiment of the present invention;

[0065] Figure 9 This is a schematic diagram showing the effect of FPGA processing image data in the second processing mode according to an embodiment of the present invention;

[0066] Figure 10 This is a second schematic diagram of the effect of FPGA processing image data in the second processing mode according to an embodiment of the present invention;

[0067] Figure 11 This is a third schematic diagram of the effect of FPGA processing image data in the second processing mode according to an embodiment of the present invention;

[0068] Figure 12 This is a fourth schematic diagram of the effect of FPGA processing image data in the second processing mode according to an embodiment of the present invention;

[0069] Figure 13 One of the flowcharts of the image processing method applied to FPGA provided by an embodiment of the present invention;

[0070] Figure 14 The second flowchart of the image processing method applied to FPGA provided by the embodiment of the present invention;

[0071] Figure 15 Flowchart 3 of the image processing method applied to FPGA provided by an embodiment of the present invention;

[0072] Figure 16 One of the flowcharts of the image processing method applied to MCU provided by an embodiment of the present invention;

[0073] Figure 17 The second flowchart of the image processing method applied to MCU provided by the embodiment of the present invention;

[0074] Figure 18 This is a schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;

[0075] Figure 19 A second structural diagram of an image processing device provided by an embodiment of the present invention;

[0076] Figure 20A schematic structural diagram of a readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention will be further described below with reference to the accompanying drawings and examples. However, the example embodiments can be implemented in various forms and should not be understood as being limited to the embodiments described herein; on the contrary, these embodiments are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the example embodiments to those skilled in the art. The same figure marks in the figures represent the same or similar structures, and their repeated descriptions will be omitted. The words expressing position and direction described in the present invention are all explained with reference to the accompanying drawings as examples, but changes can be made as needed, and the changes made are all included in the scope of protection of the present invention. The drawings of the present invention are only used to illustrate the relative position relationship and do not represent the true proportion.

[0078] It should be noted that specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The subsequent description of the specification is a preferred embodiment of the present application, but the description is for the purpose of illustrating the general principles of the present application and is not intended to limit the scope of the present application. The scope of protection of the present application shall be determined as defined by the appended claims.

[0079] The image processing method, device, electronic device, and readable storage medium provided by the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0080] In a first aspect, an embodiment of the present invention provides an electronic device, which may be a television, a computer, a mobile phone, an audio and video player, or the like. Figure 1 is a schematic diagram of an operation scenario between an electronic device and a control device according to an embodiment of the present application, such as Figure 1 As shown, a user can operate electronic device 200 via mobile terminal 300 and control device 100. Control device 100 can be a remote controller, and communication between the remote controller and the electronic device includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control electronic device 200. The user can control electronic device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc. In some embodiments, mobile terminals, tablet computers, computers, laptop computers, and other smart devices can also be used to control electronic device 200.

[0081] like Figures 2 to 5As shown, the electronic device includes a microcontroller unit (MCU), an FPGA, an external memory, and a system on chip (SOC);

[0082] Wherein, the MCU includes a memory;

[0083] The MCU is connected to the FPGA via a first bus, the FPGA is connected to the external memory via a second bus, and the SOC is connected to the FPGA;

[0084] The SOC is used to continuously send image data to be processed to the FPGA, and the FPGA and the MCU are used to collaboratively implement the image processing method described below.

[0085] During implementation, the electronic devices may vary significantly due to different configurations or performance. For example, the memory of the MCU may also include one or more operating systems, such as Windows, Mac OS, Linux, iOS, Android, Unix, FreeBSD, etc. The electronic devices may also include one or more power supplies (not shown in the figure); one or more network interfaces (not shown in the figure), including wired network interfaces and / or wireless network interfaces; and one or more input / output interfaces (not shown in the figure).

[0086] As an optional implementation, Figure 2 、 Figure 5 As shown, the SOC is directly connected to the FPGA via a third bus.

[0087] If the electronic device has a frequency multiplication function, then Figure 3 、 Figure 4 As shown, as another optional implementation, the SOC is connected to a frame rate conversion (FRC) chip via a fourth bus, and the FRC chip is connected to the FPGA via a fifth bus; the FRC chip is used to perform frequency doubling processing on the image data to be processed that is sent by the SOC to the FPGA.

[0088] The image processing method provided by the embodiment of the present invention is described below. If both the MCU and the FPGA are legitimate devices, the memory of the MCU is set to a read-protected state, and the MCU blocks the debugging function (for example, the debugging function is disabled in the computer program running on the MCU, or the debugging function is customized and modified to be unable to read the computer program stored in the memory). Figures 6 to 8 As shown, the image processing method specifically includes:

[0089] S1. MCU controls FPGA power-on.

[0090] S2. FPGA loads code from external memory for initialization.

[0091] S3. The MCU and the FPGA perform a handshake connection. During the handshake connection process, one of the MCU and the FPGA randomly generates initial synchronization information and sends it to the other.

[0092] S5-1. FPGA determines a first generation method according to initial synchronization information.

[0093] S5-2. The MCU determines a second generation method according to the initial synchronization information.

[0094] S6-1. FPGA generates first verification information according to a first generation method.

[0095] S6-2. The MCU generates second verification information according to the second generation method.

[0096] S7. The MCU sends the second verification information to the FPGA.

[0097] S9. FPGA determines whether the second verification information is the same as the first verification information.

[0098] If the result of step S9 is the same, step S10 is executed.

[0099] exist Figure 6 In the illustrated case, the MCU and FPGA are both legal devices as an example, so the judgment result of step S7 is the same.

[0100] S10. The FPGA continuously receives the image data to be processed sent by the SOC, and processes the image data to be processed using a first processing method.

[0101] After completing the processing of the image data to be processed, the FPGA sends the processed image data to the corresponding structure.

[0102] If the electronic device has the function of directly playing audio and video images, then Figure 4 As shown, optionally, the electronic device further includes a timing controller (TCON) and a display device; the TCON is connected to the FPGA via a sixth bus, and the display device is connected to the TCON via a seventh bus;

[0103] The TCON is used to receive the processed image data sent by the FPGA and convert it into a display signal and send it to the display device;

[0104] The display device is used to display an image according to the display signal sent by the TCON.

[0105] In a specific implementation process, the display device may include but is not limited to an organic electroluminescence display (OLED), a liquid crystal display (LCD), etc., which is not limited here.

[0106] If the electronic device does not have the function of directly playing audio and video images, then Figure 5 As shown, optionally, the electronic device also includes an image data output interface, which is connected to the FPGA via an eighth bus and is used to send processed image data to an external device connected to the image output interface, so that the external device displays an image based on the image data.

[0107] In a specific implementation process, the first bus to the eighth bus can be a Universal Asynchronous Receiver / Transmitter (UART) bus, a Serial Peripheral Interface (SPI) bus, a two-wire serial (Inter-Integrated Circuit, I 2 C) bus, etc., not limited here.

[0108] S14-1: After the update condition is met, the FPGA updates the first generation mode and returns to step S6-1.

[0109] S14-2: After the update condition is met, the MCU updates the second generation method and returns to step S6-2.

[0110] In a specific implementation process, the update condition may be reaching a preset update time interval; for the FPGA, it may also be receiving the second verification information sent by the MCU again; this is not limited here.

[0111] During the specific implementation process, if both the MCU and the FPGA are legitimate devices, the same set of generation methods will be pre-set in the computer program of the MCU and the code of the FPGA, including multiple different generation methods. The FPGA and the MCU will determine the same first generation method based on the same initial synchronization information, that is, the first generation method and the second generation method in the above steps are the same generation method. After the update conditions are met, the FPGA and the MCU will simultaneously update the first generation method / second generation method according to the same rules, so that the subsequently updated first generation method and the second generation method are consistent, and the first verification information generated is the same as the second verification information. The FPGA will process the image data to be processed sent by the MCU each time based on the latest first verification information and the latest second verification information being the same. For example, a set of polynomial queues are pre-set in the computer program of the MCU and the code of the FPGA, and the polynomial index number in the queue is selected for the first use according to the initial synchronization information. Then, the polynomial corresponding to the index number obtained recursively according to a specific rule based on the initial synchronization information is used to calculate the verification information. For example, N polynomials are pre-set in both the MCU's computer program and the FPGA's code. Based on the initial synchronization information, the first polynomial indexed is n. The polynomial selected for the mth iteration can then be the polynomial with index (m×n+1)%N. This ensures randomness in both the initial index and subsequent recursive polynomial indexes, making it difficult to crack.

[0112] It should be pointed out that the first processing method involved in the embodiment of the present invention is the core technical secret of the audio and video playback device, so it is not explained in this application. Those skilled in the art can set it according to actual needs.

[0113] Since the FPGA code in the external memory is a binary file, it is difficult for crackers to reversely infer the working principle of the FPGA code, so the first processing method in the FPGA code is difficult to crack at the working principle level.

[0114] Furthermore, a cracker might abandon the principle of cracking the first processing method and instead directly copy the code from the external memory and port it to an illegal device for use. Thus, with the above-mentioned image processing method, the FPGA must work in conjunction with a legitimate MCU in order to utilize the first processing method. If the FPGA code from the external memory is copied to an illegal device, and the illegal MCU is unaware of the existence of a verification mechanism between the legitimate MCU and the FPGA, the illegal MCU will be unable to provide the second verification information when the initial synchronization information is generated by the FPGA. Furthermore, the illegal MCU will be unable to provide the initial synchronization information when the initial synchronization information is generated by the MCU, and thus the FPGA will be unable to compare the verification information. Therefore, the first processing method can be avoided when the comparison is impossible. Even if the illegal MCU is aware of the existence of a verification mechanism between the legitimate MCU and the FPGA and obtains the initial synchronization information generated by the FPGA or generates and provides it to the FPGA, the legitimate MCU's memory is set to read-protected mode and the legitimate MCU's debugging function is disabled. This prevents the cracker from obtaining the MCU's computer program and thus learning the specific verification method for the verification information. Consequently, the cracker cannot provide the correct second verification information during the subsequent verification process. However, the FPGA can still be prevented from utilizing the first processing method to process the image.

[0115] Furthermore, in addition to verifying with the FPGA, the MCU in this solution also controls the FPGA. Therefore, the communication between the MCU and the FPGA includes not only the information required for verification but also other information (such as MCU control instructions for FPGA pins). This makes it more difficult for crackers to attempt to defraud the verification mechanism by eavesdropping on the communication between the MCU and FPGA and then using the intercepted information to send to the FPGA.

[0116] In this way, the solution provided by the embodiment of the present invention can effectively protect the first processing method in the FPGA code from being cracked or stolen.

[0117] During the verification process between MCU and FPGA, in addition to verification failures caused by cracking or theft, there is also a certain possibility that the generated verification information will be incorrect due to hardware interference in the working environment, clock frequency asynchrony, etc. between MCU and FPGA. The verification information error caused by this situation is a recoverable failure different from the case of cracking or theft. Some means can be used to recover it to avoid affecting the normal operation of the device. Optionally, such as Figures 7 and 8 As shown, after step S9, the method further includes:

[0118] S11. The FPGA sends a determination result of whether the first verification information and the second verification information are the same to the MCU.

[0119] The method further comprises:

[0120] S12. The MCU determines whether the determination result is received within a second set time period after sending the second verification information.

[0121] If the result of step S12 is yes, execute step S13; if the result of step S12 is no, execute step S15.

[0122] S13. The MCU determines whether the received determination result indicates that the second verification information is different from the first verification information.

[0123] If the result of step S13 is no, execute step S14-2; if the result of step S13 is yes, execute step S15.

[0124] S15. The MCU determines that the FPGA working state is abnormal, and determines whether the number of consecutive determinations of the FPGA working state being abnormal is less than a threshold.

[0125] If the result of step S15 is yes, step S16 is executed.

[0126] S16, MCU resets the working status.

[0127] S17: The MCU sends a reset instruction to the FPGA, and the process returns to step S3.

[0128] S18: The FPGA resets its working state in response to the reset instruction of the MCU, and then returns to step S3.

[0129] In a specific implementation process, the MCU and FPGA may reset their own working states by resetting only registers and program functions related to generating the first verification information / second verification information, or reinitializing the MCU / FPGA, which is not limited here.

[0130] After step S3, the method further includes:

[0131] After step S3, the method further includes:

[0132] S4-1. FPGA determines whether the handshake connection with MCU is successful.

[0133] If the result of step S4-1 is yes, execute step S5-1; if the result of step S4-1 is no, execute step S20.

[0134] S4-2, MCU determines whether the handshake connection with FPGA is successful.

[0135] If the result of step S4-2 is yes, execute step S5-2; if the result of step S4-2 is no, execute step S15.

[0136] In this way, the MCU can be reset after verification fails or verification times out without receiving feedback from the FPGA, thereby eliminating errors caused by accidents.

[0137] Correspondingly, before step S9, the method further includes:

[0138] S8. The FPGA determines whether the second verification information is received within a first set time period after the first verification information is generated.

[0139] If the result of step S8 is yes, execute step S9; if the result of step S8 is no, execute step S20.

[0140] The method further comprises:

[0141] If the result of step S9 is different, step S20 is executed.

[0142] S20 , the FPGA ignores the received image data to be processed; or continues to receive the image data to be processed sent by the SOC, and processes the image data to be processed using the second processing method.

[0143] In a specific implementation process, the second processing method is an image processing method different from the first processing method, for example, the image quality of the image to be processed may be degraded (eg Figure 9 As shown, mosaic is added to the image to be processed; or as Figure 10 As shown, reduce the resolution of the image to be processed), add graphics to the image to be processed (for example Figure 11 As shown, a graphic is overlaid on the image to be processed and the text "Equipment failure, please contact official after-sales service as soon as possible!" is displayed in it, and a preset pattern (such as Figure 12 If the FPGA ignores the received image data to be processed, the MCU will not be able to receive the processed image data and thus will not be able to output the image to the display device.

[0144] Furthermore, in order to prevent occasional recoverable errors from affecting the normal operation of the electronic device, optionally, as Figure 8 As shown, before executing step S20, the method further includes: S19, the FPGA determines that the first processing method cannot be used for image processing, and determines whether the number of consecutive determinations that the image processing cannot be performed is equal to a threshold.

[0145] If the result of step S19 is yes, step S20 is executed.

[0146] Almost at the same time when the result of FPGA executing step S19 is yes and executing step S20 is no, the MCU executes step S15 and obtains a no result, so that the FPGA continues to ignore the image data sent by the SOC or processes the image data sent by the SOC in the second processing method, so that after multiple consecutive verification failures, the FPGA cannot process the image data in the first processing method, thereby protecting the first processing method in the FPGA from being stolen.

[0147] When the device is operating for a long time, the clock frequency of the MCU and FPGA may be out of sync, which may cause the clock frequency to accumulate and affect the generation of verification information by both parties, resulting in verification failure. In order to correct the clock frequency error, the method may also include:

[0148] Every third set time interval, the MCU sends a clock synchronization signal to the FPGA.

[0149] The FPGA corrects its own clock signal according to the clock synchronization signal sent by the MCU.

[0150] During the specific implementation process, the FPGA may further send a clock synchronization feedback signal to the MCU after the correction, so as to feedback to the MCU that the clock signal synchronization is successful.

[0151] In this way, the clock frequency error between the MCU and FPGA can be continuously corrected through the regular clock heartbeat mechanism to avoid inconsistency in the verification information generated by both parties due to clock error.

[0152] Furthermore, an embodiment of the present invention also provides a voltage management mechanism to prevent FPGA from being damaged due to voltage anomalies.

[0153] Before step S1, the method further includes (the following steps are not shown in the figure):

[0154] S1-1. The MCU controls the voltage management device to power on. The voltage management device is used to control the input voltage to the FPGA.

[0155] S1-2. The MCU determines whether the input voltage of the voltage management device is stable and whether the input voltage of the voltage management device reaches the threshold voltage.

[0156] If the result of step S1-2 is yes, execute step S1-3; if the result of step S1-2 is no, continue waiting until the result is yes.

[0157] S1-3, MCU controls the voltage management device to output the operating voltage to the FPGA.

[0158] S1-4, the MCU determines whether the voltage received by the FPGA is within the normal range.

[0159] In a specific implementation process, the MCU may utilize an analog-to-digital converter (ADC) sampling method, or use a voltage sensor to monitor the rated voltage received by the FPGA.

[0160] If the result of step S1-4 is yes, execute step S3; if the result of step S1-4 is no, execute step S1-5.

[0161] S1-5, the MCU controls the voltage management device to stop outputting voltage to the FPGA.

[0162] Based on the same inventive concept, embodiments of the present invention further provide image processing methods applied to FPGA and MCU respectively.

[0163] In a second aspect, an embodiment of the present invention further provides an image processing method, which is applied to FPGA, such as Figure 13-15 Shown, including:

[0164] S101, loading code from external memory for initialization.

[0165] S102 , performing a handshake connection with the microcontroller unit MCU. During the handshake connection process, one of the MCU and the FPGA randomly generates initial synchronization information and sends it to the other.

[0166] S104: Generate first verification information according to the first generation method, and receive second verification information sent by the MCU.

[0167] The first generation method used after handshaking with the MCU is determined according to the initial synchronization information.

[0168] S105: Determine whether the second verification information is received within a first set time period after the first verification information is generated.

[0169] If the result of step S105 is yes, step S106 is executed.

[0170] S106: Determine whether the second verification information is the same as the first verification information.

[0171] If the judgment result of step S106 is yes, step S107 is executed.

[0172] S107: Determine whether the update condition is met.

[0173] If the result of step S107 is no, execute step S108; if the result of step S107 is yes, execute step S109.

[0174] S108: Continue to receive the image data to be processed sent by the SOC, and process the image data to be processed using the first processing method. Return to step S107.

[0175] S109: Update the first generation method for generating the first verification information. Return to step S104.

[0176] Alternatively, as Figure 14-15 As shown, after step S102, the method further includes:

[0177] S103: Determine whether the handshake connection with the MCU is successful.

[0178] If the result of step S103 is yes, execute step S104; if the result of step S103 is no, execute step S111.

[0179] If the result of step S105 is no, step S111 is executed.

[0180] The step S106 further includes: sending the determination result to the MCU.

[0181] If the determination result of step S106 is different, step S111 is executed.

[0182] S111 , ignoring the received image data to be processed; or continuously receiving the image data to be processed sent by the SOC, and processing the image data to be processed using a second processing method.

[0183] S112 resets the working state in response to the reset instruction sent by the MCU and returns to step S102.

[0184] The reset instruction is an instruction sent by the MCU when it determines that the working state of the FPGA is abnormal, and the number of times the FPGA working state is continuously determined to be abnormal is less than a threshold.

[0185] The MCU determines that the FPGA operating state is abnormal after one of the following situations occurs:

[0186] The received judgment result is different;

[0187] The determination result is not received within a second set time period after the second verification information is sent;

[0188] The handshake connection with the FPGA failed.

[0189] Alternatively, as Figure 15 As shown, before step S111, the method further includes:

[0190] S110: Determine that the first processing method cannot be used for image processing, and judge whether the number of times that the first processing method cannot be used for image processing is continuously determined is equal to a threshold.

[0191] If the result of step S110 is yes, execute step S111 ; if the result of step S110 is no, execute step S112 .

[0192] Optionally, the method further includes:

[0193] Receiving a clock synchronization signal sent by the MCU at every preset time interval;

[0194] The device corrects its own clock signal according to the clock synchronization signal, and sends a clock synchronization feedback signal to the MCU after correction.

[0195] In a third aspect, an embodiment of the present invention further provides an image processing method, which is applied to an MCU, wherein the MCU includes a memory, the memory of the MCU is set to a read protection state, and the MCU shields a debugging function. Figure 16-17 As shown, the method includes:

[0196] S201 , control FPGA to power on.

[0197] S202 , performing a handshake connection with the FPGA. During the handshake connection process, one of the MCU and the FPGA randomly generates initial synchronization information and sends it to the other.

[0198] S204: Generate second verification information according to the second generation method and send it to the FPGA, so that the FPGA determines whether to use the first processing method to process the image data to be processed sent by the SOC based on the result of determining whether the second verification information is identical to the first verification information generated by itself.

[0199] S207: Determine whether the update condition is met.

[0200] If the result of step S207 is no, continue to wait until the result is yes; if the result of step S207 is yes, execute step S208.

[0201] S208: Update the second generation method for generating the second verification information. Return to step S204.

[0202] Alternatively, as Figure 17 As shown, after step S202, the method further includes:

[0203] S203: Determine whether the handshake connection with the FPGA is successful.

[0204] If the result of step S203 is yes, execute step S204; if the result of step S203 is no, execute step S209.

[0205] After step S204, the method further includes:

[0206] S205: Determine whether the determination result is received within a second set time period after sending the second verification information.

[0207] If the result of step S205 is yes, execute step S206; if the result of step S205 is no, execute step S209.

[0208] S206: Determine whether the judgment result received from the FPGA is different.

[0209] If the result of step S206 is no, execute step S207; if the result of step S206 is yes, execute step S209.

[0210] S209: Determine that the FPGA operating state is abnormal, and judge whether the number of times the FPGA operating state is continuously determined to be abnormal is less than a threshold.

[0211] The FPGA operating state is determined to be abnormal when one of the following situations occurs:

[0212] The judgment result received from the FPGA is different;

[0213] The determination result is not received within a second set time period after the second verification information is sent;

[0214] The handshake connection with the FPGA failed.

[0215] If the result of step S209 is yes, step S210 is executed.

[0216] S210: Reset the working state of the MCU.

[0217] S211: Send a reset instruction to the FPGA to reset the FPGA to a working state. Return to step S202.

[0218] Optionally, the method further includes:

[0219] At preset time intervals, a clock synchronization signal is sent to the FPGA, so that the FPGA corrects its own clock signal according to the clock synchronization signal.

[0220] In a fourth aspect, based on the same inventive concept, an embodiment of the present invention further provides an image processing device, such as Figure 18 Shown, including:

[0221] The receiving unit M101 is configured to generate first verification information according to a first generation method after performing a handshake connection with the MCU, and receive second verification information sent by the MCU;

[0222] a verification unit M102, configured to determine whether the second verification information is the same as the first verification information;

[0223] The first processing unit M103 is configured to, if the judgment result is the same and the update condition is not met, continue to receive the image data to be processed sent by the SOC, and process the image data to be processed using a first processing method;

[0224] An updating unit M104 is configured to update the first generation method for generating the first verification information after determining that the update condition is satisfied, and return to the step of generating the first verification information according to the first generation method;

[0225] The first generation method used after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection.

[0226] Optionally, the image processing device further includes:

[0227] Feedback unit M105, used to send the judgment result to the MCU;

[0228] a reset unit M106, configured to reset the working state and reconnect with the MCU in response to a reset instruction sent by the MCU; wherein the reset instruction is an instruction sent by the MCU when it determines that the working state of the FPGA is abnormal and the number of consecutive determinations of the abnormal working state of the FPGA is less than a threshold;

[0229] The second processing unit M107 is configured to, if it is determined that the first processing method cannot be used for image processing, and the number of consecutive determinations that the first processing method cannot be used for image processing is equal to a threshold, ignore the image data to be processed sent by the SOC; or continue to receive the image data to be processed sent by the SOC and process the image data to be processed using the second processing method;

[0230] Wherein, it is determined that the first processing method cannot be used for image processing after one of the following situations occurs:

[0231] The judgment result is different;

[0232] The second verification information is not received within a first set time period after the first verification information is generated;

[0233] The handshake connection with the MCU fails;

[0234] The MCU determines that the FPGA operating state is abnormal after one of the following situations occurs:

[0235] The received judgment result is different;

[0236] The determination result is not received within a second set time period after the second verification information is sent;

[0237] The handshake connection with the FPGA failed.

[0238] Optionally, the image processing device further includes:

[0239] The clock synchronization unit M108 is configured to receive a clock synchronization signal sent by the MCU at every preset time interval and to correct its own clock signal according to the clock synchronization signal.

[0240] In a fifth aspect, based on the same inventive concept, an embodiment of the present invention further provides an image processing device, wherein the memory of the image processing device is set to a read-protected state, and the image processing device shields a debugging function;

[0241] like Figure 19 As shown, the device includes:

[0242] The verification unit M201 is configured to generate second verification information according to a second generation method and send the second verification information to the FPGA after a handshake connection is established with the FPGA; the FPGA is configured to determine whether to use the first processing method to process the unprocessed image data sent by the SOC based on a determination result of whether the second verification information is identical to the first verification information generated by the FPGA itself; wherein the first second generation method used after a handshake connection with the MCU is determined based on initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during a handshake connection;

[0243] The updating unit M202 is configured to update the second generation method and return to the step of generating the second verification information according to the second generation method when the update condition is met.

[0244] Optionally, the image processing device further includes:

[0245] The self-correction unit M203 is configured to determine that the working state of the FPGA is abnormal, and if the number of consecutive determinations of the abnormal working state of the FPGA is less than a threshold, reset the working state of the MCU, send a reset instruction to the FPGA to reset the working state of the FPGA, and reconnect with the FPGA by handshaking;

[0246] The FPGA operating state is determined to be abnormal when one of the following situations occurs:

[0247] The judgment result received from the FPGA is different;

[0248] The determination result is not received within a second set time period after the second verification information is sent;

[0249] The handshake connection with the FPGA failed.

[0250] Optionally, the image processing device further includes:

[0251] The clock synchronization unit M204 is configured to send a clock synchronization signal to the FPGA at preset time intervals, so that the FPGA corrects its own clock signal according to the clock synchronization signal.

[0252] In the several embodiments provided in this application, it should be understood that the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division, and there may be other division methods in actual implementation.

[0253] Since the working principle of the above-mentioned image processing device is basically the same as the working principle of the image processing method described above, reference may be made to the corresponding implementation of the above-mentioned image processing method, which will not be repeated here.

[0254] In the sixth aspect, based on the same inventive concept, an embodiment of the present invention further provides a readable storage medium, wherein the readable storage medium stores FPGA code, and the FPGA code is used to implement the data processing method as described in the second aspect.

[0255] Alternatively, the readable storage medium stores a computer program, and the computer program is used to implement the data processing method as described in the third aspect.

[0256] In the specific implementation process, Figure 20 As shown, it can adopt portable compact disk read only memory (CD-ROM). However, the program product of the present invention is not limited thereto, and in this document, the readable storage medium can be any tangible medium containing or storing computer program or FPGA code.

[0257] The readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a readable program code or FPGA code. This propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. The computer program or FPGA code contained on the readable storage medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, optical cable, RF, etc., or any suitable combination of the above. The computer program for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., as well as conventional procedural programming languages ​​such as "C", assembly language, or similar programming languages. The FPGA code for performing the operations of the present invention may be written in any combination of one or more hardware description languages, such as Verilog HDL, VHDL, System Verilog, etc.

[0258] The image processing method, device, electronic device, and readable storage medium provided by the embodiments of the present invention utilize an FPGA and an MCU to generate and compare verification information according to the same rules. During operation, the MCU and FPGA continuously update the method for generating verification information, ensuring that the device must work in conjunction with a legitimate MCU for normal operation. Furthermore, by setting the MCU's memory to read-protected mode and disabling the MCU's debugging function, the method for generating verification information between the FPGA and MCU is prevented from being disclosed, effectively protecting the technical secrets contained in the FPGA code from being cracked or misused.

[0259] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0260] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0261] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0262] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0263] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: Applied to field programmable gate arrays (FPGAs), including: After performing a handshake connection with the microcontroller unit MCU, generating first verification information according to a first generation method, and receiving second verification information sent by the MCU; determining whether the second verification information is the same as the first verification information; If the judgment result is the same and the update condition is not met, the image data to be processed sent by the system-on-chip (SOC) is continuously received, and the image data to be processed is processed in a first processing manner; if it is determined that the first processing manner cannot be used for image processing, and the number of consecutive determinations that the first processing manner cannot be used for image processing is equal to a threshold, the image data to be processed sent by the SOC is ignored; or the image data to be processed sent by the SOC is continuously received, and the image data to be processed is processed in a second processing manner; wherein, it is determined that the first processing manner cannot be used for image processing after one of the following situations occurs: the judgment result is different; the second verification information is not received within a first set time period after the first verification information is generated; the handshake connection with the MCU fails; After determining that the update condition is met, updating the first generation method, and returning to the step of generating the first verification information according to the first generation method; The first generation method used after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection.

2. The method according to claim 1, wherein The method further comprises: Sending the judgment result to the MCU; In response to a reset instruction sent by the MCU, the working state is reset, and a handshake connection with the MCU is reestablished; wherein the reset instruction is an instruction sent by the MCU when it determines that the working state of the FPGA is abnormal, and the number of consecutive determinations of the abnormal working state of the FPGA is less than a threshold value; The MCU determines that the FPGA operating state is abnormal after one of the following situations occurs: The received judgment result is different; The determination result is not received within a second set time period after the second verification information is sent; The handshake connection with the FPGA failed.

3. The method according to claim 1, wherein The method further comprises: Receiving a clock synchronization signal sent by the MCU at every preset time interval; The clock signal thereof is corrected according to the clock synchronization signal.

4. An image processing method, characterized in that: Applied to an MCU, the MCU includes a memory, the memory of the MCU is set to a read-protected state, and the MCU shields a debugging function; The method comprises: After establishing a handshake connection with the FPGA, second verification information is generated according to a second generation method and sent to the FPGA; so that the FPGA determines whether to use the first processing method to process the image data to be processed sent by the SOC based on the judgment result of whether the second verification information is the same as the first verification information generated by itself, and ignores the image data to be processed sent by the SOC after determining that the first processing method cannot be used for image processing, and the number of consecutive determinations that the first processing method cannot be used for image processing is equal to a threshold; or continuously receives the image data to be processed sent by the SOC, and uses the second processing method to process the image data to be processed; wherein, it is determined that the first processing method cannot be used for image processing after one of the following situations occurs: the judgment result is different; the second verification information is not received within a first set time period after the first verification information is generated; the handshake connection with the MCU fails; wherein the first second generation method adopted after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection; When the update condition is met, the second generation method is updated, and the process returns to the step of generating the second verification information according to the second generation method.

5. The method according to claim 4, wherein The method further comprises: Determining that the working state of the FPGA is abnormal, and the number of consecutive determinations that the working state of the FPGA is abnormal is less than a threshold, resetting the working state of the MCU and sending a reset instruction to the FPGA to reset the working state of the FPGA; Reconnect with the FPGA by handshake; The FPGA operating state is determined to be abnormal when one of the following situations occurs: The judgment result received from the FPGA is different; The determination result is not received within a second set time period after the second verification information is sent; The handshake connection with the FPGA failed.

6. The method according to claim 4, wherein The method further comprises: At preset time intervals, a clock synchronization signal is sent to the FPGA, so that the FPGA corrects its own clock signal according to the clock synchronization signal.

7. An image processing device, characterized in that: include: A receiving unit, configured to generate first verification information according to a first generation method after performing a handshake connection with the MCU, and receive second verification information sent by the MCU; a verification unit, configured to determine whether the second verification information is the same as the first verification information; The first processing unit is configured to, if the judgment result is the same and the update condition is not met, continuously receive the image data to be processed sent by the SOC and process the image data to be processed in a first processing manner; if it is determined that the first processing manner cannot be used for image processing, and the number of consecutive determinations that the first processing manner cannot be used for image processing is equal to a threshold, ignore the image data to be processed sent by the SOC; or continuously receive the image data to be processed sent by the SOC and process the image data to be processed in a second processing manner; wherein, it is determined that the first processing manner cannot be used for image processing after one of the following situations occurs: the judgment result is different; the second verification information is not received within a first set time period after the first verification information is generated; or the handshake connection with the MCU fails; an updating unit, configured to update the first generation method for generating the first verification information after determining that the update condition is satisfied, and return to the step of generating the first verification information according to the first generation method; The first generation method used after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection.

8. An image processing device, characterized in that: The memory of the image processing device is set to a read protection state, and the image processing device shields a debugging function; The device comprises: A verification unit, configured to generate second verification information according to a second generation method and send it to the FPGA after a handshake connection with the FPGA, and continuously send image data to be processed to the FPGA; so that the FPGA determines whether to use the first processing method to process the image data to be processed based on a judgment result of whether the second verification information is the same as the first verification information generated by itself, and ignores the image data to be processed sent by the SOC after determining that the first processing method cannot be used for image processing, and the number of consecutive determinations that the first processing method cannot be used for image processing is equal to a threshold value; or continuously receive the image data to be processed sent by the SOC and use the second processing method to process the image data to be processed; wherein, it is determined that the first processing method cannot be used for image processing after one of the following situations occurs: the judgment result is different; the second verification information is not received within a first set time period after the first verification information is generated; the handshake connection with the MCU fails; wherein the first second generation method used after the handshake connection with the MCU is determined according to the initial synchronization information; the initial synchronization information is information randomly generated by one of the MCU and the FPGA and sent to the other during the handshake connection; The updating unit is configured to update the second generation method and return to the step of generating the second verification information according to the second generation method when the update condition is met.

9. An electronic device, characterized in that: include: MCU, FPGA, external memory, SOC; Wherein, the MCU includes a memory; The MCU is connected to the FPGA via a first bus, the FPGA is connected to the external memory via a second bus, and the SOC is connected to the FPGA; The SOC is used to continuously send image data to be processed to the FPGA, the FPGA is used to implement the image processing method according to any one of claims 1 to 3, and the MCU is used to implement the image processing method according to any one of claims 4 to 6.

10. A readable storage medium, characterized in that: The readable storage medium stores an FPGA code, and the FPGA code is used to implement the image processing method according to any one of claims 1 to 3; Alternatively, the readable storage medium stores a computer program, and the computer program is used to implement the image processing method according to any one of claims 4 to 6.

Images