FPGA firmware upgrading method and system based on image control

Abstract

The invention discloses an FPGA (Field Programmable Gate Array) firmware upgrading method and system based on image control, and relates to the technical field of image control. A first row of pixel data, a second row of pixel data and a third row of pixel data are gradually upgraded in sequence; and determining the upgrading path of the FPGA firmware according to the first layer upgrading content of the second row of pixel data, the second layer upgrading content of the third row of pixel data and the upgrading version number of the FPGA firmware. The accuracy of the upgrading path of the FPGA firmware is improved. According to the upgrading content of the plurality of upgrading nodes, the current upgrading progress of the FPGA firmware and the current working event of the FPGA firmware, the upgrading mode of the FPGA firmware at each upgrading node is determined, and the execution of the FPGA firmware on the current working event is not influenced; in the FPGA firmware upgrading process, the upgrading process table of the FPGA firmware is gradually updated, and the dynamic optimization strategy of the FPGA firmware is determined based on the upgrading process table of the FPGA firmware and the abnormal upgrading data of the FPGA firmware, so that the accuracy of the dynamic optimization strategy of the FPGA firmware is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image control, and in particular to an FPGA firmware upgrade method and system based on image control. BACKGROUND

[0002] With the development of technology, FPGA firmware is the core of FPGA chip operation, and is widely used in communication, image processing, industrial control and other fields by defining logic functions and implementing hardware acceleration. Its flexibility and high performance make it an indispensable part of many complex systems. With the continuous development of technology, FPGA firmware will play an important role in more frontier fields, such as artificial intelligence, Internet of Things, etc. In the prior art, when it is found that the FPGA function is abnormal and needs to be upgraded to correct the abnormal problem, it is usually necessary to upgrade through a special JTAG interface. For general image processing application scenarios, the image format combination of the collected firmware data image only has first row pixel data, and cannot introduce first layer upgrade content of second row pixel data and second layer upgrade content of third row pixel data, affecting the accuracy of the FPGA firmware upgrade path, thereby affecting the upgrade of the FPGA firmware. SUMMARY

[0003] The present application provides an FPGA firmware upgrade method and system based on image control.

[0004] The present application provides an FPGA firmware upgrade method based on image control, comprising: According to the image detection of the firmware data image, the image format combination of the firmware data image is marked; According to the analysis of the image format combination of the firmware data image, multiple rows of pixel data are determined, and based on the division of the multiple rows of pixel data, first row pixel data, second row pixel data and third row pixel data are determined; The first row pixel data, the second row pixel data and the third row pixel data are sequentially upgraded step by step, and the upgrade path of the FPGA firmware is determined according to the first layer upgrade content of the second row pixel data, the second layer upgrade content of the third row pixel data and the upgrade version number of the FPGA firmware; Based on the detection of the upgrade path of the FPGA firmware, multiple upgrade nodes are determined, the upgrade mode of the FPGA firmware at each upgrade node is determined according to the upgrade content of the multiple upgrade nodes, the current upgrade progress of the FPGA firmware and the current working event of the FPGA firmware, and the execution of the FPGA firmware on the current working event is not affected; In the upgrading process of the FPGA firmware, the upgrading progress table of the FPGA firmware is updated step by step, and the dynamic optimization strategy of the FPGA firmware is determined based on the upgrading progress table of the FPGA firmware and the abnormal upgrading data of the FPGA firmware.

[0005] The embodiment of the application provides an FPGA firmware upgrading system based on image control, which is applied to the FPGA firmware upgrading method based on image control. An image format combination module is configured to mark the image format combination of the firmware data image according to the image detection of the firmware data image. A pixel data module is configured to determine the multi-line pixel data according to the analysis of the image format combination of the firmware data image, and determine the first-line pixel data, the second-line pixel data and the third-line pixel data based on the division of the multi-line pixel data. An upgrading path module is configured to sequentially upgrade the first-line pixel data, the second-line pixel data and the third-line pixel data step by step, and determine the upgrading path of the FPGA firmware according to the first-layer upgrading content of the second-line pixel data, the second-layer upgrading content of the third-line pixel data and the upgrading version number of the FPGA firmware. An upgrading mode module is configured to determine a plurality of upgrading nodes based on the detection of the upgrading path of the FPGA firmware, determine the upgrading mode of the FPGA firmware at each upgrading node according to the upgrading content of the plurality of upgrading nodes, the current upgrading progress of the FPGA firmware and the current working event of the FPGA firmware, and not affect the execution of the FPGA firmware on the current working event. A dynamic optimization strategy module is configured to update the upgrading progress table of the FPGA firmware step by step in the upgrading process of the FPGA firmware, and determine the dynamic optimization strategy of the FPGA firmware based on the upgrading progress table of the FPGA firmware and the abnormal upgrading data of the FPGA firmware.

[0006] Compared with the prior art, the application has the following advantages: In the embodiment of the application, the method in the embodiment of the application determines the upgrading path of the FPGA firmware according to the first-layer upgrading content of the second-line pixel data, the second-layer upgrading content of the third-line pixel data and the upgrading version number of the FPGA firmware, introduces the multi-line pixel data, and compatible with the overall consideration of the first-layer upgrading content of the second-line pixel data, the second-layer upgrading content of the third-line pixel data and the upgrading version number of the FPGA firmware, thereby improving the accuracy of the upgrading path of the FPGA firmware.

[0007] Therefore, based on the detection of the FPGA firmware upgrade path, the multiple upgrade nodes are determined, the FPGA firmware upgrade mode at each upgrade node is determined according to the upgrade content of the multiple upgrade nodes, the current upgrade progress of the FPGA firmware and the current working event of the FPGA firmware, and the execution of the FPGA firmware on the current working event is not affected; in the FPGA firmware upgrade process, the FPGA firmware upgrade progress table is updated gradually, and the dynamic optimization strategy of the FPGA firmware is determined based on the FPGA firmware upgrade progress table and the abnormal upgrade data of the FPGA firmware, so that the FPGA firmware upgrade mode at each upgrade node is controlled, the overall consideration based on the FPGA firmware upgrade progress table and the abnormal upgrade data of the FPGA firmware is further realized, and the accuracy of the dynamic optimization strategy of the FPGA firmware is improved. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a flowchart of the FPGA firmware upgrade method based on image control in the embodiment of the application; Figure 2 is a flowchart of step S11 in the FPGA firmware upgrade method based on image control in the embodiment of the application; Figure 3 is a flowchart of step S12 in the FPGA firmware upgrade method based on image control in the embodiment of the application; Figure 4 is a flowchart of step S13 in the FPGA firmware upgrade method based on image control in the embodiment of the application; Figure 5 is a flowchart of step S14 in the FPGA firmware upgrade method based on image control in the embodiment of the application; Figure 6 is a flowchart of step S15 in the FPGA firmware upgrade method based on image control in the embodiment of the application; Figure 7 is a structural composition diagram of the FPGA firmware upgrade system based on image control in the embodiment of the application. DETAILED DESCRIPTION

[0009] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application.

[0010] Please refer to Figures 1 to 7 A FPGA firmware upgrade method based on image control is applied to the scene of FPGA firmware upgrade; the FPGA firmware upgrade method based on image control comprises the following steps: Step S11: image format combinations of firmware data images are marked according to image detection of the firmware data images; Step S12: Determine the multi-row pixel data according to the image format combination of the firmware data image, and determine the first row pixel data, the second row pixel data and the third row pixel data based on the division of the multi-row pixel data; Step S13: Upgrade the first row pixel data, the second row pixel data and the third row pixel data in sequence, determine the upgrade path of the FPGA firmware according to the first layer upgrade content of the second row pixel data, the second layer upgrade content of the third row pixel data and the upgrade version number of the FPGA firmware; Step S14: Determine the multiple upgrade nodes based on the detection of the upgrade path of the FPGA firmware, determine the upgrade mode of the FPGA firmware at each upgrade node according to the upgrade content of the multiple upgrade nodes, the current upgrade progress of the FPGA firmware and the current working event of the FPGA firmware, and do not affect the execution of the FPGA firmware on the current working event; Step S15: In the upgrade process of the FPGA firmware, update the upgrade progress table of the FPGA firmware step by step, and determine the dynamic optimization strategy of the FPGA firmware based on the upgrade progress table of the FPGA firmware and the abnormal upgrade data of the FPGA firmware.

[0011] Reference Figure 2 In step S11, the specific steps are: S111: When upgrading the FPGA firmware, collect the firmware data image corresponding to the FPGA firmware, and perform image detection on the firmware data image to output the data set of the FPGA firmware; S112: Determine the multiple image format data of the FPGA firmware based on the screening of the data set of the FPGA firmware, determine the image format combination of the firmware data image according to the synthesis of the multiple image format data of the FPGA firmware, and mark the image format combination of the firmware data image.

[0012] In the embodiment of the present application, when the system needs to upgrade the FPGA firmware (for example, by user instruction, remote server notification or internal timer trigger), it is necessary to first determine the source of the firmware data image. This image comes from the following ways: a certain module (dedicated firmware management module) inside the FPGA encodes the firmware binary data to be upgraded into one or more image files according to the pre-defined rules, and the image file is then stored in the on-chip memory (such as BRAM) or off-chip memory (such as DDR) for subsequent processing.

[0013] According to the data source determined in the previous step, the actual pixel data of the firmware data image is collected into the FPGA for processing, which involves data reading or receiving; at this time, the data stream is directly read from the module that generates the image or the memory location that stores the image.

[0014] Optionally, assuming an industrial camera system needs to upgrade its image processing FPGA's firmware; the system receives an upgrade instruction from the factory network, and downloads an image file containing firmware data (for example, a PNG image) from the network server, this PNG file is the "firmware data image", which is collected into the FPGA system through the network interface, first stored in the system's RAM; the FPGA's network interface module receives all the data packets of the PNG file and assembles them into a complete image data stream, which is written into a BRAM in the FPGA for subsequent steps; at this time, the FPGA has "collected" the original byte stream of the firmware data image.

[0015] The collected raw image data (usually a byte stream) needs to be identified and parsed, the goal of this step is to confirm that the received data is indeed a "firmware data image" that meets expectations, and to extract its basic image properties; at the same time, the image processing module inside the FPGA begins to read the data in the BRAM; first, it checks the first 8 bytes to confirm that it is indeed the signature of a PNG file, thus identifying it as a PNG image; then, it parses the information in the PNG file header to obtain the image's width (e.g., 1920 pixels), height (e.g., 1080 pixels), color channel number (e.g., RGB, 3 channels), and bit depth (e.g., 8 bits per channel); at the same time, it calculates the CRC value of the entire data block and compares it with the CRC stored in the file to confirm data integrity; assuming this PNG image is specially designed for firmware upgrade, its pixel data is specially encoded and contains firmware upgrade information.

[0016] Organize the results of the previous step of detection and preprocessing into a structured "data set", this set not only contains the properties of the image itself, but also contains firmware-related information preliminarily extracted or confirmed from the image data; image properties: resolution (width x height), color format (such as RGB888, YUV420), total number of pixels, etc.; firmware information (preliminary): extract the firmware version number, size, checksum, etc. preliminary information from specific areas of the image (such as the first row of pixels, image metadata area, hidden watermark, etc.), for verification in subsequent steps; status information: image detection success, data integrity, etc. status flags.

[0017] Further, based on the screening of the FPGA firmware data set, a plurality of image format data of the FPGA firmware is determined, an image format combination of the firmware data image is determined according to the synthesis of the plurality of image format data of the FPGA firmware, and the image format combination of the firmware data image is marked, which is compatible with the overall consideration of the synthesis of the plurality of image format data of the FPGA firmware, and ensures the accuracy of the image format combination of the firmware data image.

[0018] At this time, specific information related to the image format is filtered out from the structured data set output from S111, which includes: image resolution: the width and height of the image (e.g., 1920x1080); pixel encoding: how each pixel is represented (e.g., RGB888 represents that each pixel consists of 3 bytes, representing the red, green, and blue color channels respectively, each with 8 bits); data arrangement: how the pixel data is arranged in memory or data stream (e.g., stored row by row, each row continuous); compression or encoding: whether the image data uses a certain compression algorithm (such as JPEG, PNG) or a specific encoding method (such as LSB steganography for hiding firmware data); firmware data embedding location / rule: where the firmware data is specifically stored in the image (e.g., a specific pixel row, a specific color channel, a hidden data area, etc.).

[0019] The extracted multiple image format data items are integrated and associated to form a complete definition of the overall structure description of the firmware data image. This "image format combination" describes how to interpret all the pixel data that constitutes the firmware data image, especially how to extract the firmware data from it, which requires understanding the logical relationship between various format data; for example, resolution and pixel encoding define the basic structure of the image, while embedding rules indicate the specific location and extraction method of the firmware data.

[0020] A marker or identifier is created for the determined "image format combination" that is easy to identify and reference. This marker can be a unique ID, a short name, or an encoded string, which represents the structured description of the entire firmware data image. This marker will be used in subsequent upgrade processes (such as S12, S13, S14, S15) to quickly identify and process the format of the firmware data image currently being processed, ensuring that subsequent steps can correctly parse and manipulate image data for firmware upgrade.

[0021] Reference Figure 3 In step S12, the specific steps are as follows: S121: Collect the image format combination of the firmware data image, perform row and column detection on the image format combination of the firmware data image, and determine multiple rows of pixel data based on the row and column detection of the image format combination of the firmware data image, at this time, the multiple rows of pixel data are arranged in sequence; S122: Divide the multiple rows of pixel data and mark multiple data markers in the multiple rows of pixel data, output first, second and third rows of pixel data based on the tracing of the multiple data markers; the first, second and third rows of pixel data are arranged in sequence and present corresponding information, and the first, second and third rows of pixel data are all different.

[0022] In the embodiment of the present application, the image format combination tag determined in step S112 is read, which contains key information on how to extract firmware data from the image, such as the resolution of the image, which bit of which color channel the firmware data is embedded in, the starting row number, the total number of rows containing data, etc.; the system needs to read this tag from a storage location (memory, register or file), which is the result of the processing in step S112, and which defines the structure of the image data.

[0023] Using the acquired image format combination tag, the firmware data image is actually analyzed to determine the specific distribution of the firmware data on which pixel rows; "row and column detection" here mainly refers to "row detection", i.e. locating to the row range in the image that needs to be processed according to the row information in the tag; for columns, the image is usually stored in row-major order, and the column information is implicitly contained in the pixel data structure; this process involves reading the metadata of the image file (such as resolution) to verify the accuracy of the tag, or directly preparing to read the pixel data of the specified rows according to the row range information in the tag.

[0024] According to the row range determined in the previous step, the pixel data of these rows is actually extracted from the firmware data image; the "multi-row pixel data" extracted is a continuous data block containing all the pixel information in the specified row range; "sequentially arranged" means that these rows maintain their order in the original image in the extracted data block (the first row is in the front, and the 100th row is in the back).

[0025] Optionally, the system already knows that the pixel data of the first row to the 100th row needs to be extracted; the system reads the pixel data of these 100 rows from the image file or memory according to the calculated address range; assuming that each row has 1920 pixels and each pixel has 3 bytes (RGB), then the total pixel data of these 100 rows is 100*1920*3=576,000 bytes; the 576,000 bytes of data extracted is the "multi-row pixel data"; in this data block, the data of the first row is at the front, followed by the data of the second row, and so on, until the data of the 100th row is at the back, and this data block is "sequentially arranged".

[0026] Further, operations are performed on the continuous pixel data stream (multi-line pixel data") extracted in S121; the division is usually not based on the physical row boundaries (as S121 has already extracted by physical rows), but based on the embedding protocol of the firmware data in the image or the characteristics of the firmware itself; the ways include: division by data block size: if the protocol specifies that the first line of pixel data contains the first N bytes, the second line contains the next M bytes, and the third line contains the remaining K bytes; division by specific identifier: special delimiter or marker bytes are included in the data stream, indicating the end of one part and the beginning of another part; division by firmware logical structure: the first part is the bootloader, the second part is the core function, and the third part is the configuration data, and the division point corresponds to different segments of the firmware file.

[0027] Specifically, assume that the multi-line pixel data output by S121 is the aforementioned 576,000 bytes (from the blue channel LSB of a 100-line 1080p image); according to the firmware upgrade protocol, it is specified that the first 512,000 bytes (approximately equal to 100 / 3*576,000) are the first line of pixel data"; the next 128,000 bytes are the second line of pixel data"; and the last 16,000 bytes are the third line of pixel data".

[0028] The system performs the division operation: extracts the first 512,000 bytes of the multi-line pixel data as the first line of pixel data"; extracts the next 128,000 bytes as the second line of pixel data"; and extracts the last 16,000 bytes as the third line of pixel data".

[0029] After the three logical parts are divided, the system needs to assign an identifier or marker to each part so that they can be distinguished in subsequent steps, these markers can be simple (such as "PART1", "PART2", "PART3"), or can contain more information (such as "BOOTLOADER", "CORE", "CONFIG"), these markers can be stored in memory, associated with the corresponding data block, or written into a data structure, recording the starting position and length of each marker corresponding data block in the original multi-line pixel data.

[0030] Specifically, the system labels the three parts: the 512,000-byte block is labeled "PART1"; the 128,000-byte block is labeled "PART2"; and the 16,000-byte block is labeled "PART3"; these markers "PART1", "PART2", "PART3" are now associated with the respective data blocks.

[0031] The system finds and extracts the corresponding data blocks according to the previously marked markers. Here, "tracing back" can be understood as finding or locating data according to the markers. Finally, the system needs to explicitly output (or make available for subsequent steps) the three marked data blocks and their order and content. It needs to ensure that the output three data blocks are indeed arranged in order (i.e., the first block is in front, the second block is in the middle, and the third block is at the back), and they are all inconsistent (different in content and size).

[0032] Specifically, the system traces back and outputs according to the markers: according to the marker "PART1", the system locates and outputs the first row of pixel data (512,000 bytes); according to the marker "PART2", the system locates and outputs the second row of pixel data (128,000 bytes); according to the marker "PART3", the system locates and outputs the third row of pixel data (16,000 bytes); the system confirms that the three data blocks output in order (512K>128K>16K), and their sizes and contents are different, and these three data blocks can now be used by the S13 step for phased firmware upgrade.

[0033] Reference Figure 4 In step S13, the specific steps are: S131: sequentially compare the first row of pixel data, the second row of pixel data, and the third row of pixel data, and take the first row of pixel data as the frame start data for upgrading the firmware to present the frame start position and frame number of the FPGA firmware; S132: determine the first layer of upgrade firmware data based on the comparison of the second row of pixel data and the first row of pixel data, and determine the first layer of upgrade content of the FPGA firmware according to the analysis of the first layer of upgrade firmware data, at this time, determine the first sub-upgrade path according to the first layer of upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware; S133: determine the second layer of upgrade firmware data based on the comparison of the third row of pixel data and the second row of pixel data, and determine the second layer of upgrade content of the FPGA firmware according to the analysis of the second layer of upgrade firmware data, at this time, determine the second sub-upgrade path according to the second layer of upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware, and determine the upgrade path of the FPGA firmware based on the first sub-upgrade path, the second sub-upgrade path, and the model of the FPGA firmware.

[0034] In the embodiments of the present application, the first row of pixel data, the second row of pixel data, and the third row of pixel data are sequentially compared, and the first row of pixel data is taken as the frame start data for upgrading the firmware to present the frame start position and frame number of the FPGA firmware. The first row of pixel data is introduced as the frame start data for upgrading the firmware to present the frame start position and frame number of the FPGA firmware.

[0035] At this point, the system compares the first row of pixel data with the content of the second and third rows. This comparison can be a simple byte-level comparison or a pattern-based match (e.g., checking if the first row contains a known bootloader signature or specific control characters, while the second and third rows do not). The first row of pixel data should be significantly different from the second and third rows. For example, the first row contains a fixed bootloader header, while the second and third rows contain variable data or configuration information. If the comparison finds that the first row of data is too similar to the subsequent rows, it means that the data tagging or format recognition (S112) has a problem, and the system needs to issue a warning or take corrective action.

[0036] The first row of pixel data is designated as the core of the upgrade process. It is the beginning of the entire firmware data stream and contains the most basic information needed to start the upgrade process. The system designates the first row of pixel data (extracted and tagged in S122) as "frame start data," which means that it will be read and processed from this row of data in the subsequent upgrade process. The system considers it as the beginning of a logical "frame" or "block." This row of data usually contains key fields required by the upgrade protocol, such as: frame start flag: a special byte sequence indicating the beginning of a new frame; frame type: indicating the role of this frame data (e.g., bootloader, configuration data, or application logic); frame length: indicating how many bytes this frame data contains; checksum / crc: used to verify the integrity of data transmission.

[0037] From the first row of pixel data (frame start data), important metadata about the firmware structure is parsed, such as the frame start position and frame number. These pieces of information help the FPGA understand the overall layout of the firmware and the progress of the current processing. The system extracts specific fields from the first row of pixel data according to the predefined protocol or format.

[0038] The frame start position indicates the starting address of this frame data in the overall logical structure of the firmware (e.g., offset in the internal memory of the FPGA), which helps the FPGA know where to write this frame data physically. The frame number is a simple counter (e.g., 0, 1, 2…) indicating the number of this frame in the entire firmware data stream, which helps the FPGA track the upgrade progress or resume after a data transmission interruption.

[0039] The extraction process usually involves converting the pixel data back to binary format and then reading according to the field offset and length defined by the protocol. For example, the protocol specifies that the 4 bytes of the frame start position are located at the 5th to 8th bytes of the pixel data.

[0040] Specifically, assuming that we have a firmware data image for upgrading a specific model of FPGA (e.g., model XC7Z020); after processing S111 and S112, we determine that the 100th row, 101st row, and 102nd row of pixel data in the image correspond to the first row, second row, and third row of pixel data, respectively, and these data have been extracted in order.

[0041] The system compares the first row of pixel data (100th row data) with the second row (101st row) and third row (102nd row) data; assuming the protocol specifies that the first 16 pixels (assuming 8-bit grayscale values) of the first row data should be a fixed boot header signature, such as [0xAA, 0x55, 0xAA, 0x55, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]; the system checks the first 16 pixel values of the 100th row and finds that they indeed match this signature; the system then checks the first 16 pixel values of the 101st row and 102nd row and finds that they do not contain this signature, but rather contain other data or configuration information that looks like it; in comparison, it is confirmed that the first row of data indeed has a unique boot header feature and is suitable as a frame start.

[0042] The system explicitly specifies all pixel data of the 100th row (assuming 960 pixels, each 8 bits, totaling 960 bytes) as the "frame start data" for this firmware upgrade; the system parses information from the 960-byte frame start data according to the protocol; the protocol specifies that the frame start position is located at the 5th to 8th byte of the frame start data (i.e., the 5th to 8th pixel values of the 100th row pixel data); assuming the read value is 0x0000F000, this means that this frame data should be written to the address 0xF000 in the FPGA internal memory; the frame number is located at the 9th to 10th byte of the frame start data; assuming the read value is 0x00, this means that this is the first frame in the firmware data stream; the system successfully parses the frame start position 0xF000 and frame number 0x00 from the first row of pixel data.

[0043] Further, based on the comparison of the second row of pixel data and the first row of pixel data to determine the first layer of upgrade firmware data, and based on the parsing of the first layer of upgrade firmware data to determine the first layer of upgrade content of the FPGA firmware, at this time, according to the first layer of upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware, a first sub-upgrade path is determined, which takes into account the compatibility of the first layer of upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware, ensuring the accuracy of the first sub-upgrade path.

[0044] At this time, the system compares the first row of pixel data (representing the current or base state) with the second row of pixel data (representing the target state), and this comparison is usually based on the binary representation of the firmware data; for example, if the first row and the second row both contain firmware code, the system compares the code bytes at the corresponding positions; the system records all the mismatched bytes and their positions in the data stream; the system generates a "difference data set" containing all the data segments and their position information that need to be updated from the first row state to the second row state, and this difference data set is the "first layer upgrade firmware data".

[0045] The "first layer upgrade firmware data" (i.e. difference data) obtained in the previous step is parsed into meaningful information to determine which parts of the FPGA firmware are modified in this upgrade; the system analyzes the content and structure of the "first layer upgrade firmware data", which involves: identifying code / data segments: according to the position and characteristics of the data, determine which functional module (such as video processing core, communication interface logic, configuration register, etc.) is modified; disassemble or parse part of the instructions if the difference data contains executable code to understand what function is modified; if it is configuration data, it needs to be parsed for its meaning; refer to firmware documentation / address mapping table: need to combine the firmware documentation or pre-defined address mapping table to map the data position to specific hardware function or software module; the system knows exactly what the "first layer upgrade firmware content" is; for example, "upgrade the algorithm parameters of the core video processing module" or "update the baud rate setting logic of the UART communication interface".

[0046] The system compares the version number of the current FPGA firmware (stored in the FPGA or represented by the first row of data) with the version number of the target firmware (contained in the firmware data or represented by the second row of data); according to the version number and the first layer upgrade content, the system decides which upgrade strategy to use; for example: direct writing: if the version number difference is small and the first layer content is only parameter update, directly write the difference data to the specified address; Bootloader assistance: if the version difference is large or the first layer content involves core logic modification, the Bootloader (represented by the first row of data in S131) is needed to safely update this part of the code; specific sequence: some registers need to be updated first, and then other registers are updated to avoid unstable intermediate state; the system generates a series of specific instructions or control signals to guide the FPGA on how to perform this first stage upgrade; the system determines a "first sub-upgrade path", which is a detailed execution plan that explains how to apply the first layer upgrade content to the FPGA.

[0047] Specifically, assuming that the current FPGA firmware version is V1.0 and the target firmware version is V1.1; the first layer upgrade content is "update the color conversion coefficient, noise reduction intensity threshold and frame buffer management logic of the video processing core"; the system confirms that the upgrade from V1.0 to V1.1 allows direct updating of these parameters without resetting the entire system or through a complex boot process; since it is only a parameter update, and the target address 0xF100-0xF114 is a programmable register area, the system decides to use the "direct write" strategy; however, in order to ensure that the image processing core does not produce abnormal images during the update process, it is necessary to first suspend image processing, and then restore after the update is complete; generate path instructions: the system generates the following instruction sequence as the "first sub-upgrade path": Send a command to the FPGA to suspend the operation of the video processing core; Write a 15-byte data block [0xAA, 0xBB, 0xCC,...] to addresses 0xF100 to 0xF114 in turn; Wait for the write operation to complete; Send a command to the FPGA to resume the operation of the video processing core; Record the completion of this upgrade.

[0048] Therefore, the second layer upgrade firmware data is determined based on the comparison of the third row of pixel data and the second row of pixel data, and the second layer upgrade content of the FPGA firmware is determined according to the analysis of the second layer upgrade firmware data, at this time, the second sub-upgrade path is determined according to the second layer upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware, the upgrade path of the FPGA firmware is determined based on the first sub-upgrade path, the second sub-upgrade path and the model of the FPGA firmware, which takes into account the compatibility of the first sub-upgrade path, the second sub-upgrade path and the model of the FPGA firmware, ensuring the accuracy of the upgrade path of the FPGA firmware, and at the same time, multiple rows of pixel data are introduced, which takes into account the overall consideration of the first layer upgrade content of the second row of pixel data, the second layer upgrade content of the third row of pixel data and the upgrade version number of the FPGA firmware, improving the accuracy of the upgrade path of the FPGA firmware.

[0049] At this point, the system compares the second row of pixel data (representing the state of the first layer of upgrade) and the third row of pixel data (representing the final target state), and this comparison is also based on the binary representation of the firmware data; the system needs to identify which data blocks are parameters or configuration information, and which are code; for example, if both the second and third rows contain firmware parameters, the system compares the parameter values at the corresponding positions; optionally, the firmware state parsed from the second row of pixel data contains parameters: threshold = 50, filter strength = 2; the target firmware state parsed from the third row of pixel data contains parameters: threshold = 75, filter strength = 3; the comparison result: the system finds that the threshold changes from 50 to 75, and the filter strength changes from 2 to 3, and the difference between these two parameters is the second layer of upgrade firmware data.

[0050] The system needs to parse the found difference data, which usually involves understanding the firmware version number and parameter table; the system will consult the parameter mapping table or configuration file according to the second layer of upgrade firmware data (such as threshold 75, filter strength 3) and the current firmware version number, to determine which registers or storage areas in the FPGA should be written to; optionally, the system parses the second layer of upgrade firmware data as: threshold = 75, filter strength = 3; combined with the current firmware version number (assuming it is v2.1) and the parameter mapping table, the system determines that: threshold 75 should be written to the register at address 0xF200; filter strength 3 should be written to the register at address 0xF201; the second layer of upgrade content is to update the values of these two parameter registers.

[0051] The system determines how to safely and efficiently update these parameters according to the second layer of upgrade content (parameters that need to be updated and their addresses) and the firmware version number, which involves specific writing protocols, timing requirements or dependency relationships (for example, some parameters need to be written in a specific order); at this point, the system determines that the second layer of upgrade content is to update the registers at addresses 0xF200 and 0xF201; combined with the firmware version number v2.1 and the FPGA model (assuming it is Xilinx Artix-7), the system consults the configuration strategy library and determines that: these parameter registers can be updated through the JTAG interface or the internal configuration bus.

[0052] A configuration enable command needs to be sent first; parameters can be written simultaneously or in sequence, but it is recommended to write in address order to ensure compatibility; a configuration confirmation command needs to be sent after writing; generate path instructions: the system generates the following instruction sequence as the second sub-upgrade path: send a configuration enable command to the FPGA; write value 75 to address 0xF200; write value 3 to address 0xF201; send a configuration confirmation command to the FPGA; record this upgrade as complete.

[0053] The system combines the first sub-upgrade path (from S132) and the second sub-upgrade path (from this step) in order; it also needs to consider the specific requirements or limitations brought by the FPGA model, such as different configuration interfaces, timing requirements, or security mechanisms; the system will ensure that the integrated path is logical and reasonable in time and fully adapts to the target FPGA; optionally, the integrated path: first sub-upgrade path: pause video processing > write core parameters [0xAA, 0xBB, …] to 0xF100-0xF114 > resume video processing; second sub-upgrade path: send configuration enable > write 75 to 0xF200 > write 3 to 0xF201 > send configuration confirmation; FPGA model: Xilinx Artix-7; the system combines the two paths in order: first sub-path is executed, then the second sub-path is executed, checks if there are special requirements for Xilinx Artix-7: for example, after the core logic is updated (first sub-path is completed), a short stabilization time is needed before parameter configuration (before the second sub-path starts); the system inserts a short delay (e.g. 100ms) between "resume video processing" of the first sub-path and "send configuration enable" of the second sub-path; the final determined FPGA firmware upgrade path: pause video processing core - write [0xAA, 0xBB, …] to 0xF100-0xF114 - resume video processing core - wait 100ms - send configuration enable command - write 75 to 0xF200 - write 3 to 0xF201 - send configuration confirmation command - complete upgrade.

[0054] Reference Figure 5 In step S14, the specific steps are: S141: Collect the upgrade path of the FPGA firmware, determine the multiple upgrade regions of the FPGA firmware according to the detection of the upgrade path of the FPGA firmware, determine the corresponding upgrade nodes according to the detection of each upgrade region, and collect multiple upgrade nodes; S142: Determine the corresponding upgrade content based on the content tracing of the multiple upgrade nodes, collect the current upgrade progress of the FPGA firmware and the current working event of the FPGA firmware; determine the first upgrade mode coefficient according to the upgrade content of the multiple upgrade nodes and the current upgrade progress of the FPGA firmware; S143: Determine the second upgrade mode coefficient according to the upgrade content of the multiple upgrade nodes and the current working event of the FPGA firmware, determine the upgrade mode of the FPGA firmware at each upgrade node based on the first upgrade mode coefficient, the second upgrade mode coefficient, and the upgrade mode mapping relationship, at this time, the FPGA firmware in the upgrade process real-time controls the execution of the FPGA firmware on the current working event to avoid stopping affecting the execution of the FPGA firmware on the current working event.

[0055] In the embodiments of the present application, the upgrade path of the FPGA firmware is collected, the system analyzes the collected upgrade path, and divides it into several logically or physically related "areas" according to certain rules; the basis for division can be various, and the common ones include: physical address continuity: operations with continuous or close addresses are classified into an area; for example, multiple sectors of continuous writing to the Flash; functional module relevance: configuration or parameter update operations belonging to the same functional module (such as video processing core, network interface, control logic) are classified into an area, which requires the upgrade path data itself or additional metadata to indicate the corresponding functional module of each operation; operation type similarity: similar operations (such as all configuration loading, all parameter writing) are classified into an area; upgrade priority: parts that need to be upgraded first (such as the boot loader) are separately divided into an area; a long and chaotic operation sequence is organized into several structured parts, facilitating subsequent management and optimization; each area represents a group of related upgrade tasks.

[0056] In each determined upgrade area, the specific operations are further decomposed into smaller, independently executable units, i.e. "upgrade nodes"; an upgrade node usually represents one or a group of closely related operations that can be completed in a relatively independent time period; determining nodes can consider: atomicity of operations: operations within a node should be a logically indivisible or executable as a whole task; for example, writing a complete configuration block; execution time / resource occupation: operations that take a long time or occupy a lot of system resources are separately considered as a node for more detailed scheduling; dependency relationship: if some operations must be executed after other operations are completed, they can be divided into different nodes and the dependency relationship is considered in subsequent steps; checkpoint / validation point: nodes can be set after key operations to roll back to a certain known state in case of upgrade failure; each upgrade area is refined into specific execution steps to provide a basis for subsequent dynamic mode selection and progress tracking; all determined upgrade nodes are collected to form a complete and ordered upgrade node list, which will be used in subsequent steps (S142 and S143) to determine how to execute the upgrade according to the specific content of each node and the current state of the system; to generate the basic data of the final structured upgrade execution plan.

[0057] Further, the corresponding upgrade content is determined based on the content of the plurality of upgrade nodes, the current upgrade progress of the FPGA firmware and the current working event of the FPGA firmware are collected; the first upgrade mode coefficient is determined according to the upgrade content of the plurality of upgrade nodes and the current upgrade progress of the FPGA firmware, which comprehensively considers the upgrade content of the plurality of upgrade nodes and the current upgrade progress of the FPGA firmware, ensuring the accuracy of the first upgrade mode coefficient.

[0058] At this time, we need to analyze the specific operations and data contained in each upgrade node in depth; we need to trace the role of these data within the FPGA, determine which part of the firmware they belong to, and what impact updating them will have on the functionality of the FPGA, which involves: data analysis: analyze the binary data contained in the node, identify whether they represent configuration parameters, logic code fragments, register values, or other information; function mapping: associate the parsed data with the specific function or module it controls in the FPGA; for example, determine whether a certain data block is used to configure the video processing unit parameters or to update the network interface card control logic; impact assessment: assess the consequences of updating the contents of this node; for example, updating a critical clock control parameter causes the system to be temporarily unstable; updating a non-critical configuration parameter has little impact; updating the bootloader requires a complete system restart.

[0059] Collecting the current upgrade progress of the FPGA firmware requires obtaining the execution status of the upgrade process so far, which usually includes: completed nodes: which upgrade nodes have been successfully executed; current node: which node is being executed or about to be executed; progress percentage: the completion ratio of the entire upgrade task; time information: including the time spent, estimated remaining time, etc.; understanding the current stage of the upgrade process is crucial for determining the next operation mode; for example, the system can tolerate different risks in the early and late stages of the upgrade.

[0060] Collecting the current working events of the FPGA firmware involves understanding the tasks or events the FPGA is currently executing, which can be achieved by monitoring system status, reading specific registers, or logs; working events include: normal tasks such as video stream processing, data acquisition, communication transmission, etc.; abnormal events such as error handling, interrupt response, etc.; system status such as idle, high load, low power mode, etc.; understanding the "busy degree" and "importance" of the FPGA will be combined with the upgrade content in the next step to determine the upgrade mode.

[0061] Combine the information obtained above (impact assessment of upgrade content + current upgrade progress) to generate a quantitative indicator - the first upgrade mode coefficient, the determination logic of this coefficient is as follows: content weight: for each upgrade node, assign a weight according to the impact assessment result of its content (such as "critical", "important", "general", "non-critical"); for example, "critical" update weight is high, "non-critical" update weight is low; progress factor: the current upgrade progress also affects the coefficient; for example, in the early stage of the upgrade (low progress), it is more inclined to choose a safer and slower mode; in the late stage of the upgrade (high progress), if the previous steps are successful, you can slightly relax the restrictions and speed up.

[0062] At this time, the first upgrade mode coefficient = Σ (the weight of each node * the state factor of the node at the current progress) / the total number of nodes; the state factor can be dynamically adjusted according to the progress, for example, the lower the progress, the smaller the factor, so that the coefficient as a whole is low, tending to be in a conservative mode; the higher the progress, the larger the factor, the coefficient rises, tending to be in a more aggressive mode; the first upgrade mode coefficient is usually designed within a certain range (for example, 0 to 1, or 0 to 100), the higher the value, the more urgent or critical the current upgrade task, or the current progress allows a more aggressive strategy; the first upgrade mode coefficient serves as a quantitative indicator that can reflect "based on the upgrade content and the current progress, what kind of upgrade urgency / risk tolerance should be adopted".

[0063] Specifically, suppose we are upgrading the previously mentioned video processing FPGA, and the current state is as follows: upgrade path and nodes: determined (same as the S141 example): node 1 (bootloader), node 2 (VPU basic configuration), node 3 (VPU parameter A), node 4 (VPU parameter B), node 5 (NIC configuration); current execution node: preparing to execute node 3 (VPU parameter A); current upgrade progress: 40% (nodes 1 and 2 have been completed); FPGA current work event: processing a medium-priority video stream.

[0064] Determine the upgrade content of node 3: node 3 contains parameter A of VPU (video processing unit); analyzing these data, it is found that parameter A controls a certain threshold of the color space conversion algorithm of the video.

[0065] Collect the current upgrade progress: completed nodes: node 1, node 2; current node: node 3; progress: 40%; collect the current work event: processing a medium-priority video stream, which means that the VPU is working, but it is not the highest priority task; determine the first upgrade mode coefficient: content weight: node 3 weight = 5; progress factor: current progress 40%, in the middle stage.

[0066] We set a simple factor: progress < 30%, factor = 0.7; 30% ≤ progress < 70%, factor = 1.0; progress ≥ 70%, factor = 1.3, here factor = 1.0; calculation: assuming we only consider the current node 3 to be executed, the first upgrade mode coefficient = (node 3 weight * progress factor) / 1 = (5 * 1.0) / 1 = 5.

[0067] Therefore, the second upgrade mode coefficient is determined according to the upgrade content of the plurality of upgrade nodes and the current working event of the FPGA firmware, and the upgrade mode of the FPGA firmware at each upgrade node is determined based on the first upgrade mode coefficient, the second upgrade mode coefficient and the upgrade mode mapping relationship, at this time, the FPGA firmware controls the execution of the FPGA firmware on the current working event in real time during the upgrade process, so as to avoid stopping the execution of the FPGA firmware on the current working event, and the overall consideration of the first upgrade mode coefficient, the second upgrade mode coefficient and the upgrade mode mapping relationship is compatible, and the accuracy of the upgrade mode of the FPGA firmware at each upgrade node is ensured.

[0068] At this time, it is determined that the content of the node to be upgraded at present (such as the VPU parameter A analyzed in S142) mainly affects which functional modules or registers of the FPGA; the key task (such as real-time video stream processing, key data transmission, sensor data acquisition, etc.) that the FPGA is currently processing is identified, and how high the requirements for real-time and continuity of these tasks are; it is judged whether the upgrade operation will affect the execution of the current key task; for example, if the upgrade content is a parameter of the VPU, and the task being performed at present is video encoding relying on this VPU, then the upgrade operation needs to be particularly careful to avoid interrupting the encoding process; according to the results of the conflict analysis, the second upgrade mode coefficient is calculated, which reflects the severity of the potential impact of the upgrade operation on the current working event; a simple factor can be set: no conflict: if the upgrade content is completely irrelevant to the current working event, the coefficient is set to a low value (for example, 0.2); slight conflict: if the upgrade content has some association with the current working event, but the impact is small (for example, updating a rarely used parameter), the coefficient is set to a medium-low value (for example, 0.5); serious conflict: if the upgrade content directly affects the execution of the current key task (for example, updating a register that is being frequently read), the coefficient is set to a high value (for example, 1.0).

[0069] The upgrade mode mapping relationship is a strategy table that defines the specific upgrade execution mode corresponding to different coefficient combinations; for example: (first coefficient low, second coefficient low) - mode A: fast write; can be completed quickly and almost does not affect the system; (first coefficient low, second coefficient high) - mode B: pause-update-resume; pause the affected key task, quickly update, and then restore the task; (first coefficient high, second coefficient low) - mode C: slow write and verification; although the content is not critical, but the progress is sensitive, so the writing speed is slowed down, and the verification step is increased to ensure safety; (first coefficient high, second coefficient high) - mode D: most conservative mode; pause all related tasks, perform update, complete verification, and then gradually restore; it needs more time.

[0070] Specifically, the calculated first coefficient (for example, 5) and second coefficient (for example, 1.0) are substituted into the mapping relationship; it is assumed that the mapping relationship is defined: if the first coefficient > 4 and the second coefficient > 0.8, mode D (the most conservative mode) is selected; in our example, the first coefficient 5 > 4 and the second coefficient 1.0 > 0.8, so it is determined that the upgrade mode of node 3 is mode D: the most conservative mode; the FPGA firmware controls the execution of the FPGA firmware on the current working event in real time during the upgrade process to avoid stopping the execution of the FPGA firmware on the current working event.

[0071] Reference Figure 6 In step S15, the specific steps are: S151: Real-time monitoring of the FPGA firmware upgrade process, and collecting the FPGA firmware upgrade progress table, which is updated step by step with the FPGA firmware upgrade to mark the upgraded content of the FPGA firmware, and collecting the un-upgraded content of the FPGA firmware; S152: Determining the abnormal upgrade area of the FPGA firmware according to the comparison of the upgraded content and the un-upgraded content of the FPGA firmware and the order of the multiple upgrade nodes, and determining the abnormal upgrade data of the FPGA firmware based on the detection of the abnormal upgrade area of the FPGA firmware; S153: Determining the dynamic optimization strategy of the FPGA firmware according to the abnormal upgrade data of the FPGA firmware, the current upgrade progress of the FPGA firmware, and the FPGA firmware upgrade progress table, and triggering the dynamic optimization based on the dynamic optimization strategy of the FPGA firmware.

[0072] In the embodiments of the present application, the FPGA firmware upgrade process is monitored in real time, and the monitoring can include: task state checking: checking whether the currently executing upgrade node (for example, the node determined in S141) is still in progress; hardware state feedback: checking the state register of the FPGA internal interface (such as JTAG, SPIFlash controller) used for upgrading to confirm whether an error flag is set (such as write timeout, verification failure); progress indication: if the upgrade operation supports, the completion percentage or the number of bytes written of the current operation is obtained; timestamp recording: recording the start and end time of each upgrade operation or stage; the frequency of monitoring needs to be high enough to discover abnormalities (such as write failure) in time, but not too frequent to affect the performance of the upgrade itself or increase unnecessary overhead; usually at the millisecond level or when the operation is completed.

[0073] At the beginning of the upgrade, the system needs to initialize a data structure (i.e., the "upgrade progress table"), which is usually a memory data structure, to record the detailed status of the upgrade; it contains the following information: version information: the version number of the new firmware; upgrade node list: contains all nodes that need to be upgraded (result from S141), each node has a unique identifier; status field: reserve a status field for each upgrade node to record the upgrade status of the node (e.g., not started, in progress, completed, failed); timestamp field: record the time of each node status change; verification information (optional): contains the checksum or hash value of each node data for subsequent verification; at the same time, the progress table is created and initialized before the upgrade starts; it is not dynamically created during the upgrade process, but an empty table is prepared to be filled.

[0074] The FPGA firmware upgrade progress table is updated step by step with the FPGA firmware upgrade, and the progress table needs to be updated to reflect the latest actual situation whenever the upgrade process changes; the update is usually triggered by the upgrade controller after performing an operation or detecting a status change, this process is automatic, ensuring that the progress table is always synchronized with the actual upgrade progress.

[0075] The nodes in the progress table with the status of completed represent which parts of the FPGA firmware have successfully applied the new content; by checking the progress table, it can be clearly known which function modules, configuration parameters, etc. have been updated to the new version, this "marker" is realized through the status field of the node in the progress table (such as marked as completed); it provides a clear way to identify the results of the upgrade.

[0076] Through the progress table, it is also easy to identify which parts have not been upgraded, including: nodes with the status of not started; nodes with the status of in progress (indicating that this part is being upgraded, but has not been completed); nodes with the status of failed (indicating that this part of the upgrade attempt has failed, and needs to be retried or skipped); by traversing the progress table and filtering out nodes with a status other than completed, a list of un-upgraded content can be obtained, which provides a basis for subsequent upgrade decisions (such as whether to retry, whether to skip, whether to terminate).

[0077] Further, the abnormal upgrade area of the FPGA firmware is determined according to the comparison of the upgraded content and the un-upgraded content of the FPGA firmware and the order of multiple upgrade nodes, and the abnormal upgrade data of the FPGA firmware is determined based on the detection of the abnormal upgrade area of the FPGA firmware, which is compatible with the overall consideration of the detection of the abnormal upgrade area of the FPGA firmware, and ensures the accuracy of the abnormal upgrade data of the FPGA firmware.

[0078] At this time, the current "upgraded content" list and "non-upgraded content" list are obtained from the upgrade process table of S151; the predetermined execution order list of the plurality of upgrade nodes determined in S141 (for example, [node A, node B, node C, node D]) is obtained; it is checked whether the nodes in the "upgraded content" all appear before the nodes in the "non-upgraded content" (according to the predetermined order); at the same time, it is checked whether there is a node in the "non-upgraded content" whose subsequent node appears in the "upgraded content"; if inconsistency is found, for example: a certain node X is not completed, but a node Y after X in the predetermined order has been completed > the node X and the subsequent dependent node Y constitute an abnormal area; a certain node X fails, but a node Y after X in the predetermined order has been completed > the node X and the subsequent dependent node Y constitute an abnormal area; a certain node X fails, and all subsequent nodes have not started > the node X itself constitutes an abnormal area; a certain node X is not completed, and has exceeded the predetermined time threshold > the node X itself constitutes an abnormal area; and the like; based on the checking results, one or more "abnormal upgrade areas" are determined, and the "area" here can be a node or a series of related nodes.

[0079] Once the abnormal area (usually one or more specific upgrade nodes) is determined, the next step is to go deep into the nodes to find out which data in the upgrade process has a problem, which requires: determining which upgrade node (or which upgrade nodes) in the abnormal area causes the problem; reviewing the "upgrade content" of the node determined in S141 (for example, whether a register is updated, a configuration parameter is updated, or an entire IP core is replaced); combining the specific error information (for example, error code, failed operation type, failed location address, etc.) monitored in real time in S151; and corresponding the error information to the upgrade content of the node to accurately locate the specific "abnormal upgrade data", which is a specific register value that fails to be correctly written, a bit field error of a configuration parameter, or a specific data block in a Flash memory that fails to be written, and the like.

[0080] Specifically, the abnormal upgrade area is determined: the upgraded content: [A, C]; the non-upgraded content: [B, D] (note that B fails and D is in progress); the node B (non-upgraded / failure) is predetermined before the node C (upgraded), which does not comply with the predetermined order! This is a clear abnormal signal; the node D (in progress / non-upgraded) is predetermined after the node C (upgraded), which complies with the order because D has not been completed and C has been completed, which is the normal order of execution.

[0081] The failure of the node B leads to inconsistency in the sequence; the node B itself constitutes an abnormal area; in addition, since B fails but C is completed, it means that the upgrade logic continues to execute C after B fails or the completion status of C is unreliable; therefore, the status of node C is also suspected and can be included in the abnormal area for further checking; the most core abnormal area is node B.

[0082] Abnormal node: node B (video processing unit driver); review S141, the upgrade content of node B is "write new VPU driver code to a specific BRAM area (for example, address range 0x1000-0x1FFF) in the FPGA"; it is assumed that this driver contains the core image processing algorithm; the monitoring record of S151 shows that when trying to write data at address 0x12C0, a "write timeout" error (error code 0x04) occurs; combined with the node content and monitoring information, it can be determined that the "abnormal upgrade data" is the specific 32-bit data word located at address 0x12C0 (assuming that one word is written each time), and this data word is a certain instruction or a key parameter in the driver code; since the data at this specific location fails to be successfully written, the upgrade of the entire node B fails.

[0083] The system not only identifies that the upgrade node "video processing unit driver" has a problem (abnormal area), but also further locates the problem to the specific operation "write data at address 0x12C0" (abnormal upgrade data), which provides accurate information for the subsequent step S153 to formulate a targeted dynamic optimization strategy (such as retrying the write at this address, switching to a backup write interface, or skipping this node and marking the system as a degraded state, etc.); in this example, the abnormal area is node B, and the abnormal upgrade data is the data at address 0x12C0 in node B.

[0084] Therefore, according to the abnormal upgrade data of the FPGA firmware, the current upgrade progress of the FPGA firmware and the upgrade process table of the FPGA firmware, the dynamic optimization strategy of the FPGA firmware is determined, the dynamic optimization of the dynamic optimization strategy of the FPGA firmware is triggered based on the dynamic optimization strategy of the FPGA firmware, the overall consideration of the dynamic optimization strategy trigger of the FPGA firmware is compatible, the accuracy of the dynamic optimization of the dynamic optimization strategy of the FPGA firmware is guaranteed, at the same time, the control of the upgrade mode of each upgrade node of the FPGA firmware is realized, further implementation of the overall consideration based on the upgrade process table of the FPGA firmware and the abnormal upgrade data of the FPGA firmware is realized, and the accuracy of the dynamic optimization strategy of the FPGA firmware is improved.

[0085] At this time, the abnormal upgrade data of the FPGA firmware, the current upgrade progress of the FPGA firmware, and the upgrade process table of the FPGA firmware are introduced. The abnormal upgrade data: the specific problem point located in S152; for example, is the data write failure of a certain specific address, or the configuration verification failure of a certain functional module, which determines the nature of the problem.

[0086] The current upgrade progress: which node has been upgraded to, which nodes have been successfully completed, and which nodes are still in progress or have failed, which determines the overall state of the system at present.

[0087] The upgrade process table: this table not only records the completed and incomplete content, but also implies the sequence and dependency of the upgrade; for example, if a node fails, does it affect the execution of subsequent nodes; based on this information, the system will calculate and select a most appropriate dynamic optimization strategy from a predefined strategy library; the selection logic of the strategy includes: Problem severity: is it a fatal error (such as bootloader failure) or a non-fatal error (such as configuration failure of a non-critical parameter); impact range: does the error affect the execution of subsequent nodes, does it cause damage to the core function of the system; Current progress: if most of the upgrade has been completed, it will tend to try to repair or bypass; if it just starts, it will tend to roll back or abort; resource availability: whether there are backup resources to switch to, whether there is additional processing time to retry.

[0088] Specific strategies include: Retry: retry a limited number of times for a specific operation that has failed (such as writing data to a specific address); Fallback Mode: if the upgrade of a critical function fails, temporarily switch to the old version or a simplified function mode to ensure basic system availability; Skip: if the upgrade of a non-critical node fails and does not affect subsequent nodes, skip the node and mark it as failed, and continue to execute subsequent upgrades; Rollback: if a serious error occurs during the upgrade process, and the system supports it, restore the successfully upgraded part to the old version; Switch Resource / Interface: if the failure reason is that a specific hardware resource (such as a certain memory controller) is busy or faulty, try to switch to a backup resource; Adjust Parameters: for example, reduce the write speed to reduce the error rate, or increase the verification step; Abort Upgrade: if the error cannot be recovered and there is no safe fallback option, abort the entire upgrade process.

[0089] Once a specific optimization strategy is determined (e.g., the decision to "retry writing data at address 0x12C0 three times"), the system needs to immediately execute this strategy, which involves: sending instructions to the upgrade controller or related hardware modules within the FPGA to execute the strategy; temporarily changing the working state of the FPGA, such as pausing certain non-critical tasks, to free up resources for the upgrade process; if the strategy involves adjusting parameters (such as reducing write speed), the values of related registers need to be modified; during the execution of the optimization strategy, its effect is continuously monitored to see if the problem is resolved.

[0090] Specifically, the abnormal upgrade data: data write at address 0x12C0 fails (error code 0x04: write timeout); current upgrade progress: upgrade to node B (video processing unit driver), progress about 50% (assuming address 0x12C0 is approximately in the middle of the data block to be written in node B); node A (boot loader) has been completed, node C (network interface controller configuration) has not started; upgrade process table: shows that node A has been completed, node B is in progress (failure), node C has not started; node order is A > B > C; the failure of node B should not theoretically affect node C (assuming that NIC configuration is independent of VPU driver).

[0091] The error is "write timeout", and the reasons include: the target BRAM area is temporarily busy, bus conflict, or the data at this address itself has a problem; current progress: just over half, node A has been completed, node C has not started; process table: node B failure does not affect subsequent node C; strategy selection: considering that the error is "timeout", first try non-invasive repair; strategy options are: retry writing this address; temporarily pause other non-critical tasks to release bus resources, then retry.

[0092] Retry after reducing write speed; if it still fails after several retries, consider skipping this node B (if VPU driver is not an absolute core function) and marking the system as a degraded state (some video processing functions are limited); select "first retry writing data at address 0x12C0 three times, with 50ms interval between each retry"; if all three fail, switch strategy to "skip node B, mark VPU driver as old version, continue executing node C".

[0093] The upgrade controller receives the instruction and starts executing the "retry 3 times" strategy; attempt 1: write data to address 0x12C0 again; result: failure (timeout); wait 50ms; attempt 2: write again; result: failure (timeout); wait 50ms; attempt 3: write again; result: success.

[0094] The third retry is successful, the data at address 0x12C0 is correctly written, the upgrade controller continues to complete the remaining write operation of node B, and it is assumed that the completion is successful, then, according to the process table, the upgrade of node C (network interface controller configuration) is started.

[0095] Specifically, an optimization strategy matching table is defined in advance; when specific abnormal data is detected, the matching table is searched according to the current progress and the process table to find the corresponding optimization strategy; the optimization strategy matching table is shown in Table 1: Table 1 Optimization strategy matching table

[0096] It is known that: abnormal upgrade data: the data at address 0x12C0 in node B (video processing unit driver) fails to be written; current upgrade progress: node B is being executed, and about 50% has been completed; upgrade process table: the node sequence is A>B>C; after node B fails, node C (network interface controller configuration) can be independently run.

[0097] In the matching table, the line with the "abnormal type" as "specific address write failure (retry <3)", the "current node state" as "node in progress", and the "impact on subsequent nodes" as "none" is searched; the optimization strategy suggested by the matching table is "retry writing (3 times, interval 50 ms)"; the upgrade controller receives the instruction and executes the "retry writing (3 times, interval 50 ms)" strategy; if the retry is successful, node B is continued; if the retry fails 3 times, according to the matching table, the strategy will be automatically switched to "skip this node, mark downgrade".

[0098] Please refer to Figure 7 , Figure 7 is a structural composition diagram of the FPGA firmware upgrade system based on image control in the embodiment of the application; the FPGA firmware upgrade system based on image control comprises: An image format combination module 21 is configured to mark the image format combination of the firmware data image according to the image detection of the firmware data image. A pixel data module 22 is configured to determine multiple rows of pixel data according to the analysis of the image format combination of the firmware data image, and determine first row pixel data, second row pixel data and third row pixel data based on the division of the multiple rows of pixel data. An upgrade path module 23 is configured to sequentially perform step-by-step upgrade on the first row pixel data, the second row pixel data and the third row pixel data, and determine the upgrade path of the FPGA firmware according to the first layer upgrade content of the second row pixel data, the second layer upgrade content of the third row pixel data and the upgrade version number of the FPGA firmware. The upgrade mode module 24 is configured to determine a plurality of upgrade nodes based on the detection of the upgrade path of the FPGA firmware, determine the upgrade mode of the FPGA firmware at each upgrade node according to the upgrade content of the plurality of upgrade nodes, the current upgrade progress of the FPGA firmware and the current working event of the FPGA firmware, and not affect the execution of the FPGA firmware on the current working event; The dynamic optimization strategy module 25 is configured to update the upgrade progress table of the FPGA firmware step by step during the upgrade of the FPGA firmware, and determine the dynamic optimization strategy of the FPGA firmware based on the upgrade progress table of the FPGA firmware and the abnormal upgrade data of the FPGA firmware.

[0099] Any combination of the technical features in the above embodiments is possible. In order to make the description concise, not all combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist, they should be considered as the scope of the present disclosure.

Claims

1. A method for upgrading FPGA firmware based on image control, characterized in that, include: The image format combination of the firmware data image is marked based on image detection of the firmware data image; The multi-row pixel data is determined by parsing the image format combination of the firmware data image, and the first row of pixel data, the second row of pixel data, and the third row of pixel data are determined based on the division of the multi-row pixel data; The first row of pixel data, the second row of pixel data, and the third row of pixel data are upgraded sequentially. The upgrade path of the FPGA firmware is determined based on the first-level upgrade content of the second row of pixel data, the second-level upgrade content of the third row of pixel data, and the upgrade version number of the FPGA firmware. Multiple upgrade nodes are determined based on the detection of the upgrade path of the FPGA firmware. The upgrade mode of the FPGA firmware at each upgrade node is determined according to the upgrade content of multiple upgrade nodes, the current upgrade progress of the FPGA firmware, and the current working event of the FPGA firmware, without affecting the execution of the current working event of the FPGA firmware. During the FPGA firmware upgrade process, the FPGA firmware upgrade progress table is updated step by step, and the dynamic optimization strategy of the FPGA firmware is determined based on the FPGA firmware upgrade progress table and abnormal upgrade data of the FPGA firmware.

2. The FPGA firmware upgrade method based on image control according to claim 1, characterized in that, The image format combination for marking firmware data images based on image detection of firmware data images includes: When upgrading the FPGA firmware, the firmware data image corresponding to the FPGA firmware is acquired, and image detection is performed on the firmware data image to output the FPGA firmware data set. Based on the filtering of the FPGA firmware data set, multiple image format data of the FPGA firmware are determined. Based on the synthesis of the multiple image format data of the FPGA firmware, the image format combination of the firmware data image is determined, and the image format combination of the firmware data image is marked.

3. The FPGA firmware upgrade method based on image control according to claim 1, characterized in that, The step of determining multiple rows of pixel data by parsing the image format combination of the firmware data image, and determining the first row of pixel data, the second row of pixel data, and the third row of pixel data based on the division of the multiple rows of pixel data, includes: The image format combination of the firmware data image is acquired, the row and column detection of the image format combination of the firmware data image is performed, and the multi-row pixel data is determined based on the row and column detection of the image format combination of the firmware data image. At this time, the multi-row pixel data is arranged sequentially. The multi-row pixel data is divided and labeled with multiple data markers. Based on the tracing of multiple data markers, the first row of pixel data, the second row of pixel data, and the third row of pixel data are output. The first row of pixel data, the second row of pixel data, and the third row of pixel data are arranged in sequence and present corresponding information. The first row of pixel data, the second row of pixel data, and the third row of pixel data are all different.

4. The FPGA firmware upgrade method based on image control according to claim 1, characterized in that, The process involves sequentially upgrading the first row of pixel data, the second row of pixel data, and the third row of pixel data. The upgrade path for the FPGA firmware is determined based on the first-level upgrade content of the second row of pixel data, the second-level upgrade content of the third row of pixel data, and the FPGA firmware upgrade version number. This includes: The first row of pixel data, the second row of pixel data, and the third row of pixel data are compared sequentially, and the first row of pixel data is used as the frame start data of the upgrade firmware to present the frame start position and frame sequence number of the FPGA firmware.

5. The FPGA firmware upgrade method based on image control according to claim 4, characterized in that, The step-by-step upgrade of the first row of pixel data, the second row of pixel data, and the third row of pixel data, and the determination of the FPGA firmware upgrade path based on the first-level upgrade content of the second row of pixel data, the second-level upgrade content of the third row of pixel data, and the FPGA firmware upgrade version number, also includes: The first-level upgrade firmware data is determined by comparing the second row of pixel data with the first row of pixel data, and the first-level upgrade content of the FPGA firmware is determined by parsing the first-level upgrade firmware data. At this time, the first sub-upgrade path is determined based on the first-level upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware. The second-layer upgrade firmware data is determined by comparing the third row of pixel data with the second row of pixel data. The second-layer upgrade content of the FPGA firmware is determined by parsing the second-layer upgrade firmware data. At this time, the second sub-upgrade path is determined based on the second-layer upgrade content of the FPGA firmware and the upgrade version number of the FPGA firmware. The upgrade path of the FPGA firmware is determined based on the first sub-upgrade path, the second sub-upgrade path and the model of the FPGA firmware.

6. The FPGA firmware upgrade method based on image control according to claim 1, characterized in that, The method involves detecting the upgrade path of the FPGA firmware to determine multiple upgrade nodes. Based on the upgrade content of these nodes, the current upgrade progress of the FPGA firmware, and the current working events of the FPGA firmware, the upgrade mode of the FPGA firmware at each upgrade node is determined without affecting the execution of the current working events by the FPGA firmware. This includes: The upgrade path of the FPGA firmware is collected, and multiple upgrade regions of the FPGA firmware are determined based on the detection of the upgrade path. The corresponding upgrade nodes are determined based on the detection of each upgrade region, so as to collect multiple upgrade nodes.

7. The FPGA firmware upgrade method based on image control according to claim 6, characterized in that, The method of determining multiple upgrade nodes based on the detection of the FPGA firmware upgrade path, determining the upgrade mode of the FPGA firmware at each upgrade node based on the upgrade content of the multiple upgrade nodes, the current upgrade progress of the FPGA firmware, and the current working event of the FPGA firmware, without affecting the execution of the current working event by the FPGA firmware, also includes: The corresponding upgrade content is determined by tracing the content of multiple upgrade nodes, and the current upgrade progress and current working events of the FPGA firmware are collected; the first upgrade mode coefficient is determined based on the upgrade content of multiple upgrade nodes and the current upgrade progress of the FPGA firmware. The second upgrade mode coefficient is determined based on the upgrade content of multiple upgrade nodes and the current working event of the FPGA firmware. The upgrade mode of the FPGA firmware at each upgrade node is determined based on the first upgrade mode coefficient, the second upgrade mode coefficient and the upgrade mode mapping relationship. At this time, the FPGA firmware monitors the execution of the current working event in real time during the upgrade process to avoid stopping and affecting the execution of the current working event.

8. The FPGA firmware upgrade method based on image control according to claim 1, characterized in that, During the FPGA firmware upgrade process, the FPGA firmware upgrade progress table is updated progressively, and a dynamic optimization strategy for the FPGA firmware is determined based on the FPGA firmware upgrade progress table and abnormal upgrade data, including: The upgrade process of the FPGA firmware is monitored in real time, and the upgrade progress table of the FPGA firmware is collected. The upgrade progress table of the FPGA firmware is updated step by step as the FPGA firmware is upgraded to mark the upgraded content of the FPGA firmware, and the unupgraded content of the FPGA firmware is collected. The abnormal upgrade area of ​​the FPGA firmware is determined by comparing the upgraded and non-upgraded content of the FPGA firmware and the order of multiple upgrade nodes, and the abnormal upgrade data of the FPGA firmware is determined based on the detection of the abnormal upgrade area of ​​the FPGA firmware.

9. The FPGA firmware upgrade method based on image control according to claim 8, characterized in that, The process of gradually updating the FPGA firmware upgrade progress table during the FPGA firmware upgrade process, and determining the dynamic optimization strategy for the FPGA firmware based on the FPGA firmware upgrade progress table and abnormal upgrade data, also includes: Based on the abnormal upgrade data of the FPGA firmware, the current upgrade progress of the FPGA firmware, and the upgrade process table of the FPGA firmware, the dynamic optimization strategy of the FPGA firmware is determined, and the dynamic optimization of the FPGA firmware is triggered based on the dynamic optimization strategy of the FPGA firmware.

10. An FPGA firmware upgrade system based on image control, characterized in that, The image-controlled FPGA firmware upgrade system is applied to the image-controlled FPGA firmware upgrade method as described in any one of claims 1-9, wherein the image-controlled FPGA firmware upgrade system comprises: An image format combination module is used to mark the image format combination of firmware data images based on image detection of firmware data images; The pixel data module is used to determine multiple rows of pixel data based on the parsing of the image format combination of the firmware data image, and to determine the first row of pixel data, the second row of pixel data, and the third row of pixel data based on the division of the multiple rows of pixel data; The upgrade path module is used to upgrade the first row of pixel data, the second row of pixel data, and the third row of pixel data in sequence. The upgrade path of the FPGA firmware is determined based on the first-level upgrade content of the second row of pixel data, the second-level upgrade content of the third row of pixel data, and the upgrade version number of the FPGA firmware. The upgrade mode module is used to determine multiple upgrade nodes based on the detection of the upgrade path of the FPGA firmware. It determines the upgrade mode of the FPGA firmware at each upgrade node according to the upgrade content of multiple upgrade nodes, the current upgrade progress of the FPGA firmware, and the current working event of the FPGA firmware, without affecting the execution of the current working event by the FPGA firmware. The dynamic optimization strategy module is used to gradually update the FPGA firmware upgrade process table during the FPGA firmware upgrade process, and determine the dynamic optimization strategy of the FPGA firmware based on the FPGA firmware upgrade process table and abnormal upgrade data of the FPGA firmware.

Images