An image processing method, device, medium and product for data stream reorganization

Abstract

An image processing method, device, medium and product for data stream reorganization. In the method, original image data stream is acquired, and the row coordinate and column coordinate of each pixel data in a preset pixel block are determined; a corresponding write cache address is generated for each pixel data according to the row coordinate and column coordinate; each pixel data is written into a target cache unit in a plurality of preset cache units according to the write cache address; a read cache address is generated according to a global read row count value and a global read column count value; the pixel data is read from the target cache unit and aggregated into a reorganized pixel row; the reorganized pixel row is written into a continuous storage area in a preset memory to obtain reorganized image data; and the reorganized image data is sequentially read out from the continuous storage area to obtain a reorganized image video stream. The technical solution provided in the application improves the overall throughput of data processing, and is particularly suitable for real-time processing of high-resolution and high-frame-rate video streams.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method, device, medium and product for data stream reorganization. BACKGROUND

[0002] In the field of high-definition video processing, real-time image analysis, etc., image data is often input in the form of a continuous pixel stream, and usually follows the order of raster scanning. In order to meet the requirements of specific algorithms (such as convolution operation, feature extraction, etc.) downstream for data arrangement format, the input pixel stream often needs to be reorganized. A common processing method is to use frame buffer technology. Specifically, this technology first allocates a continuous memory space with sufficient capacity as a frame buffer area, and the input pixel data stream is written into the buffer area row by row and sequentially according to its original row-column order until a complete image frame data is completely stored. Then, when data reorganization is needed, the frame buffer area with stored data is read at a random address according to the target address after reorganization to obtain the required pixel data.

[0003] However, the above-mentioned frame buffer-based technical solution has significant limitations in system resource utilization. Specifically, the solution causes a phased, non-uniform occupation of memory bandwidth resources. In the data writing phase, the system bus is used for continuous writing operation for a long time; and in the subsequent data reading phase, due to the large-scale random address access required by the reorganization operation, it will also cause a high-intensity access pressure on the bus for a moment. This time-sharing, high-intensity occupation mode of bus resources not only reduces the overall throughput of data processing, but also makes the system prone to performance bottlenecks when dealing with high-resolution, high-frame-rate data streams. SUMMARY

[0004] To solve the above technical problems, the present application provides an image processing method, device, medium and product for data stream reorganization.

[0005] In the first aspect of the present application, an image processing method for data stream reorganization is provided, which adopts the following technical solution:

[0006] Obtain an original image data stream, and determine the row coordinate and column coordinate of each pixel data in the original image data stream in a preset pixel block;

[0007] According to the row coordinate and the column coordinate, generate a corresponding write cache address for each pixel data;

[0008] According to the write cache address, write each pixel data to a target cache unit in a plurality of preset cache units;

[0009] The read cache address is generated based on the preset global read row count value and the preset global read column count value;

[0010] Based on the read cache address, the pixel data is read from the target cache unit and aggregated into a reconstructed pixel row;

[0011] The reconstructed pixel rows are sequentially written into a continuous storage area in a preset memory to obtain reconstructed image data;

[0012] The reconstructed image data is read sequentially from the contiguous storage area to obtain a reconstructed image video stream.

[0013] By adopting the above technical solution, and introducing multiple preset buffer units (such as FIFO (First Input First Output)) for distributed storage and pipelined processing of pixel data, the problem of phased and unbalanced occupation of memory bandwidth resources caused by frame buffering schemes in the background technology is effectively solved. Specifically, in the data writing stage, the system generates a write address in real time based on the row and column coordinates of the pixel within a preset pixel block, and distributes the pixel data to multiple buffer units, avoiding the long-term bus occupation caused by continuous writing of a single frame buffer, and realizing the transformation from "burst" bus occupation to "stream" bus occupation; in the data reading stage, a read address is generated by a preset global counter, and pixel data is read from the buffer units in parallel or sequentially and aggregated into reconstructed pixel rows, reducing the instantaneous bus pressure caused by random access. This design allows write and read operations to be executed concurrently, realizing balanced utilization of memory bandwidth, thereby improving the overall throughput of data processing, especially suitable for real-time processing of high-resolution, high-frame-rate video streams, and avoiding performance bottlenecks.

[0014] Optionally, acquiring the original image data stream and determining the row and column coordinates of each pixel data in the original image data stream within a preset pixel block includes:

[0015] The global row coordinates and global column coordinates of each pixel data are decomposed into multiple preset row coordinate bit groups and column coordinate bit groups, and the row coordinate bit groups and column coordinate bit groups are combined to obtain a set of bit groups;

[0016] According to the preset bit recombination rules, a first target bit group is determined, and the first target bit group is selected from the set of bit groups for cross-combination to generate a recombined row coordinate bit sequence;

[0017] According to the bit recombination rule, a second target bit group is selected from the plurality of row coordinate bit groups and the plurality of column coordinate bit groups for cross-combination to generate recombined column coordinate bits. The second target bit group is the set of bit groups minus the first target bit group.

[0018] The recombined row coordinate bit sequence and the recombined column coordinate bit sequence are respectively determined as the row coordinate and the column coordinate.

[0019] By employing the aforementioned technical solution, global coordinates are decomposed into bit groups and cross-recombined, achieving efficient distribution and flexible mapping of pixel data within cache units. This bit-level recombining rule ensures that pixel data is evenly distributed across multiple cache units based on its row and column characteristics, avoiding centralized bandwidth occupation caused by continuous writes in traditional frame buffers, while also reducing random access conflicts during subsequent reads. By optimizing the data storage mode, this method effectively improves the parallelism and data locality of cache access, thereby reducing bus pressure, supporting smooth execution of pipelined operations, and collaboratively addressing the technical problem of unbalanced memory bandwidth resource occupation in conjunction with the overall solution.

[0020] Optionally, generating a corresponding write cache address for each pixel data based on the row coordinates and the column coordinates includes:

[0021] Based on the column coordinates, the target cache unit is determined from the plurality of preset cache units, and the preset base address of the target cache unit is obtained;

[0022] The row coordinates are determined as the write offset within the target cache unit;

[0023] The preset base address and the write offset are combined using a preset address operation to generate the corresponding write cache address.

[0024] By adopting the above technical solution, and utilizing an address generation mechanism that directly maps column coordinates to target cache units and uses row coordinates as offsets, distributed storage and rapid positioning of pixel data are achieved. This address combination method disperses the originally continuous pixel stream into multiple cache units according to its characteristics, effectively avoiding access conflicts in a single memory bank. Simultaneously, the simple operation of adding the base address and offset reduces the complexity of address calculations. This design makes the data writing process deterministic and parallel, laying the foundation for subsequent pipelined read operations, significantly improving data reassembly efficiency, and working in conjunction with the overall solution to achieve balanced utilization of memory bandwidth.

[0025] Optionally, generating the read cache address based on the preset global read row count value and the preset global read column count value includes:

[0026] Based on the preset global read row count value and the preset global read column count value, the location identifier and read offset of the target cache unit are generated respectively through the preset address decomposition rule;

[0027] Based on the location identifier, obtain the corresponding cache unit base address;

[0028] The read cache address is generated by performing a preset address combination operation on the base address of the cache unit and the read offset.

[0029] By adopting the above technical solution, the address decomposition rule based on the global read counter achieves accurate generation and pipelined control of read addresses. This design decomposes global coordinates into buffer unit location identifiers and read offsets, enabling the system to access multiple buffer units in parallel, effectively avoiding the random access bottleneck in traditional frame buffering schemes. The hardware-friendly nature of address combination operations significantly reduces access latency, ensuring that read and write operations are executed synchronously in a pipelined manner. This dynamic address generation mechanism effectively balances memory bandwidth usage, improves the data reassembly efficiency of high-resolution video streams, and provides stable timing guarantees for real-time image processing.

[0030] Optionally, the step of performing a preset address combination operation on the base address of the cache unit and the read offset to generate the read cache address includes:

[0031] According to the preset offset bit width, the base address of the cache unit is left-shifted to obtain the shifted base address;

[0032] By using a preset offset mask, a bitwise AND operation is performed on the read offset to obtain the masked read offset;

[0033] The shifted base address and the masked read offset are bitwise ORed to generate the read cache address.

[0034] By adopting the above technical solution, address combination operations implemented using bitwise operations significantly improve address generation efficiency. Left shift operation achieves bit-field isolation between the base address and the offset, masking operation ensures effective bit-width control of the offset, and bitwise OR operation completes rapid address synthesis. This purely hardware-friendly address calculation method avoids complex multiplication operations, greatly reducing logic latency and resource consumption. This design enables read address generation to be completed within one clock cycle, effectively supporting high-frequency pipelined operations, providing stable timing guarantees for real-time video processing, and optimizing memory access efficiency in conjunction with the overall solution.

[0035] Optionally, reading the pixel data from the target cache unit and aggregating it into a reconstructed pixel row based on the read cache address includes:

[0036] Based on the read cache address and the preset cache unit scheduling sequence, a subset of target cache units is determined from the target cache units;

[0037] Data is read in parallel from a subset of the target cache units to obtain multiple pixel data blocks;

[0038] According to the preset pixel arrangement rules and the preset pixel characteristic mapping table, the multiple pixel data blocks are merged to obtain a combined pixel data stream;

[0039] Determine the seam region formed by splicing the pixel data blocks in the combined pixel data stream, and adjust the seam region using a preset smoothing algorithm to obtain a smoothed pixel data stream;

[0040] The smoothed pixel data stream is aggregated to obtain the recombined pixel row.

[0041] By employing the above technical solutions, efficient reassembly and quality optimization of pixel data are achieved. The parallel reading mechanism based on cache unit scheduling sequences effectively improves data throughput and avoids access conflicts inherent in traditional single-cache systems. Data block merging guided by a pixel feature mapping table ensures the logical consistency of the reassembled pixel arrangement. Intelligent smoothing processing of seam areas significantly improves visual continuity and eliminates image artifacts caused by data block stitching. This series of operations maintains pipeline efficiency while improving the quality of the reassembled image, making this solution particularly suitable for real-time processing scenarios with high image quality requirements.

[0042] Optionally, sequentially reading the reconstructed image data from the contiguous storage area to obtain the reconstructed image video stream includes:

[0043] The preset starting address of the contiguous storage region is initialized as a read address pointer;

[0044] According to the preset read clock, data units are continuously read from the continuous storage area based on the read address pointer, and a preset step increment operation is performed on the read address pointer after each read.

[0045] All the read data units are combined in the order they were read to generate the reconstructed image video stream.

[0046] By adopting the above technical solution, a sequential address pointer-based reading mechanism achieves efficient output of reconstructed image data. This design optimizes the memory access mode to sequential reading through address increment operations with preset steps, fully leveraging the burst transmission advantages of modern memory architectures and significantly improving data reading efficiency. The sequential reading method avoids the address switching overhead of random access, ensuring balanced and sustainable utilization of memory bandwidth and effectively solving the problem of phased bus resource occupancy in the background technology. This reading strategy works in conjunction with the aforementioned pipelined processing stage to ensure that the entire system maintains a stable data throughput when processing high-resolution, high-frame-rate video streams.

[0047] A second aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0048] A third aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0049] A fourth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the method as described in any of the preceding claims.

[0050] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0051] By constructing a distributed caching unit and pipelined processing architecture, and combining multi-layered technological innovations such as bit-level coordinate reconstruction, hardware-optimized address generation mechanisms, parallel read scheduling, and intelligent seam processing, efficient reconstruction of pixel data is achieved. This solution effectively solves the problem of phased memory bandwidth occupation in traditional frame buffering technology, significantly improves system throughput through balanced data access patterns, and ensures the quality of the reconstructed image. The entire technical solution, while maintaining hardware friendliness, provides a complete and feasible solution for real-time processing of high-resolution, high-frame-rate video streams, demonstrating significant technical advantages and application value. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the system architecture of an embodiment of an image processing method for data stream reconstruction according to this application;

[0053] Figure 2 This is a schematic flowchart of an image processing method for data stream reconstruction disclosed in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0055] Explanation of reference numerals in the attached figures: 100, System architecture; 101, First terminal device; 102, Second terminal device; 103, Third terminal device; 104, Network; 105, Server; 301, Processor; 302, Communication bus; 303, User interface; 304, Network interface; 305, Memory. Detailed Implementation

[0056] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0057] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0058] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0059] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, network 103, and server 105. Network 103 is used as a medium to provide communication links between terminal devices 101, 102, and 103 and server 105. Network 103 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0060] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 103 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as model training applications, video recognition applications, web browser applications, social platform software, etc.

[0061] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP3 (Moving Picture Experts Group Audio Layer IV) players, laptops, and desktop computers, etc. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0062] This embodiment discloses an image processing method for data stream reconstruction. Figure 2 This is a flowchart illustrating an image processing method for data stream reconstruction disclosed in an embodiment of this application. The following embodiments use the implementation of this method on an FPGA (Field Programmable Gate Array) as an example for detailed explanation. However, those skilled in the art should understand that the technical solutions of this application can also be applied to other hardware or hardware / software combined platforms such as ASIC (Application-Specific Integrated Circuit) chips and multi-core processor systems, and these variations all fall within the protection scope of this application. Figure 2 As shown, the method includes the following steps:

[0063] S201. Obtain the original image data stream and determine the row and column coordinates of each pixel data in the original image data stream within a preset pixel block.

[0064] Specifically, the system receives raw image data streams via standard MIPI CSI-2 or HDMI interfaces and uses built-in synchronous row and column counters to generate unique global coordinates for each pixel entering the processing pipeline within a complete image frame in real time. Subsequently, to map these global coordinates to local row and column coordinates within a predefined logical pixel block (its size N, for example, N×N, related to the number of subsequent distributed buffer units), the core of the system performs efficient modular arithmetic. Specifically, to meet the stringent timing requirements of real-time hardware processing, this modular arithmetic employs a highly optimized implementation: when the pixel block size N is a power of 2 (N=2k, where k is a positive integer representing the number of bits required to represent size N), the operation can be greatly simplified by hardware to extract only the low k bits of the global coordinates, producing almost no logical delay; for sizes other than powers of 2, a lookup table (LUT) pre-installed in on-chip memory (such as BRAM) is used to perform parallel lookups in a single clock cycle to obtain the local coordinates. To ensure that the entire coordinate generation process does not create processing bottlenecks, this solution is built on a two-stage pipeline architecture. The first-stage register temporarily stores the raw pixel data and its global coordinates, while the second-stage register caches the calculated local coordinates. This ensures that a set of local row and column coordinates strictly aligned with the pixel data can be stably output in each pixel clock cycle. Its fixed computation latency (e.g., two clock cycles in this embodiment) fully meets the high throughput processing requirements of high-resolution, high-frame-rate video streams. For example, assuming the processing of a 1920x1080 resolution image, the preset pixel block size is 8x8 (i.e., N=8, k=3). For a pixel with global coordinates (row 10, column 20), its local coordinates within the pixel block are calculated as follows: row coordinate _local=10mod8=2; column coordinate _local=20mod8=4.

[0065] Optionally, acquiring the original image data stream and determining the row and column coordinates of each pixel data in the original image data stream within a preset pixel block includes:

[0066] The global row coordinates and global column coordinates of each pixel data are decomposed into multiple preset row coordinate bit groups and column coordinate bit groups, and the row coordinate bit groups and column coordinate bit groups are combined to obtain a set of bit groups;

[0067] According to the preset bit recombination rules, a first target bit group is determined, and the first target bit group is selected from the set of bit groups for cross-combination to generate a recombined row coordinate bit sequence;

[0068] According to the bit recombination rule, a second target bit group is selected from the plurality of row coordinate bit groups and the plurality of column coordinate bit groups for cross-combination to generate recombined column coordinate bits. The second target bit group is the set of bit groups minus the first target bit group.

[0069] The recombined row coordinate bit sequence and the recombined column coordinate bit sequence are respectively determined as the row coordinate and the column coordinate.

[0070] Specifically, the bit width k = log2(n) is determined according to the preset pixel block size n. Then, a parallel bit extraction circuit is used in the FPGA: the binary representation of the global row coordinate v_cnt is grouped into multiple row coordinate bit groups from low bit to high bit (e.g., v_cnt[1:k] is the first group, v_cnt[k+1:2k] is the second group, and so on). At the same time, the same operation is performed on the global column coordinate h_cnt to obtain multiple column coordinate bit groups. The bits of each group are connected in row priority order through the hardware splicing unit to form a complete bit group set. This set is used as a temporary register with a bit width of 2×m×k (m is the number of groups) and is latched and output in the next clock cycle to ensure that the coordinate decomposition and combination are completed in a single clock cycle.

[0071] Furthermore, a programmable bit mapping table is pre-set in the FPGA, which stores the bit selection sequence pattern; the contents of the mapping table are parsed by the address decoding circuit to generate the control signal of the multiplexer; a crossbar switch array is used to dynamically select the specified bit from the bit group set, wherein the row coordinate bit group and the column coordinate bit group are alternately input to the selector through time division multiplexing; the selected bit is rearranged in bit order through the shift register, and the cyclic shift operation of the bit is implemented by the barrel shifter; finally, the recombined bit sequence is latched and output through the parallel loading register to form a new row coordinate bit sequence. The entire recombining process is completed within 3 clock cycles, and the throughput of processing one pixel per clock cycle is maintained by the pipeline structure. For example, for a 12-bit global row coordinate v_cnt[11:0] and a 12-bit global column coordinate h_cnt[11:0], if the pixel block size is 16x16, the local coordinates require 4 bits. One possible bit recombination rule is that the new row coordinates are formed by the cross combination of {v_cnt[3:0], h_cnt[3:0]}, and the new column coordinates are formed by the cross combination of {v_cnt[7:4], h_cnt[7:4]}.

[0072] Furthermore, a complementary selection logic circuit is designed in the FPGA. After receiving the gating signal of the first target bit group, the circuit generates the enable signal of the second target bit group through bit inversion. The remaining bit groups not occupied by the first target bit group are filtered from the bit group set using a multiplexer array, and the arrangement order of these remaining bit groups is determined by a priority encoder. The filtered bit groups are processed by a cross-combination unit, and bit interleaving technology is used to rearrange the bits from different sources in an even-odd interval manner. Finally, the recombined column coordinate bit sequence is aligned and output through a parallel shift register. The entire process is completed within 2 clock cycles and is synchronized with the row coordinate generation process.

[0073] Furthermore, a coordinate synthesis circuit is designed in the FPGA. This circuit contains two independent binary-to-integer conversion modules, which process the bit sequences of row coordinates and column coordinates respectively. Each conversion module adopts a parallel adder tree structure to sum the weighted bits of the bit sequence, where the least significant bit has a weight of 1, and the weights increase in powers of 2. The converted integer value is sent to the range verification unit, and a comparator ensures that the coordinate value falls within the preset valid range (1 to n). If it exceeds the range, it is automatically modulo-based. The final result is latched into the output register, and a valid coordinate signal is generated at the same time. The entire coordinate determination process is completed within 2 clock cycles, and timing consistency is ensured by a pipelined register.

[0074] S202. Generate a corresponding write cache address for each pixel data according to the row coordinates and the column coordinates.

[0075] Specifically, an address generation pipeline is constructed in the FPGA. The column coordinate values ​​are mapped to an n-to-one cache cell selection signal by a column coordinate decoding circuit. This decoding circuit is implemented using a priority encoder and reads the preset base address of the corresponding cache cell from the base address register file. Then, the row coordinate values ​​are input into the offset calculation unit and expanded to a bit width matching the base address by a bit width expansion circuit to form a complete write offset. Finally, a parallel adder is used to add the base address and the write offset to generate the physical write address. The address stability is ensured by a two-stage pipeline register. The entire address generation process is completed within 3 clock cycles, and each cycle can process the address calculation of one pixel, meeting the timing requirements of real-time video processing.

[0076] Optionally, generating a corresponding write cache address for each pixel data based on the row coordinates and the column coordinates includes:

[0077] Based on the column coordinates, the target cache unit is determined from the plurality of preset cache units, and the preset base address of the target cache unit is obtained;

[0078] The row coordinates are determined as the write offset within the target cache unit;

[0079] The preset base address and the write offset are combined using a preset address operation to generate the corresponding write cache address.

[0080] Specifically, a cache cell index mapping circuit is designed in the FPGA. This circuit takes column coordinates as input and maps them directly to the corresponding cache cell identifiers through a pre-programmed lookup table (LUT). The contents of the lookup table are pre-configured so that each column coordinate value (range 1 to n) corresponds to a unique cache cell number (e.g., FIFO_1 to FIFO_n). At the same time, the preset base address of each cache cell is stored in the register file. These base addresses are loaded through the configuration interface during system initialization. When a column coordinate is input, the index mapping circuit outputs the cache cell identifier and triggers the address selector to read the corresponding base address from the register file. The entire process is implemented using combinational logic to ensure that the base address is acquired within one clock cycle and to eliminate metastability risks through a synchronization register.

[0081] Furthermore, an address generation unit is designed in the FPGA, which includes an offset calculation module and an address synthesis module. The offset calculation module directly uses the row coordinate value as the base address offset, and at the same time, it ensures that the offset bit width is aligned with the base address through a bit extension circuit. The address synthesis module adopts a parallel adder structure, which performs binary addition operation on the preset base address and the write offset to generate the physical address. In order to eliminate the timing risk of address calculation, a two-stage pipeline structure is designed. The first stage completes the offset preprocessing, and the second stage performs the address addition operation. The finally generated write cache address is latched through the output register, and the address stability is ensured with the address valid signal. The entire address generation process is completed within 2 clock cycles and is synchronized with the pixel data stream.

[0082] S203. Based on the write cache address, write each pixel data into the target cache unit among multiple preset cache units.

[0083] Specifically, a distributed write control logic is constructed in the FPGA. First, a corresponding write enable signal is generated based on the generated write cache address. This signal is converted into a chip select signal for a specific cache unit through an address decoding circuit. Simultaneously, a data path controller is designed to synchronously transmit the input pixel data and address signal to the write port of the target cache unit. A multiplexer architecture is used to dynamically allocate the data flow, ensuring that only one cache unit is in an active write state each clock cycle. The entire write process is implemented through a three-stage pipeline. The first stage completes address decoding and chip select signal generation, the second stage performs data buffering and timing alignment, and the third stage executes the actual write operation and generates a write confirmation signal. All operations are completed under the pixel clock drive, ensuring the accuracy and timing consistency of data writing.

[0084] Optionally, generating the read cache address based on the preset global read row count value and the preset global read column count value includes:

[0085] Based on the preset global read row count value and the preset global read column count value, the location identifier and read offset of the target cache unit are generated respectively through the preset address decomposition rule;

[0086] Based on the location identifier, obtain the corresponding cache unit base address;

[0087] The read cache address is generated by performing a preset address combination operation on the base address of the cache unit and the read offset.

[0088] Specifically, an address decomposition pipeline is designed in the FPGA. First, the global read row count value is modulo-n to obtain the read row rotation value, which is used as the high-order part of the positioning identifier. At the same time, the global read column count value is divided by H to obtain the read column rotation value, which is used as the low-order part of the positioning identifier. The two are connected by a bit concatenation circuit to form a complete cache cell positioning identifier. Then, the low-order field is extracted from the global read column count value as the read offset by modulo operation. At the same time, an offset range verification circuit is designed to ensure that the offset does not exceed the cache cell depth. The entire decomposition process adopts a three-stage pipeline structure. The first stage performs modulo operation and integer division operation, the second stage performs identifier concatenation and offset extraction, and the third stage performs range verification and result latching to ensure stable output of positioning identifier and read offset within 3 clock cycles.

[0089] Furthermore, a base address lookup circuit is designed in the FPGA. This circuit takes the location identifier as the address input and reads the corresponding cache unit base address in parallel from the pre-configured base address memory. At the same time, the read offset is input to the offset processing unit, and the bit width alignment circuit ensures that the offset matches the bit width of the base address. The address synthesis module adopts a three-stage pipeline structure. The first stage completes the base address reading and offset preprocessing. The second stage performs a left shift operation on the base address (the number of shift bits is determined by the offset bit width). The third stage performs a bitwise OR operation between the shifted base address and the offset to generate the final read address. The entire address generation process is kept in sync with the pipeline register and the address calculation is completed within 3 clock cycles to ensure strict alignment with the read control signal.

[0090] S204. Generate the read cache address based on the preset global read row count value and the preset global read column count value.

[0091] Specifically, a read address generation pipeline is constructed in the FPGA. First, the address decomposition module performs a modulo-n operation on the preset global read row count value to obtain the read row rotation value, and simultaneously performs an integer division operation on the preset global read column count value to obtain the read column rotation value. The two are concatenated to form the target cache cell location identifier. Then, the base address of the corresponding cache cell is read in parallel according to the location identifier through the base address lookup table, and the low-order field is extracted from the global read column count value as the read offset. Finally, the address synthesis unit performs a bitwise OR operation on the base address shifted left by the offset bit width and the masked offset to generate the final read cache address. The entire address generation process adopts a four-stage pipeline structure, and the timing alignment circuit ensures data synchronization at each stage. The address calculation is completed and output to the cache controller within 4 clock cycles.

[0092] Optionally, the step of performing a preset address combination operation on the base address of the cache unit and the read offset to generate the read cache address includes:

[0093] According to the preset offset bit width, the base address of the cache unit is left-shifted to obtain the shifted base address;

[0094] By using a preset offset mask, a bitwise AND operation is performed on the read offset to obtain the masked read offset;

[0095] The shifted base address and the masked read offset are bitwise ORed to generate the read cache address.

[0096] Specifically, a dedicated shifter circuit is designed in the FPGA. This circuit uses the offset bit width as a control signal input to the configuration port of the barrel shifter. The base address data is input to the shifter through a 32-bit wide data bus, and the shifter generates the corresponding mask pattern based on the offset bit width value. A multi-stage multiplexer architecture is used to implement the variable bit width left shift operation, with each stage processing 1 bit shift. The maximum shift requirement is achieved through cascading. After the shift is completed, the empty low bits are automatically filled with zeros, and the overflow data in the high bits is discarded. The entire shift operation is completed within a single clock cycle, and the shift result is latched through the output register. At the same time, a shift bit verification circuit is designed to ensure that the offset bit width does not exceed the effective bit width of the base address to prevent data loss.

[0097] Furthermore, a mask processing unit is designed in the FPGA. This unit stores the preset offset mask (such as a 32-bit all-1 mask) in the configuration register. After reading the offset input, it performs a bitwise AND operation with the mask through a parallel bitwise AND logic gate array, where each bit uses an independent AND gate circuit for synchronous processing. A mask dynamic configuration interface is designed to support real-time updates of the mask value via the APB bus to adapt to different cache unit depths. The operation result is output as a valid offset through a data selector. An overflow detection circuit is also designed to generate an alarm signal when the offset exceeds the mask range. The entire mask processing process is completed within a single clock cycle, ensuring a strict match between the offset bit width and the subsequent address synthesis requirements.

[0098] Furthermore, an address synthesis unit is designed in the FPGA. This unit contains a parallel bit processing logic array and uses a 32-bit wide bitwise OR gate circuit to achieve address synthesis. The shifted base address and the masked offset are respectively input to the two input terminals of the OR gate array, and each bit is independently subjected to a logical OR operation. A timing alignment circuit is designed to ensure that the setup time and hold time of the two input signals before entering the OR gate meet the timing requirements through two-stage D flip-flops. The address synthesis result is latched by the output register, and an address validity detection circuit is designed to generate an error flag when the synthesized address exceeds the preset range. The entire address synthesis process is completed within a single clock cycle, and the synthesized read cache address is directly output to the address bus of the cache unit to ensure the accurate execution of the read operation.

[0099] S205. Based on the read cache address, read the pixel data from the target cache unit and aggregate it into a recombined pixel row.

[0100] Specifically, a multi-stage pipelined read and aggregation architecture is constructed in the FPGA. First, read enable signals for a subset of target cache units are generated based on the read cache address and a preset cache unit scheduling sequence. Pixel data blocks are then extracted synchronously from multiple cache units through a parallel read interface. Subsequently, the data blocks enter the rearrangement engine, where they are reassembled in real time according to the pixel characteristic mapping table stored in the BRAM. Logical rearrangement of pixel data is achieved through a cross-switching network. The reassembled data stream passes through a seam detection and smoothing module, which uses a configurable Gaussian filter core to adaptively smooth the seam region, eliminating discontinuities between blocks. Finally, the row aggregation controller combines the processed pixel data into complete pixel rows according to a preset format. The entire processing flow adopts a seven-stage pipeline structure, achieving a throughput of processing one row of pixel data per clock cycle under the pixel clock drive, ensuring the real-time performance and data integrity of the reassembly process.

[0101] Optionally, reading the pixel data from the target cache unit and aggregating it into a reconstructed pixel row based on the read cache address includes:

[0102] Based on the read cache address and the preset cache unit scheduling sequence, a subset of target cache units is determined from the target cache units;

[0103] Data is read in parallel from a subset of the target cache units to obtain multiple pixel data blocks;

[0104] According to the preset pixel arrangement rules and the preset pixel characteristic mapping table, the multiple pixel data blocks are merged to obtain a combined pixel data stream;

[0105] Determine the seam region formed by splicing the pixel data blocks in the combined pixel data stream, and adjust the seam region using a preset smoothing algorithm to obtain a smoothed pixel data stream;

[0106] The smoothed pixel data stream is aggregated to obtain the recombined pixel row.

[0107] Specifically, a scheduling control module is designed in the FPGA. This module pre-stores the preset cache unit scheduling sequence in a circular shift register and generates a sequence index based on the low-order bits of the global read row count value. Simultaneously, the high-order part of the read cache address is converted into a cache unit selection mask through an address decoding circuit, and a bitwise AND operation is performed with the enable signal output from the scheduling sequence to generate the target cache unit subset enable signal for the current clock cycle. This enable signal is routed to the corresponding cache unit control port through a multiplexer to ensure that only the cache units within the subset are in an active read state each clock cycle. The entire subset determination process is completed within two clock cycles. The first cycle performs address decoding and sequence indexing, and the second cycle completes the logic operation and enable signal output, and eliminates timing hazards through a synchronization register.

[0108] Furthermore, a multi-port parallel read architecture is designed in the FPGA. This architecture includes an independent data channel controller, with each channel corresponding to a cache unit interface. When the cache unit subset enable signal is valid, multiple cache units are synchronously triggered to read. Data from the read clock domain is stably transmitted to the processing clock domain through a cross-clock domain synchronization circuit. The read pixel data blocks are formatted uniformly by the bit width adjustment unit and then temporarily stored in a multi-level buffer register group. At the same time, a data validity verification circuit is designed to verify the integrity of each data block through parity check codes, ensuring that multiple pixel data blocks read in parallel are acquired and buffered within 3 clock cycles, providing synchronous and complete data input for subsequent data merging processing.

[0109] Furthermore, a data rearrangement engine is designed in the FPGA. This engine stores a preset pixel characteristic mapping table in the on-chip BRAM. The mapping table defines the output position index of each pixel data block based on its characteristics (such as polarization state or spectrum). The parallel input pixel data blocks are first sorted by a multiplexer array according to the selection signal generated by the mapping table. The sorted data blocks enter a cross-switching network, which is implemented by a configurable switch matrix and can dynamically adjust the data path according to preset arrangement rules. Finally, the data stream passes through a serialization unit, which uses a shift register structure to convert the parallel data blocks into a continuous combined pixel data stream. The entire process is completed under a 4-stage pipeline control to ensure the real-time performance and accuracy of data merging.

[0110] Furthermore, a seam detection and smoothing module is designed in the FPGA. This module first stores multiple consecutive rows of pixel data through a row buffer, and uses a gradient calculation circuit to detect the first and second derivatives of the pixel values. When the gradient value exceeds a preset threshold, it is marked as a seam boundary. Subsequently, the smoothing unit uses a configurable Gaussian filter kernel to dynamically adjust the filtering parameters according to the seam width, and performs a weighted average of the pixels in the seam area through convolution operations. At the same time, an interpolation compensation circuit is designed to perform bilinear interpolation on the smoothed pixels to maintain the integrity of details. The entire processing is completed in a five-stage pipeline, including data buffering, gradient detection, filtering calculation, interpolation compensation, and output latching, ensuring that the smoothed pixel data stream is output within 5 clock cycles, effectively eliminating inter-block discontinuities.

[0111] Furthermore, a row aggregation controller is designed in the FPGA. This controller includes a configurable-depth row buffer array for temporarily storing smoothed pixel data. A pixel counter tracks the current pixel position within a row in real time, and generates a row end signal when the counter reaches a preset row width. The data reconstruction unit writes the pixel data in the buffer into the output FIFO queue in a specified order according to a preset pixel arrangement format. At the same time, a flow control circuit is designed to ensure that the data throughput matches the receiving capacity of the downstream processing module through a backpressure signal mechanism. The entire aggregation process is implemented using a three-stage pipeline, including data buffering, format reconstruction, and flow control. Finally, under the trigger of the row synchronization signal, a complete reconstructed pixel row is output, along with a corresponding row validity flag signal.

[0112] S206. The reconstructed pixel rows are sequentially written into a continuous storage area in a preset memory to obtain reconstructed image data.

[0113] Specifically, a memory write controller is designed in the FPGA, which includes an address generator and a data path manager. The address generator calculates the starting address of the current row based on the preset memory base address and row offset, where the row offset is dynamically generated by multiplying the row counter by the number of pixels per row. The data path manager packages the recombined pixel row data into burst transmission transactions through the AXI4 bus master interface, adopting a pipelined operation mode where the write address channel precedes the write data channel. At the same time, a write response processing circuit is designed to monitor the acknowledgment signal of the AXI write response channel to ensure the reliability of data writing. The entire writing process achieves rate matching through a depth-configurable write buffer FIFO. Transmission is started after a row valid signal is detected. The row counter is automatically incremented after each row of data is written until the entire frame of image data is stored in a continuous area of ​​the memory.

[0114] S207. The reconstructed image data is read sequentially from the continuous storage area to obtain a reconstructed image video stream.

[0115] Specifically, a video stream output engine is built in the FPGA. This engine includes an address pointer controller, an AXI4 bus read interface, and a video timing generator. After receiving the frame synchronization signal, the address pointer controller loads the preset start address into the address register, and then initiates a burst read request through the AXI4 read address channel in each pixel clock cycle. The read data is stored in the streaming FIFO buffer after verification, while the address pointer automatically increments by a preset step size. The video timing generator generates precise line synchronization, field synchronization, and data valid signals according to the target video standard (such as 1080p@60Hz). The streaming FIFO outputs pixel data under the control of the data valid signal and is strictly synchronized with the timing signal. Finally, the parallel pixel data is converted into a serial video stream conforming to the HDMI or DisplayPort standard by the video encoder, completing the complete output link from memory to video interface.

[0116] Optionally, sequentially reading the reconstructed image data from the contiguous storage area to obtain the reconstructed image video stream includes:

[0117] The preset starting address of the contiguous storage region is initialized as a read address pointer;

[0118] According to the preset read clock, data units are continuously read from the continuous storage area based on the read address pointer, and a preset step increment operation is performed on the read address pointer after each read.

[0119] All the read data units are combined in the order they were read to generate the reconstructed image video stream.

[0120] Specifically, an address pointer control module is designed in the FPGA. This module includes a 32-bit reloadable address register and a configuration interface. After the system powers on, the preset start address is written to the address register via the APB configuration bus. At the same time, an address verification circuit is designed to ensure that the start address falls within the valid range of the memory. The initialization process is controlled by a state machine. When a frame synchronization signal is received, the state machine switches to the initialization state, clears the address register, loads the preset start address, and generates an address valid flag signal. The entire initialization process is completed within 3 clock cycles, including address verification, register loading, and state confirmation, ensuring that the read address pointer accurately points to the starting position of the storage area at the beginning of the video frame.

[0121] Furthermore, a clock-synchronized read state machine is designed, which triggers the read operation on the rising edge of the read clock. The address pointer register outputs the current address to the AXI4 bus read address channel and simultaneously issues a read enable signal. After the memory controller returns the read data, it verifies the data integrity through a data verification circuit and stores the valid data into the output FIFO. The address increment module automatically increases the address pointer by a preset step size (the step size is determined by the pixel width and bus width) after each read cycle. The entire read process adopts a pipelined design, with address generation, data reading, and pointer update operations overlapping to ensure that a data read operation is completed once per clock cycle, and flow control logic prevents FIFO overflow.

[0122] Furthermore, a video stream synthesis engine is designed in the FPGA, which includes a data reconstruction unit and a video timing generator. The data reconstruction unit stores data units read from memory into a depth-configurable streaming FIFO buffer according to the first-in-first-out principle, while ensuring the sequential integrity of the data units through a packet counter. The video timing generator generates precise line synchronization, field synchronization, and data valid signals according to preset video format parameters (such as resolution, frame rate, and blanking interval). When the data valid signal is activated, the streaming FIFO outputs data units according to the pixel clock beat and is strictly aligned with the timing signals. The final output video stream passes through a format conversion circuit to convert the parallel data into a serial data stream conforming to a standard video interface (such as DisplayPort), completing the entire process of generating the reconstructed image video stream.

[0123] This embodiment also discloses an electronic device, as shown in the reference. Figure 3 The electronic device may include: at least one processor 301, at least one communication bus 302, user interface 303, network interface 303, and at least one memory 305.

[0124] The communication bus 302 is used to enable communication between these components.

[0125] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0126] The network interface 303 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0127] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0128] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. Figure 3 As shown, the memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for an image processing method that reconstructs data streams.

[0129] exist Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 301 can be used to call an application program that stores an image processing method for data stream reconstruction in the memory 305. When executed by one or more processors 301, the electronic device performs one or more methods as described in the above embodiments.

[0130] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0131] In some embodiments of this application, a computer-readable storage medium is provided, including instructions that, when executed on the electronic device, cause the electronic device to perform a QR code level assessment method according to an embodiment of this application.

[0132] In some embodiments of this application, a computer program product is also provided, which, when run on an electronic device, causes the electronic device to execute a QR code level assessment method according to an embodiment of this application.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned memory 305 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0136] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the disclosure in this specification. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. An image processing method for data stream reconstruction, characterized in that, Applied to a server, the method includes: Acquire the raw image data stream and determine the row and column coordinates of each pixel data in the raw image data stream within a preset pixel block; Based on the row coordinates and column coordinates, a corresponding write cache address is generated for each pixel data; Based on the write cache address, each pixel data is written to a target cache unit among multiple preset cache units; The read cache address is generated based on the preset global read row count value and the preset global read column count value; Based on the read cache address, the pixel data is read from the target cache unit and aggregated into a reconstructed pixel row; The reconstructed pixel rows are sequentially written into a continuous storage area in a preset memory to obtain reconstructed image data; The reconstructed image data is read sequentially from the contiguous storage area to obtain a reconstructed image video stream; The step of generating the read cache address based on the preset global read row count value and the preset global read column count value includes: Based on the preset global read row count value and the preset global read column count value, the location identifier and read offset of the target cache unit are generated respectively through the preset address decomposition rule; Based on the location identifier, obtain the corresponding cache unit base address; The read cache address is generated by performing a preset address combination operation on the base address of the cache unit and the read offset. The step of performing a preset address combination operation between the base address of the cache unit and the read offset to generate the read cache address includes: According to the preset offset bit width, the base address of the cache unit is left-shifted to obtain the shifted base address; By using a preset offset mask, a bitwise AND operation is performed on the read offset to obtain the masked read offset; The shifted base address and the masked read offset are bitwise ORed to generate the read cache address. The step of reading the pixel data from the target cache unit and aggregating it into a reconstructed pixel row based on the read cache address includes: Based on the read cache address and the preset cache unit scheduling sequence, a subset of target cache units is determined from the target cache units; Data is read in parallel from a subset of the target cache units to obtain multiple pixel data blocks; According to the preset pixel arrangement rules and the preset pixel characteristic mapping table, the multiple pixel data blocks are merged to obtain a combined pixel data stream; Determine the seam region formed by splicing the pixel data blocks in the combined pixel data stream, and adjust the seam region using a preset smoothing algorithm to obtain a smoothed pixel data stream; The smoothed pixel data stream is aggregated to obtain the recombined pixel row.

2. The method according to claim 1, characterized in that, The step of acquiring the original image data stream and determining the row and column coordinates of each pixel data in the original image data stream within a preset pixel block includes: The global row coordinates and global column coordinates of each pixel data are decomposed into multiple preset row coordinate bit groups and column coordinate bit groups, and the row coordinate bit groups and column coordinate bit groups are combined to obtain a set of bit groups; According to the preset bit recombination rules, a first target bit group is determined, and the first target bit group is selected from the set of bit groups for cross-combination to generate a recombined row coordinate bit sequence; According to the bit recombination rule, a second target bit group is selected from the plurality of row coordinate bit groups and the plurality of column coordinate bit groups for cross-combination to generate recombined column coordinate bits. The second target bit group is the set of bit groups minus the first target bit group. The recombined row coordinate bit sequence and the recombined column coordinate bit sequence are respectively determined as the row coordinate and the column coordinate.

3. The method according to claim 1, characterized in that, The step of generating a corresponding write cache address for each pixel data based on the row coordinates and the column coordinates includes: Based on the column coordinates, the target cache unit is determined from the plurality of preset cache units, and the preset base address of the target cache unit is obtained; The row coordinates are determined as the write offset within the target cache unit; The preset base address and the write offset are combined using a preset address operation to generate the corresponding write cache address.

4. The method according to claim 1, characterized in that, The step of sequentially reading the reconstructed image data from the contiguous storage area to obtain the reconstructed image video stream includes: The preset starting address of the contiguous storage region is initialized as a read address pointer; According to the preset read clock, data units are continuously read from the continuous storage area based on the read address pointer, and a preset step increment operation is performed on the read address pointer after each read. All the read data units are combined in the order they were read to generate the reconstructed image video stream.

5. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-4.

6. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-4.

Images