Method for timing synchronization based on multiple data paths and related devices

Abstract

The application provides a timing synchronization method based on multiple data paths and related equipment. The method comprises the following steps: acquiring image data, and performing line storage on the image data through a line storage module; determining the processing timing of a corresponding data interface according to the processing delay of a subsequent module of each data path; constructing the window data required by each data path based on the line-stored image data through the line storage module, and outputting the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path performs data processing on the window data to obtain intermediate data; and performing aggregation processing on the intermediate data output by the multiple data paths through a data aggregation module to obtain target image data. The application outputs multiple sets of timing-adapted data interfaces through a line storage module, realizes timing synchronization of multiple data paths, and can reduce the chip area and save the chip cost.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of image processing technology, and in particular to a timing synchronization method based on multiple data paths and related equipment. Background Technology

[0002] With the development of digital chip technology, digital image processing algorithms have been widely used in various devices. These algorithms often involve calculations based on window data. When multiple modules based on window data operations need to be cascaded, row latency is introduced. In some complex algorithms, it is necessary to converge and fuse data that has been processed through different data paths, or to merge processed data with the original input data. This requires that all data paths involved in the convergence be synchronized in time.

[0003] In related technologies, one approach to achieve this timing synchronization is to merge the preceding and following memory stages together, ensuring consistent row latency across the processing modules of each data path. However, this approach has a limitation: when the logic area of ​​the first-level module is large, merging the memory leads to excessive area costs. In solutions that do not merge the memory, existing technologies must add additional row-aligned caches to store shorter-latency data paths while waiting for the longest-latency data paths. This approach increases the amount of additional memory, which in turn increases the chip area. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application provides a timing synchronization method and related equipment based on multiple data paths. By outputting multiple sets of timing-adaptive data interfaces through a row storage module, timing synchronization of multiple data paths is achieved, which can reduce chip area and save chip cost.

[0005] To achieve the above objectives, a first aspect of this application proposes a timing synchronization method based on multiple data paths, applied to an image processing system. The image processing system includes a row storage module, multiple data paths, and a data aggregation module. The method includes:

[0006] Image data is acquired and stored in rows through the row storage module; wherein the row storage module is connected to each data path through a data interface.

[0007] The processing timing of the corresponding data interface is determined based on the processing delay of the subsequent module of each data path.

[0008] The row storage module constructs the window data required for each data path based on the image data stored in the row storage, and outputs the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path processes the window data to obtain intermediate data.

[0009] The intermediate data output from the multiple data channels is aggregated by the data aggregation module to obtain the target image data.

[0010] In some embodiments, storing the image data in rows via the row storage module includes:

[0011] The image data is acquired pixel by pixel and written to the row storage module according to a predetermined row and column order.

[0012] Based on the row and column order of the image data, the pixels are constructed into multiple rows of image data for storage.

[0013] In some embodiments, determining the processing timing of the corresponding data interface based on the processing delay of the subsequent module of each data path includes:

[0014] Among the multiple data paths, identify the target data path with the longest subsequent module processing delay, and determine the longest subsequent module processing delay as the target row delay;

[0015] For each other data path besides the target data path, calculate the processing delay of the subsequent module for that other data path;

[0016] Based on the target row delay and the subsequent module processing delay of the other data paths, a compensation row delay required for the other data paths is calculated;

[0017] The compensation line delay is determined as the processing timing of the data interface corresponding to the other data paths.

[0018] In some embodiments, constructing the window data required for each data path based on the image data stored in the row storage module includes:

[0019] Determine the corresponding number of rows based on the window size required for each data path;

[0020] Based on the number of rows, determine the starting number of rows required to construct the window data;

[0021] When the number of rows of the multi-row image data stored in the row storage module reaches the starting number of rows, a multi-row image with a row size equal to the number of rows corresponding to the window size is selected from the multi-row image data stored in the row storage module.

[0022] From the selected multi-row images, extract pixel data with a column size equal to the window size to obtain the window data.

[0023] In some embodiments, the method further includes:

[0024] When the window data needs to obtain boundary pixels outside the valid data area, the boundary pixel coordinates of the boundary pixels are determined;

[0025] Based on the boundary pixel coordinates and the preset boundary, calculate the mirror coordinates of the boundary pixel coordinates within the effective data area;

[0026] Obtain the mirror pixel data corresponding to the mirror coordinates from the row storage module;

[0027] The mirrored pixel data is used as the boundary pixels to complete the construction of the window data.

[0028] In some embodiments, the step of outputting window data from the data interface to the corresponding data path based on the output timing corresponding to each data interface, so that the data path processes the window data to obtain intermediate data, includes:

[0029] According to the output timing determined for each data interface, when the output timing arrives, the window data is started to be output from the data interface to the corresponding data path;

[0030] The data path receives the window data, and the subsequent modules in the data path perform preset logical operations on the window data to obtain the intermediate data.

[0031] In some embodiments, the step of aggregating the intermediate data output from the plurality of data paths through the data aggregation module to obtain target image data includes:

[0032] The data aggregation module synchronously receives intermediate data from the multiple data paths.

[0033] A preset fusion operation is performed on the intermediate data output from the multiple data channels within the same clock cycle and corresponding to the same image coordinate point to obtain the target image data.

[0034] Secondly, embodiments of this application provide a timing synchronization device based on multiple data paths, comprising:

[0035] An acquisition module is used to acquire image data and store the image data in rows through a row storage module; wherein, the row storage module is connected to each data path through a data interface.

[0036] The determining module is used to determine the processing timing of the corresponding data interface based on the processing delay of the subsequent module of each data path;

[0037] The construction module is used to construct the window data required for each data path based on the image data stored in the row storage module, and output the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path processes the window data to obtain intermediate data.

[0038] The processing module is used to aggregate the intermediate data output from the multiple data channels through the data aggregation module to obtain target image data.

[0039] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the timing synchronization method based on multiple data paths as described in any one of the embodiments of the first aspect of this application.

[0040] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that is executed by a processor to implement the timing synchronization method based on multiple data paths as described in any one of the embodiments of the first aspect of this application.

[0041] The timing synchronization method based on multiple data paths proposed in this application is applied to an image processing system. The image processing system includes a row storage module, multiple data paths, and a data aggregation module. The method includes: acquiring image data and storing the image data in rows through the row storage module; wherein the row storage module is connected to each data path through a data interface; determining the processing timing of the corresponding data interface based on the processing delay of the subsequent module of each data path; constructing the window data required by each data path based on the row-stored image data through the row storage module, and outputting the window data from the data interface to the corresponding data path based on the processing timing of each data interface, so that the data path processes the window data to obtain intermediate data; and aggregating the intermediate data output by multiple data paths through the data aggregation module to obtain target image data.

[0042] The proposed timing synchronization method based on multiple data paths first stores image data in rows using a row storage module, providing an independent data interface for each data path, thus laying the structural foundation for subsequent differentiated data output. Then, based on the processing latency of the subsequent modules of each data path, the processing timing of the corresponding data interface is determined, ensuring that the start time of data output is controllable and calculated. Finally, after constructing the window data, the row storage module outputs the required window data to each data path based on the determined processing timing for each path. This ensures that the total row latency of all data paths is consistent when they reach the data aggregation module, solving the technical problem in the prior art where additional alignment row buffers are required for compensation due to inconsistent row latency. In summary, this application achieves timing synchronization of multiple data paths by outputting multiple sets of timing-adapted data interfaces through a single row storage module, reducing chip area and saving chip costs.

[0043] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in an embodiment of this application.

[0045] Figure 2 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in another embodiment of this application;

[0046] Figure 3 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in another embodiment of this application;

[0047] Figure 4 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in another embodiment of this application;

[0048] Figure 5 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in another embodiment of this application;

[0049] Figure 6 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in another embodiment of this application;

[0050] Figure 7 This is a flowchart illustrating a timing synchronization method based on multiple data paths provided in another embodiment of this application;

[0051] Figure 8 This is a schematic diagram of a timing synchronization device based on multiple data paths provided in an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] With the development of digital chip technology, digital image processing algorithms have been widely used in various devices. These algorithms often involve calculations based on window data. When multiple modules based on window data operations need to be cascaded, row latency is introduced. In some complex algorithms, it is necessary to converge and fuse data that has been processed through different data paths, or to merge processed data with the original input data. This requires that all data paths involved in the convergence be synchronized in time.

[0057] In related technologies, one approach to achieve this timing synchronization is to merge the preceding and following memory stages together, ensuring consistent row latency across the processing modules of each data path. However, this approach has a limitation: when the logic area of ​​the first-level module is large, merging the memory leads to excessive area costs. In solutions that do not merge the memory, existing technologies must add additional row-aligned caches to store shorter-latency data paths while waiting for the longest-latency data paths. This approach increases the amount of additional memory, which in turn increases the chip area.

[0058] Based on this, the embodiments of this application provide a timing synchronization method and related equipment based on multiple data paths. By outputting multiple sets of timing-adaptive data interfaces through a row storage module, timing synchronization of multiple data paths is realized, which can reduce chip area and save chip cost.

[0059] The timing synchronization method and related equipment based on multiple data paths provided in this application are specifically described through the following embodiments. First, the timing synchronization method based on multiple data paths in this application is described.

[0060] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0061] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0062] Figure 1 This is an optional flowchart of a timing synchronization method based on multiple data paths provided in the embodiments of this application. The method is applied to an image processing system, which includes a row storage module, multiple data paths, and a data aggregation module, and may include, but is not limited to, steps 101 to 104.

[0063] Step 101: Acquire image data and store the image data in rows through the row storage module; wherein, the row storage module communicates with each data path through a data interface.

[0064] Step 102: Determine the processing timing of the corresponding data interface based on the processing delay of the subsequent module of each data path.

[0065] Step 103: The row storage module constructs the window data required for each data path based on the image data stored in the row storage, and outputs the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path processes the window data to obtain intermediate data.

[0066] Step 104: The intermediate data output from multiple data channels is aggregated through the data aggregation module to obtain the target image data.

[0067] Steps 101 to 104, as illustrated in this embodiment, firstly, image data is stored row-wise using a row storage module, and an independent data interface is provided for each data path, providing a structural foundation for subsequent differentiated data output. Then, based on the processing latency of the subsequent modules of each data path, the processing timing of the corresponding data interface is determined, ensuring that the start time of data output is controllable and calculated. Finally, after constructing the window data, the row storage module outputs the required window data to each data path based on the processing timing determined for each path. This ensures that the total row latency of all data paths is consistent when they reach the data aggregation module, solving the technical problem in the prior art where additional alignment row buffers are required for compensation due to inconsistent row latency. In summary, this application outputs multiple sets of timing-adapted data interfaces through a single row storage module, achieving timing synchronization of multiple data paths, reducing chip area and saving chip costs.

[0068] In step 101 of some embodiments, the image processing device acquires initial image data, which is typically a pixel stream input sequentially pixel by pixel in a specific row and column order. This image data is transmitted to a core row storage module. The row storage module is a hardware buffer unit whose core function is not only to temporarily store data, but also to serve as the data source for all subsequent parallel processing tasks. The key to this step is establishing a "one-to-many" hardware connection topology: the row storage module is designed to have multiple independent output ports, i.e., data interfaces, and each data path in the image processing device is connected one-to-one to one of these data interfaces. Here, a data path refers to an independent downstream logic processing pipeline used to execute a specific image processing algorithm. This step, by establishing this physical connection structure, lays the foundation for subsequent implementation of differentiated timing control.

[0069] Please see Figure 2 In some embodiments, the step of storing image data in rows by the row storage module may include, but is not limited to, steps 201 to 202.

[0070] Step 201: Acquire image data pixel by pixel and write it to the row storage module according to the predetermined row and column order.

[0071] Step 202: Based on the row and column order of the image data, the pixels are constructed into multi-row image data for storage.

[0072] In step 201 of some embodiments, the row storage module of the image processing apparatus begins receiving raw image data from the image source. This acquisition follows a predetermined row-column order, typically starting with the first pixel of the first row of the image, then the pixels to the right of the first pixel, and so on, until the end of the row; then, starting with the first pixel of the second row, this process is repeated. This process is performed pixel-by-pixel, serially. In each clock cycle, the row storage module receives the data of one pixel and immediately writes the data of that pixel into its internal storage unit.

[0073] In step 202 of some embodiments, where pixels are written as a continuous one-dimensional data stream in step 201, this step is responsible for reconstructing this one-dimensional data stream into a two-dimensional image structure. The row storage module is internally configured with a row width parameter; for example, for a high-resolution image, this parameter might be set to 1920. When the row storage module receives pixels, it counts them. Whenever 1920 pixels are counted, this set of data is marked as a complete image row, and subsequent pixels are stored in the storage space representing the next image row. In this way, the row storage module dynamically constructs multi-row image data from the serially input pixels according to the original row and column order of the image data, and maintains and stores this multi-row data internally.

[0074] Through steps 201 to 202 described above, this embodiment of the application elaborates on the core buffering mechanism of the row storage module. Step 201 defines how data is received serially, while step 202 converts a one-dimensional time-series data stream into two-dimensional spatial data. These two steps, combined, fulfill the basic function of the row buffer, enabling the image processing device to convert serial data into parallel multi-row data. This is a prerequisite for subsequent window data construction (as described in step 103) and provides a data foundation for all downstream data paths.

[0075] In step 102 of some embodiments, a critical timing pre-computation operation is performed. The purpose of this operation is to precisely quantify the downstream module processing latency introduced by the complexity of each data path due to its internal algorithm. Downstream module processing latency represents the time required for a data path to complete all its internal computations and produce a result from the receipt of its input data; this time unit is typically row latency or clock cycles. For example, suppose data path A performs a simple 3×3 convolution, with a downstream module processing latency calculated to be 5 rows; while data path B performs a feature extraction algorithm, with a latency of 20 rows. This step analyzes all data paths, identifies the longest latency of 20 rows as the baseline latency, and then calculates a compensation latency of 15 rows (i.e., 20 rows minus 5 rows) required for data path A. This calculated 15-row compensation latency is then determined as the processing timing that the data interface corresponding to data path A must adhere to.

[0076] Please see Figure 3 In some embodiments, step 102 may include, but is not limited to, steps 301 to 304.

[0077] Step 301: Determine the target data path with the longest processing delay of the subsequent module among multiple data paths, and determine the longest processing delay of the subsequent module as the target row delay.

[0078] Step 302: For each other data path besides the target data path, calculate the processing latency of the subsequent modules for that data path.

[0079] Step 303: Calculate the required compensation row delay for other data paths based on the target row delay and the processing delay of the subsequent modules of other data paths.

[0080] Step 304: Determine the processing timing of the data interface corresponding to other data paths based on the compensation line delay.

[0081] In step 301 of some embodiments, the image processing device first performs a global latency analysis. This operation traverses all parallel data paths and evaluates the processing latency of the downstream module within each path. The downstream module processing latency refers to the time required for a data path to complete its internal computation after receiving input. The purpose of this operation is to identify the path with the longest computation time among all paths, i.e., the target data path with the longest downstream module processing latency. This path will serve as the benchmark for the entire timing synchronization. For example, if the latency of data path A is 5 lines and the latency of data path B is 20 lines, then data path B is determined as the target data path, and its 20-line latency is set as the global target line latency.

[0082] In step 302 of some embodiments, after determining the target row delay, the image processing device precisely calculates the processing delay of each subsequent module for each of the other data paths besides the target data path (i.e., data path B). This is an information acquisition process used to obtain the actual delay values ​​of all non-reference paths (data path A in this example). Through this step, the device clearly knows that the processing delay of the subsequent module of data path A is 5 rows.

[0083] In step 303 of some embodiments, the target row delay (20 rows) determined in step 301 and the subsequent module processing delays (5 rows) of other data paths calculated in step 302 are used. A difference is calculated by subtracting the delays of the other data paths from the target row delay; this difference represents the required compensation row delay for that data path. In this example, the required compensation row delay for data path A is calculated to be 15 rows.

[0084] In step 304 of some embodiments, the compensated row delay calculated in step 303 is formally assigned to its corresponding data path. Specifically, this 15-row value is set as the processing timing of the data interface corresponding to data path A. This processing timing is an execution instruction that the storage module must wait for 15 rows of time before outputting window data to the data interface of data path A. For the target data path B, its compensated row delay is zero, so its processing timing is "immediate output".

[0085] Through steps 301 to 304 described above, this embodiment first establishes a global, slowest baseline latency in step 301; then, in step 302, it measures the latency of all other "faster" paths; next, in step 303, it calculates the time difference between each path and the baseline; and finally, in step 304, it converts the time difference into a specific compensation line latency instruction. This series of steps together constitutes a complete timing pre-computation scheme, and the final processing timing output provides accurate numerical basis for achieving proactive and differentiated data distribution in step 103.

[0086] In step 103 of some embodiments, firstly, the row storage module uses the multi-row image data continuously stored in step 101 to construct the window data required by its algorithm for each data path. The window data is a pixel matrix, such as a 5×5 pixel block, which is the actual data unit for downstream data path operations. Subsequently, the row storage module performs critical timing distribution strictly following the processing timing determined for each data interface in step 102 to control when to start outputting window data from that data interface. Continuing with the data from the previous example, the row storage module will immediately output window data to the data interface of data path B (delayed by 20 rows); but will wait for a 15-row delay before starting to output the corresponding window data to the data interface of data path A (delayed by 5 rows). Once the data path receives the window data at the specified time point, it begins to execute its internal data processing and finally outputs its processing result, i.e., intermediate data.

[0087] Please see Figure 4 In some embodiments, the step of constructing the window data required for each data path based on the image data stored in the row storage module may include, but is not limited to, steps 401 to 404.

[0088] Step 401: Determine the corresponding number of rows based on the window size required for each data path.

[0089] Step 402: Determine the starting number of rows required to construct the window data based on the row size.

[0090] Step 403: When the number of rows of multi-row image data stored in the row storage module reaches the starting number of rows, select multi-row images from the multi-row image data stored in the row storage module that have a row size equal to the window size.

[0091] Step 404: Extract pixel data from the selected multi-row images, with the number of columns corresponding to the window size being equal, to obtain window data.

[0092] In step 401 of some embodiments, the image processing device first needs to determine the specific size of the window data required for each data path. For example, a data path may be configured to require a window with a row size of n. This step involves determining the row size corresponding to the window size based on the specific algorithm requirements of each data path.

[0093] In step 402 of some embodiments, an initial row number for triggering the construction operation is calculated based on the row size n determined in step 401. To construct a center-aligned window (assuming n is odd), the row storage module buffers a sufficient number of image rows before starting to construct the first valid window. The window center point needs to be located above... Rows and below The row data. This means that the row storage module must at least receive and store the row data. Only after obtaining the image data can we begin constructing the first image. A row-centric window. Therefore, this The number of rows is determined as the starting number of rows required to construct the window data.

[0094] In step 403 of some embodiments, the row storage module continuously monitors the cumulative number of rows of multi-row image data stored internally. When this cumulative number of rows (e.g., the number of rows of newly input image data) is reached... When the number of rows (after all row data has been completely stored) reaches the starting row number determined in step 402, the window construction logic is triggered. At this time, the row storage module selects the specific rows needed to construct the window from all the multi-row image data stored internally. For a window with a row size of n, when it is the nth row... When constructing the row window, the row storage module will simultaneously select the n rows of image data from row 1 to row n.

[0095] In step 404 of some embodiments, after the correct image rows (e.g., rows 1 to n) have been selected in step 403, the device performs the final step of constructing the window, namely, extracting column data. This step extracts a corresponding number of pixel data from the selected multiple rows of images based on the number of columns m corresponding to the window size. For example, when the nth row is selected... When constructing a window from a central pixel in a row, the device extracts the pixel data of the central pixel and the m columns to its left and right from row 1 to row n. The set of n×m pixels, consisting of m pixels in each of the n rows, is organized into an n×m pixel matrix, which is the final window data.

[0096] Through steps 401 to 404 described above, this embodiment of the application first clarifies when construction can begin (i.e., when the initial number of rows is reached) through steps 401 and 402; then, through steps 403 and 404, the correct rows are selected first (step 403), and the correct columns are extracted from these rows (step 404). This series of steps together ensures that the row storage module can accurately and efficiently generate the two-dimensional window data required for subsequent processing from the serial pixel input according to the needs of different data paths.

[0097] Please see Figure 5 In some embodiments, the method provided in this application may further include, but is not limited to, steps 501 to 504.

[0098] Step 501: When the window data needs to obtain boundary pixels outside the valid data area, determine the boundary pixel coordinates.

[0099] Step 502: Calculate the mirror coordinates of the boundary pixel coordinates within the valid data area based on the boundary pixel coordinates and the preset boundary.

[0100] Step 503: Obtain the mirror pixel data corresponding to the mirror coordinates from the row storage module.

[0101] Step 504: Use mirrored pixel data as boundary pixels to complete the construction of window data.

[0102] In step 501 of some embodiments, when the row storage module constructs window data for pixels located at the edge of the image, a boundary detection logic is triggered. For example, when constructing a 7×7 window for a pixel located in the first row and first column of the image, the center of the window is (1,1), but the construction of the window requires all pixels in the coordinate range (-2,-2) to (4,4). This step will detect that coordinates such as (-2,-2) are outside the valid data area of ​​the image (i.e., the area where the row number is greater than 0 and the column number is greater than 0), and these pixels are identified as boundary pixels.

[0103] In step 502 of some embodiments, after determining the boundary pixel coordinates, the device performs a mirror coordinate calculation based on these coordinates and a preset boundary (e.g., the starting boundary of the image is row 1). For row coordinate -2, it is 3 units outside boundary 1 (1-(-2)=3), so its mirror coordinate is 3 units inside boundary 1, i.e., row 4 (1+3=4). Similarly, the mirror coordinate of column coordinate -2 is also calculated as column 4. Therefore, the mirror coordinate of the boundary pixel coordinate (-2,-2) within the valid data area is calculated as (4,4).

[0104] In step 503 of some embodiments, the device uses the mirror coordinates calculated in step 502 to access the image data stored in the row storage module. Since the mirror coordinates (4,4) are coordinates located within the valid data area, the row storage module must store the pixel data corresponding to these coordinates. The device retrieves the pixel data corresponding to these (4,4) coordinates from the row storage module; this data is referred to as the mirror pixel data.

[0105] In step 504 of some embodiments, the mirrored pixel data obtained in step 503 (i.e., pixel data from (4,4)) is used to fill the positions in the window data that originally required boundary pixels (i.e., coordinates (-2,-2)). In this way, all pixels located outside the valid data area are replaced with mirrored pixel data from inside the image. This allows a window that could not be constructed due to missing data to be constructed, ensuring that even at the very edge of the image, a window with complete size and valid content can be generated.

[0106] Through steps 501 to 504 described above, this embodiment of the application first accurately identifies when the window construction exceeds the effective data area of ​​the image through step 501; then, through steps 502 and 503, based on the principle of mirror reflection, a reasonable pixel data is found from within the effective data area to replace the invalid boundary pixel; finally, the window data is filled through step 504. This series of steps ensures that the row storage module can construct a window data of complete size under any circumstances (including corners and edges of the image), avoiding calculation errors in subsequent processing modules due to missing data, and ensuring the stability and accuracy of the entire data processing flow.

[0107] Please see Figure 6 In some embodiments, based on the output timing corresponding to each data interface, the step of outputting window data from the data interface to the corresponding data path so that the data path processes the window data to obtain intermediate data may include, but is not limited to, steps 601 to 602.

[0108] Step 601: Based on the output timing determined for each data interface, when the output timing arrives, start outputting window data from the data interface to the corresponding data path.

[0109] Step 602: The data path receives window data, and the subsequent modules in the data path perform preset logical operations on the window data to obtain intermediate data.

[0110] In step 601 of some embodiments, the row storage module continuously monitors an internal row counter or clock cycle and compares it with the output timing determined for each data interface in step 102. Continuing with the previous example, the output timing for data path B (20-line delay) is "immediate output," and the output timing for data path A (5-line delay) is "wait 15 lines." When the row storage module constructs the first usable window of data, it immediately detects that the output timing for data path B has arrived and begins continuously outputting window data from the data interface of data path B. Simultaneously, it prevents data output to the data interface of data path A and starts a 15-line delay counter for path A. Only after this counter has counted to 15 lines does the row storage module determine that the output timing for data path A has arrived, and from this moment on, it begins outputting window data from the data interface of data path A to its corresponding data path.

[0111] In step 602 of some embodiments, the data path begins receiving window data output by the row storage module at the time specified in step 601. After waiting for 15 rows, data path A begins receiving window data and sends it to its internal downstream module. This downstream module performs a preset logical operation on the input window data, which itself takes 5 rows of processing time. After consuming 5 rows of processing time, the downstream module outputs the operation result, i.e., the intermediate data of data path A. Simultaneously, data path B begins receiving window data from row 0 and sends it to its internal downstream module (e.g., a complex feature extraction unit). This unit performs its preset logical operation, which takes 20 rows of processing time. After consuming 20 rows of processing time, the downstream module also outputs its operation result, i.e., the intermediate data of data path B.

[0112] Through steps 601 to 602 described above, this embodiment of the application elaborates in detail the core execution mechanism that enables timing synchronization. Step 601 is the physical execution layer for timing compensation, which transforms the compensation delay calculated in step 102 into specific waiting and releasing actions of the row storage module at the data output port, thereby achieving differentiated and interleaved data distribution at the data source. Step 602 describes the process by which data, after entering its respective data path, undergoes its inherent subsequent module processing delay and finally generates intermediate data. The technical effect of these two steps combined is that: data released with a 15-line delay (step 601), after a 5-line data processing (step 602), generates its final intermediate data after a total delay of 20 lines (15+5); while data released immediately (step 601), after a 20-line processing (step 602), also generates its final intermediate data after a total delay of 20 lines. In this way, the row delays of the intermediate data in all data paths are precisely aligned when flowing to the data aggregation module.

[0113] In step 104 of some embodiments, the data aggregation module is a downstream processing unit that simultaneously receives intermediate data streams from multiple data paths. Taking an image enhancement algorithm as an example, the intermediate data of data path A may be edge details of the image, while the intermediate data of data path B may be the base tone of the image. Since the timing distribution operation in step 103 has already compensated for the delay differences for all paths, the edge details and base tone arriving at the data aggregation module at this moment are temporally aligned, meaning they correspond to the processing results of the same original image coordinate point. Therefore, the data aggregation module can immediately perform aggregation processing on these synchronized intermediate data, such as pixel-weighted summation, without requiring any additional internal buffering to wait. The result of this aggregation processing is the final target image data output by the entire image processing device.

[0114] Please see Figure 7 In some embodiments, step 104 may include, but is not limited to, steps 701 to 702.

[0115] Step 701: Through the data aggregation module, intermediate data from multiple data channels are received synchronously.

[0116] Step 702: Perform a preset fusion operation on the intermediate data output from multiple data channels within the same clock cycle that correspond to the same image coordinate point to obtain the target image data.

[0117] In step 701 of some embodiments, the data aggregation module has multiple parallel input ports, each port corresponding to a data path. Synchronous reception means that within the same clock cycle, the data aggregation module expects to simultaneously receive a copy of intermediate data from all its input ports. Thanks to the timing adaptation operation performed in step 103, these intermediate data arriving at the same time are constructed from pixels based on the same original image coordinate point.

[0118] In step 702 of some embodiments, the data aggregation module performs a preset fusion operation on the multiple intermediate data received synchronously in step 701. Since these intermediate data all correspond to the same image coordinate point, they can be used immediately for calculation. For example, if the intermediate data of data path A is edge detail information of the image, and the intermediate data of data path B is the basic tone information of the image, a preset fusion operation may be defined as performing a pixel-level weighted average of the two data to achieve image enhancement. The data aggregation module performs this operation and outputs the result (i.e., the fused pixel value) as the target image data within the clock cycle. This process is repeated continuously as the clock cycle progresses, generating the final target image data pixel by pixel.

[0119] Through steps 701 to 702 described above, this embodiment of the application clarifies the final stage of the entire timing synchronization method. Step 701 is the verification point for the timing synchronization effect, confirming that all intermediate data streams processed through different delay paths arrive simultaneously at the input of the data aggregation module. The data aggregation module can immediately perform a wait-free fusion operation on the aligned data. Since the data aggregation module does not require an additional built-in row buffer to wait for the slowest data path, its hardware implementation can be simplified, thereby reducing chip area and saving chip costs.

[0120] The proposed timing synchronization method based on multiple data paths first stores image data in rows using a row storage module, providing an independent data interface for each data path, thus laying the structural foundation for subsequent differentiated data output. Then, based on the processing latency of the subsequent modules of each data path, the processing timing of the corresponding data interface is determined, ensuring that the start time of data output is controllable and calculated. Finally, after constructing the window data, the row storage module outputs the required window data to each data path based on the determined processing timing for each path. This ensures that the total row latency of all data paths is consistent when they reach the data aggregation module, solving the technical problem in the prior art where additional alignment row buffers are required for compensation due to inconsistent row latency. In summary, this application achieves timing synchronization of multiple data paths by outputting multiple sets of timing-adapted data interfaces through a single row storage module, reducing chip area and saving chip costs.

[0121] Please see Figure 8 This application also provides a timing synchronization device based on multiple data paths, which can implement the above-mentioned timing synchronization method based on multiple data paths, including:

[0122] The acquisition module is used to acquire image data and store the image data in rows through the row storage module; wherein, the row storage module is connected to each data path through a data interface.

[0123] The determination module is used to determine the processing timing of the corresponding data interface based on the processing delay of the subsequent modules of each data path.

[0124] The building module is used to construct the window data required for each data path based on the image data stored in the row storage module, and output the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path can process the window data to obtain intermediate data.

[0125] The processing module is used to aggregate intermediate data output from multiple data channels through the data aggregation module to obtain target image data.

[0126] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the timing synchronization method based on multiple data paths as described in any one of the embodiments of the first aspect of this application.

[0127] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that is executed by a processor to implement the timing synchronization method based on multiple data paths as described in any one of the embodiments of the first aspect of this application.

[0128] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0129] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0130] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to implement the timing synchronization method based on multiple data paths according to the embodiments of this application.

[0131] The input / output interface 903 is used to implement information input and output;

[0132] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0133] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0134] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0135] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described timing synchronization method based on multiple data paths.

[0136] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0137] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0138] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0139] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0141] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0142] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0146] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A timing synchronization method based on multiple data paths, characterized in that, Applied to an image processing system, the image processing system including a row storage module, multiple data paths, and a data aggregation module, the method includes: Image data is acquired and stored in rows through the row storage module; wherein the row storage module is connected to each data path through a data interface. Determining the processing timing of the corresponding data interface based on the processing latency of the downstream module of each data path includes: identifying the target data path with the longest downstream module processing latency among the plurality of data paths, and determining the longest downstream module processing latency as the target row latency; calculating the downstream module processing latency of each other data path besides the target data path; calculating a compensation row latency required for the other data path based on the target row latency and the downstream module processing latency of the other data path; and determining the compensation row latency as the processing timing of the data interface corresponding to the other data path. The row storage module constructs window data required for each data path based on the image data stored in the row storage, and outputs the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path processes the window data to obtain intermediate data. This includes: the row storage module continuously monitors an internal row counter and compares it with the processing timing determined for each data interface; a delay counter is started for the other data paths, and only when the delay counter reaches the compensation row delay is it determined that the processing timing of the other data path has arrived, and the window data is started to be output from the data interface to the corresponding other data path; the data path receives the window data, and the subsequent modules in the data path perform preset logical operations on the window data to obtain the intermediate data. The intermediate data output from the multiple data channels is aggregated by the data aggregation module to obtain the target image data.

2. The timing synchronization method based on multiple data paths according to claim 1, characterized in that, The process of storing the image data in rows using the row storage module includes: The image data is acquired pixel by pixel and written to the row storage module according to a predetermined row and column order. Based on the row and column order of the image data, the pixels are constructed into multiple rows of image data for storage.

3. The timing synchronization method based on multiple data paths according to claim 2, characterized in that, The step of constructing the window data required for each data path based on the image data stored in the row storage module includes: Determine the corresponding number of rows based on the window size required for each data path; Based on the number of rows, determine the starting number of rows required to construct the window data; When the number of rows of the multi-row image data stored in the row storage module reaches the starting number of rows, a multi-row image with a row size equal to the number of rows corresponding to the window size is selected from the multi-row image data stored in the row storage module. From the selected multi-row images, extract pixel data with a column size equal to the window size to obtain the window data.

4. The timing synchronization method based on multiple data paths according to claim 3, characterized in that, The method further includes: When the window data needs to obtain boundary pixels outside the valid data area, the boundary pixel coordinates of the boundary pixels are determined; Based on the boundary pixel coordinates and the preset boundary, calculate the mirror coordinates of the boundary pixel coordinates within the effective data area; Obtain the mirror pixel data corresponding to the mirror coordinates from the row storage module; The mirrored pixel data is used as the boundary pixels to complete the construction of the window data.

5. The method according to claim 1, characterized in that, The process of aggregating the intermediate data output from the multiple data paths through the data aggregation module to obtain target image data includes: The data aggregation module synchronously receives intermediate data from the multiple data paths. A preset fusion operation is performed on the intermediate data output from the multiple data channels within the same clock cycle and corresponding to the same image coordinate point to obtain the target image data.

6. A timing synchronization device based on multiple data paths, characterized in that, The device includes: An acquisition module is used to acquire image data and store the image data in rows through a row storage module; wherein, the row storage module is connected to each data path through a data interface. A determining module is configured to determine the processing timing of the corresponding data interface based on the processing latency of the downstream module of each data path, including: determining a target data path with the longest downstream module processing latency among multiple data paths, and determining the longest downstream module processing latency as the target row latency; calculating the downstream module processing latency of each other data path besides the target data path; calculating a compensation row latency required for the other data path based on the target row latency and the downstream module processing latency of the other data path; and determining the compensation row latency as the processing timing of the data interface corresponding to the other data path. A construction module is used to construct window data required for each data path based on the image data stored in the row storage module, and output the window data from the data interface to the corresponding data path based on the processing timing corresponding to each data interface, so that the data path processes the window data to obtain intermediate data. This includes: the row storage module continuously monitors an internal row counter and compares it with the processing timing determined for each data interface; a delay counter is started for the other data paths, and only when the delay counter reaches the compensated row delay is the processing timing of the other data path determined, and the window data is started to be output from the data interface to the corresponding other data path; the data path receives the window data, and the subsequent modules in the data path perform preset logical operations on the window data to obtain the intermediate data. The processing module is used to aggregate the intermediate data output from multiple data channels through the data aggregation module to obtain target image data.

7. An electronic device, characterized in that, include: The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the timing synchronization method based on multiple data paths as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program, which is executed by a processor to implement the timing synchronization method based on multiple data paths as described in any one of claims 1 to 5.

Images