High-performance image processing task scheduling method and intelligent collaborative computing platform

Abstract

The present invention relates to the technical field of computing platforms for image processing, and discloses a high-efficiency image processing task scheduling method and an intelligent collaborative computing platform, wherein a first processing module divides a single-frame image into fixed pixel blocks, performs label marking, and dynamically adjusts the communication frame granularity; the first processing module generates a global synchronization clock signal and performs protocol-level timing alignment, and when the first processing module sends a communication frame to the second processing module, it triggers a processing start signal of the second processing module; the second processing module performs target identification and labeling on each fixed pixel block in the communication frame; the first processing module sends the processed communication frame to a third processing module; the third processing module merges communication frames with the same timing label into a single-frame image to output a video stream with target image parameter marking. The present invention is conducive to solving the technical problems of insufficient real-time transmission efficiency of edge computing platforms and low overall resource utilization in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of computing platforms for image processing, and in particular to a high-performance image processing task scheduling method and an intelligent collaborative computing platform. Background Art

[0002] Cloud computing is a service model that provides on-demand computing resources (such as servers, storage, databases, networks, software, etc.) over the Internet. Users can remotely access a shared pool of computing resources without having to deploy hardware or manage infrastructure locally.

[0003] Although cloud computing does not require the deployment of excessive computing resources locally, it does require data to be transmitted to remote data centers for processing, resulting in high round-trip latency (usually over 100ms), which cannot meet the needs of scenarios requiring millisecond-level responses (such as target recognition and tracking, autonomous driving, and industrial control).

[0004] Edge computing platform is a distributed computing architecture that moves computing, storage, and data processing capabilities from the traditional cloud to the edge of the network (close to the data source or terminal device).

[0005] In target recognition and tracking technologies, edge computing platforms can process data locally to avoid round-trip delays in the cloud. However, edge computing platforms can generally only calculate key data in target recognition and tracking technologies, and the latency level of the edge computing platform's output calculation results significantly exceeds the real-time system threshold (above 200 to 500ns), seriously restricting real-time transmission efficiency. At the same time, the computing resources of the edge computing platform are oversaturated or idle, and the overall resource utilization is significantly reduced.

[0006] Therefore, it is necessary to propose a new high-efficiency image processing task scheduling method and intelligent collaborative computing platform to solve the technical problems of insufficient real-time transmission efficiency and low overall resource utilization of edge computing platforms in existing technologies. Summary of the Invention

[0007] The main purpose of the present invention is to provide a high-efficiency image processing task scheduling method and an intelligent collaborative computing platform, aiming to solve the technical problems of insufficient real-time transmission efficiency and low overall resource utilization of edge computing platforms in the existing technology.

[0008] To achieve the above objectives, the present invention provides a high-efficiency image processing task scheduling method, which is applied to an intelligent collaborative computing platform; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively communicatively connected to the first processing module; the method includes the following steps:

[0009] The first processing module divides a single frame of the received image data into fixed pixel blocks, tags each communication frame and adds a checksum, and dynamically adjusts the communication frame granularity based on the real-time bandwidth and computing load. The tag includes a timing tag, and the timing tag includes a timestamp.

[0010] The first processing module uses a phase-locked loop to generate a global synchronous clock signal and performs protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, the processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, the processing start signal of the third processing module is triggered;

[0011] The second processing module receives the communication frame sent by the first processing module, performs target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returns the communication frame after target recognition and marking to the first processing module;

[0012] The first processing module sends the communication frame after target identification and marking processing to the third processing module;

[0013] The communication frames with the same label are merged into a single frame image through the third processing module, and the set parameters of the target image are calculated to output a video stream marked with the target image parameters.

[0014] Optionally, before the step of the first processing module dividing a single frame image in the received image data into fixed pixel blocks, labeling each communication frame and adding a checksum, and dynamically adjusting the communication frame granularity according to real-time bandwidth and computing load, the step further includes:

[0015] Acquire high-speed video signal images captured by each image acquisition unit, convert the high-speed video signal images captured by each image acquisition unit into low-speed 16-bit parallel image data, and send the low-speed 16-bit parallel image data corresponding to each image acquisition unit to the first processing module through different data transmission channels, wherein each image acquisition unit is used to capture images at different monitoring angles;

[0016] The step of labeling each communication frame comprises:

[0017] The first processing module marks each communication frame with a channel identifier.

[0018] Optionally, the method further includes:

[0019] A dual-channel cache module is configured in the first processing module to perform zero-wait writing and reading of communication frames;

[0020] A ring buffer module is constructed in the third processing module, and the ring buffer module is divided into multiple independent buffer units with different priorities according to task priority. Each buffer unit corresponds to an independent processing thread, so that the operating system can set the thread scheduling priority to implement a priority execution and response guarantee mechanism for high-priority tasks;

[0021] In the first processing module, the second processing module and the third processing module, the data queues in the corresponding buffer modules are dynamically managed according to the communication frame timing tags.

[0022] Optionally, the method further includes:

[0023] The second processing module calculates priority weights based on the task type and the target state level of the external input, and optimizes the task allocation of each computing core in the second processing module through the multi-core resource preemption mechanism.

[0024] Optionally, the second processing module calculates priority weights based on task types and externally input target state levels, and optimizes task allocation for each computing core in the second processing module through a multi-core resource preemption mechanism, including:

[0025] The second processing module presets a basic priority weight for each task type according to the set task type, obtains the real-time target state level corresponding to the task type input by the external sensor, and dynamically adjusts the real-time priority weight of each task type;

[0026] The second processing module allocates the task types that reach the set priority weight to the dedicated computing core, and allocates the task types that do not reach the set priority weight to the shared computing core;

[0027] When the existence of a high-priority task type is detected, a data queue update is triggered by a hardware interrupt signal to replace the currently executing task type with the high-priority task type.

[0028] Optionally, the method further includes:

[0029] Through the embedded performance counter, the task processing time of each communication frame is counted in real time according to the timing tag of each communication frame, and the task processing time of each communication frame is judged to be timed out.

[0030] If so, trigger communication frame retransmission or computing resource reallocation.

[0031] Optionally, the step of performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds includes:

[0032] Identifying the image background type of each fixed pixel block in the communication frame;

[0033] For frame areas where the image background type is identified as sky, target region extraction, background suppression, target detection, and false target elimination are performed in sequence. Target detection includes downscaling the original image of each fixed pixel block in the communication frame and performing weak target detection or area target detection on images of each scale, depending on the target scale.

[0034] For the frame area where the image background type is identified as the water surface, the target interest area is processed in sequence: smoothing filtering, ROI extraction and target enhancement, ROI segmentation and target confirmation. Among them, the confirmation of the target interest area includes: detecting the target waterline of the ship, searching for the target vertical edge in the set area above the target waterline, and determining the target interest area based on the target waterline and the target vertical edge.

[0035] Optionally, the step of performing small target detection or area target detection on images of different scales according to different target scales includes:

[0036] When the target size is within the first size interval, the Robinson filter and morphological filter operators of the corresponding scale are used to perform background suppression filtering on the weak target;

[0037] When the target size is within the second size interval, a weak target processing process is performed on the downscaled image;

[0038] When the target size is in the third size interval, surface target segmentation and detection processing is performed on the downscaled image, and target merging is performed at the original scale, wherein the first size interval, the second size interval, and the third size interval increase in sequence.

[0039] To achieve the above-mentioned object, the present invention further proposes an intelligent collaborative computing platform, which applies the high-efficiency image processing task scheduling method; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively connected to the first processing module;

[0040] The first processing module is configured to: divide a single frame of image data received into fixed pixel blocks, label each communication frame and add a checksum, and dynamically adjust the communication frame granularity according to real-time bandwidth and computing load, wherein the label includes a timing tag, and the timing tag includes a timestamp; use a phase-locked loop to generate a global synchronous clock signal and perform protocol-level timing alignment so that when the first processing module sends a communication frame to the second processing module, a processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, a processing start signal of the third processing module is triggered;

[0041] The second processing module is configured to: receive the communication frame sent by the first processing module, perform target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and return the communication frame after target recognition and marking to the first processing module;

[0042] The first processing module is further configured to: send the communication frame after target identification and marking processing to the third processing module;

[0043] The third processing module is used to merge communication frames with the same label into a single frame image, and perform set parameter calculation on the target image to output a video stream marked with the target image parameters.

[0044] Optionally, the first processing module is connected to the second processing module through a high-speed serial interconnect interface and a general input / output interface respectively; the first processing module is connected to the third processing module through a high-speed peripheral component interconnect interface and a general input / output interface respectively; the first processing module is connected to multiple high-speed transceivers, each high-speed transceiver is used to receive a video image captured by an external image acquisition unit; the first processing module is provided with a first cache module, the second processing module is provided with a second cache module, and the third processing module is provided with a third cache module; the third processing module is connected to a first PHY chip, and the first PHY chip is connected to the debug interface, the second PHY chip and the third PHY chip respectively through a Gigabit Ethernet switching chip; the second PHY chip is connected to the first processing module, and the third PHY chip is connected to the second processing module; the third processing module is also connected to an HDMI distributor chip to support multi-channel high-definition video output.

[0045] In the technical solution of the present invention, a hybrid heterogeneous architecture is adopted to establish a modular collaborative computing system. Specifically, the first processing module is responsible for processing a single-frame image into a communication frame, adding labels and check codes to the communication frame, and performing dedicated synchronization control between modules; the second processing module performs target identification and marking on each communication frame according to different image backgrounds based on the communication frame timing, and the first processing module is responsible for sending the image after target identification and marking to the third processing module, which performs image merging and sets parameter calculations based on the target marking results, thereby outputting the processed image as a video stream marked with target image parameters. In the present invention, the first processing module adopts a communication frame transmission method, dynamically adjusts the real-time transmission capability according to the real-time bandwidth and computing load, and fully utilizes the computing resources to avoid oversaturation or no-load status through global synchronous clock signals and protocol-level timing alignment, thereby improving overall resource utilization. As a result, the present invention forms a pipelined collaborative processing architecture between heterogeneous computing units, and divides the image processing flow into three pipeline stages: the first processing module processes data communication frames, the second processing module parallel calculation, and the third processing module data integration. Each stage is executed in parallel, and the end-to-end delay is the maximum delay of each stage, which is significantly lower than the traditional serial architecture. Therefore, the end-to-end delay is compressed by parallel execution between modules, ensuring that the end-to-end processing delay of image data is controllable. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a block diagram of the intelligent collaborative computing platform of the present invention;

[0047] Figure 2 is a flow chart of a first embodiment of a high-performance image processing task scheduling method of the present invention;

[0048] Figure 3 This is a flow chart of the sky small target detection algorithm in the present invention;

[0049] Figure 4 This is a flow chart of the surface ship target detection algorithm in the present invention;

[0050] Figure 5 This is a flow chart of ship target waterline detection in the present invention;

[0051] Figure 6 This is a flowchart for extracting the ship target area of ​​interest in the present invention.

[0052] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] In the following description, suffixes such as "unit," "component," or "unit" used to represent elements are only used to facilitate the description of the present invention and have no specific meaning. Therefore, "unit," "component," or "unit" can be used interchangeably.

[0055] See also Figures 1 to 2 In a first embodiment of the present invention, a high-performance image processing task scheduling method is provided, which is applied to an intelligent collaborative computing platform; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively connected to the first processing module; the method includes the following steps:

[0056] Step S10: The first processing module divides a single frame of the received image data into fixed pixel blocks, tags each communication frame and adds a checksum, and dynamically adjusts the communication frame granularity according to the real-time bandwidth and computing load, wherein the tag includes a timing tag, and the timing tag includes a timestamp;

[0057] Step S20: The first processing module uses a phase-locked loop to generate a global synchronous clock signal and performs protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, the processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, the processing start signal of the third processing module is triggered;

[0058] Step S30: The second processing module receives the communication frame sent by the first processing module, performs target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returns the communication frame after target recognition and marking to the first processing module;

[0059] Step S40: the first processing module sends the communication frame after target identification and marking processing to the third processing module;

[0060] Step S50 , merging the communication frames with the same label into a single frame image through the third processing module, and performing set parameter calculation on the target image to output a video stream marked with the target image parameters.

[0061] In the technical solution of the present invention, a hybrid heterogeneous architecture is adopted to establish a modular collaborative computing system. Specifically, the first processing module is responsible for processing a single-frame image into a communication frame, adding labels and check codes to the communication frame, and performing dedicated synchronization control between modules; the second processing module performs target identification and marking on each communication frame according to different image backgrounds based on the communication frame timing, and the first processing module is responsible for sending the image after target identification and marking to the third processing module, which performs image merging and sets parameter calculations based on the target marking results, thereby outputting the processed image as a video stream marked with target image parameters. In the present invention, the first processing module adopts a communication frame transmission method, dynamically adjusts the real-time transmission capability according to the real-time bandwidth and computing load, and fully utilizes the computing resources to avoid oversaturation or no-load status through global synchronous clock signals and protocol-level timing alignment, thereby improving overall resource utilization. As a result, the present invention forms a pipelined collaborative processing architecture between heterogeneous computing units, and divides the image processing flow into three pipeline stages: the first processing module processes data communication frames, the second processing module parallel calculation, and the third processing module data integration. Each stage is executed in parallel, and the end-to-end delay is the maximum delay of each stage, which is significantly lower than the traditional serial architecture. Therefore, the end-to-end delay is compressed by parallel execution between modules, ensuring that the end-to-end processing delay of image data is controllable.

[0062] In the technical solution of the present invention, the image collected by the external image acquisition unit is sent to the first processing module for processing, wherein the image collected by the external image acquisition unit can be of multiple types, for example, visible light images, infrared images, ultraviolet images, spectral images or radar images.

[0063] The communication frame includes at least one fixed pixel block.

[0064] Specifically, in this embodiment, the fixed pixel block size for grid division of a single frame image can be 256×256 pixels, and a unique label is attached to each communication frame. In this embodiment, the label includes a timing label, and the timing label includes: a frame number and a timestamp marked by the first processing module. The check code can be a CRC check code. The timestamp marked by the first processing module can be used to detect transmission delays, and the image time can also be known when a subsequent user views the corresponding image.

[0065] Furthermore, in this embodiment, the real-time transmission bandwidth of each communication frame is monitored through the status flag, and in this embodiment, the memory occupancy rate corresponding to the second processing module and the third processing module can be used to feedback the computing unit load and dynamically adjust the communication frame size. When the back-end cannot handle it, the communication frame size is reduced.

[0066] The communication frame boundary is aligned with the image coding format (such as H.264, YUV), which can ensure the integrity of the communication frame data.

[0067] In the second processing module, target recognition and labeling are performed, wherein the target object type for target recognition is at least one. In a specific application scenario, the image can be an aerial image, a water surface image, or a mixed image of aerial and water surface images. Taking the aerial image as an example, the target objects for target recognition can be multiple aerial targets, such as various flying objects.

[0068] By performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, multiple target objects in each fixed pixel block can be recognized, and the recognized results can be marked one by one.

[0069] Therefore, in the third processing module, the position calculation, orientation calculation and speed calculation of the target object can be performed respectively according to each target object identified from each fixed pixel block.

[0070] Specifically, each time the first processing module acquires a single frame of image and processes it to obtain a fixed pixel block, it then processes the fixed pixel block into a communication frame. It can then send the corresponding communication frame to the second processing module without waiting, thereby improving the real-time performance of task processing. Furthermore, the first processing module triggers the processing start signal of the second processing module by sending a communication frame to the second processing module, allowing the second processing module to utilize idle computing cores to process the communication frame without waiting (performing target identification and labeling based on different image backgrounds). After processing, the processed communication frame is returned to the first processing module. The second processing module then sends the communication frame to the third processing module, and by sending the communication frame, it triggers the processing start signal of the third processing module, allowing the third processing module to merge communication frames with the same label and set parameter calculations without waiting. The entire process requires no waiting, reduces latency, and achieves high-efficiency image processing task scheduling.

[0071] Based on the first embodiment of the high-performance image processing task scheduling method of the present invention, in the second embodiment of the high-performance image processing task scheduling method of the present invention, before step S10, the method further includes:

[0072] Step S60: Acquire the high-speed video signal images captured by each image acquisition unit, convert the high-speed video signal images captured by each image acquisition unit into low-speed 16-bit parallel image data, and send the low-speed 16-bit parallel image data corresponding to each image acquisition unit to the first processing module through different data transmission channels, wherein each image acquisition unit is used to capture images from different monitoring angles;

[0073] The step of labeling each communication frame in step S10 includes:

[0074] Step S11: The first processing module marks each communication frame with a channel identifier.

[0075] The high-speed video signal image may be a high-speed CML signal image. Specifically, the multiple external image acquisition units are configured to cover a full-angle (360°) image acquisition range, with each image acquisition unit capturing images within a set angle range. For example, each image acquisition unit covers a 90° field of view, and full-angle image acquisition is performed using four image acquisition units.

[0076] The image captured by each image acquisition unit is transmitted to the first processing module via one of the data transmission channels. In this embodiment, the image captured by each image acquisition unit is sequentially transmitted to the first processing module via a uniquely corresponding video signal interface and a high-speed transceiver.

[0077] Specifically, embedding the channel identifier of the corresponding data transmission channel in each communication frame is conducive to identifying the data transmission channel to which the communication frame belongs through the channel identifier in the communication frame, thereby distinguishing each communication frame. When the third processing module merges the communication frames, the communication frames with the same timing label and the same channel identifier can be merged into a single-frame image, thereby reducing the calculation time of image merging and improving the merging efficiency; and the channel where the frame is lost can be located through the channel identifier of the communication frame.

[0078] Based on the second embodiment of the high-performance image processing task scheduling method of the present invention, in a third embodiment of the high-performance image processing task scheduling method of the present invention, the method further includes:

[0079] Step S70, configuring a dual-channel cache module in the first processing module to perform zero-wait writing and reading of communication frames;

[0080] Step S80: constructing a ring buffer module in the third processing module and dividing the ring buffer module into a plurality of independent buffer units of different priorities according to task priorities. Each buffer unit corresponds to an independent processing thread, so that the operating system can set thread scheduling priorities to implement a priority execution and response guarantee mechanism for high-priority tasks;

[0081] In step S90 , the first processing module, the second processing module, and the third processing module dynamically manage data queues in corresponding buffer modules according to the communication frame timing tags.

[0082] Specifically, the first processing module, the second processing module, and the third processing module are each provided with a corresponding cache module; the priority division method of the ring buffer module of the third processing module can be set as needed, for example, it can be divided into three independent buffer units of high, medium, and low according to task priority. The communication frames sent from the first processing module to the third processing module after target identification and marking are cached in buffer units corresponding to different priorities according to different priorities. Therefore, the operating system can prioritize the scheduling of data in the high-priority buffer unit, so that the third processing module can perform priority processing.

[0083] At the same time, the third processing module can also calculate the data access frequency in the low-priority buffer unit and dynamically release the cache space corresponding to the image data with a data access frequency lower than a preset value to prevent data accumulation.

[0084] At the same time, the first processing module, the second processing module and the third processing module dynamically manage the data queues in the corresponding cache modules according to the communication frame timing tags, so that the image can be processed strictly without interruption.

[0085] Based on the third embodiment of the high-performance image processing task scheduling method of the present invention, in a fourth embodiment of the high-performance image processing task scheduling method of the present invention, the method further includes:

[0086] In step S100 , the second processing module calculates priority weights based on the task type and the target state level of the external input, and optimizes the task allocation of each computing core in the second processing module through a multi-core resource preemption mechanism.

[0087] Specifically, the task type may be an object type for target recognition, and the target state level may be a state level determined based on data such as the speed, distance, and orientation of the target object. The target state level data may be collected by an external sensor.

[0088] Based on the fourth embodiment of the high-performance image processing task scheduling method of the present invention, in a fifth embodiment of the high-performance image processing task scheduling method of the present invention, step S100 includes:

[0089] In step S101, the second processing module presets a basic priority weight for each task type according to the set task type, obtains the real-time target state level corresponding to the task type input by the external sensor, and dynamically adjusts the real-time priority weight of each task type;

[0090] Step S102: The second processing module allocates the task types that reach the set priority weight to the dedicated computing core, and allocates the task types that do not reach the set priority weight to the shared computing core;

[0091] Step S103 : When a high-priority task type is detected, a data queue update is triggered by a hardware interrupt signal to replace the currently executed task type with the high-priority task type.

[0092] For example, the dedicated computing cores may be Core 0 to Core 5 of the second processing module, and the shared computing cores may be Core 6 to Core 7 of the second processing module.

[0093] Based on the first embodiment of the high-performance image processing task scheduling method of the present invention, in a sixth embodiment of the high-performance image processing task scheduling method of the present invention, the method further includes:

[0094] Step S110, using an embedded performance counter, based on the timing tag of each communication frame, real-time statistics are collected on the task processing time of each communication frame, and it is determined whether the task processing time of each communication frame has exceeded the time limit;

[0095] If so, execute step S120: trigger communication frame retransmission or computing resource reallocation.

[0096] The corresponding timestamp can be obtained from the timing tag of each communication frame. This allows the processing time of the communication frame to be detected based on the timestamp and the current processing progress of the communication frame. If a timeout occurs, retransmission or re-allocation of the computing core of the second processing module can be performed, effectively controlling image processing latency.

[0097] Furthermore, the method further includes:

[0098] Step S130: construct a first high-speed transmission channel (eg, PCIe channel) between the first processing module and the third processing module, and construct a second high-speed transmission channel (eg, SRIO channel) between the first processing module and the second processing module.

[0099] Specifically, the first processing module is connected to the second processing module through a high-speed serial interconnect interface (SRIO interface) and a general input and output interface (GPIO interface); the first processing module is connected to the third processing module through a high-speed peripheral component interconnect interface (PCIe interface) and a general input and output interface (GPIO interface).

[0100] Furthermore, in this embodiment, a phase-locked loop (PLL) in the first processing module is used to generate a global synchronous clock signal, and protocol-level timing alignment is achieved through the synchronization field (such as a 64-bit timestamp) in the SRIO frame header. This ensures that when the first processing module sends a communication frame, it triggers the second processing module as the processing start signal at the receiving end, thereby ensuring that the delay error is ≤10ns.

[0101] See also Figures 3 to 6 Based on the first to sixth embodiments of the high-efficiency image processing task scheduling method of the present invention, in the seventh embodiment of the high-efficiency image processing task scheduling method of the present invention, the step of identifying and marking each fixed pixel block in the communication frame according to different image backgrounds in step S30 includes:

[0102] Step S31, identifying the image background type of each fixed pixel block in the communication frame;

[0103] Step S32: for the frame area where the image background type is identified as sky, target area extraction, background suppression, target detection and false target elimination are performed in sequence (see Figure 3 ); wherein the target detection includes: downscaling the original image of each fixed pixel block in the communication frame, and performing weak target detection or surface target detection processing on the image of each scale according to the different target scales;

[0104] Step S33: for the frame area where the image background type is identified as water surface, the target interest region is sequentially subjected to smoothing filtering, ROI extraction and target enhancement, ROI segmentation and target confirmation (see Figure 4 ), wherein the confirmation of the target area of ​​interest includes: detecting the target waterline of the ship, searching for the target vertical edge in a set area above the target waterline, and determining the target area of ​​interest based on the target waterline and the target vertical edge.

[0105] Specifically, in the technical solution of the present invention, each image acquisition unit covers a 360° image acquisition range. Within the 360° image acquisition range, the acquired images include the sky, the water surface, or the junction of the water and the sky.

[0106] After splitting the image into fixed pixel blocks and processing the fixed pixel blocks into communication frames, background type recognition is performed on each fixed pixel block, and different target detection processing is performed on the fixed pixel blocks according to different image backgrounds.

[0107] In this invention, different target detection methods are used for sky targets based on their scale. Furthermore, for surface targets, the primary focus is on ships, and the region of interest for target detection is determined by the ship's waterline position and the target's vertical edge, enabling rapid target location and reducing data processing.

[0108] For the water-sky junction position, surface targets and sky targets are segmented and identified separately.

[0109] Specifically, in the high-efficiency image processing task scheduling method of the present invention, the second processing module can perform a sky target detection algorithm when the image background type is sky, and execute a surface ship target detection algorithm when the image background type is water.

[0110] Among them, according to the function of the sky target detection algorithm, the image sequence is input into the second processing module, and target area extraction, background suppression, target detection and false target elimination are performed in sequence.

[0111] Furthermore, to address the problem of frequent interference from erroneous targets in complex backgrounds, a method based on multi-feature fusion judgment is adopted, including:

[0112] Extract the characteristic quantities of the suspected target in the feature space, including statistical features (including the mean, variance, contrast and distribution of the target grayscale), structural features (including the width, height, area, contour and perimeter of the target image), transformation coefficient features (target characteristics described by seven geometric invariants), and motion features (spatial position, velocity and acceleration of the target or the distance between targets).

[0113] Determine the fusion decision conditions based on the statistical characteristic parameters;

[0114] Eliminate erroneous targets that do not meet the fusion judgment conditions.

[0115] In the scene model of infrared image sequences of ship targets with a water-sky background, although ships are generally surface targets larger than 5×5 pixels in infrared image sequences, from the detector's perspective, the chimneys of ship targets are often obscured by other parts over a wide range of angles. Overall, the ship's radiation temperature is much lower than that of an aircraft's tail plume. More importantly, the water surface background is much more complex than the sky background. Therefore, directly performing global threshold segmentation on the entire infrared image to detect targets is extremely difficult. Even if targets can be segmented, many false targets will be generated. Considering the locational characteristics of distant ship targets, which are generally located near the water-sky boundary, the visual system's attention to the region of interest, and the real-time processing system's requirements for algorithm complexity and effectiveness, an infrared ship target detection algorithm flow is proposed.

[0116] In view of the function of the surface ship target detection algorithm, the image sequence is input into the second processing module, which performs smoothing filtering, ROI extraction (Region of Interest Extraction) and target enhancement, ROI segmentation (Region of Interest Segmentation), and target confirmation processing in sequence.

[0117] Ship targets appear near or below the water-sky boundary. In this method, characteristic regions within the ship are selected. For example, a ship target has distinct waterline edge features and localized high brightness. Based on these characteristics, the method's guiding principle for finding the target region of interest is to first locate the waterline edge of the ship target below the water-sky boundary, and then determine the target region of interest based on the localized high brightness features above the waterline.

[0118] The target waterline of the ship is determined as follows:

[0119] (1) There are obvious horizontal edge features within the fixed pixel block;

[0120] (2) The area above the waterline is the target's high grayscale area, and the area below the waterline is the water's low grayscale area;

[0121] (3) The edge of the waterline is below the water-sky boundary.

[0122] According to the above characteristics of the ship's target waterline, the ship's target waterline detection process is as follows: Figure 5 As shown:

[0123] The idea of ​​extracting the ship target interest area is to search for the target vertical edge (for example, the ship target chimney, or other preset vertical edges) in a certain area above the detected target waterline. If a vertical edge exists, the target interest area is determined by the waterline and the vertical edge together; if no vertical edge exists, the extracted waterline is considered to be an incorrect target and is removed. The algorithm implementation process is as follows: Figure 6 As shown:

[0124] By segmenting the image with the target enhanced within the region of interest (ROI), the target can be effectively extracted and its shape information obtained. After enhancement, the target has the highest grayscale, occupying approximately 10% to 40% of the image area. The grayscale difference between the sky and water backgrounds is minimal, and the target occupies the low-grayscale portion of the ROI, with a uniform grayscale distribution. A fast image segmentation algorithm suitable for real-time processing employs the maximum inter-class variance method. Based on the criterion of maximizing the inter-class variance, the optimal threshold is found when this measure function reaches its maximum. The maximum inter-class variance method achieves optimal performance when the target and background occupy roughly equal proportions within the image. ROI extraction and target enhancement ensure that the target and background characteristics within the ROI to be segmented precisely meet these requirements. Furthermore, the algorithm is computationally simple, requiring only the calculation of the zero-order and first-order cumulative moments of the grayscale histogram. Hardware implementation is simple, making it suitable for real-time image segmentation applications.

[0125] Based on the seventh embodiment of the high-efficiency image processing task scheduling method of the present invention, in the eighth embodiment of the high-efficiency image processing task scheduling method of the present invention, the step of performing small target detection or area target detection on images of different scales according to different target scales in step S32 includes:

[0126] Step S321: When the target size is within the first size interval, a Robinson filter and a morphological filter operator of the corresponding scale are used to perform background suppression filtering on the weak target;

[0127] Step S322: When the target size is within the second size range, a small target processing process is performed on the downscaled image;

[0128] Step S323: When the target size is within the third size interval, surface target segmentation and detection processing is performed on the downscaled image, and target merging is performed at the original scale, wherein the first size interval, the second size interval, and the third size interval increase in sequence.

[0129] In order to solve the problems encountered in sky target detection, such as weak target detection in complex background and interference from false targets, the target detection part of the second processing module adopts a real-time weak target detection algorithm based on multi-scale filtering, which can effectively realize the effective detection of weak targets in complex low-altitude background.

[0130] Specifically, the core idea of ​​the algorithm in the present invention is to downscale the original image and perform small target detection and area target detection on the image at each scale according to the target scale. Small target detection is mainly performed on the original scale image.

[0131] Specifically, the first, second, and third size ranges can be set based on specific needs. For example, the first size range can be 2×2 to 10×10 pixels (excluding 10×10 pixels) to effectively enhance small targets and suppress background. The second size range can be 10×10 to 20×20 pixels (excluding 20×20 pixels), with the same small target processing process performed on the downscaled image to achieve algorithm modularization. The third size range can be 20×20 pixels or larger, with surface target segmentation and detection performed on the downscaled image, and ultimately, target merging performed at the original scale.

[0132] To achieve the above-mentioned object, the present invention further proposes an intelligent collaborative computing platform, which applies the high-efficiency image processing task scheduling method; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively connected to the first processing module;

[0133] The first processing module is configured to: divide a single frame of image data received into fixed pixel blocks, label each communication frame and add a checksum, and dynamically adjust the communication frame granularity according to real-time bandwidth and computing load, wherein the label includes a timing tag, and the timing tag includes a timestamp; use a phase-locked loop to generate a global synchronous clock signal and perform protocol-level timing alignment so that when the first processing module sends a communication frame to the second processing module, a processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, a processing start signal of the third processing module is triggered;

[0134] The second processing module is configured to: receive the communication frame sent by the first processing module, perform target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and return the communication frame after target recognition and marking to the first processing module;

[0135] The first processing module is further configured to: send the communication frame after target identification and marking processing to the third processing module;

[0136] The third processing module is used to merge communication frames with the same label into a single frame image, and perform set parameter calculation on the target image to output a video stream marked with the target image parameters.

[0137] Optionally, the first processing module is connected to the second processing module through a high-speed serial interconnect interface (SRIO interface) and a general input and output interface (GPIO interface); the first processing module is connected to the third processing module through a high-speed peripheral component interconnect interface (PCIe interface) and a general input and output interface (GPIO interface).

[0138] Optionally, the first processing module is connected to multiple high-speed transceivers, each high-speed transceiver is used to receive a video image captured by an external image acquisition unit; the first processing module is provided with a first cache module, the second processing module is provided with a second cache module, and the third processing module is provided with a third cache module; the third processing module is connected to a first PHY chip (used to convert RGMII into an MDI signal), and the first PHY chip is connected to the debugging interface, the second PHY chip and the third PHY chip respectively through the Gigabit Ethernet switching chip; the second PHY chip is connected to the first processing module, and the third PHY chip is connected to the second processing module; the third processing module is also connected to an HDMI distributor chip to support multi-channel high-definition video output.

[0139] In a specific embodiment, the intelligent collaborative computing platform may be provided with four high-speed transceivers corresponding to four channels of video images. Of course, the number of high-speed transceivers is not limited thereto.

[0140] Specifically, in the technical solution of the present invention, the pipelined collaborative processing architecture between heterogeneous computing units can be divided into the following pipeline stages:

[0141] Phase 1: The first processing module performs image data acquisition, communication frame encapsulation (e.g., a 1K image communication frame is 16 fixed pixel blocks), and SRIO transmission (rate ≥ 20Gbps) and PCIe transmission (rate ≥ 20Gbps).

[0142] Phase 2: Each computing core of the second processing module processes each fixed pixel block of the communication frame in parallel (for example, each core processes one fixed pixel block), executes the target detection algorithm, and transmits the results back to the first processing module via SRIO.

[0143] Phase 3: The third processing module integrates the results of each fixed pixel block, performs image stitching and trajectory fusion, and outputs a 4K@60Hz video stream. The information marked in the video stream includes: target object type, corresponding coordinate information of the target object, and speed information.

[0144] The advantages of the above pipeline are: each stage is executed in parallel, and the end-to-end latency is the maximum latency of each stage, which is significantly lower than the traditional serial architecture. In fact, the total latency is lower due to parallel processing.

[0145] Furthermore, the present invention implements measures to ensure controllable end-to-end processing delay, specifically:

[0146] The present invention realizes hierarchical optimization design:

[0147] Transport layer: The second high-speed transmission channel interface adopts short frame transmission mode (frame length ≤ 256B), with a delay of ≤ 150ns; the first high-speed transmission channel interface enables zero-copy DMA technology, with a delay of ≤ 200ns.

[0148] Processing layer: The pipeline architecture reduces module idle waiting, and the dynamic scheduling algorithm avoids resource contention (for example, the load balancing error between the cores of the second processing module is ≤5%).

[0149] The present invention adopts real-time monitoring and fault tolerance mechanism:

[0150] The first processing module has a built-in delay monitoring module that counts the time consumed in communication frame processing in real time and generates an interruption alarm.

[0151] If the communication frame processing timeout (e.g. >1ms), dynamic priority adjustment is triggered or the communication frame is retransmitted through redundant links.

[0152] The above end-to-end delay control method ensures that the image data processing delay is stable within the threshold range by optimizing the transmission, processing and synchronization mechanisms in a layered manner and combining real-time monitoring and fault-tolerance strategies.

[0153] Therefore, compared with the existing technology that relies on a single bus (such as EMIF) and results in limited bandwidth (≤6.4Gbps), the present invention achieves a total bandwidth of 40Gbps through a dual-interface architecture of a second high-speed transmission channel + a first high-speed transmission channel, which increases the bandwidth by 6.25 times.

[0154] At the same time, compared with the traditional solution that lacks dynamic priority scheduling, the present invention improves resource utilization by ≥40% through task weight calculation and multi-core preemption mechanism.

[0155] The present invention achieves an end-to-end delay of ≤700μs (traditional solutions ≥1.5ms), meeting the real-time requirements of image processing.

[0156] Furthermore, the present invention achieves the localization of the intelligent collaborative computing platform. Key components, including the first, second, and third processing modules, are domestically manufactured, ensuring independent control. The first processing module is responsible for high-speed video signal acquisition, serial-to-parallel conversion, and preliminary data preprocessing. Using a high-speed transceiver, it converts the high-speed video signal into 16-bit parallel data. The preprocessed data is then packaged in a predefined frame format and transmitted via a high-speed interface on a second high-speed transmission channel.

[0157] The second processing module uses domestic chips and relies on its eight-core parallel computing architecture to perform real-time processing of image targets (including fixed / floating-point operations), receive and parse data transmitted by the first processing module through the second high-speed transmission channel interface, and feedback the processing results after completing target detection and calculation tasks.

[0158] The third processing module realizes image stitching, target recognition and data comprehensive processing, receives the data stream processed by the second processing module through the first high-speed transmission channel interface, performs video synthesis and outputs it through the HDMI interface.

[0159] The high-speed data transmission solution of the present invention is specifically embodied in:

[0160] (1) Implementation of the second high-speed transmission channel interface between the first processing module and the second processing module: The second high-speed transmission channel point-to-point transmission architecture is adopted, and the data frame contains an identification segment, a check code and a clock synchronization field; in terms of the synchronization mechanism, a dedicated synchronization signal is configured in the first processing module, and it works in conjunction with the second processing module to ensure that the transmission delay is controlled in the range of 50 to 150ns and the transmission rate reaches 20Gbps.

[0161] (2) The first high-speed transmission channel interface is implemented between the first processing module and the third processing module to construct a first high-speed transmission channel with large bandwidth, adopting a continuous data block transmission mode to achieve a 20Gbps transmission rate and a 100 to 200ns delay index; the interface protocol strictly follows the latest PCIe technical standards to ensure the transmission efficiency between modules.

[0162] The synchronous buffering and dynamic scheduling mechanism of the present invention is specifically embodied in:

[0163] Implement image data segmentation protocol to divide large-size images into fixed pixel block units with complete timing information; configure multi-level cache structure to realize data buffering function between modules, effectively avoiding data anomalies caused by processing rate differences.

[0164] The present invention can achieve the following key technical indicators:

[0165] Low latency: The data transmission delay between modules is less than 100ns;

[0166] High bandwidth performance: The first high-speed transmission channel interface transmission rate reaches 20 Gbps;

[0167] High localization rate: key components are localized, and the first processing module, second processing module, and third processing module respectively use domestic components, and the localization rate can reach more than 95%.

[0168] Compared with traditional single-processor architectures and shallow heterogeneous solutions, this technical solution has the following advantages:

[0169] Data transmission delay is significantly reduced, and real-time performance indicators are improved by more than 30%;

[0170] Achieve system-level collaborative optimization through modular division of labor, synchronous buffering, and dynamic scheduling mechanisms;

[0171] The layered high-speed interface architecture increases the system transmission bandwidth to 2.5 times that of traditional solutions;

[0172] The composite interface design (SRIO / PCIe / MIPI / LVDS) supports multi-modal expansion applications.

[0173] Example 1:

[0174] Example 1 is a basic high-speed transmission and collaborative processing solution.

[0175] Step 1: Video data acquisition and preprocessing process, specifically:

[0176] The physical layer conversion of high-speed video signals to 16-bit parallel data is achieved based on a high-speed transceiver.

[0177] The first processing module performs frame structure analysis, data decoding and channel selection operations, and encapsulates the data into a data frame that complies with the IEEE1355 standard.

[0178] Real-time interconnection with the second processing module is achieved through the integrated SRIO transmission interface, and the end-to-end transmission delay is controlled within 100 nanoseconds.

[0179] Step 2: Image target processing mechanism, specifically:

[0180] It adopts a domestically produced multi-core second processing module and relies on its eight-core parallel architecture to execute image feature extraction and target detection algorithms.

[0181] The SRIO transmission protocol stack has a built-in CRC check module, which cooperates with the timestamp synchronization mechanism to ensure data integrity and timing consistency. The processing results are transmitted back to the first processing module through a dedicated feedback channel.

[0182] Step 3: Image fusion and output system

[0183] The first processing module delivers the pre-processed data to the third processing module via the PCIe 2.0×4 bus.

[0184] The third processing module performs multi-source image registration, adaptive stitching and target fusion processing, and finally outputs a 4K@60Hz video display signal through the HDMI 2.0 interface.

[0185] The system adopts a three-level cache architecture and a global clock tree synchronization solution, and cooperates with a dynamic frequency adjustment mechanism to coordinate the working status of each module to ensure that the power consumption of the entire board is stably maintained below the 30W threshold.

[0186] Example 2:

[0187] Example 2 is a multimodal interface expansion solution. Based on the architecture of Example 1, a MIPI / LVDS expansion configuration module is added to build a composite multi-source data acquisition system.

[0188] The first processing module integrates a multi-protocol interface PHY layer, supports 4×MIPI CSI-2 or 8-channel LVDS parallel access, and forms a complementary transmission mechanism with the main transmission channel.

[0189] The system achieves collaborative processing of heterogeneous data streams through a resource-aware task scheduling algorithm, significantly improving the system's topology adaptability and multi-scenario deployment capabilities.

[0190] Based on the core technology of high-speed data transmission and collaborative mechanisms, this invention constructs a low-latency, high-bandwidth collaborative data processing system among multiple modules within a domestically produced heterogeneous architecture, effectively resolving the bottlenecks of real-time response and autonomous controllability that exist in existing technologies. Through the innovative integration of a layered transmission architecture that integrates SRIO and PCIe interfaces, the collaborative application of dedicated synchronization buffer modules, and the systematic implementation of dynamic task scheduling strategies, the comprehensive performance indicators of the intelligent collaborative computing platform are improved while ensuring the system's low-power operation characteristics and high security protection level in edge computing environments.

[0191] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course, by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to enter the method described in each embodiment of the present invention.

[0192] Throughout this specification, references to "one embodiment," "another embodiment," "other embodiments," or "first through Xth embodiments" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, method steps, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0193] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0194] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0195] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A high-performance image processing task scheduling method, characterized in that: Applied to an intelligent collaborative computing platform; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively connected to the first processing module; the method includes the following steps: The first processing module divides a single frame image in the received image data into fixed pixel blocks, tags each communication frame and adds a checksum, and dynamically adjusts the communication frame granularity according to the real-time bandwidth and computing load, wherein the tag includes a timing tag, and the timing tag includes a timestamp. The fixed pixel blocks are processed into communication frames, and the communication frames include at least one fixed pixel block. The single frame image is gridded according to the fixed pixel block size; the real-time transmission bandwidth of each communication frame is monitored through the status flag bit, and the computing unit load is fed back through the memory occupancy rate corresponding to the second processing module and the third processing module respectively, and the communication frame size is dynamically adjusted; The first processing module uses a phase-locked loop to generate a global synchronous clock signal and performs protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, the processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, the processing start signal of the third processing module is triggered; The second processing module receives the communication frame sent by the first processing module, performs target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returns the communication frame after target recognition and marking to the first processing module; The first processing module sends the communication frame after target identification and marking processing to the third processing module; The communication frames with the same label are merged into a single frame image through the third processing module, and the set parameters of the target image are calculated to output a video stream marked with the target image parameters.

2. The high-performance image processing task scheduling method according to claim 1, characterized in that: Before the first processing module divides a single frame image in the received image data into fixed pixel blocks, tags each communication frame and adds a checksum, and dynamically adjusts the communication frame granularity according to real-time bandwidth and computing load, the process further includes: Acquire high-speed video signal images captured by each image acquisition unit, convert the high-speed video signal images captured by each image acquisition unit into low-speed 16-bit parallel image data, and send the low-speed 16-bit parallel image data corresponding to each image acquisition unit to the first processing module through different data transmission channels, wherein each image acquisition unit is used to capture images at different monitoring angles; The step of labeling each communication frame comprises: The first processing module marks each communication frame with a channel identifier.

3. The high-performance image processing task scheduling method according to claim 2, characterized in that: The method further comprises: A dual-channel cache module is configured in the first processing module to perform zero-wait writing and reading of communication frames; A ring buffer module is constructed in the third processing module, and the ring buffer module is divided into multiple independent buffer units with different priorities according to task priority. Each buffer unit corresponds to an independent processing thread, so that the operating system can set the thread scheduling priority to implement a priority execution and response guarantee mechanism for high-priority tasks; In the first processing module, the second processing module and the third processing module, the data queues in the corresponding buffer modules are dynamically managed according to the communication frame timing tags.

4. The high-performance image processing task scheduling method according to claim 3, characterized in that: The method further comprises: The second processing module calculates priority weights based on the task type and the target state level of the external input, and optimizes the task allocation of each computing core in the second processing module through the multi-core resource preemption mechanism.

5. The high-performance image processing task scheduling method according to claim 4, characterized in that: The second processing module calculates priority weights based on task types and externally input target state levels, and optimizes task allocation for each computing core in the second processing module through a multi-core resource preemption mechanism, including: The second processing module presets a basic priority weight for each task type according to the set task type, obtains the real-time target state level corresponding to the task type input by the external sensor, and dynamically adjusts the real-time priority weight of each task type; The second processing module allocates the task types that reach the set priority weight to the dedicated computing core, and allocates the task types that do not reach the set priority weight to the shared computing core; When the existence of a high-priority task type is detected, a data queue update is triggered by a hardware interrupt signal to replace the currently executing task type with the high-priority task type.

6. The high-performance image processing task scheduling method according to claim 1, characterized in that: The method further comprises: Through the embedded performance counter, the task processing time of each communication frame is counted in real time according to the timing tag of each communication frame, and the task processing time of each communication frame is judged to be timed out. If so, communication frame retransmission or computing resource reallocation is triggered.

7. The high-performance image processing task scheduling method according to any one of claims 1 to 6, characterized in that: The step of performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds includes: Identifying the image background type of each fixed pixel block in the communication frame; For frame areas where the image background type is identified as sky, target region extraction, background suppression, target detection, and false target elimination are performed in sequence. Target detection includes downscaling the original image of each fixed pixel block in the communication frame and performing weak target detection or area target detection on images of each scale, depending on the target scale. For the frame area where the image background type is identified as the water surface, the target interest area is processed in sequence: smoothing filtering, ROI extraction and target enhancement, ROI segmentation and target confirmation. Among them, the confirmation of the target interest area includes: detecting the target waterline of the ship, searching for the target vertical edge in the set area above the target waterline, and determining the target interest area based on the target waterline and the target vertical edge.

8. The high-performance image processing task scheduling method according to claim 7, characterized in that: The step of performing small target detection or area target detection on images of different scales according to different target scales includes: When the target size is within the first size interval, the Robinson filter and morphological filter operators of the corresponding scale are used to perform background suppression filtering on the weak target; When the target size is within the second size interval, a weak target processing process is performed on the downscaled image; When the target size is in the third size interval, surface target segmentation and detection processing is performed on the downscaled image, and target merging is performed at the original scale, wherein the first size interval, the second size interval, and the third size interval increase in sequence.

9. An intelligent collaborative computing platform, characterized in that: The high-efficiency image processing task scheduling method according to any one of claims 1 to 8 is applied; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively connected to the first processing module; The first processing module is used to: divide a single frame image in the received image data into fixed pixel blocks, label each communication frame and add a check code, and dynamically adjust the communication frame granularity according to the real-time bandwidth and computing load, wherein the label includes a timing label, the timing label includes a timestamp, the fixed pixel block is processed into a communication frame, the communication frame includes at least one fixed pixel block, and the single frame image is grid-divided according to the fixed pixel block size; monitor the real-time transmission bandwidth of each communication frame through a status flag, feedback the computing unit load through the memory occupancy rate corresponding to the second processing module and the third processing module respectively, and dynamically adjust the communication frame size; use a phase-locked loop to generate a global synchronous clock signal, and perform protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, the processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, the processing start signal of the third processing module is triggered; The second processing module is configured to: receive the communication frame sent by the first processing module, perform target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and return the communication frame after target recognition and marking to the first processing module; The first processing module is further configured to: send the communication frame after target identification and marking processing to the third processing module; The third processing module is used to merge communication frames with the same label into a single frame image, and perform set parameter calculation on the target image to output a video stream marked with the target image parameters.

10. The intelligent collaborative computing platform according to claim 9, characterized in that: The first processing module is connected to the second processing module through a high-speed serial interconnect interface and a general input and output interface respectively; the first processing module is connected to the third processing module through a high-speed peripheral component interconnect interface and a general input and output interface respectively; the first processing module is connected to multiple high-speed transceivers, each of which is used to receive a video image captured by an external image acquisition unit; the first processing module is provided with a first cache module, the second processing module is provided with a second cache module, and the third processing module is provided with a third cache module; the third processing module is connected to a first PHY chip, and the first PHY chip is connected to the debugging interface, the second PHY chip and the third PHY chip respectively through a Gigabit Ethernet switching chip; the second PHY chip is connected to the first processing module, and the third PHY chip is connected to the second processing module; the third processing module is also connected to an HDMI distributor chip to support multi-channel high-definition video output.

Images