High-efficiency image processing task scheduling method and intelligent cooperative computing platform

Abstract

The invention relates to the technical field of computing platforms for image processing, and discloses a high-efficiency image processing task scheduling method and an intelligent cooperative computing platform, a first processing module divides a single-frame image into fixed pixel blocks, carries out label marking, and dynamically adjusts the granularity of a communication frame; the first processing module generates a global synchronous clock signal and performs protocol-level time sequence alignment, and when the first processing module sends a communication frame to the second processing module, a processing starting signal of the second processing module is triggered; the second processing module performs target identification and marking on each fixed pixel block in the communication frame; the first processing module sends the processed communication frame to a third processing module; and combining the communication frames of the same time sequence label into a single-frame image through a third processing module so as to output a video stream with a target image parameter label. The technical problems that in the prior art, the real-time transmission efficiency of an edge computing platform is insufficient, and the overall resource utilization rate is low are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of computing platforms for image processing, and particularly to a high-performance image processing task scheduling method and an intelligent collaborative computing platform. Background Art

[0002] Cloud computing is a service model that provides on-demand computing resources (such as servers, storage, databases, networks, software, etc.) over the Internet. Users can remotely access a shared computing resource pool without local deployment of hardware or management of infrastructure.

[0003] Although cloud computing does not require excessive local deployment of computing resources, cloud computing needs to transmit data to a remote data center for processing, with high round-trip latency (usually over 100 ms), and cannot meet scenarios with millisecond-level responses (such as target recognition and tracking, autonomous driving, industrial control scenarios).

[0004] An edge computing platform is a distributed computing architecture that sinks computing, storage, and data processing capabilities from traditional clouds to the network edge (close to data sources or terminal devices).

[0005] In target recognition and tracking technology, the edge computing platform can perform local data processing to avoid cloud round-trip latency. However, in target recognition and tracking technology, the edge computing platform generally can only perform calculations on key data, and the latency level of the calculation results output by the edge computing platform significantly exceeds the real-time system threshold (above 200 to 500 ns), severely restricting the real-time transmission efficiency; at the same time, the computing resources of the edge computing platform are in an over-saturated or idle state, and the overall resource utilization rate is significantly reduced.

[0006] Therefore, it is necessary to propose a new high-performance image processing task scheduling method and an intelligent collaborative computing platform to solve the technical problems of insufficient real-time transmission efficiency and low overall resource utilization rate of the edge computing platform in the prior art. Summary of the Invention

[0007] The main purpose of the present invention is to provide a high-performance image processing task scheduling method and an intelligent collaborative computing platform, aiming to solve the technical problems of insufficient real-time transmission efficiency and low overall resource utilization rate of the edge computing platform in the prior art.

[0008] To achieve the above object, a high-performance image processing task scheduling method provided by the present invention is applied to an intelligent collaborative computing platform; the intelligent collaborative computing platform includes a first processing module, and a second processing module and a third processing module that are respectively communicatively connected to the first processing module; the method includes the following steps: The first processing module divides a single-frame image in the received image data into fixed pixel blocks, marks each communication frame with tags and adds a check code, and dynamically adjusts the communication frame granularity according to the real-time bandwidth and computing load. Among them, the tag marking includes a timing tag, and the timing tag includes a timestamp; The first processing module uses a phase-locked loop to generate a global synchronous clock signal and performs protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, it triggers the processing start signal of the second processing module, and when the first processing module sends a communication frame to the third processing module, it triggers the processing start signal of the third processing module; The second processing module receives the communication frame sent by the first processing module, performs target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returns the communication frame after the target recognition and marking process to the first processing module; The first processing module sends the communication frame after the target recognition and marking process to the third processing module; The third processing module merges the communication frames with the same tag into a single-frame image, and performs parameter calculation on the target image to output a video stream with target image parameter markings.

[0009] Optionally, before the step that the first processing module divides a single-frame image in the received image data into fixed pixel blocks, marks each communication frame with tags and adds a check code, and dynamically adjusts the communication frame granularity according to the real-time bandwidth and computing load, it further includes: Obtain the high-speed video signal images collected by each image acquisition unit, convert the high-speed video signal images collected by each image acquisition unit into low-speed 16-bit parallel image data respectively, and send the low-speed 16-bit parallel image data corresponding to each image acquisition unit to the first processing module through different data transmission channels. Among them, each image acquisition unit is used to collect images from different monitoring angles; The step of marking each communication frame with tags includes: The first processing module marks each communication frame with a channel identifier.

[0010] Optionally, the method further includes: Configure a dual-channel cache module in the first processing module for zero-wait writing and reading of communication frames; Build a circular buffer module in the third processing module, and divide the circular buffer module into multiple independent buffer units with different priorities according to the task priorities. Each buffer unit corresponds to an independent processing thread, so that the operating system can implement a priority execution and response guarantee mechanism for high-priority tasks by setting the thread scheduling priority; In the first processing module, the second processing module, and the third processing module, the data queues in the corresponding buffer modules are dynamically managed according to the communication frame timing tags respectively.

[0011] Optionally, the method further includes: The second processing module calculates the priority weights based on the task type and the target state level input externally, and optimizes the task allocation of each computing core in the second processing module through the multi-core resource preemption mechanism.

[0012] Optionally, the steps of the second processing module calculating the priority weights based on the task type and the target state level input externally, and optimizing the task allocation of each computing core in the second processing module through the multi-core resource preemption mechanism include: The second processing module presets the basic weights of the priority for each task type according to the set task type, and obtains the real-time target state level corresponding to the task type input by the external sensor, and dynamically adjusts the real-time weights of the priority of each task type; The second processing module allocates the task types that reach the set priority weights to the dedicated computing cores, and allocates the task types that do not reach the set priority weights to the shared computing cores; When it is detected that there is a high-priority task type, the data queue is updated by triggering a hardware interrupt signal to replace the currently executing task type with the high-priority task type.

[0013] Optionally, the method further includes: Through the embedded performance counter, according to the timing tags of each communication frame, the task processing time of each communication frame is statistically calculated in real time, and it is judged whether the task processing time of each communication frame times out; If so, trigger the retransmission of the communication frame or the reallocation of computing resources.

[0014] Optionally, the steps of performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds include: Identify the image background types of each fixed pixel block in the communication frame; For the frame area whose image background type is identified as the sky, perform target area extraction, background suppression, target detection, and false target elimination processing in sequence; among them, target detection includes: performing downscaling decomposition on the original images of each fixed pixel block in the communication frame, and performing small target detection or surface target detection processing on the images of each scale according to different target scales; For the frame region whose image background type is recognized as water surface, the target region of interest is successively subjected to smoothing filtering, ROI extraction and target enhancement processing, and ROI segmentation and target confirmation processing. Among them, the confirmation of the target region of interest includes: detecting the draft line of the ship target, finding the target vertical edge in the set region above the target draft line, and determining the target region of interest according to the target draft line and the target vertical edge.

[0015] Optionally, the step of performing small target detection or surface target detection processing on images of each scale according to different target scales includes: When the target size is in the first size interval, use the corresponding scale Robinson filter and morphological filter operator to perform background suppression filtering on weak targets; When the target size is in the second size interval, perform the small target processing flow on the downscaled image; When the target size is in the third size interval, perform surface target segmentation detection processing on the downscaled image and perform target merging on the original scale, where the first size interval, the second size interval, and the third size interval increase in sequence.

[0016] To achieve the above object, the present invention also proposes an intelligent collaborative computing platform that applies the high-performance image processing task scheduling method; the intelligent collaborative computing platform includes a first processing module, a second processing module, and a third processing module that are respectively communicatively connected to the first processing module; The first processing module is used for: dividing a single-frame image in the received image data into fixed pixel blocks, labeling each communication frame and adding a check code, and dynamically adjusting the communication frame granularity according to the real-time bandwidth and the computing load, where the label marking includes a timing label, and the timing label includes a timestamp; generating a global synchronous clock signal using a phase-locked loop and performing protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, it triggers the processing start signal of the second processing module, and when the first processing module sends a communication frame to the third processing module, it triggers the processing start signal of the third processing module; The second processing module is used for: receiving the communication frame sent by the first processing module, performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returning the communication frame after target recognition and marking processing to the first processing module; The first processing module is also used for: sending the communication frame after target recognition and marking processing to the third processing module; The third processing module is used for: merging the communication frames with the same label into a single-frame image, and performing set parameter calculation on the target image to output a video stream with target image parameter markings.

[0017] Optionally, the first processing module is respectively connected to the second processing module through a high-speed serial interconnect interface and a general-purpose input / output interface; the first processing module is respectively connected to the third processing module through a high-speed peripheral component interconnect interface and a general-purpose input / output interface; the first processing module is connected with a plurality of high-speed transceivers, and each high-speed transceiver is used for receiving video images collected by an external image acquisition unit; the first processing module is provided with a first buffer module, the second processing module is provided with a second buffer module, and the third processing module is provided with a third buffer module; the third processing module is connected with a first PHY chip, and the first PHY chip is respectively connected to a debugging interface, a second PHY chip and a third PHY chip through a gigabit Ethernet switch chip; the second PHY chip is connected to the first processing module, and the third PHY chip is connected to the second processing module; the third processing module is further connected with an HDMI splitter chip, supporting multi-channel high-definition video output.

[0018] In the technical solution of the present invention, a hybrid heterogeneous architecture is adopted to establish a modular collaborative computing system. Specifically, the first processing module is responsible for processing a single-frame image into a communication frame, adding tags and check codes to the communication frame, and performing dedicated synchronization control between modules; the second processing module performs target recognition and marking on each communication frame according to different image backgrounds according to the communication frame timing, and the first processing module is responsible for sending the image after target recognition and marking to the third processing module, and the third processing module performs image merging and calculates set parameters according to the target marking result, so as to output the processed image as a video stream with target image parameter markings. In the present invention, the first processing module adopts a communication frame transmission method, dynamically adjusts the real-time transmission ability according to the real-time bandwidth and computing load, and through the global synchronous clock signal and protocol-level timing alignment, the computing resources are fully utilized to avoid oversaturation or idle state, improving the overall resource utilization rate. Therefore, the present invention forms a pipeline-type collaborative processing architecture between heterogeneous computing units, divides the image processing process into three pipeline stages: the first processing module processes data communication frames, the second processing module performs parallel computing, and the third processing module performs data integration. Each stage is executed in parallel, and the end-to-end delay is the maximum delay of each stage, which is significantly lower than the traditional serial architecture. Therefore, the end-to-end delay is compressed through parallel execution between modules, ensuring that the end-to-end processing delay of image data is controllable. Description of the Drawings

[0019] Figure 1 It is a block diagram of the composition of the intelligent collaborative computing platform of the present invention; Figure 2 It is a flowchart of the first embodiment of an efficient image processing task scheduling method of the present invention; Figure 3 It is a flowchart of the sky weak target detection algorithm in the present invention; Figure 4It is the flow chart of the water surface ship target detection algorithm in the present invention; Figure 5 It is the flow chart of the draft line detection of the ship target in the present invention; Figure 6 It is the flow chart of the region of interest extraction of the ship target in the present invention.

[0020] The realization of the purpose, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0021] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] In the subsequent description, suffixes such as "unit", "component" or "module" used to represent elements are only for the convenience of explaining the present invention, and they have no specific meaning by themselves. Therefore, "unit", "component" or "module" can be used interchangeably.

[0023] Please refer to Figures 1 to 2 , in the first embodiment of the present invention, an efficient image processing task scheduling method is provided, which is applied to an intelligent collaborative computing platform; the intelligent collaborative computing platform includes a first processing module, a second processing module and a third processing module that are respectively communicatively connected to the first processing module; the method includes the following steps: Step S10, the first processing module divides a single-frame image in the received image data into fixed pixel blocks, marks each communication frame with labels and adds a check code, and dynamically adjusts the communication frame granularity according to the real-time bandwidth and computing load, wherein the label marking includes a timing label, and the timing label includes a timestamp; Step S20, the first processing module uses a phase-locked loop to generate a global synchronous clock signal and performs protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, it triggers the processing start signal of the second processing module, and when the first processing module sends a communication frame to the third processing module, it triggers the processing start signal of the third processing module; Step S30, the second processing module receives the communication frame sent by the first processing module, performs target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returns the communication frame after target recognition and marking processing to the first processing module; Step S40, the first processing module sends the communication frame after target recognition and marking processing to the third processing module; Step S50, the third processing module merges the communication frames with the same label into a single-frame image, and performs set parameter calculation on the target image to output a video stream with target image parameter markings.

[0024] In the technical solution of the present invention, a hybrid heterogeneous architecture is adopted, and a modular collaborative computing system is established. Specifically, the first processing module is responsible for processing a single-frame image into a communication frame, adding tags and check codes to the communication frame, and performing dedicated synchronization control between modules; the second processing module performs target recognition and marking on each communication frame according to different image backgrounds according to the communication frame timing. The first processing module is responsible for sending the image after target recognition and marking to the third processing module, and the third processing module performs image merging and calculates set parameters according to the target marking result. Thus, the processed image is output as a video stream with target image parameter markings. In the present invention, the first processing module adopts a communication frame transmission method, dynamically adjusts the real-time transmission ability according to the real-time bandwidth and computing load, and through the global synchronous clock signal and protocol-level timing alignment, the computing resources are fully utilized to avoid over-saturation or idle state, improving the overall resource utilization rate. Therefore, the present invention forms a pipeline-type collaborative processing architecture between heterogeneous computing units, divides the image processing process into three pipeline stages: the first processing module processes data communication frames, the second processing module performs parallel computing, and the third processing module performs data integration. Each stage is executed in parallel, and the end-to-end delay is the maximum delay of each stage, which is significantly lower than the traditional serial architecture. Thus, the end-to-end delay is compressed through parallel execution between modules, ensuring that the end-to-end processing delay of image data is controllable.

[0025] In the technical solution of the present invention, the images collected by the external image acquisition unit are sent to the first processing module for processing. Among them, the images collected by the external image acquisition unit can be various, for example, visible light images, infrared images, ultraviolet images, spectral images, or radar images.

[0026] Among them, the communication frame includes at least one of the fixed pixel blocks.

[0027] Specifically, in this embodiment, the size of the fixed pixel block for grid division of a single-frame image can be 256×256 pixels, and each communication frame is attached with a unique tag. Among them, in this embodiment, the tag includes a timing tag, and the timing tag includes: a frame number and a timestamp marked by the first processing module. The check code can be a CRC check code. The timestamp marked by the first processing module can be used to detect the transmission delay and can also know the image time when the user views the corresponding image later.

[0028] Furthermore, in this embodiment, the real-time transmission bandwidth of each communication frame is monitored through a status flag bit, and in this embodiment, the computing unit load can be fed back by the memory occupancy rates corresponding to the second processing module and the third processing module respectively, and the communication frame size is dynamically adjusted. When the backend cannot handle it, the communication frame size is reduced.

[0029] Aligning the communication frame boundary with the image coding format (such as H.264, YUV) can ensure the integrity of the communication frame data.

[0030] Target recognition and marking are performed in the second processing module, where the types of target objects for target recognition are at least one. In a specific application scenario, the image can be an aerial image, a water surface image, or a mixed image of an aerial image and a water surface image. Taking the aerial image as an example, the objects for target recognition can be various targets in the air, such as various flying objects.

[0031] Each fixed pixel block in the communication frame is subjected to target recognition and marking according to different image backgrounds, which can be to recognize multiple target objects in each fixed pixel block and mark the recognized results one by one.

[0032] Thus, in the third processing module, for each type of target object recognized from each fixed pixel block, position calculation, azimuth calculation, and speed calculation of the target object can be performed respectively.

[0033] Specifically, every time the first processing module obtains a single-frame image, processes it to obtain fixed pixel blocks, and then processes the fixed pixel blocks into a communication frame, it can send the corresponding communication frame to the second processing module without waiting, improving the real-time performance of task processing. Moreover, when the first processing module sends a communication frame to the second processing module, it can trigger the processing start signal of the second processing module. Thus, the second processing module can also use the idle computing cores to process the communication frame without waiting (perform target recognition and marking according to different image backgrounds). After processing, the processed communication frame is returned to the first processing module. The second processing module then sends the communication frame to the third processing module and triggers the processing start signal of the third processing module by sending the communication frame, enabling the third processing module to merge and set parameter calculations for the communication frames with the same label without waiting. The entire process does not require waiting, reducing latency and achieving high-performance image processing task scheduling.

[0034] Based on the first embodiment of the high-performance image processing task scheduling method of the present invention, in the second embodiment of the high-performance image processing task scheduling method of the present invention, before step S10, it further includes: Step S60, obtaining high-speed video signal images collected by each image acquisition unit, respectively converting the high-speed video signal images collected by each image acquisition unit into low-speed 16-bit parallel image data, and respectively sending the low-speed 16-bit parallel image data corresponding to each image acquisition unit to the first processing module through different data transmission channels, where each image acquisition unit is used to collect images from different monitoring angles; The step of label-marking each communication frame in step S10 includes: Step S11, the first processing module marks each communication frame with a channel identifier.

[0035] Among them, the high-speed video signal image can be a high-speed CML signal image. Specifically, multiple external image acquisition units are used to cover the full-angle (360°) image acquisition range, and each image acquisition unit acquires images within the field of view of a set angle range. For example, each image acquisition unit covers a field of view within a 90° range, and four image acquisition units are used for full-angle image acquisition.

[0036] The images acquired by each image acquisition unit are transmitted to the first processing module through one of the data transmission channels. In this embodiment, the images acquired by each image acquisition unit are sequentially transmitted to the first processing module through the uniquely corresponding video signal interface and high-speed transceiver.

[0037] Specifically, channel identifiers of the corresponding data transmission channels are embedded in each communication frame, which is beneficial to identifying the data transmission channel to which the communication frame belongs through the channel identifier in the communication frame, realizing the differentiation of each communication frame. When the third processing module merges communication frames, communication frames with the same timing tag and the same channel identifier can be merged into a single-frame image, reducing the calculation time of image merging and improving the merging efficiency; and the channel where the lost frame occurs can be located through the channel identifier of the communication frame.

[0038] Based on the second embodiment of the high-performance image processing task scheduling method of the present invention, in the third embodiment of the high-performance image processing task scheduling method of the present invention, the method further includes: Step S70, configure a dual-channel cache module in the first processing module to perform zero-wait writing and reading of communication frames; Step S80, construct a circular buffer module in the third processing module, and divide the circular buffer module into multiple independent buffer units with different priorities according to task priorities. Each buffer unit corresponds to an independent processing thread, so that the operating system can implement a priority execution and response guarantee mechanism for high-priority tasks by setting the thread scheduling priority; Step S90, in the first processing module, the second processing module, and the third processing module, respectively manage the data queues in the corresponding cache modules dynamically according to the communication frame timing tags.

[0039] Specifically, the first processing module, the second processing module, and the third processing module respectively set corresponding cache modules; the priority division method of the circular buffer module of the third processing module can be set as needed. For example, it can be divided into three independent buffer units of high, medium, and low according to task priorities. The communication frames that have been subjected to target recognition and marking processing and are sent from the first processing module to the third processing module are cached in the corresponding buffer units with different priorities according to the priorities. Thus, the operating system can preferentially schedule the data in the high-priority buffer units, enabling the third processing module to perform preferential processing.

[0040] Meanwhile, the third processing module can also calculate the data access frequency in the buffer units with low priority, and dynamically release the cache space corresponding to the image data with a data access frequency lower than a preset value to prevent data accumulation.

[0041] Meanwhile, in the first processing module, the second processing module, and the third processing module, the data queues in the corresponding cache modules are dynamically managed according to the communication frame timing tags respectively, which can enable strict continuous processing of images.

[0042] Based on the third embodiment of the high-performance image processing task scheduling method of the present invention, in the fourth embodiment of the high-performance image processing task scheduling method of the present invention, the method further includes: Step S100, the second processing module calculates the priority weights based on the task type and the externally input target state level, and optimizes the task allocation of each computing core in the second processing module through a multi-core resource preemption mechanism.

[0043] Specifically, the task type can be the object type of target recognition, the target state level can be the state level determined according to data such as the speed, distance, and orientation of the target object, and the target state level data can be collected by an external sensor.

[0044] Based on the fourth embodiment of the high-performance image processing task scheduling method of the present invention, in the fifth embodiment of the high-performance image processing task scheduling method of the present invention, the step S100 includes: Step S101, the second processing module presets the basic weights of each task type according to the set task type, and obtains the real-time target state level corresponding to the task type input by the external sensor, and dynamically adjusts the real-time weights of the priorities of each task type; Step S102, the second processing module allocates the task types that reach the set priority weights to dedicated computing cores, and allocates the task types that do not reach the set priority weights to shared computing cores; Step S103, when it is detected that there is a high-priority task type, trigger the data queue update through a hardware interrupt signal to replace the currently executing task type with the high-priority task type.

[0045] For example, the dedicated computing cores can be Core0 to Core5 of the second processing module, and the shared computing cores can be Core6 to Core7 of the second processing module.

[0046] Based on the first embodiment of the high-performance image processing task scheduling method of the present invention, in the sixth embodiment of the high-performance image processing task scheduling method of the present invention, the method further includes: Step S110: Use an embedded performance counter to statistically calculate the task processing time of each communication frame in real time according to the timing tag of each communication frame, and determine whether the task processing time of each communication frame times out. If yes, execute Step S120: Trigger communication frame retransmission or computing resource reallocation.

[0047] The corresponding timestamp can be obtained through the timing tag of each communication frame, so that the task processing time of the communication frame can be detected according to the timestamp of the communication frame and the current processing progress of the communication frame. If it times out, retransmit or reallocate the calculation cores of the second processing module, thereby effectively controlling the image processing delay.

[0048] Furthermore, the method further includes: Step S130: Construct a first high-speed transmission channel (e.g., PCIe channel) between the first processing module and the third processing module, and construct a second high-speed transmission channel (e.g., SRIO channel) between the first processing module and the second processing module.

[0049] Specifically, the first processing module is respectively connected to the second processing module through a Serial Rapid I / O (SRIO) interface and a General-Purpose Input / Output (GPIO) interface; the first processing module is respectively connected to the third processing module through a Peripheral Component Interconnect Express (PCIe) interface and a General-Purpose Input / Output (GPIO) interface.

[0050] Furthermore, in this embodiment, a phase-locked loop (PLL) of the first processing module is used to generate a global synchronization clock signal, and protocol-level timing alignment is achieved through the synchronization field (such as a 64-bit timestamp) in the SRIO frame header, ensuring that when the first processing module sends a communication frame, it triggers a processing start signal for the second processing module as the receiving end, so that the delay error ≤ 10 ns.

[0051] Please refer to Figures 3 to 6 , in the seventh embodiment of the high-performance image processing task scheduling method of the present invention, based on the first to sixth embodiments of the high-performance image processing task scheduling method of the present invention, the step of performing target recognition and marking on each fixed pixel block in the communication frame in Step S30 includes: Step S31: Identify the image background type of each fixed pixel block in the communication frame; Step S32: For the frame area whose image background type is identified as sky, perform target area extraction, background suppression, target detection, and false target elimination processing in sequence (please refer to Figure 3); wherein, the target detection includes: performing downscaling decomposition on the original images of each fixed pixel block in the communication frame, and according to different target scales, performing weak target detection or surface target detection processing on the images of each scale. Step S33, for the frame area whose image background type is recognized as water surface, sequentially perform smoothing filtering processing, ROI extraction and target enhancement processing, ROI segmentation and target confirmation processing on the target region of interest (please refer to Figure 4 ), wherein, the confirmation of the target region of interest includes: detecting the draft line of the ship target, searching for the target vertical edge in the set area above the draft line, and determining the target region of interest according to the draft line of the target and the target vertical edge.

[0052] Specifically, in the technical solution of the present invention, the image acquisition range of 360° is covered by each image acquisition unit, and within the 360° image acquisition range, the acquired images include the sky, the water surface, or the position where the sky meets the water.

[0053] After splitting the image into fixed pixel blocks and processing the fixed pixel blocks into communication frames, the background type of each fixed pixel block is recognized, and according to different image backgrounds, different target detection processing is performed on the fixed pixel blocks.

[0054] In the present invention, different target detection methods are adopted for sky targets according to different target scales. Further, for water surface targets, the ship target is mainly concerned, and the region of interest for target detection is determined by the position of the ship draft line and the target vertical edge to quickly locate the target and reduce the data processing volume.

[0055] For the position where the sky meets the water, the water surface targets and the sky targets are respectively segmented and recognized.

[0056] Specifically, in the high-performance image processing task scheduling method of the present invention, the second processing module can perform the sky target detection algorithm when the image background type is the sky, and execute the water surface ship target detection algorithm when the image background type is the water surface.

[0057] Among them, for the function of the sky target detection algorithm, the image sequence is input into the second processing module, and target region extraction, background suppression, target detection, and false target elimination are sequentially performed.

[0058] Further, for the problem of many false target interferences in complex backgrounds, a method based on multi-feature fusion decision is adopted, including: Extract the feature quantities of the suspected target in the feature spaces such as statistical features (including the mean, variance, contrast and their distributions of the target grayscale), structural features (including the width, height, area, contour line and perimeter of the target image, etc.), transform coefficient features (the target characteristics described by seven geometric invariant moments), and motion features (the spatial position, speed and acceleration of the target or the distance between targets, etc.); Determine the fusion decision condition according to the statistical feature parameters; Eliminate the wrong targets that do not meet the fusion decision condition.

[0059] In the scene model of the infrared image sequence of ship targets on the water-sky background for the water surface ship target detection algorithm, although most ships in the infrared image sequence are surface targets with an area larger than 5×5 pixels, from the perspective of the detector, the chimney part of the ship target is generally blocked by other parts within a large angular range. Generally speaking, the radiant temperature of the ship is much lower than the infrared radiant temperature of the aircraft target's tail flame; more importantly, the water surface background is much more complex than the sky background. Therefore, it is very difficult to directly detect the target by global threshold segmentation of the entire infrared image. Even if the target can be segmented, many wrong targets will be generated. Considering the position feature that the ship targets at a long distance are generally near the water-sky intersection line, and considering the visual attention of the visual system to the region of interest, and the requirements of the real-time processing system for the algorithm complexity and effectiveness, an infrared ship target detection algorithm process is proposed.

[0060] For the function of the water surface ship target detection algorithm, input the image sequence into the second processing module, and sequentially perform smoothing filtering, ROI extraction (Region of Interest Extraction) and target enhancement, ROI segmentation (Region of Interest Segmentation), and target confirmation processing.

[0061] The position where the ship target appears is near or below the water-sky intersection line. In the present invention, the characteristic region in the ship is selected: for example, the ship target has obvious horizontal edge features of the waterline and local high-brightness features. According to the above characteristics of the ship target, the guiding ideology for finding the region of interest of the target in the present invention is: first find the horizontal edge of the waterline of the ship target below the water-sky intersection line, and then jointly determine the region of interest according to the local high-brightness characteristic above the waterline.

[0062] Determine the waterline of the ship target in the following way: (1) There are obvious horizontal edge features within a fixed pixel block; (2) The area above the waterline is the high-gray region of the target, and the area below is the low-gray region of the water surface; (3) The waterline edge is located below the water-sky intersection line.

[0063] According to the above characteristics of the ship's target draft line, the ship's target draft line detection process is as follows Figure 5 shown in: The idea of extracting the ship's target region of interest is as follows: Search for the target vertical edge within a certain area above the detected target draft line (for example, the chimney part of the ship's target, or other preset vertical edges). If there is a vertical edge, the region of interest is determined jointly by the draft line and the vertical edge; if there is no vertical edge, the detected draft line is considered an incorrect target and is excluded. The algorithm implementation process is as follows Figure 6 shown in: By segmenting the image with enhanced targets in the region of interest, the targets can be effectively extracted and the target shape information can be obtained. After enhancement, the target has the highest gray level, occupies about 10% to 40% of the image area, the gray level difference between the sky background and the water surface background is not large, and it occupies the low-gray part of the region of interest with a uniform gray level distribution. Among the fast image segmentation algorithms suitable for real-time processing, the maximum inter-class variance method is adopted. Based on the measure criterion that the inter-class variance is the largest, the optimal threshold is obtained when the measure function takes the maximum value. The maximum inter-class variance method achieves the best performance when the proportion of the target and the background in the image is approximately the same. Through the extraction of the region of interest and the target enhancement processing, it is ensured that the characteristics of the target and the background in the region of interest to be segmented exactly meet this characteristic. Moreover, this algorithm is simple to calculate, only needs to calculate the zero-order and first-order cumulative moments of the gray level histogram, and is easy to implement in hardware, suitable for the application occasions of real-time image segmentation.

[0064] Based on the seventh embodiment of the high-performance image processing task scheduling method of the present invention, in the eighth embodiment of the high-performance image processing task scheduling method of the present invention, the step of performing small target detection or surface target detection processing on images of each scale according to different target scales in step S32 includes: Step S321, when the target size is in the first size interval, use the corresponding scale Robinson filter and morphological filter operator to perform background suppression filtering on weak targets; Step S322, when the target size is in the second size interval, perform the small target processing process on the downscaled image; Step S323, when the target size is in the third size interval, perform surface target segmentation detection processing on the downscaled image and perform target merging on the original scale, where the first size interval, the second size interval, and the third size interval increase in sequence.

[0065] Aiming at the problems of complex background weak target detection and many false target interferences encountered in sky target detection, the target detection part of the second processing module adopts a real-time weak target detection algorithm based on multi-scale filtering, which can effectively detect weak targets under complex low-altitude backgrounds.

[0066] Specifically, the core idea of the algorithm in the present invention is as follows: The original image is subjected to downscaling decomposition. According to different target scales, small target detection and surface target detection are respectively performed on the images of each scale. The small target detection is mainly carried out on the image of the original scale.

[0067] Specifically, the first size interval, the second size interval, and the third size interval can be set according to specific needs. For example, the first size interval can be a target with a size of 2×2 to 10×10 pixels (excluding 10×10 pixels) to effectively enhance the suppression of small targets against the background; the second size interval can be a target with a size of 10×10 to 20×20 pixels (excluding 20×20 pixels). The same small target processing process is carried out on the downscaled image to realize the modularization of the algorithm; while the third size interval can be a target with a size of more than 20×20 pixels. The surface target segmentation and detection are carried out on the downscaled image, and finally the target merging is carried out on the original scale.

[0068] To achieve the above object, the present invention also proposes an intelligent collaborative computing platform, which applies the high-performance image processing task scheduling method; the intelligent collaborative computing platform includes a first processing module, a second processing module, and a third processing module that are respectively communicatively connected to the first processing module; The first processing module is used for: dividing a single-frame image in the received image data into fixed pixel blocks, marking each communication frame with a label and adding a check code, and dynamically adjusting the communication frame granularity according to the real-time bandwidth and computing load, wherein the label marking includes a timing label, and the timing label includes a timestamp; generating a global synchronous clock signal by using a phase-locked loop and performing protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, it triggers the processing start signal of the second processing module, and when the first processing module sends a communication frame to the third processing module, it triggers the processing start signal of the third processing module; The second processing module is used for: receiving the communication frame sent by the first processing module, performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returning the communication frame after the target recognition and marking process to the first processing module; The first processing module is further used for: sending the communication frame after the target recognition and marking process to the third processing module; The third processing module is used for: merging the communication frames with the same label into a single-frame image, and performing set parameter calculation on the target image to output a video stream with target image parameter markings.

[0069] Optionally, the first processing module is connected to the second processing module through a Serial Rapid I / O (SRIO) interface and a General-Purpose Input / Output (GPIO) interface respectively; the first processing module is connected to the third processing module through a Peripheral Component Interconnect Express (PCIe) interface and a General-Purpose Input / Output (GPIO) interface respectively.

[0070] Optionally, the first processing module is connected with multiple high-speed transceivers, and each high-speed transceiver is used to receive video images collected by an external image acquisition unit; the first processing module is provided with a first buffer module, the second processing module is provided with a second buffer module, and the third processing module is provided with a third buffer module; the third processing module is connected with a first PHY chip (for converting RGMII into MDI signals), and the first PHY chip is respectively connected to a debugging interface, a second PHY chip, and a third PHY chip through a gigabit Ethernet switch chip; the second PHY chip is connected to the first processing module, and the third PHY chip is connected to the second processing module; the third processing module is further connected with an HDMI splitter chip, supporting multi-channel high-definition video output.

[0071] In a specific embodiment, the intelligent collaborative computing platform may be provided with 4 high-speed transceivers corresponding to 4 video images. Of course, the number of high-speed transceivers is not limited thereto.

[0072] Specifically, in the technical solution of the present invention, the pipeline-type collaborative processing architecture between heterogeneous computing units can be divided into the following pipeline stages: The first stage: The first processing module performs image data acquisition, communication frame encapsulation (such as a 1K image communication frame is 16 fixed pixel blocks), SRIO transmission (rate ≥ 20 Gbps), and PCIe transmission (rate ≥ 20 Gbps).

[0073] The second stage: Each computing core of the second processing module processes each fixed pixel block of the communication frame in parallel (for example, each core processes 1 fixed pixel block), executes the target detection algorithm, and the result is transmitted back to the first processing module through SRIO.

[0074] The third stage: The third processing module integrates the results of each fixed pixel block, performs image stitching and trajectory fusion, and outputs a 4K@60Hz video stream. The information marked in the video stream includes: target object type, coordinate information corresponding to the target object, speed information, etc.

[0075] The advantages of the above pipeline are: Each stage is executed in parallel, and the end-to-end delay is the maximum delay of each stage, which is significantly lower than the traditional serial architecture, and the actual total delay is even lower due to parallel processing.

[0076] Furthermore, the present invention realizes the safeguard measures for controllable end-to-end processing delay. Specifically: The present invention realizes hierarchical optimization design: Transport layer: The second high-speed transmission channel interface adopts a short frame transmission mode (frame length ≤ 256B), with a delay ≤ 150ns; the first high-speed transmission channel interface enables zero-copy DMA technology, with a delay ≤ 200ns.

[0077] Processing layer: The pipeline architecture reduces the idle waiting of modules, and the dynamic scheduling algorithm avoids resource contention (such as the inter-core load balancing error of the second processing module ≤ 5%).

[0078] The present invention adopts a real-time monitoring and fault tolerance mechanism: The first processing module is built-in with a delay monitoring module, which statistically calculates the communication frame processing time in real time and generates an interrupt alarm.

[0079] If the communication frame processing times out (such as > 1ms), trigger dynamic priority adjustment or retransmit the communication frame through the redundant link.

[0080] The above end-to-end delay control method ensures that the image data processing delay is stably within the threshold range by hierarchically optimizing the transmission, processing, and synchronization mechanisms, and combining real-time monitoring and fault tolerance strategies.

[0081] Therefore, compared with the prior art that relies on a single bus (such as EMIF) resulting in limited bandwidth (≤ 6.4Gbps), the present invention realizes a total bandwidth of 40Gbps through the dual-interface architecture of the second high-speed transmission channel + the first high-speed transmission channel, and the bandwidth is increased by 6.25 times.

[0082] At the same time, compared with the traditional scheme lacking dynamic priority scheduling, the present invention improves the resource utilization rate by ≥ 40% through task weight calculation and multi-core preemptive mechanism.

[0083] The present invention realizes an end-to-end delay ≤ 700μs (the traditional scheme ≥ 1.5ms), meeting the real-time requirements of image processing.

[0084] Furthermore, the present invention also realizes the localization of the intelligent collaborative computing platform. The key components, the first processing module, the second processing module, and the third processing module, adopt domestic devices to ensure autonomy and controllability. The first processing module undertakes the functions of high-speed video signal acquisition, serial-to-parallel conversion, and primary data preprocessing. Based on the high-speed transceiver, it realizes the conversion processing of the high-speed video signal to 16-bit parallel data, encapsulates the preprocessed data according to the predefined frame format, and implements data transmission through the high-speed interface of the second high-speed transmission channel.

[0085] The second processing module adopts domestic chips and relies on its eight-core parallel computing architecture to perform real-time processing of image targets (including fixed-point / floating-point operations), receive and parse the data transmitted by the first processing module through the second high-speed transmission channel interface, and feedback the processing results after completing the target detection and calculation tasks.

[0086] The third processing module implements image stitching, target recognition, and data comprehensive processing. It receives the processed data stream of the second processing module through the first high-speed transmission channel interface, performs video synthesis, and then outputs it through the HDMI interface.

[0087] The high-speed data transmission solution of the present invention is specifically embodied in: (1) Implementation of the second high-speed transmission channel interface between the first processing module and the second processing module: Adopting a point-to-point transmission architecture of the second high-speed transmission channel, the data frame includes an identification segment, a check code, and a clock synchronization field; in terms of the synchronization mechanism, a dedicated synchronization signal is configured in the first processing module to work in coordination with the second processing module to ensure that the transmission delay is controlled within the range of 50 to 150 ns, and the transmission rate reaches 20 Gbps.

[0088] (2) Implementation of the first high-speed transmission channel interface between the first processing module and the third processing module, constructing a large-bandwidth transmission channel of the first high-speed transmission channel, adopting a continuous data block transmission mode, achieving a transmission rate of 20 Gbps and a delay index of 100 to 200 ns; the interface protocol strictly follows the latest PCIe technical standard to ensure the transmission efficiency between modules.

[0089] The synchronization buffer and dynamic scheduling mechanism of the present invention is specifically embodied in: Implement an image data segmentation protocol to divide large-sized images into fixed pixel block units containing complete timing information; configure a multi-level high-speed cache structure to achieve the data buffering function between modules, effectively avoiding data anomalies caused by differences in processing rates.

[0090] The present invention can achieve the following key technical indicators: Low-latency characteristic: The data transmission delay between modules is less than 100 ns; High-bandwidth performance: The transmission rate of the first high-speed transmission channel interface reaches 20 Gbps; High localization rate: The key components are domesticated. The first processing module, the second processing module, and the third processing module respectively use domestic components, and the domesticization rate can reach more than 95%.

[0091] Compared with the traditional single-processor architecture and shallow heterogeneous solutions, the present technical solution has the following advantages: The data transmission delay is significantly reduced, and the real-time index is improved by more than 30%; System-level collaborative optimization is achieved through modular division of labor, synchronization buffer, and dynamic scheduling mechanism; The hierarchical high-speed interface architecture increases the system transmission bandwidth to 2.5 times that of the traditional solution; The composite interface design (SRIO / PCIe / MIPI / LVDS) supports multi-modal extended applications.

[0092] Example 1: Example 1 is a basic high-speed transmission and collaborative processing solution.

[0093] Step 1: Video data acquisition and preprocessing process, specifically: Based on a high-speed transceiver, physical layer conversion of high-speed video signals to 16-bit parallel data is achieved.

[0094] The first processing module performs frame structure parsing, data decoding, and channel gating operations, and encapsulates them into data frames compliant with the IEEE 1355 standard.

[0095] Through an integrated SRIO transmission interface, real-time interconnection with the second processing module is achieved, and the end-to-end transmission delay is controlled within 100 nanoseconds.

[0096] Step 2: Image target processing mechanism, specifically: A domestic multi-core second processing module is adopted, and image feature extraction and target detection algorithms are executed relying on its eight-core parallel architecture.

[0097] The SRIO transmission protocol stack incorporates a CRC check module, which, in conjunction with the timestamp synchronization mechanism, ensures data integrity and timing consistency. The processing results are transmitted back to the first processing module through a dedicated feedback channel.

[0098] Step 3: Image fusion and output system The first processing module delivers the preprocessed data to the third processing module through a PCIe 2.0×4 bus.

[0099] The third processing module performs multi-source image registration, adaptive stitching, and target fusion processing, and finally outputs a 4K@60Hz video display signal through an HDMI 2.0 interface.

[0100] The system adopts a three-level cache architecture and a global clock tree synchronization scheme, and coordinates the working states of each module with a dynamic frequency adjustment mechanism to ensure that the overall board power consumption is stably maintained below the 30W threshold.

[0101] Example 2: Example 2 is a multi-modal interface expansion solution. Based on the architecture of Example 1, an MIPI / LVDS expansion configuration module is added to build a composite multi-source data acquisition system.

[0102] The first processing module integrates the PHY layer of multiple protocols, supports 4×MIPI CSI-2 or 8-channel LVDS parallel access, and forms a complementary transmission mechanism with the main transmission channel.

[0103] The system realizes collaborative processing of heterogeneous data streams through a resource-aware task scheduling algorithm, significantly improving the system's topology adaptability and multi-scenario deployment capabilities.

[0104] Based on the core technical path of high-speed data transmission and collaboration mechanism, the present invention constructs a low-latency and high-bandwidth collaborative data processing system among multiple modules under the domestic heterogeneous architecture, effectively solving the bottleneck problems of real-time response and self-control existing in the prior art. Through the innovative integration of the hierarchical transmission architecture of SRIO and PCIe interfaces, the collaborative application of dedicated synchronization buffer modules, and the systematic implementation of dynamic task scheduling strategies, while improving the comprehensive performance indicators of the intelligent collaborative computing platform, it ensures the low-power operation characteristics and high security protection level of the system in the edge computing environment.

[0105] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium as described above (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions to enable a terminal device to enter the methods described in the various embodiments of the present invention.

[0106] In the description of this specification, the descriptions referring to terms such as "one embodiment", "another embodiment", "other embodiments", or "the first embodiment to the Xth embodiment" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, method steps, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0107] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or system including the element.

[0108] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0109] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present invention.

Claims

1. A high-performance image processing task scheduling method, characterized in that: Applied to an intelligent collaborative computing platform; the intelligent collaborative computing platform includes a first processing module and a second processing module and a third processing module respectively connected to the first processing module in communication; the method includes the following steps: The first processing module divides a single frame image in the received image data into fixed pixel blocks, tags each communication frame and adds a checksum, and dynamically adjusts the communication frame granularity according to the real-time bandwidth and computing load, wherein the tag includes a timing tag, and the timing tag includes a timestamp; The first processing module uses a phase-locked loop to generate a global synchronous clock signal and performs protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, the processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, the processing start signal of the third processing module is triggered; The second processing module receives the communication frame sent by the first processing module, performs target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and returns the communication frame after the target recognition and marking processing to the first processing module; The first processing module sends the communication frame after the target identification and marking processing to the third processing module; The communication frames with the same label are merged into a single frame image through the third processing module, and the set parameters of the target image are calculated to output a video stream marked with the target image parameters.

2. The high-performance image processing task scheduling method according to claim 1, characterized in that: Before the step of the first processing module dividing a single frame image in the received image data into fixed pixel blocks, labeling each communication frame and adding a checksum, and dynamically adjusting the communication frame granularity according to the real-time bandwidth and computing load, the step further includes: Acquire high-speed video signal images acquired by each image acquisition unit, and convert the high-speed video signal images acquired by each image acquisition unit into low-speed 16-bit parallel image data, and send the low-speed 16-bit parallel image data corresponding to each image acquisition unit to the first processing module through different data transmission channels, wherein each image acquisition unit is used to acquire images at different monitoring angles; The step of labeling each communication frame comprises: The first processing module marks each communication frame with a channel identifier.

3. The high-performance image processing task scheduling method according to claim 2, characterized in that: The method further comprises: A dual-channel cache module is configured in the first processing module to perform zero-wait writing and reading of communication frames; A ring buffer module is constructed in the third processing module, and the ring buffer module is divided into a plurality of independent buffer units with different priorities according to the task priority, each buffer unit corresponds to an independent processing thread, so that the operating system can realize the priority execution and response guarantee mechanism of high priority tasks by setting the thread scheduling priority; In the first processing module, the second processing module and the third processing module, the data queues in the corresponding cache modules are dynamically managed according to the communication frame timing tags.

4. The high-performance image processing task scheduling method according to claim 3, characterized in that: The method further comprises: The second processing module performs priority weight calculation based on the task type and the target state level of the external input, and optimizes the task allocation of each computing core in the second processing module through the multi-core resource preemption mechanism.

5. The high-performance image processing task scheduling method according to claim 4, characterized in that: The second processing module calculates the priority weight based on the task type and the target state level of the external input, and optimizes the task allocation of each computing core in the second processing module through the multi-core resource preemption mechanism, including: The second processing module presets a basic weight of the priority for each task type according to the set task type, obtains the real-time target state level corresponding to the task type input by the external sensor, and dynamically adjusts the real-time weight of the priority for each task type; The second processing module allocates the task types that reach the set priority weight to the dedicated computing core, and allocates the task types that do not reach the set priority weight to the shared computing core; When the existence of a high-priority task type is detected, a data queue update is triggered through a hardware interrupt signal to replace the currently executed task type with the high-priority task type.

6. The high-performance image processing task scheduling method according to claim 1, characterized in that: The method further comprises: Through the embedded performance counter, the task processing time of each communication frame is counted in real time according to the timing tag of each communication frame, and it is determined whether the task processing time of each communication frame has exceeded the time limit. If so, communication frame retransmission or computing resource reallocation is triggered.

7. The high-performance image processing task scheduling method according to any one of claims 1 to 6, characterized in that: The step of performing target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds includes: Identify the image background type of each fixed pixel block in the communication frame; For the frame area where the image background type is identified as the sky, target area extraction, background suppression, target detection and false target elimination are performed in sequence; wherein the target detection includes: downscaling the original image of each fixed pixel block in the communication frame, and performing weak target detection or surface target detection on the image of each scale according to the different target scales; For the frame area where the image background type is identified as the water surface, the target interest area is processed in sequence by smoothing filtering, ROI extraction and target enhancement, ROI segmentation and target confirmation. Among them, the confirmation of the target interest area includes: detecting the target waterline of the ship, searching for the target vertical edge in the set area above the target waterline, and determining the target interest area according to the target waterline and the target vertical edge.

8. The high-performance image processing task scheduling method according to claim 7, characterized in that: The step of performing small target detection or area target detection processing on images of different scales according to different target scales includes: When the target size is in the first size interval, the Robinson filter and morphological filter operator of the corresponding scale are used to perform background suppression filtering on the weak target; When the target size is within the second size interval, a weak target processing flow is performed on the downscaled image; When the target size is in the third size interval, surface target segmentation detection processing is performed on the downscaled image, and target merging is performed on the original scale, wherein the first size interval, the second size interval, and the third size interval increase in sequence.

9. An intelligent collaborative computing platform, characterized in that: The high-performance image processing task scheduling method according to any one of claims 1 to 8 is applied; the intelligent collaborative computing platform comprises a first processing module and a second processing module and a third processing module respectively connected to the first processing module in communication; The first processing module is used to: divide a single frame image in the received image data into fixed pixel blocks, label each communication frame and add a check code, and dynamically adjust the communication frame granularity according to the real-time bandwidth and computing load, wherein the label label includes a timing label, and the timing label includes a timestamp; use a phase-locked loop to generate a global synchronous clock signal, and perform protocol-level timing alignment, so that when the first processing module sends a communication frame to the second processing module, the processing start signal of the second processing module is triggered, and when the first processing module sends a communication frame to the third processing module, the processing start signal of the third processing module is triggered; The second processing module is used to: receive the communication frame sent by the first processing module, perform target recognition and marking on each fixed pixel block in the communication frame according to different image backgrounds, and return the communication frame after target recognition and marking to the first processing module; The first processing module is also used to: send the communication frame after the target identification and marking processing to the third processing module; The third processing module is used to merge the communication frames with the same label into a single frame image, and perform set parameter calculation on the target image to output a video stream marked with the target image parameters.

10. The intelligent collaborative computing platform according to claim 9, characterized in that: The first processing module is connected to the second processing module through a high-speed serial interconnect interface and a general input-output interface respectively; the first processing module is connected to the third processing module through a high-speed peripheral component interconnect interface and a general input-output interface respectively; the first processing module is connected to a plurality of high-speed transceivers, each of which is used to receive a video image acquired by an external image acquisition unit; the first processing module is provided with a first cache module, the second processing module is provided with a second cache module, and the third processing module is provided with a third cache module; the third processing module is connected to a first PHY chip, and the first PHY chip is respectively connected to the debugging interface, the second PHY chip and the third PHY chip through a Gigabit Ethernet switching chip; the second PHY chip is connected to the first processing module, and the third PHY chip is connected to the second processing module; the third processing module is also connected to an HDMI distributor chip to support multi-channel high-definition video output.

Images