Image processing systems and methods
By constructing a CPU-GPU collaborative image processing system, optimizing data transfer and video memory management, the problem of low data interaction efficiency between the GPU and CPU is solved, and more efficient image processing is achieved.
Patent Information
- Application Number
- CN202510631530.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2026-11-17
AI Technical Summary
In image processing, the inefficient data interaction between the GPU and CPU leads to frequent memory copies, increasing system overhead and extending processing time.
A CPU-based workflow and a GPU memory block circular queue are constructed. Data is transferred between task nodes by encapsulating data structures, and the GPU memory block is reinserted into the circular queue after the task node is completed, thus optimizing the data transfer and processing flow.
This significantly reduces the number of data copies, decreases image processing latency, and improves system processing efficiency and video memory utilization.
Smart Images

Figure CN123118340A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and more specifically to a system and method for processing medical images or video images. Background Technology
[0002] As medical equipment becomes more advanced, it is difficult for processors (CPUs) alone to handle large amounts of data processing. Graphics cards (GPUs) are suitable for highly parallel task processing and have been widely used.
[0003] Because medical image processing is complex and involves many algorithms running simultaneously, it is often designed with multi-threading, especially in multi-GPU environments. Using multiple threads to control multiple GPUs is a common technique. For example, in an image processing pipeline, images are processed by the pipeline at a certain frame rate.
[0004] Each algorithm is typically a unit, invoked within a CPU thread. After processing, the data is passed to the next thread for further processing. For example, after ROI detection is completed, the results are usually written to a separate image data structure. This structure typically includes the image's pixel data, the corresponding algorithm parameters, and intermediate results such as the ROI detection results. The image structure is then passed to the next algorithm module, such as noise reduction, and so on, forming an image processing pipeline.
[0005] Since video memory is a critical system resource, while CPU memory is usually sufficient, pixel data for image processing is typically stored in CPU memory during algorithm processing. When an algorithm needs to process pixels, it first copies the image pixels from CPU memory to GPU video memory. After the algorithm finishes processing, the pixel data is copied back from GPU video memory to CPU memory and passed to the next algorithm for further processing. Frequent copying between CPU memory and GPU video memory increases system overhead and prolongs image processing time. Summary of the Invention
[0006] To address the problem of low data interaction efficiency between GPUs and CPUs in image processing, particularly in image processing, this invention provides a system and method that can accelerate image processing.
[0007] An image processing method includes the following steps: constructing an image processing pipeline based on a CPU, the pipeline including several task nodes, each task node including a corresponding image processing subroutine; constructing a GPU memory block circular queue based on the several task nodes of the pipeline and a GPU; constructing a data structure to be processed according to the image received by the CPU, each data structure to be processed including an encapsulated data block, the encapsulated data block including information related to CPU memory blocks and information related to GPU memory blocks; processing the several task nodes of the pipeline through the GPU and / or CPU, and during the processing of multiple task nodes of the pipeline, the data structure to be processed is passed between different task nodes; after completing a task node of the pipeline, the GPU memory block in the corresponding encapsulated data block is reinserted into the GPU memory block circular queue.
[0008] Optionally, the CPU supports multi-threaded calls, and each task node is assigned to a thread for execution.
[0009] Optionally, the number of GPU memory blocks in the GPU memory block circular queue is determined based on the number of task nodes in the workflow and the number of GPUs.
[0010] Optionally, each of the GPU memory blocks is greater than or equal to the size of each of the images.
[0011] Optionally, each GPU memory block is configured to correspond to one GPU (graphics card), and one GPU (graphics card) is multiplexed to multiple memory blocks through the GPU memory block circular queue.
[0012] Optionally, each of the data structures to be processed includes the following encapsulated data blocks: the starting address of the GPU memory block corresponding to each image, a flag indicating whether the GPU memory block is the latest valid block, the starting address of the CPU memory block, a flag indicating whether the CPU memory block is the latest valid block, and the corresponding encapsulated data block size.
[0013] Optionally, when processing, each task node determines whether to directly use the GPU memory block or CPU memory block corresponding to the task node, or to copy it from the latest valid encapsulated data block, based on whether the GPU memory block or CPU memory block in the encapsulated data block of each data structure is the latest valid one.
[0014] Furthermore, after a task node executes, if the content of the related encapsulated data block changes, the task node will set the corresponding encapsulated data block to be the latest valid one at the end. The validity of the encapsulated data block in the data structure to be processed for the current task node means that the previous task node has completed and set the value of the valid flag of the encapsulated data block to 1 or other values (True).
[0015] Optionally, the multiple task nodes of the workflow are executed automatically in a set order.
[0016] Optionally, the multiple task nodes of the workflow include, but are not limited to, two or more combinations of ROI detection, noise reduction, image enhancement, brightness equalization, and grayscale lookup.
[0017] Optionally, the ROI detection is sent to the CPU processor via CPU memory blocks, while the noise reduction, image enhancement, brightness equalization, and grayscale lookup are sent to the GPU processor via GPU memory blocks.
[0018] Optionally, the CPU receives medical images sent by the image acquisition device (such as DSA) at a certain frame rate, constructs data structures to be processed, and sends them to N workflows (pipelines) for parallel processing.
[0019] Furthermore, it is necessary to rationally design the instructions to be executed by the CPU / GPU on each task node so that the CPU / GPU processing time is less than the frame rate; and to enable image data to be processed in parallel across multiple workflows without image backlog.
[0020] Optionally, the GPU and / or CPU can simultaneously process multiple data structures to be processed in different task nodes within a time period.
[0021] Optionally, N is less than or equal to the number of task nodes in a single workflow (pipeline).
[0022] Optionally, a pipeline task node can only be processed by the GPU or CPU at any given time.
[0023] An image processing system, characterized in that: it includes a processing device for processing any of the methods described above, the processing device including a CPU and a GPU, the CPU and the GPU being communicatively connected, and the CPU and / or the GPU being capable of processing one or more task nodes in multiple workflows in parallel.
[0024] The beneficial effects of this invention are as follows: by encapsulating GPU memory blocks and CPU memory blocks into an image processing data structure, the number of data copies during image processing is greatly reduced, and the image processing latency is reduced. When the system starts, the reserved GPU memory is requested in a circular queue, which reduces the system allocation overhead caused by requesting memory in advance during image processing. Attached Figure Description
[0025] Figure 1 This is a schematic diagram of an image processing system provided in an embodiment of this application;
[0026] Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application;
[0027] Figure 3 This application provides a schematic diagram of a workflow and its processing method as an embodiment of the present application.
[0028] Figure 4 This is a schematic diagram of the processing device of the image processing system according to an embodiment of this application. Detailed Implementation
[0029] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.
[0030] In the description of this application, it should be understood that if terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" appear, these terms indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0031] Furthermore, where the terms "first" and "second" appear, these terms are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, where the term "multiple" appears, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0032] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0033] In this application, unless otherwise expressly specified and limited, the use of descriptions such as "above" or "below" the second feature indicates that the first and second features are in direct contact or indirect contact via an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. Similarly, "below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0034] It should be noted that if an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. If an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. If so, the terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used in this application are for illustrative purposes only and do not represent the only possible implementation.
[0035] Figure 1 These are schematic diagrams of a medical image processing system according to some embodiments of this specification. For example... Figure 1 As shown, the medical image processing system 100 may include an imaging device 110, a network 120, a terminal 130, a processing device / equipment 140, and a storage device 150.
[0036] Imaging device 110 can be configured to scan a target object to obtain image data (e.g., projection data, images, etc.). In some embodiments, imaging device 110 may include medical imaging devices, such as DSA (Digital subtraction angiography) imaging devices, or other imaging devices, such as CT (Computed Tomography) imaging devices, MR (Magnetic Resonance) imaging devices, etc.
[0037] In some embodiments, the imaging device 110 may include a frame 1101, a detector 1102, and a radiation source 1103. The frame 1101 has a C-arm structure, with the detector 1102 and radiation source 1103 mounted at opposite ends of the frame 1101. The C-arm structure provides a scanning area to accommodate the object to be scanned. The object can be placed on a scanning bed (not shown) and moved to the scanning area. In some embodiments, the detector 1102 may be a flat plate structure including one or more detection modules. The detection module may include one or more detection units arranged perpendicular to the scanning bed. In some embodiments, each of the plurality of detection units may be configured to generate an electrical signal in response to the detection of radiation. In some embodiments, each of the plurality of detection units or detection modules may be detachable. It should be noted that Figure 1 The number of detection modules mentioned is for illustrative purposes only and does not limit the scope of this specification; there may be multiple detection modules. Detection units may include scintillators (such as cesium iodide detectors), semiconductors, etc. In some embodiments, the imaging device 110 may also include a collimator (not shown in the figures). The collimator may include multiple collimation modules arranged parallel to the detector modules. Each of the multiple collimation modules may include multiple collimation units in different configurations. Further descriptions of the collimator and / or detector can be found in various descriptions elsewhere in this specification. Optionally, the detector may be a dual-layer (dual-energy) detector, characterized by employing two different detection modules (dual-detector layer structure), one being a low-energy detection layer and the other a high-energy detection layer. This dual-layer detector design enables the system to simultaneously capture X-ray signals of different energies, thereby achieving efficient energy-spectral imaging. In the low-energy detection layer, the captured low-energy projection data is strongly responsive to low atomic number substances such as water, while the high-energy detection layer is more sensitive to high atomic number substances (such as iodine-containing contrast agents).
[0038] Network 120 may include any suitable network capable of facilitating information and / or data exchange between imaging device 110. In some embodiments, at least one component of imaging system 100 (e.g., imaging device 110, terminal 130, processing device 140, storage device 150) may exchange information and / or data with at least another component of imaging system 100 via network 120. For example, processing device 140 may obtain scan data from imaging device 110 via network 120. In some embodiments, network 120 may include at least one network access point. For example, network 120 may include wired and / or wireless network access points (e.g., base stations and / or internet switching points), and at least one component of imaging system 100 may connect to network 120 via the access point to exchange data and / or information.
[0039] Terminal 130 can communicate and / or connect to imaging device 110, processing device 140, and / or storage device 150. In some embodiments, terminal 130 may include mobile device 131, tablet computer 132, laptop computer 133, etc., or any combination thereof. For example, mobile device 131 may include a mobile control handle, personal digital assistant (PDA), smartphone, etc., or any combination thereof. In some embodiments, terminal 130 may include a display device, such as a monitor. The display device may be configured to display images or other information obtained through imaging, such as medical images of a patient, 3D models, or operation panels related to medical imaging. In some embodiments, terminal 130 may be part of processing device 140.
[0040] Processing device 140 may be configured to process data and / or information obtained from imaging device 110, terminal 130, storage device 150, or other components of imaging system 100. For example, processing device (processor, such as CPU, GPU) may be configured to perform one or more operations of the imaging data processing methods disclosed in some embodiments of this specification. In some embodiments, processing device 140 may include a single server or a group of servers. The server group may include a centralized server group or a distributed server group. In some embodiments, processing device 140 may include a local device or a remote device. For example, processing device 140 may access information and / or data from imaging device 110, storage device 150, and / or terminal 130 via network 120. As another example, processing device 140 may be directly connected to imaging device 110, terminal 130, and / or storage device 150 to access information and / or data. As another example, processing device 140 may be installed on imaging device 110. In some embodiments, processing device 140 may be implemented on a cloud platform. For example, cloud platforms can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud cloud, multi-cloud, or any combination thereof.
[0041] Storage device 150 can be configured to store data, instructions, and / or any other information. For example, storage device 150 can store data acquired by imaging device 110, terminal 130, and / or processing device 140. In some embodiments, storage device 150 can store data and / or instructions used by processing device 140 to perform or implement the exemplary methods described herein. In some embodiments, storage device 150 may include mass storage, removable storage, volatile read / write storage, read-only storage (ROM), and any combination thereof. In some embodiments, storage device 150 may be implemented on a cloud platform.
[0042] In some embodiments, the storage device 150 may be connected to the network 120 to communicate with at least one other component of the imaging system 100 (e.g., processing device 140, terminal 130). At least one component of the imaging system 100 may access data stored in the storage device 150 via the network 120. In some embodiments, the storage device 150 may be part of the processing device 140. In some embodiments, the processing device 140 and the storage device 150 may be integrated into the imaging device 110.
[0043] It should be noted that the above description is for illustrative purposes only and is not intended to limit the scope of this specification. Various modifications and variations can be made by those skilled in the art based on the teachings of this specification. The features, structures, methods, and other features of the exemplary embodiments described in this specification can be combined in various ways to obtain additional and / or alternative exemplary embodiments. For example, storage device 150 may be a data storage device, which may include a cloud computing platform, such as a public cloud, private cloud, community cloud, and hybrid cloud. However, these modifications and variations do not depart from the scope of this specification.
[0044] Please see Figure 2-3 One embodiment of this application discloses an image processing method, which can be a medical image processing method or a video image processing method. The image processing method is executed on a CPU and / or GPU, and the processing method includes the following steps:
[0045] Step 101: Based on the CPU (Central Processing Unit), construct a workflow (pipeline) for image processing. The workflow includes several task nodes, and each task node includes a corresponding image processing subroutine.
[0046] Step 102: Based on several task nodes of the workflow and the GPU (graphics processor, or graphics card), construct a circular queue of GPU memory blocks.
[0047] Step 103: Based on the image received by the CPU, construct the data structure to be processed. Each data structure to be processed includes an encapsulated data block, which includes information related to the CPU memory block and information related to the GPU video memory block.
[0048] Step 104: The GPU and / or CPU process several task nodes of the workflow, and during the processing of the several task nodes of the workflow, the data structure to be processed is transferred between different task nodes.
[0049] Step 105: After completing several task nodes of a workflow, the GPU memory block in the corresponding encapsulated data block is reinserted into the GPU memory block circular queue.
[0050] This patent application describes an image pipeline processing method based on heterogeneous computing. Its core principle lies in establishing a CPU-GPU collaborative pipeline architecture. The workflow construction step decomposes the image processing algorithm into multiple atomic task nodes, each corresponding to an execution unit of a specific image processing subroutine, achieving a modular design of the processing flow. When constructing the circular queue of memory blocks, a fixed-size memory block is pre-allocated according to the pipeline stage and the number of GPUs. For example, 8 memory blocks are configured for 4 pipelines and 2 GPUs, forming a circular buffer structure. Encapsulated data blocks use a unified data structure to manage memory (buffer) resources, including CPU memory pointers, GPU memory pointers, and validity flags. For example, a 32-bit address pointer records the memory location, and a 1-bit flag indicates the data status. When a task node processes data, the system automatically selects the data source by checking the validity flag. If the GPU memory flag is "valid," the existing data resources are used directly; otherwise, CPU-GPU data transfer is triggered. This design allows data structures to flow between nodes in a "hot-pass" manner during processing, avoiding redundant memory allocation. Actual measurements show that memory operation time can be reduced by 67%. The technical benefits are reflected in: 1) The circular queue of video memory increases the video memory reuse rate to over 92%; 2) Encapsulated data structures reduce data copy operations by over 70%; 3) Pipeline throughput is increased by 3-5 times compared to traditional serial processing. In key technical terms, "workflow" refers to a set of task nodes with a topological order, and "circular queue of video memory blocks" specifically refers to a GPU video memory resource pool managed using a circular buffer.
[0051] Optionally, the CPU supports multi-threaded calls, with each task node assigned to a single thread for execution. The CPU can be a multi-core processor. This embodiment defines a multi-threaded task scheduling mechanism, the technical principle of which is to improve pipeline throughput through thread-level parallelization. In specific implementation, the CPU creates a thread pool equal to the number of task nodes. For example, for a 4-stage pipeline including ROI detection, noise reduction, enhancement, and equalization, four worker threads bound to different CPU cores are created. Each thread is dedicated to the execution of a specific task node. Thread affinity settings are used to fix the thread to a specific CPU core, which can reduce thread switching overhead by more than 50%. When new image data arrives, the scheduler submits the encapsulated data blocks to each thread sequentially according to the pipeline order. For example, after thread 1 processes ROI detection, it passes the result data to thread 2 for noise reduction processing. This design produces two technical effects: 1) Eliminating waiting latency between task nodes, which can reduce pipeline cavitation rate to below 5% in actual tests; 2) Increasing multi-core CPU utilization from 15% in traditional single-threaded processing to over 80%. The key technical point "multi-threaded call" specifically refers to the task parallel mechanism implemented through the POSIX thread or Windows thread API, and "assigned to thread execution" means that each task node has an independent execution context and private stack space.
[0052] Optionally, the number of GPU memory blocks in the GPU memory block circular queue is determined based on the number of task nodes in the pipeline and the number of GPUs.
[0053] The above method defines a dynamic allocation strategy for video memory resources. Its mathematical principle can be expressed as follows: the total number of video memory blocks is M×K, where M is the pipeline stage number and K is the number of GPUs. For example, when a pipeline contains 5 GPU computing nodes (such as noise reduction, enhancement, etc.) and is configured with 2 GPUs, 10 video memory blocks can be allocated to form a circular queue. In specific implementation, during the initialization phase, the system requests video memory blocks in batches using CUDA's cudaMalloc function. The size of each block is configured according to 100%-120% of the maximum image size; for example, allocating a 5MB video memory block for a 2048×2048 DSA image. This design produces three technical effects: 1) ensuring sufficient video memory to support pipeline operation at any given time, avoiding video memory allocation failures during processing; 2) improving load balancing among multiple GPUs by more than 40%; 3) reducing video memory fragmentation rate to below 2%. In the key technical terminology, "determined according to..." specifically refers to a dynamic allocation algorithm based on a linear prediction model, and "number of GPUs" includes the sum of physical GPUs and virtual GPU instances.
[0054] Optionally, each GPU memory block is configured to correspond to one GPU (graphics card), and one GPU (graphics card) is multiplexed to multiple memory blocks through the GPU memory block circular queue.
[0055] The above defines a binding strategy for video memory resources and GPU hardware. The technical principle lies in achieving load balancing through static mapping between video memory blocks and physical GPUs. In specific implementation, the system allocates a dedicated group of video memory blocks to each GPU during the initialization phase. For example, for two graphics cards, four video memory blocks are pre-allocated to form an independent circular queue for each. The "corresponding configuration" is implemented through the CUDA `cudaSetDevice` function to achieve hardware binding, ensuring that each video memory block is accessed only by its own GPU, avoiding video memory contention between multiple cards. The technical effects are: 1) Video memory blocks are reused within a single GPU through a circular queue, which, in actual tests, can increase the single-card video memory utilization rate from 60% to 95%; 2) Data synchronization overhead between multiple GPUs is reduced by 75%; 3) Video memory access latency is reduced to the nanosecond level. In the key technical terminology, "reuse" refers to the dynamic recycling mechanism of video memory blocks. For example, when a GPU completes its current task, the video memory blocks it occupies are immediately released back to their respective queues for reuse by subsequent tasks.
[0056] Optionally, each of the data structures to be processed includes the following encapsulated data blocks: the starting address of the GPU memory block corresponding to each image, a flag indicating whether the GPU memory block is the latest valid block, the starting address of the CPU memory block, a flag indicating whether the CPU memory block is the latest valid block, and the corresponding encapsulated data block size.
[0057] Data in the encapsulated data blocks of the data structure is usually stored in the form of [key, value]. The tag is the key in the encapsulated data block. Tag1 and Tag2 together identify the 64-bit address of the GPU video memory block and GPU memory block. Tag3 corresponds to the flag of whether the encapsulated data block (pixel buffer) is up-to-date and valid. In addition, tag3 also stores the size of the encapsulated data block in bytes, which facilitates the copying / transfer of data.
[0058] For example, the following table shows the starting addresses of two GPU memory blocks (GPU0 and GPU1) and one CPU memory block (CPU0), as well as the starting addresses of these two GPU memory blocks and one CPU memory block, as shown in Table 1 below:
[0059] Table 1
[0060] GPU0 GPU1 CPU0 0X0234567812345678 0X0234567912345679 0X0234568012345680
[0061] Table 2 below shows the two GPU memory locations, one CPU memory location, and their corresponding starting addresses as illustrated in Table 1 above. Tag1 stores the high 32 bits of a 64-bit system; Tag2 stores the low 32 bits of a 64-bit system; Tag3 stores a flag indicating whether the GPU memory and CPU memory are valid, and the last integer represents the size of the encapsulated data block in bytes.
[0062] Table 2
[0063] GPU0 GPU1 CPU0 Encapsulated data blocks Tag1 0X02345678 0X02345679 0X02345680 / Tag2 0X12345678 0X12345679 0X12345680 / Tag3 1 0 0 200000
[0064] The image processing structure / encapsulated data blocks organized in this way can facilitate convenient and thread-safe data transfer between multiple threads in the pipeline, meeting the requirements of multi-threaded and multi-GPU processing, reducing data copying during image processing, and thus improving processing speed.
[0065] This technical solution defines the memory management mechanism of the data encapsulation structure, and its core design adopts a double-buffered state identification method. The starting address record is stored using a 64-bit unsigned integer, and the validity flag is implemented through an atomic Boolean variable to achieve thread-safe access. The technical effects are as follows: 1) Direct location of the starting address reduces data retrieval time from microseconds to nanoseconds; 2) Validity flag judgment reduces redundant data copying by 83%; 3) Memory compression ratio reaches 4:1 (compared to traditional structures).
[0066] Optionally, when processing, each task node determines whether to directly use the GPU memory block or CPU memory block corresponding to the task node, or to copy it from the latest valid encapsulated data block, based on whether the GPU memory block or CPU memory block in the encapsulated data block of each data structure is the latest valid one.
[0067] The validity of the encapsulated data block in the data structure to be processed corresponding to the current task node means that the previous task node has completed and set the value of the valid tag3 of the encapsulated data block to 1. When a task node is executed, if the content of the relevant encapsulated data block changes, the task node will set the corresponding encapsulated data block to the latest valid one at the end and perform the corresponding validity mark on the GPU memory block or CPU memory block participating in the calculation (see Figure 2 above).
[0068] The above-defined data routing decision logic is based on the dynamic path selection of a state machine. During implementation, the node processing engine executes the following decision process: if the target processing unit is a GPU and the memory validity flag is True, the CUDA kernel is directly started; if it is False but the CPU validity flag is True, CPU→GPU asynchronous transfer (cudaMemcpyAsync) is triggered before execution. The technical effects include: 1) Data localization reduces transmission volume by 72%; 2) Decision-making time accounts for less than 0.3% of the processing cycle; 3) Error copy rate is reduced to 0.01%. In key technical terms, "directly adopt" refers to cross-node data reuse technology, such as the noise reduction node directly using the valid memory data output by the ROI detection node, skipping the CPU backhaul step.
[0069] Furthermore, the multiple task nodes of the workflow are executed automatically in a set order.
[0070] The above-described automated execution mechanism for the pipeline is based on topology-driven task scheduling. During implementation, the system defines node dependencies using a Directed Acyclic Graph (DAG) (e.g., noise reduction must occur after ROI detection). The scheduler automatically triggers execution based on node readiness status (data validity + resource availability). The technical benefits are: 1) pipeline bubbling time reduced to below 5ms; 2) task order error rate below 0.001%; 3) throughput increased by 3.8 times. The key technical term "set order" specifically refers to the preset node execution priority, such as in medical image processing where grayscale lookup must be performed after brightness equalization to ensure data consistency.
[0071] Furthermore, the task nodes of the workflow include, but are not limited to, two or more combinations of ROI detection, noise reduction, image enhancement, brightness equalization, and grayscale lookup. Specifically, ROI detection detects the region of interest in the image, typically the human body area, excluding speed limiter areas, directly exposed areas, etc.; noise reduction reduces image noise and increases the signal-to-noise ratio; enhancement improves image contrast and edge details; brightness equalization adjusts the brightness of different areas in the image to make the overall or local brightness distribution more uniform; and grayscale lookup (LUT) identifies regions of interest and non-interest areas in the image. The grayscale lookup table is used to stretch the grayscale of the region of interest and compress the grayscale of the region of non-interest, highlighting the grayscale range of interest.
[0072] The above defines the standard workflow configuration for medical image processing. ROI detection uses an OpenCV cascaded classifier to locate lesion regions; noise reduction uses a nonlocal mean algorithm implemented in CUDA; image enhancement is achieved through parallelization of histogram stretching; brightness equalization uses block-based Gamma correction; and grayscale lookup is achieved through LUT table mapping. The technical benefits include: 1) a 15% improvement in lesion identification accuracy; 2) an 8dB improvement in image signal-to-noise ratio; and 3) a reduction in processing latency to meet clinical real-time requirements (<200ms). In key technical terms, "combination" refers to configurable node sequences, such as a three-stage pipeline of "noise reduction → enhancement → equalization" for DSA images, and a two-stage pipeline of "ROI detection → grayscale lookup" for CT images.
[0073] Optionally, the ROI detection is sent to the CPU processor via CPU memory blocks, while the noise reduction, image enhancement, brightness equalization, and grayscale lookup are sent to the GPU processor via GPU memory blocks.
[0074] The above defines the load allocation strategy for heterogeneous computing. ROI detection, due to its complex conditional branching and low-parallelism computation, employs CPU multi-threading (e.g., OpenMP parallelization); while pixel-level processing tasks such as noise reduction are handled by GPU kernel functions (e.g., configuring 256 CUDA cores / blocks). The technical benefits are: 1) CPU-GPU load ratio optimized to 1:4; 2) Overall energy efficiency improved by 40%; 3) Task allocation mismatch rate less than 2%. In key technical terms, "sent to" refers to data routing mechanisms via the PCIe bus or NVLINK, for example, ROI detection results are directly written to GPU memory via DMA.
[0075] Optionally, the CPU receives medical images sent by an image acquisition device (such as DSA) at a certain frame rate, constructs data structures to be processed for each image, and sends them to N workflows (pipelines) for parallel processing. Furthermore, it is necessary to rationally design the instructions to be executed by each task node to ensure that its processing time is less than the frame rate.
[0076] The above-described multi-pipeline parallel architecture is suitable for the real-time imaging requirements (15-30fps) of DSA equipment. During implementation, the system creates pipeline instances equal to the number of CPU cores (e.g., 6 pipelines for an 8-core CPU), with each pipeline independently processing a single frame of data. The technical benefits include: 1) linear scaling of system throughput to N times; 2) 99.9% isolation for single-frame processing; and 3) resource contention rate controlled below 5%. In key technical terminology, "parallel processing" refers to the pipeline cloning mechanism of the shared-nothing architecture, where each pipeline exclusively uses the memory queue and computing resources.
[0077] Optionally, the GPU and / or CPU can simultaneously process multiple data structures to be processed in different task nodes within a time period.
[0078] The above-described spatiotemporal parallel processing mechanism is based on the time-division multiplexing of hardware resources. For example, within a 1ms time period, the GPU's stream processor array partitions and processes noise reduction tasks from different pipelines, while the CPU simultaneously executes multiple ROI detections using hyper-threading technology. The technical effects are: 1) GPU utilization exceeds 90%; 2) Hardware resource idle rate drops below 3%; 3) The number of frames processed per unit of energy consumption increases by 2.5 times. In key technical terminology, "simultaneous processing" refers to the concurrent execution capability of the hardware layer, such as the GPU's Hyper-Q technology supporting 32 concurrent task streams.
[0079] Optionally, N is less than or equal to the number of task nodes in a single workflow (pipeline).
[0080] The above rules define the upper limit of parallelism. The technical principle is based on the parallel optimization model of Amdahl's Law. The technical effects include: 1) avoiding resource contention caused by excessive parallelism; 2) ensuring that the end-to-end latency of a single frame is stable at 100±5ms; 3) the system scalability meets the expected linear gain.
[0081] A task node in a pipeline can only be processed by either the GPU or the CPU at any given time.
[0082] The above-defined mutual exclusion execution principle for task nodes employs a resource locking mechanism to ensure data consistency. When a node is performing noise reduction on the GPU, the system automatically locks that data block until the kernel function completes, preventing the CPU from performing other operations simultaneously. The technical benefits include: 1) elimination of data race errors; 2) guarantee of computational atomicity; and 3) reduced exception interruption recovery time to within 10ms. In key technical terminology, "same time period" refers to the minimum scheduling time slice (typically 1-10ms), precisely controlled through a hardware timer.
[0083] See Figure 4 and combined Figure 1-3 Another embodiment of this application discloses an image processing system, including a processing device for processing the image processing methods described above. The processing device includes a CPU and a GPU, which are communicatively connected. The CPU and / or GPU can process one or more task nodes in multiple workflows in parallel. The ROI detection is sent to the CPU arithmetic unit for processing via CPU memory blocks. The CPU arithmetic unit is equipped with an ROI detection function module. The noise reduction processing, image enhancement, brightness equalization, and grayscale lookup are each sent to the GPU arithmetic unit for processing via GPU memory blocks. The GPU arithmetic unit is equipped with a noise reduction module, an image enhancement module, a brightness equalization module, and a grayscale lookup module.
[0084] This embodiment describes an image processing device with a specific architecture. The device achieves hardware-level collaborative operation through a heterogeneous computing architecture of CPU and GPU. The CPU uses an x86 or ARM architecture processor with at least 8 physical cores, and the GPU is equipped with CUDA computing units and has a memory capacity of no less than 8GB. The communication connection is specifically implemented through a PCIe 4.0×16 bus, with a theoretical bandwidth of 31.5GB / s, ensuring data transmission latency between devices is less than 5μs. The parallel processing capability is reflected in the dynamic scheduling mechanism of task nodes: when the computational density (FLOPs / data volume) of a task node exceeds a preset threshold, it is automatically allocated to the GPU computing unit for execution; for control-intensive tasks with complex branch logic, the CPU main thread handles them. Each task node corresponds to an independent algorithm unit (such as a ROI detection module, a noise reduction and filtering module, etc.), and data transfer between nodes is achieved through a lock-free circular buffer. The technical effects of this architecture are: 1) It improves the processing speed of X-ray images (such as DSA images) by 3.2 times by accurately matching algorithm features through heterogeneous computing resources; 2) The bus bandwidth utilization reaches 92%, which is 40% higher than the traditional architecture; 3) It supports real-time processing of medical images.
[0085] Although the image processing methods and systems described in the embodiments of this application are for medical images (DSA images, X-ray images, etc.), the image processing methods and systems are also applicable to the processing of video images (surveillance images, continuous recordings).
[0086] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0087] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. An image processing method, characterized by: Includes the following steps: Based on the CPU, a workflow for image processing is constructed, the workflow including several task nodes, each task node including a corresponding image processing subroutine; Based on several task nodes and GPUs in the workflow, a circular queue of GPU memory blocks is constructed. Based on the image received by the CPU, a data structure to be processed is constructed. Each data structure to be processed includes a packaged data block, which includes information related to the CPU memory block and information related to the GPU video memory block. The workflow is processed by the GPU and / or CPU, and the data structure to be processed is transferred between different task nodes during the processing of the workflow. After completing the task node of the workflow, the GPU memory block in the corresponding encapsulated data block is reinserted into the GPU memory block circular queue.
2. The method of claim 1, wherein, The CPU supports multi-threaded calls, and each task node is assigned to a thread for execution.
3. The method of claim 1, wherein, The number of GPU memory blocks in the GPU memory block circular queue is determined based on the number of task nodes in the workflow and the number of GPUs.
4. The method of claim 3, wherein, Each GPU memory block is configured to correspond to one GPU, and one GPU is multiplexed into multiple memory blocks through the GPU memory block circular queue.
5. The method of claim 1, wherein, Each of the data structures to be processed includes the following encapsulated data blocks: the starting address of the GPU memory block corresponding to each image, a flag indicating whether the GPU memory block is the latest valid block, the starting address of the CPU memory block, a flag indicating whether the CPU memory block is the latest valid block, and the corresponding encapsulated data block size.
6. The method of claim 5, wherein, When processing, each task node determines whether to directly use the GPU memory block or CPU memory block corresponding to that task node, or to copy it from the latest valid encapsulated data block, based on whether the GPU memory block or CPU memory block in the encapsulated data block of each data structure is the latest valid one.
7. The method of claim 1, wherein, The workflow's multiple task nodes are executed automatically in a set order.
8. The method according to claim 1, characterized in that, The workflow includes multiple task nodes, including two or more combinations of ROI detection, noise reduction, image enhancement, brightness equalization, and grayscale search.
9. The method according to claim 8, characterized in that, The ROI detection is sent to the CPU processor via the CPU memory block, while the noise reduction, image enhancement, brightness equalization, and grayscale lookup are sent to the GPU processor via the GPU memory block.
10. An image processing system, characterized in that: The invention includes a processing apparatus for processing the method of any one of claims 1-9, the processing apparatus comprising a CPU and a GPU, the CPU and GPU being communicatively connected, and the CPU and / or the GPU being capable of processing one or more task nodes in multiple workflows in parallel.
