Ultrahigh-speed ultrasonic imaging system based on real-time resource collaborative scheduling of CPU and GPU

Abstract

The invention discloses an ultra-high-speed ultrasonic imaging system based on CPU and GPU real-time resource collaborative scheduling, the hardware of the system comprises an FPGA, a CPU and a GPU, the FPGA is combined with an ultrasonic receiving / transmitting circuit to form an ultrasonic plate, ultrasonic transmitting and receiving are completed, and data preprocessing is carried out; the CPU and the GPU cooperate to complete data processing, high-speed ultrasonic imaging is achieved, and mutual transmission of ultrasonic data is completed between the FPGA and the CPU and between the FPGA and the GPU through a PCIe high-speed data path. According to the invention, data processing is cooperatively completed by using the computing advantages of the CPU and the GPU, the computing operation is adjusted in real time through real-time resource cooperative scheduling of the CPU and the GPU, and ultrasonic imaging processing is rapidly completed, so that ultra-high-speed ultrasonic imaging is realized.

Description

TECHNICAL FIELD

[0001] The application relates to a super-high-speed ultrasonic imaging system based on real-time resource coordination scheduling of CPUs and GPUs, and belongs to the field of ultrasonic imaging. BACKGROUND

[0002] As a non-invasive and non-radiation medical imaging technology, ultrasonic imaging plays an important role in medical diagnosis and treatment. It uses the reflection and scattering characteristics of high-frequency sound waves in human tissues to generate real-time dynamic images of organs, blood vessels and soft tissues, and is widely used in the examination of heart, abdomen, obstetrics, blood vessels and musculoskeletal system. Ultrasonic imaging has the advantages of non-invasiveness, non-radiation, easy operation and low cost, and is especially suitable for emergency, surgical guidance and long-term monitoring.

[0003] In clinical practice, ultrasonic imaging is not only used for disease diagnosis, but also plays a key role in interventional therapy and intraoperative navigation. With the advancement of technology, such as three-dimensional ultrasonic imaging, elastic imaging and contrast-enhanced ultrasonic imaging, new modes further expand its clinical applications, making ultrasonic imaging an indispensable tool in modern medicine.

[0004] Super-high-speed ultrasonic imaging breaks through the time and space resolution limit of traditional ultrasonic technology by greatly improving the imaging frame rate, bringing revolutionary progress to clinical diagnosis and treatment and scientific research. At the same time, combined with advanced plane wave transmission and soft beam technology, super-high-speed imaging can acquire data of the entire field of view in a single scan, significantly improving blood flow sensitivity and micro-lesion detection rate, especially in evaluating early vascular lesions and microcirculation dysfunction.

[0005] However, super-high-speed ultrasonic imaging requires a large amount of data to be calculated in a short time, which requires higher computing power than traditional ultrasonic imaging systems. In addition to selecting higher-performance computing components, improvements need to be made in system architecture, resource coordination, algorithm optimization and other aspects. CPU and GPU are two common computing components, each with its own characteristics. Combining the advantages of both can achieve better results. SUMMARY

[0006] The purpose of the present application is to address the problem of how to organically combine CPU and GPU and coordinate resources in real time to complete super-high-speed ultrasonic imaging. The present application provides a super-high-speed ultrasonic imaging system based on real-time resource coordination scheduling of CPUs and GPUs.

[0007] To achieve the above object, the technical scheme adopted by the present application is: a super-high-speed ultrasonic imaging system based on real-time resource cooperative scheduling of CPU and GPU, the hardware composition of the system includes three components of FPGA, CPU and GPU, wherein the FPGA combines with the ultrasonic transmitting / receiving circuit to form an ultrasonic board, which completes the transmission and reception of ultrasonic waves and performs data preprocessing; the CPU and the GPU cooperatively complete data processing to realize high-speed ultrasonic imaging, and the FPGA and the CPU and the GPU complete mutual transmission of ultrasonic data through the PCIe high-speed data path.

[0008] Further, the CPU and the GPU of the system complete real-time resource cooperative scheduling, the scheduling method divides and distributes the data processing and computing tasks to different computing components, and real-time scheduling of computing resources is performed according to the execution situation to realize efficient data processing and higher ultrasonic imaging frame rate. Specifically, the whole process of ultrasonic data is divided into CPU-friendly steps and GPU-friendly steps according to the computing parallelism, corresponding flag bits are set, and the corresponding devices are pushed for execution when the steps are executed; for each data processing step, the computing amount of the step is budgeted, the data amount contained in a single computing thread is determined, and the thread amount pushed to the computing component in this phase is set; the computing resource threshold is set for the CPU and the GPU respectively, the CPU is based on the number of computing cores and the number of computations per core per unit time, and the GPU is based on the number of computations per unit time, the memory bandwidth and the register capacity; the computing flag bit and the count variable of the completed computing amount are set in the program, new computing threads are supplemented according to the number of completed computing threads, and the step is completed until all the computing is completed; if there is spare computing resource in the computing component and there is no new computing thread to be supplemented in this phase, the computing of the next unrelated step is started until the data processing is completed.

[0009] Further, the real-time resource cooperative scheduling process between the CPU and the GPU is as follows:

[0010] Step one, divide the whole ultrasonic data flow of ultrasonic imaging into several different computing steps;

[0011] Step two, type division is performed on each computing step;

[0012] Step three, estimate the data amount and the computing amount in each computing step to determine the data amount pushed to the computing component each time;

[0013] Step four, according to the number of computing cores of the CPU and the unit computing amount of each computing core, determine the computing threads and the computing data amount pushed to the CPU at the same time;

[0014] Step five, according to the number of computations per unit time, the memory bandwidth and the register capacity of the GPU, determine the computing threads and the computing data amount pushed to the GPU at the same time;

[0015] Step six, a statistical variable of completed calculation data and a calculation completion flag are set in the data processing program for each step, for identifying the calculation state;

[0016] Step seven, complete all ultrasound data processing according to the above steps, and finally obtain the ultrasound image.

[0017] Further, the type division in step two is specifically:

[0018] CPU-friendly step, refers to the calculation of the step cannot be parallel or the number of parallel threads is not more than 2 times the number of cores of the CPU used in the system;

[0019] GPU-friendly step, refers to the calculation of the step can be parallel, and the number of parallel threads is more than 2 times the number of cores of the CPU used in the system.

[0020] Further, in step three, when estimating the data volume, the data volume of the temporary variables generated in this step is also calculated. For example, if a m*n matrix and an n*p matrix are multiplied, the calculation volume is about m*p*(2n-1), and the storage space capacity occupied is about (m*n+n*p+m*p+4)*single data byte; if a m*n matrix and a m*n matrix are added, the calculation volume is about m*n, and the storage space capacity occupied is about m*n*3*single data byte.

[0021] Further, in step four, the unit time calculation volume estimation method of each calculation core of the CPU is:

[0022] FLOP=CLK*IPC

[0023] Where FLOP is the unit time calculation volume of a single calculation core, CLK is the running clock frequency of the CPU, and IPC is the calculation volume per cycle (inverse of clock frequency) of a single calculation core. IPC varies according to different CPU types, and the estimation basis is:

[0024] Scalar operation (no SIMD): 1 FLOP / cycle;

[0025] SSE / AVX vectorization:

[0026] SSE (128-bit): 4 FLOP / cycle (single precision) or 2 FLOP / cycle (double precision);

[0027] AVX / AVX2 (256-bit): 8 FLOP / cycle (single precision) or 4 FLOP / cycle (double precision);

[0028] AVX-512 (512-bit): 16 FLOP / cycle (single precision) or 8 FLOP / cycle (double precision)

[0029] In actual calculation, the number of calculation threads pushed to the CPU at the same time should not be greater than the number of its calculation cores, and the data volume allocated to a single calculation thread should be an integer multiple of the calculation volume corresponding to a single core of 1 microsecond.

[0030] Further, in step five, the maximum number of calculation threads that can be carried in the GPU is calculated based on the number of calculations per unit time as follows:

[0031]

[0032] where T is the maximum number of calculation threads based on the number of calculations, FLOP is the calculation volume per unit time of the GPU, and FLOP is the calculation volume of each thread. F T

[0033] The maximum number of calculation threads that can be carried in the GPU is calculated based on the memory bandwidth and register capacity as follows:

[0034]

[0035] where T is the maximum number of calculation threads based on the memory bandwidth and register capacity, Data is the total data volume corresponding to all registers in all stream calculators in the working state, Data is the data volume of shared memory corresponding to all data blocks in one stream calculator in the GPU, and BD is the memory bandwidth of the GPU. BD T B

[0036] In actual calculation, the number of calculation threads pushed to the GPU at the same time is based on the condition that 85% of the stream calculators in the GPU are in the working state.

[0037] Further, the specific steps of step six are as follows: a statistical variable of completed calculation data is set for each step in the data processing program; after each calculation thread ends, the variable is incremented by one, and new data to be calculated is pushed, or if there is no new calculation task in this step, the calculation data of the next non-associated step is pushed, so that the CPU or GPU is always in the maximum calculation effect range until all data is calculated; after all calculations are completed, the corresponding flag bit is set to "true", and this flag bit is effective for the situation where the next calculation step cannot start until this step is completed.

[0038] ​​​​​In the above description process, the computing thread refers to the thread at the application software level, and each stream calculator in GPU computing also manages multiple data computing threads, which are not the same meaning and belong to different levels. In addition, steps 1 to 3 are completed in advance, and the remaining steps are completed in real time during actual imaging.

[0039] The beneficial effects of the present application are: the present application utilizes the computing advantages of both CPU and GPU to cooperatively complete data processing, adjusts the computing operation in real time based on the real-time resource cooperative scheduling method of CPU and GPU, quickly completes ultrasonic imaging processing, and thus realizes ultrahigh-speed ultrasonic imaging. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a schematic diagram of an ultrahigh-speed ultrasonic imaging system architecture based on real-time resource cooperative scheduling of CPU and GPU;

[0041] Figure 2 is a flow chart of real-time resource cooperative scheduling of CPU and GPU;

[0042] Figure 3 is a comparison diagram of execution before real-time resource cooperative scheduling (a) and after real-time resource cooperative scheduling (b). DETAILED DESCRIPTION

[0043] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0044] This embodiment adopts one physical channel number of 64, a sampling rate of 50MHz, a sampling depth of 12bit, an FPGA of AMD Kintex7 type, a CPU of Inteli7-13700kf (16 computing cores, supporting AVX2 instruction set extension), a GPU of NVIDIA GTX3060 (4864 computing cores, memory bandwidth 360.0GB / s, each computing block (block) has 65536 available registers, and 49152 bytes of shared memory), and a PCIe3.0 X4 rate interface for data transmission in the system. The overall architecture is as shown in Figure 1 The FPGA controls the ultrasonic transceiver circuit to complete the emission and reception control of ultrasonic waves, and performs simple direct current filtering, time gain adjustment and other basic data preprocessing. Then the ultrasonic wave data is transmitted to the CPU and GPU heterogeneous data processing circuit through the PCIe high-speed data path.

[0045] The method for completing real-time resource cooperative scheduling between the CPU and the GPU in this embodiment specifically includes:

[0046] Step one, the entire ultrasound data flow of ultrasound imaging is divided into several different computing steps. In this embodiment, it is divided into: modulation, beam synthesis, singular value decomposition, energy spectrum calculation, image reconstruction, image rendering. Corresponding to Figure 3 Mid-red, grass green, yellow, light blue, orange, brown color block represented by color block. There are also component device latency represented by gray color block.

[0047] Step two, each computing step is classified by type, specifically: CPU-friendly step, which means the calculation of this step cannot be parallel or the number of parallel threads is no more than twice the number of CPU cores used in the system; GPU-friendly step, which means the calculation of this step can be parallel and the number of parallel threads is more than twice the number of CPU cores used in the system. In this embodiment, modulation, beam synthesis, image reconstruction, image rendering are CPU-friendly steps, and singular value decomposition, energy spectrum calculation are GPU-friendly steps.

[0048] Step three, the data volume and calculation volume of each computing step are estimated to determine the data volume pushed to the computing component each time. When estimating the data volume, the data volume of temporary variables generated in this step should also be calculated. For example, if a m*n matrix and an n*p matrix are multiplied, the calculation volume is approximately m*p*(2n-1), and the storage space capacity occupied is approximately (m*n+n*p+m*p+4)*single data byte number; if a m*n matrix and a m*n matrix are added, the calculation volume is approximately m*n, and the storage space capacity occupied is approximately m*n*3*single data byte number. In this embodiment, taking modulation as an example, the received ultrasound echo data of each channel has 1k (corresponding to an image field depth of 1.5 cm), and there are 64 channels. The corresponding calculation volume is 1k*1k*64=64M times, and the storage space occupied is 1k*2*64*2=256kB; taking singular value decomposition as an example, the received ultrasound echo data of each channel has 1k (corresponding to an image field depth of 1.5 cm), and there are 64 channels. The corresponding calculation volume is 1k*1k*64=64M times, and the storage space occupied is 1k*4*64*2=512kB;

[0049] Step four, the calculation volume per computing core of CPU per unit time is estimated as follows:

[0050] FLOP=CLK*IPC

[0051] Where FLOP is the calculation volume per computing core per unit time, CLK is the running clock frequency of CPU, and IPC is the calculation volume per cycle (inverse of clock frequency) of a single computing core. IPC varies depending on the type of CPU, and the estimation basis is as follows:

[0052] Scalar operations (no SIMD): 1 FLOP / cycle

[0053] SSE / AVX vectorization:

[0054] SSE (128-bit): 4 FLOP / cycle (single precision) or 2 FLOP / cycle (double precision).

[0055] AVX / AVX2 (256-bit): 8 FLOP / cycle (single precision) or 4 FLOP / cycle (double precision).

[0056] AVX-512 (512-bit): 16 FLOP / cycle (single precision) or 8 FLOP / cycle (double precision).

[0057] In actual calculation, the number of calculation threads pushed to the CPU at the same time should not be greater than the number of its calculation cores, and the data volume allocated to a single calculation thread should not be greater than the calculation volume corresponding to its single core 1 microsecond. In the embodiment, taking modulation as an example, the number of calculation threads pushed to the CPU at the same time should not exceed 16. The standard frequency of the CPU is 3.4 GHz, and the calculation volume after AVX2 vectorization is 27.2 G single precision calculations, and the calculation volume pushed each time should not exceed 27.2 k single precision calculations, and the corresponding data volume is 0.1 kB integer times

[0058] Step five, the maximum number of calculation threads that can be carried in the GPU is calculated based on the calculation times per unit time as:

[0059]

[0060] wherein, T F is the maximum number of calculation threads based on the calculation times, FLOP is the calculation volume of the GPU per unit time, and FLOP T is the calculation volume of each stream calculator.

[0061] The maximum number of calculation threads that can be carried in the GPU is calculated based on the memory bandwidth and the register capacity as:

[0062]

[0063] wherein, T BD is the maximum number of calculation threads based on the memory bandwidth and the register capacity, Data T is the total data volume corresponding to all registers in all stream calculators in the working state, and Data BBD is the data amount of shared memory corresponding to all data blocks in one stream processor in the GPU, and BD is the memory bandwidth of the GPU. The two values are different for each GPU, and the information can be obtained by consulting the user manual of the GPU.

[0064] In actual calculation, the number of calculation threads pushed to the GPU at the same time should be such that 85% of the stream processors in the GPU are in working condition. The standard calculation capacity of the GPU in this embodiment is 12.74 TFLOP, and the calculation capacity of each stream processor is 0.227 TFLOP. According to the calculation, the number of calculation threads is 12.74 / 0.227≈56. The total capacity of the registers included in each stream processor is 64 kB, and the capacity of the shared memory is 48 kB. Therefore, the number of calculation threads calculated according to the memory bandwidth and the register capacity is about 480. In summary, the number of calculation threads pushed to the GPU at the same time is 56*0.85≈48.

[0065] Step six, a statistical variable of completed calculation data is set for each step in the data processing program. The variable is increased by one after each calculation thread ends, and new to-be-calculated data is pushed, or the calculation data of the next non-associated step is pushed if there is no new calculation task in this step. This makes the CPU or GPU always in the maximum calculation effect range until all data are calculated. After the completion of this step, the corresponding flag bit is set to “true”. This flag bit is effective for the situation that the next calculation step can start only after the completion of this step.

[0066] Step seven, the above steps are completed to obtain the ultrasound image. In this embodiment, the imaging frame rate before real-time resource cooperative scheduling is about 100 fps, and the imaging frame rate after real-time resource cooperative scheduling is up to 300 fps, realizing ultrahigh-speed ultrasound imaging. Figure 3 As shown in the comparison diagram of the calculation before and after real-time resource cooperative scheduling, the real-time cooperative scheduling method reduces the waiting time of the components, thereby realizing higher calculation efficiency. In the diagram, the acceleration effect is not very large because the cores of the two components cannot be displayed. In the actual process, better acceleration effect can be achieved.

[0067] In the above description, the calculation thread refers to the thread at the application software level, and each stream processor in the GPU calculation also manages multiple data calculation threads. The two are not the same meaning and belong to different levels. In addition, steps one to three are completed in advance, and the remaining steps are completed in real time during actual imaging. Figure 2

[0068] ​The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the above examples do not limit the protection scope of the present application in any form, and any technical solutions obtained by equivalent replacement or the like fall within the protection scope of the present application. The parts not involved in the present application are the same as or can be realized by using the prior art.

Claims

1. A real-time resource cooperative scheduling based on CPU and GPU for ultra-high-speed ultrasonic imaging system, characterized in that, The hardware components of the system include FPGA, CPU and GPU, wherein the FPGA combines with the ultrasonic transmitting / receiving circuit to form an ultrasonic board, complete ultrasonic transmission and reception, and perform data preprocessing; the CPU and the GPU cooperatively complete data processing to realize high-speed ultrasonic imaging, and the FPGA and the CPU and the GPU complete mutual transmission of ultrasonic data through a PCIe high-speed data path.

2. The CPU and GPU real-time resource collaborative scheduling based ultrasonic imaging system of claim 1, wherein, The CPU and the GPU of the system complete real-time resource cooperative scheduling, and the specific process is as follows: Step one, divide the entire ultrasonic data flow of ultrasonic imaging into several different calculation steps; Step two, classify each calculation step; Step three, estimate the data volume and the calculation volume in each calculation step to determine the data volume pushed to the calculation component each time; Step four, according to the number of calculation cores of the CPU and the unit calculation volume of each calculation core, determine the calculation threads and the calculation data volume pushed to the CPU at the same time; Step five, according to the unit time calculation frequency, the memory bandwidth and the register capacity of the GPU, determine the calculation threads and the calculation data volume pushed to the GPU at the same time; Step six, in the data processing program, a statistical variable of completed calculation data and a calculation completion flag are set for each step to identify the calculation state; Step seven, complete all ultrasonic data processing according to the above steps, and finally obtain an ultrasonic image.

3. The CPU and GPU real-time resource collaborative scheduling based ultrasonic imaging system of claim 2, wherein, The classification in step two is as follows: CPU-friendly step, which means that the calculation of the step cannot be parallel or the number of parallel threads is not more than twice the number of cores of the used CPU; GPU-friendly step, which means that the number of parallel threads of the calculation of the step is more than twice the number of cores of the used CPU.

4. The CPU and GPU real-time resource collaborative scheduling based ultrasonic imaging system of claim 2, wherein, In step three, when estimating the data volume, the data volume of temporary variables generated in this step is also calculated.

5. The CPU and GPU real-time resource collaborative scheduling based ultrasonic imaging system of claim 2, wherein, In step four, the unit time calculation volume of each calculation core of the CPU is estimated as follows: FLOP=CLK*IPC Wherein, FLOP is the unit time calculation volume of a single calculation core, CLK is the running clock frequency of the CPU, and IPC is the calculation volume per cycle of a single calculation core; In actual calculation, the number of calculation threads pushed to the CPU at the same time should not be more than the number of its calculation cores, and the data volume of a single calculation thread should be an integer multiple of the calculation volume corresponding to a single core of 1 microsecond.

6. The CPU and GPU real-time resource collaborative scheduling based ultrasonic imaging system of claim 2, wherein, In step five, the maximum number of calculation threads that can be carried in the GPU is calculated based on the unit time calculation frequency as follows: wherein T F is the maximum number of computing threads based on the number of calculations, FLOP is the amount of computation per unit time of the GPU, and FLOP T is the amount of computation per thread. In the GPU, the maximum number of calculation threads that can be carried is calculated based on the memory bandwidth and the register capacity as follows: wherein T BD is the maximum number of computing threads based on the memory bandwidth and the register capacity, Data T is the total amount of data corresponding to all registers in all stream processors in the working state, Data B is the amount of data corresponding to the shared memory of all data blocks in one stream processor in the GPU, BD is the memory bandwidth of the GPU; In actual calculation, the number of calculation threads pushed to the GPU at the same time should be such that 85% of the stream processors in the GPU are in working state.

7. The CPU and GPU real-time resource collaborative scheduling based ultrasonic imaging system of claim 2, wherein, The specific step of the step six is: setting a statistical variable of completed calculation data for each step in the data processing program; adding one to the variable after each calculation thread ends and pushing new data to be calculated, or pushing the calculation data of the next non-associated step if there is no new calculation task in the current step, so that the CPU or GPU is always in the maximum calculation effect range until all data are completed; after completing all calculations, setting the corresponding flag bit to "true", which is effective for the situation that the next calculation step needs the current step to be completed before starting.

Images