Vascular ultrasound enhancement method and device, electronic equipment and storage medium

Abstract

The invention discloses a blood vessel ultrasonic enhancement method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring imaging data of subcutaneous blood vessels; performing reconstruction processing on the imaging data to obtain an imaging image; training data are obtained based on imaging image construction; performing iterative training on a pre-constructed network model by using the training data based on the binary label to obtain a target enhancement model; the network model comprises a coding structure and a decoding structure; and performing enhancement processing on the to-be-enhanced vascular ultrasound image by using the target enhancement model to obtain an ultrasound enhancement result. According to the method, photoacoustic imaging is utilized to make up for the defects of contrast and resolution of a clinical diasonograph in recognition of a blood vessel image, the processes of deep learning and manual annotation removal are combined, an algorithm model for enhancing the blood vessel ultrasonic image is trained with the photoacoustic image as a fact benchmark, efficient and accurate blood vessel ultrasonic enhancement is achieved, and the accuracy of blood vessel ultrasonic enhancement is improved. The method can be widely applied to the technical field of ultrasonic image processing.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image processing, and in particular to a method, device, electronic equipment and storage medium for blood vessel ultrasonic enhancement. Background Art

[0002] At present, the use of ultrasound-guided injection needles and other invasive devices to invasively intervene in the human body's subcutaneous superficial veins and arterial vessels is a common imaging method in preclinical diagnosis, examination, treatment and even surgery. This is the most direct and common method to visualize vascular tissue and invasive devices. The principle is to contact the surface of the skin with an ultrasound probe through a coupling agent, and use the piezoelectric transducer installed on the probe surface to generate an ultrasonic signal of a certain frequency. Different ultrasonic echo signals are obtained according to the differences in the acoustic characteristics of different vascular tissue components and injection needles, thereby obtaining a two-dimensional cross-section of the blood vessel containing the vascular structure and a real-time tomographic image of the injection needle cross-section, thereby realizing the identification of the blood vessel position and the guidance of the injection needle into the blood vessel. However, due to the low contrast of ultrasound images and the great influence of noise and various artifacts, it is still difficult to distinguish the tissue characteristics of blood vessels. In addition, the resolution of two-dimensional ultrasound (B-mode) is about 100μm to 1000μm, which is difficult to distinguish blood vessels with a diameter of less than 100μm.

[0003] Deep learning has emerged as a powerful tool for handling a variety of signal and image processing tasks, making it increasingly prevalent in the field of medical imaging. Convolutional neural network (CNN)-based architectures are widely used to effectively solve complex imaging challenges with robustness and efficiency. This has led to its application in enhancing the visualization of vascular structures in ultrasound images by leveraging the power of data. However, deep learning-based methods still require large datasets with accurate needle annotations for effective training, which is difficult to achieve in practice. Although simulations mimic known vascular structures, simulated data, which are often used as substitutes, may not translate well to real clinical situations. Images from in vitro or in vivo experiments can also be used as training datasets, while the true location of blood vessels remains elusive. Manual annotation by clinical experts provides an alternative, but may require manual annotation one by one and may introduce bias. In addition, deep learning-based methods still rely mainly on manual annotation on datasets, which is more dependent on experienced clinical ultrasound experts, time-consuming and labor-intensive, and has certain biases. Summary of the invention

[0004] The present invention proposes a method, device, electronic device and storage medium for vascular ultrasonic enhancement, aiming to solve one of the technical problems in the related art at least to a certain extent. The embodiments of the present invention can achieve vascular ultrasonic enhancement efficiently and accurately.

[0005] In one aspect, an embodiment of the present invention provides a method for blood vessel ultrasound enhancement, comprising:

[0006] Acquire imaging data of subcutaneous blood vessels; the imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data;

[0007] Reconstructing the imaging data to obtain an imaging image; the imaging image includes a data pair of a photoacoustic image and an ultrasound image registration; the imaging image is marked with binary labels of blood vessels and non-blood vessels based on the photoacoustic image and the ultrasound image;

[0008] Training data is obtained based on the imaging images;

[0009] Based on binary labels, the pre-built network model is iteratively trained using training data to obtain a target enhancement model; the network model includes an encoding structure and a decoding structure;

[0010] The target enhancement model is used to enhance the vascular ultrasound image to obtain the ultrasound enhancement result.

[0011] Optionally, acquiring imaging data of subcutaneous blood vessels comprises the following steps:

[0012] Using a preset multimodal imaging device to image and collect subcutaneous blood vessels at multiple locations of the target object, and obtaining photoacoustic radio frequency data and ultrasonic radio frequency data of the tomographic plane of the subcutaneous blood vessels at each location;

[0013] Among them, imaging acquisition is achieved by collecting subcutaneous photoacoustic radio frequency signals and subcutaneous ultrasound radio frequency signal data of a preset number of images in a preset imaging range; the photoacoustic tomography image is obtained based on the corresponding subcutaneous photoacoustic radio frequency data, and the ultrasound tomography image is obtained based on the corresponding subcutaneous ultrasound radio frequency data.

[0014] Optionally, reconstructing the imaging data to obtain an imaging image comprises the following steps:

[0015] Filtering the photoacoustic scanning lines in the photoacoustic radio frequency data, and then extracting the signal envelope through Hilbert transform to obtain the photoacoustic signal;

[0016] Reconstructing the photoacoustic signal into a blood vessel tomographic image according to the arrangement of the photoacoustic scanning lines;

[0017] Performing blood vessel filtering post-processing on the blood vessel tomographic image to extract a gold standard image with only blood vessel features; using the gold standard image as a photoacoustic image;

[0018] Performing time gain compensation on the ultrasonic scanning lines in the ultrasonic radio frequency data to obtain a time series signal;

[0019] A fixed frequency reference signal is multiplied with each captured time series signal to obtain a secondary modulation signal, and then a low-pass filter is applied and the signal envelope is extracted to remove high-frequency changes to obtain an ultrasonic signal;

[0020] Corresponding to the photoacoustic scanning line, performing ultrasound image mapping post-processing on the ultrasound signal to obtain an ultrasound image registered with the photoacoustic image;

[0021] Among them, binary labels are distinguished based on the vascular features of photoacoustic images.

[0022] Optionally, performing blood vessel filtering post-processing on the blood vessel tomographic image to extract a gold standard image with only blood vessel features includes the following steps:

[0023] The clutter filtering method based on singular value decomposition is used to filter the non-vascular signals in the vascular tomographic image, and then grayscale processing is performed to generate a grayscale image;

[0024] Binarization is performed on the grayscale image based on the threshold value determined by the adaptive method to obtain a blood vessel photoacoustic grayscale image;

[0025] Marking the connected areas of the vascular photoacoustic grayscale image to serve as the annotation image of the corresponding ultrasound image;

[0026] The annotated image is subjected to nonlinear median filtering to obtain a gold standard image with only vascular features.

[0027] Optionally, corresponding to the photoacoustic scanning line, performing ultrasound image mapping post-processing on the ultrasound signal to obtain an ultrasound image registered with the photoacoustic image comprises the following steps:

[0028] Based on the ultrasonic signal corresponding to each ultrasonic scanning line, logarithmic compression is performed through a preset compression ratio to obtain compressed data with a reduced brightness dynamic range;

[0029] Based on the compressed data, the sampling distance of the preset sampling multiple corresponding to the photoacoustic scanning line is spatially remapped to obtain a two-dimensional tomographic image registered with the photoacoustic image; and the two-dimensional tomographic image is used as the ultrasound image.

[0030] Optionally, constructing training data based on the imaging images comprises the following steps:

[0031] Performing offline data expansion on the photoacoustic image and the corresponding registered ultrasound image to obtain training data;

[0032] Among them, offline data expansion includes rotation, translation, noise addition and scaling transformation; training data includes training set, validation set and test set divided according to preset proportions.

[0033] Optionally, based on the binary labels, the pre-built network model is iteratively trained using the training data to obtain a target enhancement model, including the following steps:

[0034] An encoding path of an encoding structure and a decoding path of a decoding structure are constructed based on a multi-layer scale, and a network model is pre-constructed except that the convolution kernel of the scale of the last preset layer is replaced by a series of convolution kernels with different expansion rates;

[0035] Among them, in the encoding path: except for the convolution kernel of the last preset number of layers, which is replaced by a series of convolution kernels with different expansion rates, the convolution kernel of each layer of the scale is obtained based on the residual U-Net network setting of different depths, the number of input image channels and the number of pre-extracted feature channels are set before each layer of the scale, and a maximum pooling layer is set after each layer of the scale; in the decoding path: except for the convolution kernel of the last preset number of layers, which is replaced by a series of convolution kernels with different expansion rates, the convolution kernel of each layer of the scale is obtained based on the residual U-Net network setting, and a transposed convolution layer is set after each layer of the scale for upsampling; each layer of the scale in the encoding path and the decoding path performs feature encoding and context information capture based on the interactive cross attention module, and in the resolution maintenance deep supervision module, the residual U-Net is integrated into the deep network to maintain the resolution;

[0036] Input the training samples in the training data into the network model for classification processing to obtain the output result;

[0037] Based on the binary labels corresponding to the output results and training samples, the loss function is calculated using binary cross entropy;

[0038] The model weights of the network model are optimized and adjusted according to the result of the loss function, and the step of inputting the training samples in the training data into the network model for classification processing to obtain the output results is returned until the preset training conditions are met, and the target enhancement model is obtained according to the optimized and adjusted model weight settings.

[0039] On the other hand, an embodiment of the present invention provides a blood vessel ultrasound enhancement device, comprising:

[0040] A data acquisition module, used to obtain imaging data of subcutaneous blood vessels; the imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data;

[0041] A preprocessing module is used to reconstruct the imaging data to obtain an imaging image; the imaging image includes a data pair of photoacoustic image and ultrasound image registration; the imaging image is marked with binary labels of blood vessels and non-blood vessels based on the photoacoustic image and the ultrasound image;

[0042] A data construction module, used for constructing training data based on imaging images;

[0043] The model training module is used to iteratively train the pre-built network model based on binary labels using training data to obtain a target enhancement model; the network model includes an encoding structure and a decoding structure;

[0044] The enhancement module is used to perform enhancement processing on the vascular ultrasound image to be enhanced using the target enhancement model to obtain an ultrasound enhancement result.

[0045] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the above-mentioned blood vessel ultrasonic enhancement method.

[0046] On the other hand, an embodiment of the present invention provides a computer storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned blood vessel ultrasonic enhancement method.

[0047] The embodiment of the present invention obtains imaging data of subcutaneous blood vessels; the imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data; the imaging data is reconstructed to obtain an imaging image; the imaging image includes a data pair of photoacoustic image and ultrasonic image registration; the imaging image is labeled with binary labels of blood vessels and non-blood vessels and ultrasonic images based on the photoacoustic image; training data is constructed based on the imaging image; based on the binary labels, the pre-constructed network model is iteratively trained using the training data to obtain a target enhancement model; the network model includes a coding structure and a decoding structure; the target enhancement model is used to enhance the vascular ultrasonic image to be enhanced to obtain an ultrasonic enhancement result. The present invention uses photoacoustic imaging to make up for the lack of contrast and resolution in the recognition of vascular images of clinical ultrasonic diagnostic instruments, and combines deep learning to remove the process of manual annotation, and uses photoacoustic images as factual benchmarks to train an algorithm model for enhancing vascular ultrasonic images, thereby achieving efficient and accurate vascular ultrasonic enhancement. The technical solution of the present invention is applicable to medical and other fields, and is conducive to the promotion of interventional treatment scenarios of subcutaneous blood vessels under ultrasound guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0049] Figure 1 A schematic diagram of an implementation environment for performing vascular ultrasound enhancement provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a flow chart of a blood vessel ultrasound enhancement method provided by an embodiment of the present invention;

[0051] Figure 3 A schematic flow chart of an overall implementation scheme of a blood vessel ultrasonic enhancement method provided by an embodiment of the present invention;

[0052] Figure 4A schematic diagram of the process of preprocessing photoacoustic images in the vascular ultrasound intelligent enhancement method provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the process of preprocessing an ultrasound image in the vascular ultrasound intelligent enhancement method provided by an embodiment of the present invention;

[0054] Figure 6 A flowchart of an implementation example of the vascular ultrasound intelligent enhancement method provided by an embodiment of the present invention;

[0055] Figure 7 A schematic diagram of the architecture of the UIU-Net network model provided in an embodiment of the present invention;

[0056] Figure 8 A schematic diagram of an example of image comparison of a vascular ultrasound image, a corresponding photoacoustic ultrasound superimposed image, and a superimposed image after UIU-Net recognition involved in the vascular ultrasound intelligent enhancement process provided by an embodiment of the present invention;

[0057] Fig. 9 A schematic structural diagram of a blood vessel ultrasonic enhancement device provided by an embodiment of the present invention;

[0058] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0060] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0061] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present invention may be combined with other embodiments.

[0062] like Figure 1FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0063] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0064] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0065] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0066] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides a method for ultrasonic enhancement of a blood vessel. The following is explained by taking the application of the method for ultrasonic enhancement of a blood vessel in a terminal 102 as an example. It can be understood that the method for ultrasonic enhancement of a blood vessel can also be applied to a server 101.

[0067] Reference Figure 2 , Figure 2 The flowchart of the blood vessel ultrasonic enhancement method applied to the terminal provided by the embodiment of the present invention, the execution subject of the blood vessel ultrasonic enhancement method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 , the method comprises the following steps:

[0068] S100, acquiring imaging data of subcutaneous blood vessels;

[0069] The imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data;

[0070] It should be noted that, in some embodiments, step S100 may include the following steps: using a preset multimodal imaging device to perform imaging and acquisition of subcutaneous blood vessels in multiple parts of the target object, and obtain photoacoustic radio frequency data and ultrasonic radio frequency data of the tomographic plane of the subcutaneous blood vessels in each part; wherein the imaging acquisition is achieved by acquiring subcutaneous photoacoustic radio frequency signals and subcutaneous ultrasonic radio frequency signal data of a preset number of images in a preset imaging range; the photoacoustic tomographic image is obtained based on the corresponding subcutaneous photoacoustic radio frequency data, and the ultrasonic tomographic image is obtained based on the corresponding subcutaneous ultrasonic radio frequency data.

[0071] Exemplarily, in some specific embodiments, the data acquisition method includes: the original radio frequency data corresponding to the subcutaneous ultrasound and photoacoustic tomography images of the XZ plane (i.e., tomographic plane) of M pieces of N*O (unit mm2) imaging range of a specific part can be collected from a multimodal imaging device (such as a photoacoustic ultrasound microscopy dual-modal imaging device), where M≥200, N≥2.4, O≥8, and the setting of specific parameter values ​​can be adaptively adjusted according to actual application requirements.

[0072] S200, reconstructing the imaging data to obtain an imaging image;

[0073] The imaging image includes a data pair of photoacoustic image and ultrasound image registration; the imaging image is marked with binary labels of blood vessels and non-blood vessels based on the photoacoustic image and the ultrasound image;

[0074] It should be noted that, in some embodiments, step S200 may include the following steps: filtering the photoacoustic scan lines in the photoacoustic radio frequency data, and then extracting the signal envelope through Hilbert transform to obtain the photoacoustic signal; reconstructing the photoacoustic signal into a vascular tomographic image according to the arrangement of the photoacoustic scan lines; performing vascular filtering post-processing on the vascular tomographic image to extract a gold standard image with only vascular features; using the gold standard image as the photoacoustic image; performing time gain compensation on the ultrasonic scan lines in the ultrasonic radio frequency data to obtain a time series signal; multiplying each captured time series signal with a fixed frequency reference signal to obtain a secondary modulated signal, and then applying a low-pass filter and extracting the signal envelope to remove high-frequency changes to obtain an ultrasonic signal; corresponding to the photoacoustic scan lines, performing ultrasonic image mapping post-processing on the ultrasonic signal to obtain an ultrasonic image aligned with the photoacoustic image; wherein the binary label is based on the vascular feature distinction marker of the photoacoustic image.

[0075] For example, in some specific implementations, the data preprocessing method includes: according to the photoacoustic and ultrasonic division, each scanning line of the XZ plane obtained in step (1) is subjected to corresponding filtering processing. The photoacoustic part extracts the signal envelope, and the formula is:

[0076]

[0077] Where x′(t) represents the photoacoustic time series signal after signal envelope, It represents the impulse response of the system at time t.

[0078] Then the filtered signals are directly reconstructed into vascular tomographic images according to the scanning line arrangement, and the tomographic images are subjected to vascular filtering post-processing algorithm to extract the gold standard image with only obvious vascular features.

[0079] The ultrasonic part first adds time gain compensation to the scan line, the formula is:

[0080]

[0081] in represents the scan line time series signal after gain compensation, α represents the attenuation coefficient in dB / (MHz mm), f represents the transmission frequency in MHz, and d represents the total round-trip distance in mm. Then, a reference signal with a fixed frequency is multiplied with the scan line to obtain a modulated signal, and then a bandpass filter is applied to obtain a harmonic signal for tissue harmonic imaging. Then, the Hilbert transform is used to remove more high-frequency changes in the scan line. When mapped to the ultrasound image, the pixel value at the same position in the image matrix as the photoacoustic image matrix is ​​converted to the sampling amplitude of the ultrasound scan line according to twice the sampling distance of the photoacoustic scan line, and the scan line post-processing is remapped to an ultrasound image that is aligned with the photoacoustic image and suitable for visual perception and display using logarithmic compression and upsampling; wherein, the setting of the specific parameter values ​​can be adaptively adjusted according to the actual application requirements.

[0082] In some embodiments, the vascular tomographic image is subjected to vascular filtering post-processing to extract a gold standard image with only vascular features, which may include the following steps: filtering non-vascular signals in the vascular tomographic image based on a clutter filtering method of singular value decomposition, and then performing grayscale processing to generate a grayscale image; binarizing the grayscale image based on a threshold determined by an adaptive method to obtain a vascular photoacoustic grayscale image; marking the connected area of ​​the vascular photoacoustic grayscale image to serve as an annotated image of the corresponding ultrasound image; and adding cross-frame matching filtering and nonlinear median filtering to the annotated image to obtain a gold standard image with only vascular features.

[0083] For example, in some specific implementations, the detailed method of post-processing of photoacoustic image blood vessel filtering includes: first, adding a clutter filtering method based on singular value decomposition to the tomographic image to filter non-vascular signals, then performing grayscale processing, and mapping the original numerical matrix proportionally to the range of 0 to 1 to generate a grayscale image. Then, an adaptive method is used to determine the threshold for binarization processing to reduce the amount of calculation for feature extraction. The formula is:

[0084]

[0085] Where f(x) represents the binary vascular photoacoustic grayscale image, and T represents the threshold of binary division. Then the connected areas of the image are marked and serve as the annotation images of the corresponding ultrasound images. Then nonlinear median filtering is performed to replace the pixel values ​​of the annotation image with the median values ​​of the adjacent pixels to maintain the consistency of the edges of the vascular image. The formula is:

[0086] y(n)=median(x(nl),…,x(n),…,x(n+k)),

[0087] Where y represents the filtered image matrix, x represents the original image matrix, and finally a cross-frame matching filter is added to retain the vascular features.

[0088] In some embodiments, corresponding to the photoacoustic scanning line, the ultrasound signal is subjected to ultrasound image mapping post-processing to obtain an ultrasound image registered with the photoacoustic image, which may include the following steps: based on the ultrasound signal corresponding to each ultrasound scanning line, logarithmic compression is performed through a preset compression ratio to obtain compressed data with a reduced brightness dynamic range; based on the compressed data, spatially remapping the sampling distance of a preset sampling multiple of the photoacoustic scanning line to obtain a two-dimensional tomographic image registered with the photoacoustic image; and the two-dimensional tomographic image is used as the ultrasound image.

[0089] For example, in some specific implementations, the detailed method of ultrasound image mapping post-processing (the setting of specific parameter values ​​can be adaptively adjusted according to actual application requirements) includes: first setting the compression ratio to 3, performing logarithmic compression on the scan lines to reduce the dynamic range, and the formula is:

[0090] y=c*log(x),

[0091] Where y represents the compressed data and x represents the compression factor. Finally, the compressed scan lines are upsampled twice to improve the image resolution.

[0092] S300, constructing and obtaining training data based on the imaging image;

[0093] It should be noted that, in some embodiments, step S300 may include the following steps: performing offline data expansion on the photoacoustic image and the corresponding registered ultrasound image to obtain training data; wherein the offline data expansion includes rotation, translation, noise addition and scaling transformation; the training data includes a training set, a validation set and a test set divided according to a preset ratio.

[0094] For example, in some specific implementations, the method for constructing a data set includes: rotating, translating, adding noise, and scaling the preprocessed photoacoustic image and the corresponding registered ultrasound image to achieve offline data expansion, and finally dividing it into a training set and a validation set at a ratio of 14:1 (the test set is independent of the corresponding batch data, such as images obtained by collecting and preprocessing subcutaneous blood vessels in other parts). In the training process, after reading the data set, random transformation can also be performed to achieve online enhancement and reduce memory pressure.

[0095] S400, based on the binary label, using the training data to iteratively train the pre-built network model to obtain a target enhancement model;

[0096] Among them, the network model includes an encoding structure and a decoding structure;

[0097] It should be noted that, in some embodiments, step S400 may include the following steps: constructing an encoding path of an encoding structure and a decoding path of a decoding structure based on a multi-layer scale, pre-constructing a network model except that the convolution kernel of the last preset number of layers of scale is replaced by a series of convolution kernels with different expansion rates; wherein, in the encoding path: except that the convolution kernel of the last preset number of layers of scale is replaced by a series of convolution kernels with different expansion rates, the convolution kernel of each layer of scale is obtained based on a residual U-Net network setting of different depths, the number of input image channels and the number of pre-extracted feature channels are set before each layer of scale, and a maximum pooling layer is set after each layer of scale; in the decoding path: except that the convolution kernel of the last preset number of layers of scale is replaced by a series of convolution kernels with different expansion rates, the convolution kernel of each layer of scale is obtained based on a residual U-Net network setting of different depths. t network setting is obtained, and a transposed convolution layer is set after each layer of rulers for upsampling; each layer of rulers in the encoding path and the decoding path are respectively used for feature encoding and context information capture based on the interactive cross attention module, and in the resolution maintenance deep supervision module, the residual U-Net is integrated into the deep network to maintain the resolution; the training samples in the training data are input into the network model for classification processing to obtain the output result; based on the binary labels corresponding to the output results and the training samples, the loss function is calculated using the binary cross entropy; the model weights of the network model are optimized and adjusted according to the result of the loss function, and the step of inputting the training samples in the training data into the network model for classification processing to obtain the output result is returned to execute until the preset training conditions are met, and the target enhancement model is obtained according to the optimized and adjusted model weight settings. The preset training conditions include but are not limited to reaching the preset training rounds and the error of the loss function reaching the preset threshold.

[0098] Exemplarily, in some specific implementations, the method for constructing the UIU-Net network model includes: first constructing an encoding structure, where the convolution kernel of each layer of the encoding path is replaced by a small residual U-Net network to form multiple scales. The last two scales retain a series of convolutions with different expansion rates. The number of feature channels extracted by the first scale is 64, and the number of input image channels is 3. After each scale convolution, the maximum pooling layer size for downsampling is passed, and the step size of the maximum pooling layer is 2×2. Each downsampling step doubles the number of feature channels, and the encoding process is completed after 6 scales. Then construct a decoding structure, and the decoding path follows a similar structure to the encoding. Except for the last two scales, each scale contains a small residual U-Net network, followed by a transposed convolution layer with an upsampling factor of 2. After each upsampling, the interactive cross attention module encodes the high-level and low-level features from the encoder path, and restores the image size and number of channels layer by layer. Then construct the loss function of the network, and use binary cross entropy to measure the classification error. The formula is:

[0099]

[0100] in, It is a binary label 0 or 1, which represents the binary classification of blood vessels and non-blood vessels in the present invention. is the probability that the output belongs to the label, N represents the number of groups of objects predicted by the model, and Loss represents the gap between the predicted image and the actual image. The smaller the loss, the better the prediction result. A total of 7 loss values ​​are output, and the average value is taken as the final loss value. Finally, the training parameters are set (the specific parameter values ​​can be adaptively adjusted according to the actual application requirements), the training iteration is 20,000 rounds, the batch size is 32, and the model weight with the smallest average loss value is saved every 100 rounds until the iteration is completed and the weight saved at this time is used as the optimal weight.

[0101] S500 , using the target enhancement model to perform enhancement processing on the vascular ultrasound image to be enhanced, to obtain an ultrasound enhancement result.

[0102] Exemplarily, in some specific implementations, the trained UIU-Net network model can be used to input the collected vascular ultrasound images of any cross-sections such as XZ (fault plane) and YZ (vertical cross-section) that are not in the training set and the validation set into the network model to obtain enhanced vascular ultrasound images.

[0103] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0104] First of all, it should be noted that photoacoustic imaging is a new biomedical imaging technology that combines optical excitation and acoustic detection. It reconstructs photoacoustic images by detecting the ultrasonic signals generated after irradiation with a short-pulse laser beam. It has the high contrast and high resolution characteristics of optical imaging, thus making up for the shortcomings of insufficient image contrast in clinical ultrasound imaging. Hemoglobin in vascular blood has better optical absorption characteristics than human tissue. Using the photoacoustic images obtained from it as a supplement to ultrasound images can provide high-resolution and high-contrast image quality for blood vessels.

[0105] Existing algorithm-driven vascular ultrasound image enhancement methods, such as singular value decomposition clutter filtering, Doppler ultrasound blood flow imaging, and data-driven deep learning methods, can significantly enhance the visibility of blood vessels in ultrasound, but it is difficult to apply these technologies to real-time ultrasound scanning scenarios with inhomogeneous backgrounds.

[0106] In view of this, the present invention provides a vascular ultrasound intelligent enhancement method based on photoacoustic imaging, which uses photoacoustic imaging to make up for the lack of contrast and resolution of clinical ultrasound images in vascular identification, and combines deep learning to remove the process of manual annotation, and uses photoacoustic images to annotate vascular ultrasound images to train an algorithm model for enhancing vascular ultrasound images. The method realizes the identification and enhancement of blood vessels in real-time displayed vascular ultrasound images, and provides a new method for vascular visualization in clinical scenarios such as subcutaneous vascular injection.

[0107] In some specific implementations, a method for intelligent enhancement of vascular ultrasound based on photoacoustic imaging has the following basic steps:

[0108] Collect photoacoustic and ultrasound data of blood vessels in different locations;

[0109] Preprocess and enhance the acquired photoacoustic and ultrasound RF data;

[0110] Construct training set, validation set, and test set;

[0111] Construct and train the UIU-Net network model for vascular ultrasound image enhancement;

[0112] Input the ultrasound image of the test set into the trained UIU-Net network model and output the enhanced vascular ultrasound image.

[0113] like Figure 3 As shown, the present invention mainly provides a vascular ultrasound intelligent enhancement method based on photoacoustic imaging, and the method mainly includes five steps: data acquisition, data preprocessing, data set construction, model training, and test comparison and evaluation.

[0114] In some specific application scenarios, the step of collecting data may include: collecting original photoacoustic and ultrasonic radio frequency signals of m different fault planes from a photoacoustic microscopy multimodal imaging device under 532 wavelength laser excitation. The imaging target can be collected under different conditions such as different vascular locations and different body positions. At the same time, the data is classified according to the blood vessels in different locations.

[0115] In some specific application scenarios, such as Figure 4 As shown in the figure, the steps of photoacoustic data preprocessing include: first, filtering the original signal of each A-line scanning line of the photoacoustic data. Sending the received sound pressure signal to the bandpass filter to remove high-frequency and low-frequency signals outside the bandwidth range of the transducer center frequency. Then adding wavelet transform to remove Gaussian noise with a smaller transformation scale. Then extracting the signal envelope, and then adding the singular value decomposition algorithm to remove non-vascular signals such as background noise and high-frequency oscillation noise caused by random motors with higher amplitudes. This completes the preprocessing of the photoacoustic signal.

[0116] Among them, it should be noted that Figure 4 In , x represents the photoacoustic image obtained by directly projecting the photoacoustic signal of the original distribution; y represents the photoacoustic image obtained by migrating the photoacoustic signal of the original distribution after adding bandpass filtering, wavelet transform and signal envelope extraction; z represents the photoacoustic image obtained by adding singular value decomposition, cross-frame matching algorithm, thresholding and binarization to the photoacoustic signal obtained by preprocessing y and then migrating it to obtain the final photoacoustic image as the ultrasound label; f(x) represents the construction of a generator of bandpass filtering, wavelet transform and signal envelope extraction to complete the distribution migration from x to y; G(y) represents the construction of a generator of singular value decomposition, cross-frame matching algorithm, thresholding and binarization to complete the distribution migration from y to z. Specifically, Figure 4 The data image example shown in x→f(x)→y→G(y) corresponds to the photoacoustic data processing part of step S200 in the specific implementation method, that is, the photoacoustic data preprocessing, reconstruction processing and blood vessel filtering post-processing parts.

[0117] In some specific application scenarios, the photoacoustic signals received and preprocessed by multiple scanning lines can be directly projected into a two-dimensional tomographic image containing obvious vascular features. Then, an adaptive algorithm is used to determine the threshold, and each pixel point of the tomographic image is thresholded to obtain a tomographic image containing blood vessels of uniform intensity. Then, the image pixel values ​​are mapped proportionally to a binary grayscale image of 0 or 255, and then cross-frame matching and median filtering algorithms are added to retain vascular signals and sharpen the edges of vascular images, which are used as the gold standard for blood vessels in ultrasound images to establish a data set.

[0118] In some specific application scenarios, such as Figure 5As shown in the figure, the steps of ultrasound data preprocessing include: first, time gain compensation is performed on the original signal of each A-line scan line of ultrasound data. We assume that the attenuation of sound waves in tissues follows a certain physical equation, and then the received sound pressure signal is compensated and corrected by inverse solution. Then, a reference signal with a fixed frequency is multiplied with the scan line to obtain a modulated signal. In order to reduce the influence of noise outside the transmission frequency range, filtering is performed by specifying the required filter center frequency and percentage bandwidth. Then, the signal envelope is extracted by Hilbert transform, and the high-frequency changes of the scan line are removed during the demodulation process of envelope detection. Then, the compression ratio is set to 3 to perform logarithmic compression on the scan line to reduce the dynamic range to meet the grayscale range of display or visual perception. Finally, the scan line is upsampled by 2 times, and spatially remapped at 2 times the sampling distance of the corresponding photoacoustic scan line to provide a two-dimensional tomographic image registered with the photoacoustic.

[0119] Among them, it should be noted that Figure 5 In the first secondary demodulation, each captured modulated signal is multiplied by a fixed frequency reference signal, which is synchronized with the carrier frequency of the modulated signal. After this, a low-pass filter is applied and the envelope signal is obtained. Considering the wide dynamic range of amplitude inherent in US signals, logarithmic compression is used to reduce the dynamic range. Specifically, Figure 5 In , x represents the ultrasonic image obtained by directly projecting the ultrasonic signal of the original distribution; y represents the ultrasonic image obtained by migration after adding gain compensation and signal demodulation (involving multiplying each captured modulated signal with a reference signal of a fixed frequency, which is synchronized with the carrier frequency of the modulated signal) to the ultrasonic signal of the original distribution; z represents the ultrasonic signal obtained by preprocessing y by adding low-pass filtering and signal envelope extraction to perform mapping reconstruction, and the ultrasonic reconstruction image is migrated after adding logarithmic compression post-processing to obtain the final ultrasonic image; f(x) represents the construction of gain compensation and signal demodulation generator to complete the distribution migration from X to Y; G(y) represents the construction of high-pass filtering, signal envelope extraction and logarithmic compression generator to complete the distribution migration from Y to Z. Among them, Figure 5 The data image example shown in x→f(x)→y→G(y) corresponds to the ultrasound data processing part of step S200 in the specific implementation method, that is, the preprocessing, reconstruction processing and post-processing of the ultrasound data.

[0120] Further, such as Figure 6 and Figure 7As shown in the figure, the steps of constructing the UIU-Net network model include: first, constructing the encoding structure, the convolution kernel of each layer of the encoding path is replaced by a small residual U-Net network to form multiple scales. The last two scales retain a series of convolutions with different expansion rates. The number of feature channels extracted by the first scale is 64, and the number of input image channels is 3. After each scale convolution, the maximum pooling layer size for downsampling is passed. The step size of the maximum pooling layer is 2×2. Each downsampling step will double the number of feature channels. After 6 scales, the encoding process is completed. Then construct the decoding structure. The decoding path follows a similar structure to the encoding. Except for the last two scales, each scale contains a small residual U-Net network, followed by a transposed convolution layer with an upsampling factor of 2. After each upsampling, the interactive cross attention module encodes the high-level and low-level features from the encoder path and restores the image size and number of channels layer by layer. Then construct the loss function of the network, use binary cross entropy to measure the classification error, and the model outputs 7 loss values, and the average is taken as the final loss value. Finally, set the training parameters, train for 20,000 rounds, batch size 32, save the model weight with the minimum average loss value every 100 rounds until the iteration is completed and use the saved weight as the optimal weight.

[0121] Further, such as Figure 8 As shown, the present invention provides a specific implementation case of a vascular ultrasound intelligent enhancement method based on photoacoustic imaging, and the specific steps include: deploying a trained UIU-Net network model, inputting the collected vascular ultrasound images of any sections such as XZ, YZ that are not in the training set and the validation set into the network model, and obtaining the identified and segmented vascular ultrasound images in real time. And superimposing with the ultrasound image to obtain the enhanced image.

[0122] Here we show the final effect of the implementation of the present invention, the results are as follows Figure 4 , Figure 5 and Figure 8 As shown. Figure 4 and Figure 5 The figure shows the data preprocessing result diagram of the training set and the validation set provided by the embodiment of the present invention. Photoacoustic displays the label map, and ultrasound displays the two preprocessing modes of ultrasound images. Figure 8 is a schematic diagram of an original image, an original photoacoustic gold standard image, and a recognition enhanced image provided by an embodiment of the present invention, wherein Figure 8 The left side is the original image. Figure 8 The middle one is the gold standard result of photoacoustic imaging. Figure 8 On the right is the intelligent enhancement result.

[0123] In summary, the present invention provides a method for vascular ultrasound enhancement based on photoacoustic imaging, which belongs to the technical field of ultrasound image processing and medical information processing. First, the photoacoustic ultrasound bimodal images of subcutaneous blood vessels are collected, and the vascular areas of the ultrasound images are annotated with photoacoustic images for preprocessing and image enhancement; a data set of enhanced vascular ultrasound images is constructed, and the data set is divided into a training set, a validation set, and an original test set in proportion. Then, a UIU-Net convolutional neural network model is constructed; the training set is input into the UIU-Net convolutional neural network model to learn the inherent laws and characteristics of the image, and the model parameters are dynamically adjusted based on the validation set. The model weight with the least average loss value within the set number of iterations is retained; the best model weight obtained after training is loaded into the network model, and the vascular ultrasound images in the original test set images are tested to complete automatic enhancement and perform comparative evaluation. This method can more accurately and objectively identify and enhance the vascular structure in the actual ultrasound image by combining photoacoustic image annotation and deep learning technology, and can solve the technical problems of difficulty in annotating medical data sets and difficulty in identifying blood vessels.

[0124] On the other hand, Fig. 9 As shown, an embodiment of the present invention provides a blood vessel ultrasound enhancement device 900, comprising:

[0125] The data acquisition module 901 is used to obtain imaging data of subcutaneous blood vessels; the imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data;

[0126] The preprocessing module 902 is used to reconstruct the imaging data to obtain an imaging image; the imaging image includes a data pair of the photoacoustic image and the ultrasound image registration; the imaging image is marked with binary labels of blood vessels and non-blood vessels based on the photoacoustic image and the ultrasound image;

[0127] A data construction module 903 is used to construct training data based on the imaging images;

[0128] The model training module 904 is used to iteratively train the pre-built network model based on the binary label using the training data to obtain a target enhancement model; the network model includes an encoding structure and a decoding structure;

[0129] The enhancement module 905 is used to perform enhancement processing on the vascular ultrasound image to be enhanced using the target enhancement model to obtain an ultrasound enhancement result.

[0130] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0131] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above sensitive information method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0132] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0133] like Fig.10 As shown, Fig.10 A specific example of the hardware structure of an electronic device 1000 according to an embodiment is shown. The electronic device 1000 includes:

[0134] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0135] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0136] Input / output interface 1003, used to implement information input and output;

[0137] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0138] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0139] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0140] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0142] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0143] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0144] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0145] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0146] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0148] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0149] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented by the present invention. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0150] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of various functional modules in the device disclosed in the present invention, the actual implementation of the module will be understood within the conventional skills of engineers. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0151] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0152] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0153] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0154] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0155] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0156] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0157] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for blood vessel ultrasonic enhancement, characterized in that: The following steps are involved: Acquiring imaging data of subcutaneous blood vessels; the imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data; Reconstructing the imaging data to obtain an imaging image; the imaging image includes a data pair of a photoacoustic image and an ultrasound image registration; the imaging image is marked with binary labels of blood vessels and non-blood vessels based on the photoacoustic image and the ultrasound image; Constructing training data based on the imaging images; Based on the binary label, the pre-constructed network model is iteratively trained using the training data to obtain a target enhancement model; the network model includes an encoding structure and a decoding structure; The target enhancement model is used to perform enhancement processing on the blood vessel ultrasound image to be enhanced to obtain an ultrasound enhancement result.

2. The method for blood vessel ultrasonic enhancement according to claim 1, characterized in that: The step of obtaining imaging data of subcutaneous blood vessels comprises the following steps: Using a preset multimodal imaging device to perform imaging acquisition on subcutaneous blood vessels at multiple locations of the target object, and obtaining the photoacoustic radio frequency data and the ultrasonic radio frequency data of the tomographic plane of the subcutaneous blood vessels at each location; Among them, the imaging acquisition is achieved by collecting subcutaneous photoacoustic radio frequency signals and subcutaneous ultrasound radio frequency signal data of a preset number of images in a preset imaging range; the photoacoustic tomography image is obtained based on the subcutaneous photoacoustic radio frequency data, and the ultrasound tomography image is obtained based on the subcutaneous ultrasound radio frequency data.

3. The method for blood vessel ultrasonic enhancement according to claim 1, characterized in that: The reconstructing the imaging data to obtain an imaging image comprises the following steps: Performing filtering processing on the photoacoustic scanning lines in the photoacoustic radio frequency data, and then extracting the signal envelope by Hilbert transform to obtain the photoacoustic signal; reconstructing the photoacoustic signal into a blood vessel tomographic image according to the arrangement of the photoacoustic scanning lines; Performing blood vessel filtering post-processing on the blood vessel tomographic image to extract a gold standard image with only blood vessel features; and using the gold standard image as the photoacoustic image; Performing time gain compensation on the ultrasonic scan lines in the ultrasonic radio frequency data to obtain a time series signal; A fixed frequency reference signal is multiplied with each captured time series signal to obtain a secondary modulation signal, and then a low-pass filter is applied and the signal envelope is extracted to remove high-frequency changes to obtain an ultrasonic signal; Corresponding to the photoacoustic scanning line, performing ultrasound image mapping post-processing on the ultrasound signal to obtain the ultrasound image registered with the photoacoustic image; The binary tag is based on the blood vessel feature distinction mark of the photoacoustic image.

4. The method for blood vessel ultrasonic enhancement according to claim 3, characterized in that: The step of performing blood vessel filtering post-processing on the blood vessel tomographic image to extract a gold standard image having only blood vessel features comprises the following steps: filtering the non-vascular signals in the vascular tomographic image by a clutter filtering method based on singular value decomposition, and then performing grayscale processing to generate a grayscale image; Binarize the grayscale image based on a threshold value determined by an adaptive method to obtain a blood vessel photoacoustic grayscale image; Marking the connected areas of the blood vessel photoacoustic grayscale image to serve as a labeled image corresponding to the ultrasound image; Nonlinear median filtering is performed on the annotated image to obtain the gold standard image with only blood vessel features.

5. The method for blood vessel ultrasonic enhancement according to claim 3, characterized in that: The corresponding photoacoustic scanning line, performing ultrasound image mapping post-processing on the ultrasound signal to obtain the ultrasound image registered with the photoacoustic image, comprises the following steps: Based on the ultrasonic signal corresponding to each ultrasonic scanning line, logarithmic compression is performed using a preset compression ratio to obtain compressed data with a reduced brightness dynamic range; Based on the compressed data, a sampling distance corresponding to a preset sampling multiple of the photoacoustic scanning line is spatially remapped to obtain a two-dimensional tomographic image registered with the photoacoustic image; and the two-dimensional tomographic image is used as the ultrasound image.

6. The method for blood vessel ultrasonic enhancement according to claim 1, characterized in that: The step of constructing training data based on the imaging image comprises the following steps: Performing offline data expansion on the photoacoustic image and the corresponding registered ultrasound image to obtain the training data; The offline data expansion includes rotation, translation, noise addition and scaling transformation; the training data includes a training set, a validation set and a test set divided according to a preset ratio.

7. The method for blood vessel ultrasonic enhancement according to claim 1, characterized in that: The method of iteratively training the pre-built network model based on the binary label using the training data to obtain a target enhancement model includes the following steps: Constructing an encoding path of the encoding structure and a decoding path of the decoding structure based on a multi-layer scale, except that the convolution kernel of the scale of the last preset layer is replaced by a series of convolution kernels with different expansion rates to pre-construct the network model; Among them, in the encoding path: except for the convolution kernel of the ruler of the last preset number of layers, which is replaced by a series of convolution kernels with different expansion rates, the convolution kernel of each layer of the ruler is obtained based on the residual U-Net network setting of different depths, the number of input image channels and the number of pre-extracted feature channels are set before each layer of the ruler, and a maximum pooling layer is set after each layer of the ruler; in the decoding path: except for the convolution kernel of the ruler of the last preset number of layers, which is replaced by a series of convolution kernels with different expansion rates, the convolution kernel of each layer of the ruler is obtained based on the residual U-Net network setting, and a transposed convolution layer is set after each layer of the ruler for upsampling; the rulers of each layer in the encoding path and the decoding path perform feature encoding and context information capture based on the interactive cross attention module, and in the resolution maintenance deep supervision module, the residual U-Net is integrated into the deep network to maintain the resolution; Inputting the training samples in the training data into the network model for classification processing to obtain an output result; Calculating a loss function using binary cross entropy based on the output result and the binary labels corresponding to the training samples; The model weights of the network model are optimized and adjusted according to the result of the loss function, and the step of inputting the training samples in the training data into the network model for classification processing to obtain output results is returned to execute until the preset training conditions are met, and the target enhancement model is obtained according to the model weight settings after optimization and adjustment.

8. A blood vessel ultrasound enhancement device, characterized in that: include: A data acquisition module, used to obtain imaging data of subcutaneous blood vessels; the imaging data includes photoacoustic radio frequency data and ultrasonic radio frequency data; A preprocessing module, configured to reconstruct the imaging data to obtain an imaging image; the imaging image comprises a data pair of a photoacoustic image and an ultrasound image registration; the imaging image is marked with binary labels of blood vessels and non-blood vessels based on the photoacoustic image and the ultrasound image; A data construction module, used for constructing training data based on the imaging images; A model training module, used to iteratively train a pre-built network model based on the binary label using the training data to obtain a target enhancement model; the network model includes an encoding structure and a decoding structure; The enhancement module is used to perform enhancement processing on the vascular ultrasound image to be enhanced by using the target enhancement model to obtain an ultrasound enhancement result.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images