Ultrasonic image processing method and device, ultrasonic equipment and network training method

Abstract

The invention relates to an ultrasonic image processing method and device, ultrasonic equipment and a network training method. The method comprises the following steps: acquiring a plurality of ultrasonic images output when ultrasonic equipment scans a target object at multiple angles, inputting the plurality of ultrasonic images into a weight prediction network to obtain a weight map corresponding to each ultrasonic image, and fusing each ultrasonic image and the corresponding weight map to obtain a target ultrasonic image of the target object. By adopting the method, the imaging effect of the target ultrasonic image can be improved.

Description

Technical Field

[0001] This application relates to the field of ultrasound image processing technology, and in particular to an ultrasound image processing method, apparatus, ultrasound equipment, and network training method. Background Technology

[0002] With the continuous advancement of medical testing equipment and the increasing awareness of safety among users, users are increasingly inclined to conduct medical examinations in a safe and non-invasive manner. Based on this, ultrasound imaging equipment has been widely used in medical examinations due to its advantages such as real-time dynamic imaging, non-invasive safety, lightweight equipment, and outstanding economy. However, ultrasound imaging technology also has problems such as speckle noise and multiple artifact interference.

[0003] In existing technologies, spatial composite technology is commonly used to process ultrasound medical images. However, the continuous acquisition process of spatial composite technology is time-consuming and has problems such as low resolution and blurriness, which affect the imaging effect of ultrasound medical images. Summary of the Invention

[0004] Therefore, it is necessary to provide an ultrasound image processing method, apparatus, ultrasound equipment, computer-readable storage medium, and computer program product that can improve the imaging effect of ultrasound medical images in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides an ultrasound image processing method, comprising: acquiring multiple ultrasound images output by an ultrasound device scanning a target object from multiple angles; inputting the multiple ultrasound images into a weight prediction network to obtain a weight map corresponding to each ultrasound image; and fusing each ultrasound image and its corresponding weight map to obtain a target ultrasound image of the target object.

[0006] In one embodiment, the target ultrasound image of the target object is obtained by fusing each ultrasound image and its corresponding weight map, including: weighted fusing each ultrasound image and its corresponding weight map to obtain a weighted ultrasound image corresponding to each ultrasound image; and image fusing all the weighted ultrasound images to obtain the target ultrasound image.

[0007] In one embodiment, the weight prediction network includes a mapping unit and an output unit. Multiple ultrasound images are input into the weight prediction network to obtain a weight map corresponding to each ultrasound image. This includes: inputting multiple ultrasound images into the mapping unit to map image matrix values ​​to obtain weight values ​​corresponding to each ultrasound image; and inputting each weight value into the output unit to construct a weight map to obtain a weight map corresponding to each ultrasound image.

[0008] In one embodiment, acquiring multiple ultrasound images output by an ultrasound device when scanning a target object from multiple angles includes: scanning the target object with an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images; and spatially registering all candidate ultrasound images to obtain multiple ultrasound images.

[0009] In one embodiment, the training process of the weight prediction network includes: acquiring sample ultrasound images and labeled ultrasound images, and inputting the sample ultrasound images into an initial weight prediction network to obtain a predicted weight map; determining a first loss value based on the predicted weight map, the sample ultrasound images, the labeled ultrasound images, and a first loss function; determining a second loss value based on the predicted weight map, the sample ultrasound images, the labeled ultrasound images, and a second loss function; and adjusting the parameters of the initial weight prediction network based on the first loss value and the second loss value until the first loss value and the second loss value converge to obtain the trained weight prediction network.

[0010] In one embodiment, determining a first loss value based on a predicted weight map, a sample ultrasound image, a labeled ultrasound image, and a first loss function includes: fusing the predicted weight map and the sample ultrasound image to obtain a sample fused image; calculating the pixel difference matrix between the sample fused image and the labeled ultrasound image based on the first loss function, and weighting the pixel difference matrix according to the edge feature map corresponding to the labeled ultrasound image to obtain the first loss value.

[0011] Secondly, this application also provides an ultrasound image processing apparatus, comprising: an ultrasound image acquisition module for acquiring multiple ultrasound images output by an ultrasound device scanning a target object from multiple angles; a weight map acquisition module for inputting the multiple ultrasound images into a weight prediction network to obtain a weight map corresponding to each ultrasound image; and a fusion module for fusing each ultrasound image and its corresponding weight map to obtain a target ultrasound image of the target object.

[0012] Thirdly, this application also provides an ultrasound device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0015] Sixthly, this application provides a network training method, comprising: acquiring a training sample set, the training sample set including multiple sample ultrasound image groups and label ultrasound images corresponding to each sample ultrasound image group, the sample ultrasound image group containing a second number of sample ultrasound images, and the label ultrasound images being generated based on a first number of ultrasound images, the first number being greater than the second number; for each sample ultrasound image group and its corresponding label ultrasound image, inputting the sample ultrasound image group into an initial weight prediction network to obtain a predicted weight map; fusing the predicted weight map and the sample ultrasound image group to obtain a sample fused image; and training the initial weight prediction network based on the sample fused image, the label ultrasound image, and a loss function to obtain a trained weight prediction network.

[0016] In a seventh aspect, this application provides another ultrasound image processing method, comprising: scanning a target object according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple ultrasound images; inputting each ultrasound image into a weight prediction network to obtain a weight map corresponding to each ultrasound image output by the weight prediction network; performing weighted fusion of each ultrasound image and the weight map to obtain multiple weighted ultrasound images; performing image fusion of each weighted ultrasound image to obtain a target ultrasound image, and displaying the target ultrasound image on a screen.

[0017] The aforementioned ultrasound image processing method, apparatus, ultrasound equipment, computer-readable storage medium, and computer program product involve an ultrasound equipment acquiring multiple ultrasound images output when scanning a target object from multiple angles. These multiple ultrasound images are then input into a weighted prediction network to obtain a weight map corresponding to each ultrasound image. The weight map enables the reconstruction of key anatomical details from the ultrasound images. By fusing each ultrasound image with its corresponding weight map, a target ultrasound image of the target object is obtained. By predicting pixel-level weight maps, differentiated utilization of information from different angles is achieved. For each pixel location, the network determines the optimal angle based on local imaging quality and assigns it the highest weight, thereby preserving the clearest and most reliable tissue details during the fusion process. This ensures the reliability and accuracy of the fused target ultrasound image and improves the imaging effect of the ultrasound image. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is an application environment diagram of an ultrasound image processing method in one embodiment;

[0020] Figure 2 This is a flowchart illustrating an ultrasound image processing method in one embodiment;

[0021] Figure 3 This is a flowchart illustrating step 203 in one embodiment;

[0022] Figure 4 This is a flowchart illustrating step 202 in one embodiment;

[0023] Figure 5 This is a flowchart illustrating step 201 in one embodiment;

[0024] Figure 6 This is a flowchart illustrating the training steps of a weight prediction network in one embodiment.

[0025] Figure 7 This is a flowchart illustrating step 602 in one embodiment;

[0026] Figure 8 This is a flowchart illustrating the ultrasound image processing method in another embodiment;

[0027] Figure 9 This is a structural block diagram of an ultrasound image processing device in one embodiment;

[0028] Figure 10 This is a diagram of the internal structure of an ultrasonic device in one embodiment;

[0029] Figure 11 This is a flowchart illustrating a network training method in one embodiment;

[0030] Figure 12 This is a flowchart illustrating the ultrasound image processing method in another embodiment. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0032] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0033] The ultrasound image processing method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown includes at least an ultrasound device 101 and a weighted prediction network 101-1.

[0034] The ultrasound device 101 is used to scan the target object from multiple angles to obtain multiple ultrasound images. These multiple ultrasound images are input into a weighted prediction network 101-1 to obtain weight maps corresponding to each ultrasound image output by the network. The ultrasound images and their corresponding weight maps are then fused to obtain the target ultrasound image of the target object. The ultrasound device 101 may include components such as a control unit, a power module, a signal processing unit, an ultrasound probe, and a display. The probe can be connected to the control unit's interface via a cable; the display can be connected to the control unit via a video cable or an integrated connection; the control unit is connected to the signal processing unit; and the power module is connected to the control unit, the signal processing unit, the ultrasound probe, and the display.

[0035] The weight prediction network 101-1 is used to generate a weight map of multiple ultrasound images input to the ultrasound device 101, and sends each weighted image to the ultrasound device 101. The weight prediction network 101-1 can be deployed inside the ultrasound device 101 and transmit data with the ultrasound device 101 through interface calls. The weight network 101-1 can be a pre-trained neural network.

[0036] In real-world scenarios, ultrasound imaging, as a core technology for clinical diagnosis and treatment, has been widely applied in diverse settings such as obstetric monitoring, cardiovascular function assessment, musculoskeletal system detection, and abdominal organ exploration due to its advantages of real-time dynamic imaging, non-invasive safety, lightweight equipment, and outstanding cost-effectiveness. Its dynamic real-time imaging characteristics enable physicians to instantly capture organ motion and hemodynamic features; the non-invasive detection mechanism ensures zero radiation exposure for patients, and the high portability of the equipment allows it to break through the limitations of traditional radiology departments, making it widely used in emergency care, primary care, and mobile healthcare, demonstrating strong clinical adaptability.

[0037] However, ultrasound imaging systems face significant technical bottlenecks in their widespread application, with the core challenge lying in image quality optimization. Speckle noise and multiple artifacts are the main obstacles affecting image clarity. Speckle noise originates from the coherent superposition effect of ultrasound waves propagating within tissues, manifesting as a granular interference pattern in the image that reduces the signal-to-noise ratio. Multiple artifacts arise from the complex reflections and refractions of sound waves at tissue interfaces, such as reverberation, shadowing, and sidelobe interference. These interferences not only severely obscure minute anatomical structures and reduce image contrast but can also significantly reduce the visibility of critical areas (such as early-stage tumors and microcalcifications). For example, in early tumor screening, the combined effect of speckle noise and low-contrast tissue boundaries can easily lead to missed or misdiagnosed sub-centimeter lesions, thus affecting the accuracy of clinical diagnosis.

[0038] In existing technologies, spatial fusion is commonly used as the mainstream solution for speckle reduction. This involves transmitting and receiving ultrasound beams from multiple angles and fusing the acquired multi-frame echo data to suppress noise. The underlying principle is to utilize the incoherence of speckle noise at different incident angles, weakening its intensity through statistical averaging. However, this technology has inherent drawbacks. First, it significantly reduces temporal resolution. The continuous acquisition of multiple frames takes a long time, leading to a decrease in the imaging frame rate, making it difficult to meet the imaging requirements of dynamic scenes demanding high temporal resolution (such as rapid heart valve movement and dynamic blood flow observation). Second, it introduces structural blurring. Information fusion operations in traditional fusion processes (such as simple pixel averaging) easily cause image structural blurring, resulting in decreased edge sharpness of key subtle features such as microcalcifications and blood vessel walls, thus losing diagnostic value. Third, it introduces motion artifacts. For moving tissues (such as moving organs), the tissue position changes during multi-angle scanning, causing spatial misalignment between multiple frames. If traditional fusion algorithms fail to properly handle this misalignment, motion artifacts will be introduced, forming new sources of diagnostic interference. Therefore, how to effectively suppress speckle noise, maintain or even improve the clarity of image details, while overcoming the problems of reduced temporal resolution, structural blurring and motion artifacts caused by traditional spatial composite technology has become a core technical challenge restricting the further improvement of ultrasound image quality.

[0039] To address this, this application first reduces the angle of the acquired ultrasound images, avoiding spatial misalignment caused by too many images. Furthermore, a weight map of each ultrasound image is obtained through a pre-trained weight prediction network. The weight map is used to enhance key features in the ultrasound images and weaken edge features, thereby improving the effectiveness of image fusion. While significantly reducing the number of angles required for fusion, it achieves or even surpasses the noise reduction effect of traditional multi-angle fusion, while greatly improving image sharpness and temporal resolution and effectively suppressing motion artifacts, ultimately improving the ultrasound imaging effect.

[0040] In one exemplary embodiment, such as Figure 2 As shown, an ultrasound image processing method is provided, which can be applied to... Figure 1 The following steps are used as an example of ultrasonic equipment, including steps 201 to 203.

[0041] Step 201: Acquire multiple ultrasound images output by the ultrasound device when scanning the target object from multiple angles.

[0042] In this application, the target object refers to the object being scanned by an ultrasound device. During implementation, the ultrasound device performs ultrasound scans on the target object from multiple angles, resulting in multiple ultrasound images output after scanning the target object from multiple angles. Furthermore, the ultrasound device can also perform spatial registration processing on each ultrasound image, aligning the pixels corresponding to the scanned subject in each ultrasound image. Based on the aligned ultrasound images, an ultrasound image group is constructed, which is then used as the processing object to improve data processing efficiency. Alternatively, the ultrasound device can also perform geometric transformations on each ultrasound image. During the transformation process, the transformation can be performed according to a preset oblique scan coordinate system to align the acquired skewed image data with the non-skewed image data.

[0043] During the acquisition process, the ultrasound equipment can perform ultrasound scanning of the target object from multiple angles through the ultrasound probe. The ultrasound probe can cover the ultrasound scan from multiple angles through electronic deflection and focusing. The ultrasound equipment can acquire ultrasound images of one angle at the same time until the scanning of multiple angles is completed, and then obtain multiple ultrasound images corresponding to multiple angles.

[0044] It should be noted that the number of ultrasound images acquired in this application is less than the number of ultrasound images acquired by existing spatial compositing techniques. This is because this application uses a weighted graph approach, where ultrasound images are weighted and fused together before being fused again to obtain the target ultrasound image. This ensures that, even with fewer ultrasound images acquired by existing spatial compositing techniques, the target ultrasound image has similar image quality to the ultrasound images generated by existing spatial compositing techniques. This fundamentally avoids the ghosting and motion blur problems caused by acquiring too many images in existing spatial compositing techniques. In other words, compared to existing spatial compositing techniques, this application reduces image acquisition time, improves ultrasound image generation efficiency, reduces the number of ultrasound images acquired, avoids ghosting and motion blur, and improves the image quality of ultrasound images.

[0045] Step 202: Input multiple ultrasound images into the weight prediction network to obtain the weight map corresponding to each ultrasound image.

[0046] During implementation, the ultrasound equipment inputs multiple ultrasound images into the weight prediction network, which then predicts and outputs the weight map of each ultrasound image. The ultrasound equipment then obtains the weight map corresponding to each ultrasound image output by the weight prediction network.

[0047] The weight map is used to characterize the criticality of each pixel or region in the ultrasound image. The higher the criticality, the larger the weight value, and the lower the criticality, the smaller the weight value. The pixel size of the weight map is the same as the pixel size of the ultrasound image. The weight prediction network can be a pre-trained neural network. A lightweight network can be used to reduce the computational power required, so that the weight prediction network can run smoothly in the ultrasound equipment.

[0048] In real-world scenarios, existing technologies typically apply a uniform weighted average to all input images, but this approach weakens anatomical details that are only clearly visible from certain angles. To address this, this application predicts a weight map of the ultrasound image, which assigns weights to each image region / pixel in the ultrasound image. This preserves the clearest and most reliable image details in the ultrasound image while suppressing the influence of less important image regions / pixels on image fusion, thereby improving the imaging effect of the target ultrasound image.

[0049] Step 203: Fuse each ultrasound image with its corresponding weight map to obtain the target ultrasound image of the target object.

[0050] During the implementation process, the ultrasound equipment first performs weighted processing based on each ultrasound image and its corresponding weight map to obtain multiple weighted ultrasound images. Then, image fusion processing is performed on each weighted ultrasound image to obtain the target ultrasound image of the target object.

[0051] This application first preserves key features and constrains secondary features of ultrasound images through a weighted graph. For non-edge regions, even with a large amount of speckle noise, their weights are significantly reduced, which can smooth and suppress image noise. This avoids the influence of speckle noise in the ultrasound image on the fused target ultrasound image, ensuring the reliability and accuracy of the target ultrasound image. Under the guidance of the edge weighting mechanism, the network effectively avoids excessive sensitivity to noise. At the same time, combined with the global consistency constraint of the structural similarity index, the image noise of the target ultrasound image is reduced, while ensuring the high fidelity of the details of the target ultrasound image.

[0052] It should be noted that ultrasound equipment has limited computing power. A lightweight encoder-decoder convolutional neural network variant can be used to balance feature extraction capability and computational complexity, enabling the model to run on resource-constrained ultrasound equipment. This convolutional neural network can also be combined with the TensorRT engine for 16-bit floating-point quantization inference. That is, the ultrasound equipment provides computing power to the TensorRT engine, which drives the weight prediction network for prediction processing. This requires less computation than 32-bit floating-point calculations. Furthermore, CUDA (Compute Unified Device Architecture) streaming technology enables parallel pipelined processing of data preprocessing, model inference, and post-processing. Based on these operations, the processing time for a single high-resolution image frame is consistently below 8 milliseconds, fully meeting the performance requirements of clinical ultrasound for real-time dynamic imaging, improving computing power utilization, and achieving fast, stable, and high-fidelity ultrasound image generation.

[0053] In the aforementioned ultrasound image processing method, the ultrasound device acquires multiple ultrasound images output when scanning the target object from multiple angles. These multiple ultrasound images are input into a weighted prediction network to obtain a weight map corresponding to each ultrasound image. The weight map is used to reconstruct key anatomical details of the ultrasound image. The ultrasound images and their corresponding weight maps are fused to obtain the target ultrasound image of the target object. By predicting pixel-level weight maps, differentiated utilization of information from different angles is achieved. For each pixel position, the network determines the optimal angle based on local imaging quality and assigns it the highest weight. This preserves the clearest and most reliable tissue details in the local area during the fusion process, thereby ensuring the reliability and accuracy of the fused target ultrasound image and improving the imaging effect of the ultrasound image.

[0054] Based on the above exemplary embodiment, the following provides an ultrasound image processing method in one or more exemplary embodiments, which is applied to... Figure 1 Taking ultrasound equipment as an example, the explanation includes the following:

[0055] During the fusion process, the ultrasound image and the weighted image can be fused first to obtain a weighted ultrasound image, and then the weighted ultrasound image can be fused to obtain the target ultrasound image; in one optional implementation provided by this application, such as Figure 3 As shown, step 203 includes steps 301 to 302:

[0056] Step 301: Weighted fusion of each ultrasound image and its corresponding weight map is performed to obtain the weighted ultrasound image corresponding to each ultrasound image.

[0057] During the process, the ultrasound equipment performs weighted fusion on each ultrasound image and its corresponding weight map to obtain a weighted ultrasound image corresponding to the ultrasound image, until all ultrasound images are weighted and fused to obtain a weighted ultrasound image corresponding to multiple ultrasound images.

[0058] During the execution process, the ultrasound equipment can perform pixel-by-pixel weighted calculations on the ultrasound image based on the weights recorded in the weight map to obtain a weighted ultrasound image; for example, refer to formula (1) for pixel-by-pixel weighted calculation:

[0059] I weighted (x,y,θ)=W θ (x,y)×I θ (x,y) formula (1);

[0060] Among them, W θ Let I be the weight matrix of the weighted graph. θ Let x be the image matrix of the ultrasound image, y be the height of the matrix, θ be the direction angle of the matrix, and θ be the weight value along direction θ at (x, y) of the matrix.

[0061] It should be noted that, when an ultrasound image group is constructed from ultrasound images, each weighted ultrasound image can also be constructed into a weighted image group I. weighted ∈R {H×W×3} Where R represents the image group, H represents the height of the matrix, W represents the width of the matrix, and 3 indicates that the image group includes 3 weighted images.

[0062] Step 302: Perform image fusion on all weighted ultrasound images to obtain the target ultrasound image.

[0063] During implementation, the ultrasound equipment can spatially composite all weighted ultrasound images to obtain the target ultrasound image; during execution, the ultrasound equipment can spatially composite weighted ultrasound images from multiple angles to enhance the salient features of the tissue by averaging the strongest echo signal, thereby obtaining the target ultrasound image.

[0064] For example, fusion can be performed by referring to formula (2):

[0065] k Formula (2);

[0066] Where N=n Deg For the total angle, I weighted denoted as a weighted ultrasound image, k represents the k-th angle, and sort represents the fusion function.

[0067] One optional implementation method provided in this application involves first performing weighted fusion of ultrasound images, and then performing spatial composite of the weighted ultrasound images. Compared with traditional networks that generate composite images in one go, this embodiment enhances the main features of the image while suppressing irrelevant noise by performing weighted calculations on the ultrasound images. At the same time, the weighted ultrasound images are further fused to enhance the significant features of the tissue, which conforms to the actual process of multi-angle utilization of ultrasound imaging, thereby improving the imaging effect of the target ultrasound image.

[0068] In the process of generating a weight graph using a weight prediction network, the weight prediction network can obtain the weight graph layer by layer through a hierarchical structure; wherein, the weight prediction network may include a mapping unit and an output unit; in one optional embodiment provided in this application, such as Figure 4 As shown, step 202 includes steps 401 to 402:

[0069] Step 401: Map the image matrix values ​​of multiple ultrasound images into the mapping unit to obtain the weight values ​​corresponding to each ultrasound image.

[0070] During implementation, multiple ultrasound images are input into the mapping unit of the weight prediction network. The mapping unit predicts the weight values ​​corresponding to the image matrix values ​​based on the image matrix values ​​of the ultrasound images. The mapping unit can be a unit for weight value prediction, and it may include an encoder and a decoder. For each ultrasound image, the encoder in the mapping unit maps each image matrix value of the ultrasound image to obtain a mapping result, and then the decoder decodes the mapping result to obtain the weight value corresponding to each image matrix value.

[0071] In real-world scenarios, the weight prediction network is a pre-trained convolutional neural network. The convolutional neural network maps input data to output data through parameter mapping. The output data can be a pixel-level angle weight map W. θ W θ It contains sub-channels with the same number of channels as the ultrasound images, and each sub-channel corresponds to a weight for an angle view. During execution, the ultrasound device can also input an ultrasound image group containing multiple ultrasound images into the weight prediction network. The mapping layer of the weight prediction network maps the image matrix values ​​of the ultrasound images in the ultrasound image group to obtain the weight values ​​corresponding to the image matrix values ​​of all ultrasound images.

[0072] For example, if an ultrasound image is a 2x2 image matrix, then the ultrasound image includes four sub-channels, which are (x1, y1), (x1, y2), (x2, y1), and (x2, y2), respectively. Then W θIt also includes four sub-channels, and the weight values ​​of the four sub-channels are as follows: w1 for (x1, y1), w2 for (x1, y2), w3 for (x2, y1), and w4 for (x2, y2).

[0073] Step 402: Input and output the weight values ​​of each weight to construct a weight map, and obtain the weight map corresponding to each ultrasound image.

[0074] During implementation, the weight prediction network inputs weight values ​​to the output unit of the next layer of the mapping unit. The output unit constructs a weight map based on the weight values, obtaining the weight map corresponding to each ultrasound image, and then outputs the weight map to the ultrasound device. The output unit can be used for data processing and output, and can be used to: construct the weight map corresponding to each ultrasound image based on its weight values, and output the weight map.

[0075] One optional implementation provided in this application is that the weight prediction network obtains the weight map of the ultrasound image through hierarchical processing, and the lightweight design ensures the usability of the network on ultrasound equipment.

[0076] During the acquisition of ultrasound images, spatial registration processing can be performed on candidate ultrasound images acquired by the ultrasound equipment to obtain multiple ultrasound images; in one optional embodiment provided in this application, such as Figure 5 As shown, step 201 includes steps 501 to 502:

[0077] Step 501: Scan the target object using an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images.

[0078] In practical applications, ultrasound probes in ultrasound equipment can typically acquire ultrasound images from multiple angles. These images can then be spatially composited. Existing technologies often acquire ultrasound images from a large number of angles, such as 5-9, which is time-consuming and produces more motion artifacts. To address this, this application proposes a method based on the principles of ultrasound physics. Larger angle intervals significantly enhance the decorrelation of speckle patterns and alter the echo intensity of specular reflectors. Therefore, compared to existing technologies, a probe with a larger angle can be selected, while simultaneously reducing the number of acquired angles and the image acquisition time, thus reducing artifacts at their source.

[0079] During implementation, the ultrasound equipment scans the target object sequentially at each angle according to the preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images; optionally, the multiple acquisition angles include three angles: -15°, 0° and +15°; furthermore, the acquisition angles may also include only any two of the above angles, or any one angle.

[0080] Step 502: Spatial registration is performed on all candidate ultrasound images to obtain multiple ultrasound images.

[0081] During implementation, the ultrasound equipment performs spatial registration based on the feature distribution of each candidate ultrasound image to obtain multiple ultrasound images, thereby unifying the feature dimensions of each candidate ultrasound image and facilitating subsequent processing.

[0082] One optional implementation provided in this application uses fewer angles to acquire ultrasound images, thereby avoiding motion artifacts at the source. At the same time, it reduces the image acquisition time, further avoiding motion artifacts, and achieves dimensional uniformity through spatial registration, improving the usability of ultrasound images, and thus improving the usability of the target ultrasound image.

[0083] In practical scenarios, the weight prediction network also needs to be trained to generate a weight map based on the trained network. During network training, network parameters can be adjusted from two dimensions: visual similarity and weight map, to ensure the high availability of the fusion results of the target ultrasound images. One optional implementation provided in this application is as follows: Figure 6 As shown, the training process of the weight prediction network includes steps 601 to 604:

[0084] Step 601: Obtain the sample ultrasound image and the label ultrasound image, and input the sample ultrasound image into the initial weight prediction network to obtain the prediction weight map.

[0085] During implementation, the network training device acquires sample ultrasound images and label ultrasound images, and inputs the sample ultrasound images into an initial weight prediction network. The initial weight prediction network then predicts weights based on the image matrix values ​​of the sample ultrasound images to obtain a predicted weight map. The sample ultrasound images may include multiple groups of sample ultrasound images, each group may include multiple candidate sample ultrasound images, and each group may correspond to a label ultrasound image. The weight prediction network may include a mapping unit and an output unit. Optionally, the label ultrasound image is generated based on a first number of ultrasound images; it may be generated by spatial composite of the first number of ultrasound images. Optionally, each group of sample ultrasound images may include a second number of candidate sample ultrasound images; the first number is greater than the second number.

[0086] For example, the second number of candidate sample ultrasound images can be ultrasound images acquired at three angles: -15°, 0°, and +15°, in which case the second number is 3; the first number of ultrasound images can be ultrasound images acquired at five angles: -15°, -7.5°, 0°, 7.5°, and +15°, in which case the first number is 5.

[0087] Regarding the acquisition angle, there are certain requirements; it's an inclusion relationship. The acquisition angle range for the first number of ultrasound images is [-a, a], and the acquisition angle range for the second number of candidate sample ultrasound images is [-b, b], where b is less than or equal to a. Furthermore, the acquisition angle corresponding to the label ultrasound image can include the acquisition angle corresponding to the candidate sample ultrasound images in the sample ultrasound image group. Continuing with the previous example: the five angles corresponding to the five ultrasound images used to generate the label ultrasound image include the three angles corresponding to the three candidate sample ultrasound images in the sample ultrasound image group.

[0088] During the execution process, there can be multiple pairs of sample ultrasound images and label ultrasound images. For each pair of sample ultrasound images and label ultrasound images, steps 601 to 604 can be repeated until the network training is completed.

[0089] Step 602: Determine the first loss value based on the predicted weight map, sample ultrasound image, labeled ultrasound image, and first loss function.

[0090] During implementation, the network training device performs fusion processing based on the predicted weight map and the sample ultrasound images, and then inputs the fused ultrasound images and the labeled ultrasound images into the first loss function to obtain the first loss value. The fusion processing procedure can be referred to step 203 above and its specific implementation, and will not be repeated here.

[0091] During execution, the first loss function can be a loss function that calculates the difference in noise between the fused ultrasound image and the labeled ultrasound image. The first loss value is used to enhance the noise suppression capability of the weight prediction network.

[0092] Step 603: Determine the second loss value based on the predicted weight map, the sample ultrasound image, the label ultrasound image, and the second loss function.

[0093] During implementation, the network training device performs fusion processing based on the predicted weight map and the sample ultrasound images, and then inputs the fused ultrasound images and the labeled ultrasound images into the second loss function to obtain the second loss value.

[0094] During execution, the second loss function can be a loss function that calculates the difference in structural similarity between the fused ultrasound image and the labeled ultrasound image. For example, the structural similarity (SSIM) loss can be calculated as the second loss value to evaluate I. fused (x, y) and I target The consistency of (x, y) in image structure and brightness is used to guide the network to maintain the overall visual similarity between the fusion result and the target image. Where, I fused (x, y) can be the fused sample ultrasound image, I target(x, y) represents the labeled ultrasound image, where x represents the height of the image matrix and y represents the width of the image matrix.

[0095] It should be noted that either step 602 or step 603 can be executed according to actual needs. If it is necessary to obtain the first loss value and the second loss value, steps 602 and 603 can be replaced by: fusing the predicted weight map and the sample ultrasound image to obtain the weighted sample ultrasound image; determining the first loss value based on the weighted sample ultrasound image, the labeled ultrasound image and the first loss function; and determining the second loss value based on the weighted sample ultrasound image, the labeled ultrasound image and the second loss function.

[0096] Step 604: Adjust the parameters of the initial weight prediction network according to the first loss value and / or the second loss value until the first loss value and / or the second loss value converge to obtain the trained weight prediction network.

[0097] During implementation, the model training device can first adjust the parameters of the initial weight prediction network based on a first loss value until the first loss value converges, and / or adjust the parameters of the initial weight prediction network based on a second loss value until the second loss value converges, thus obtaining the trained weight prediction network. Specifically, during training, the parameters of the mapping units of the weight prediction network can be adjusted, particularly the encoders included in the mapping units, until the second loss value converges, resulting in trained mapping units and ultimately, the trained weight prediction network.

[0098] The following explanation uses an actual training process as an example. For each set of sample ultrasound images and label ultrasound images, the sample ultrasound images consist of x1 ultrasound images, and the label ultrasound image is a spatial composite of x2 ultrasound images, where x2 is greater than x1. During training, the x1 ultrasound images are input into the initial weight prediction network, resulting in prediction weight maps corresponding to each of the x1 ultrasound images output by the initial weight prediction network. Weighted calculations are then performed based on the x1 ultrasound images and the prediction weight maps to obtain a weighted ultrasound image. Finally, the weighted ultrasound images are fused to obtain the fused ultrasound image. First, based on the fused ultrasound image, the label ultrasound image, and the first loss function, a first loss value is calculated to evaluate the image structure and brightness of the fused ultrasound image and the label ultrasound image. First, the consistency of the surface is ensured. Second, based on the fused ultrasound image, the labeled ultrasound image, and the second loss function, a second loss value is calculated to evaluate the fidelity of the fused ultrasound image and the labeled ultrasound image at important edges and the degree of suppression of irrelevant noise. The parameters of the initial weight prediction network are adjusted according to the first and second loss values, and the network is trained again according to subsequent samples until the first and second loss values ​​converge, resulting in the trained weight prediction network. The weight map output by the weight prediction network achieves the edge fidelity effect of multi-angle ultrasound image fusion with fewer-angle ultrasound image fusion. In addition, the weight map output by the weight prediction network also suppresses irrelevant noise and improves the efficiency of ultrasound image fusion.

[0099] One optional implementation provided in this application guides the weight prediction network to learn the weight map by combining loss functions, resulting in better imaging effect of the target ultrasound image and improving the usability and reliability of the weight prediction network.

[0100] In determining the first loss value, the predicted weight map and the sample ultrasound image can be fused first to obtain a fused sample image. Then, the pixel difference matrix can be calculated based on the fused sample image and the labeled ultrasound image, and the first loss value can be determined based on the pixel difference matrix. In one optional implementation provided by this application, such as... Figure 7 As shown, step 602 includes steps 701 to 702:

[0101] Step 701: The predicted weight map and the sample ultrasound image are fused to obtain the sample fused image.

[0102] During implementation, the network training device performs pixel-by-pixel weighted multiplication based on the predicted weight map and the sample ultrasound image to obtain the sample fusion image.

[0103] In the actual training process, there can be multiple sample ultrasound images, and correspondingly, there can also be multiple prediction weight maps. The sample ultrasound images and the corresponding prediction weight maps can be fused together, and the fused images can be fused again to obtain the sample fused image. The specific fusion process can be referred to step 203 and the steps of the lower embodiment corresponding to step 203, which will not be repeated here.

[0104] Step 702: Calculate the pixel difference matrix between the sample fused image and the label ultrasound image based on the first loss function, and perform weighted calculation on the pixel difference matrix according to the edge feature map corresponding to the label ultrasound image to obtain the first loss value.

[0105] During implementation, the model training device can calculate the pixel difference matrix between the sample fused image and the label ultrasound image based on the first loss function, and weight the pixel difference matrix based on the edge feature map corresponding to the label ultrasound image to obtain the first loss value.

[0106] During execution, the model training device can calculate I. fused (x, y) and I target The pixel difference at (x, y), and according to I target The true edge map (x, y) is weighted by the error, giving higher weights to diagnostically relevant true edge regions, prompting the model to prioritize the recovery of these details; and giving lower weights to regions dominated by speckle noise, enabling the model to smoothly suppress noise.

[0107] One optional implementation provided in this application enhances the weight prediction network by using the true edge map of the labeled ultrasound image for training, thereby improving the network's ability to suppress speckle noise and enhancing its reliability and usability.

[0108] In one embodiment, see Figure 8 The document illustrates a flowchart of an ultrasound image processing method provided in an embodiment of this application, which can be applied to... Figure 1 In the ultrasonic equipment shown. For example... Figure 8 As shown, the ultrasound image processing method may include the following steps:

[0109] Step 801: Scan the target object using an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images.

[0110] Optionally, multiple acquisition angles are available: –15°, 0°, and +15°.

[0111] Step 802: Spatial registration is performed on all candidate ultrasound images to obtain multiple ultrasound images.

[0112] Step 803: Input multiple ultrasound images into the mapping unit included in the weight prediction network to map the image matrix values ​​and obtain the weight value corresponding to each ultrasound image.

[0113] Step 804: Input each weight value into the output unit of the weight prediction network to construct a weight map, and obtain the weight map corresponding to each ultrasound image.

[0114] Step 805: Weighted fusion of each ultrasound image and its corresponding weighted image is performed to obtain the weighted ultrasound image corresponding to each ultrasound image.

[0115] Step 806: Perform image fusion on all weighted ultrasound images to obtain the target ultrasound image.

[0116] Step 807: Display the target ultrasound image on the screen.

[0117] It should be noted that any one or more of steps 801 to 807 can be combined to form a new implementation method according to the needs of implementation and deployment. Furthermore, any one or more technical features in the technical solution composed of steps 801 to 807 can also be combined to form a new implementation method according to the actual deployment needs, or technical features in one or more optional implementations provided by one or more of the above embodiments can be combined to form a new implementation method. These will not be elaborated on here.

[0118] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0119] Based on the same inventive concept, this application also provides an ultrasound image processing apparatus for implementing the ultrasound image processing method described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more ultrasound image processing apparatus embodiments provided below can be found in the limitations of the ultrasound image processing method described above, and will not be repeated here.

[0120] In one exemplary embodiment, such as Figure 9 As shown, an ultrasound image processing device is provided, including: an ultrasound image acquisition module 901, a weight map acquisition module 902, and a fusion module 903, wherein: the ultrasound image acquisition module 901 is used to acquire multiple ultrasound images output by an ultrasound device scanning a target object from multiple angles; the weight map acquisition module 902 is used to input the multiple ultrasound images into a weight prediction network to obtain a weight map corresponding to each ultrasound image; and the fusion module 903 is used to fuse each ultrasound image and its corresponding weight map to obtain a target ultrasound image of the target object.

[0121] In one embodiment, the fusion module 903 includes a first fusion unit and a second fusion unit, wherein: the first fusion unit is used to perform weighted fusion of each ultrasound image and the corresponding weight map to obtain a weighted ultrasound image corresponding to each ultrasound image; the second fusion unit is used to perform image fusion of all weighted ultrasound images to obtain a target ultrasound image.

[0122] In one embodiment, the weight map acquisition module 902 includes a matrix mapping unit and an image output unit, wherein: the matrix mapping unit is used to map multiple ultrasound images into the mapping unit to obtain image matrix values, thereby obtaining weight values ​​corresponding to each ultrasound image; the image output unit is used to input each weight value into the output unit to construct a weight map, thereby obtaining a weight map corresponding to each ultrasound image.

[0123] In one embodiment, the ultrasound image acquisition module 901 includes an acquisition unit and a registration unit, wherein: the acquisition unit is used to scan the target object using an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images; the registration unit is used to spatially register all the candidate ultrasound images to obtain multiple ultrasound images.

[0124] In one embodiment, the apparatus further includes a network training module, which comprises a training set acquisition unit, a first loss determination unit, a second loss determination unit, and a parameter tuning unit. The training set acquisition unit acquires sample ultrasound images and label ultrasound images, and inputs the sample ultrasound images into an initial weight prediction network to obtain a predicted weight map. The first loss determination unit determines a first loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a first loss function. The second loss determination unit determines a second loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a second loss function. The parameter tuning unit adjusts the parameters of the initial weight prediction network based on the first and second loss values ​​until the first and second loss values ​​converge, thus obtaining the trained weight prediction network.

[0125] In one embodiment, the first loss determination unit includes a sample fusion image acquisition unit and a loss value calculation unit, wherein: the sample fusion image acquisition unit is used to fuse the predicted weight map and the sample ultrasound image to obtain a sample fusion image; the loss value calculation unit is used to calculate the pixel difference matrix between the sample fusion image and the label ultrasound image based on a first loss function, and to perform weighted calculation on the pixel difference matrix based on the edge feature map corresponding to the label ultrasound image to obtain a first loss value.

[0126] Each module in the aforementioned ultrasound image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the ultrasound device in hardware form or independent of it, or stored in the memory of the ultrasound device in software form, so that the processor can call and execute the corresponding operations of each module.

[0127] In one exemplary embodiment, an ultrasonic device is provided, the internal structure of which can be shown in the diagram below. Figure 10As shown, the ultrasound device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an ultrasound image processing method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the ultrasound device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the ultrasound device, or external keyboards, touchpads, or mice, etc.

[0128] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the ultrasonic device to which the present application is applied. A specific ultrasonic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one exemplary embodiment, an ultrasound device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring multiple ultrasound images output by the ultrasound device when scanning a target object from multiple angles; inputting the multiple ultrasound images into a weight prediction network to obtain a weight map corresponding to each ultrasound image; and fusing the ultrasound images and the corresponding weight maps to obtain a target ultrasound image of the target object.

[0130] In one embodiment, when the processor executes the computer program, it further performs the following steps: weighted fusion of each ultrasound image and its corresponding weight map to obtain a weighted ultrasound image corresponding to each ultrasound image; and image fusion of all weighted ultrasound images to obtain a target ultrasound image.

[0131] In one embodiment, when the processor executes the computer program, it further performs the following steps: mapping the image matrix values ​​of multiple ultrasound image input mapping units to obtain the weight values ​​corresponding to each ultrasound image; and constructing a weight map by inputting and outputting each weight value to obtain the weight map corresponding to each ultrasound image.

[0132] In one embodiment, when the processor executes the computer program, it further performs the following steps: scanning the target object with an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images; and spatially registering all candidate ultrasound images to obtain multiple ultrasound images.

[0133] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring sample ultrasound images and label ultrasound images, and inputting the sample ultrasound images into an initial weight prediction network to obtain a predicted weight map; determining a first loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a first loss function; determining a second loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a second loss function; and adjusting the parameters of the initial weight prediction network based on the first loss value and the second loss value until the first loss value and the second loss value converge to obtain a trained weight prediction network.

[0134] In one embodiment, when the processor executes the computer program, it further performs the following steps: fusing the predicted weight map and the sample ultrasound image to obtain a sample fused image; calculating the pixel difference matrix between the sample fused image and the label ultrasound image based on a first loss function, and performing weighted calculation on the pixel difference matrix based on the edge feature map corresponding to the label ultrasound image to obtain a first loss value.

[0135] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps: acquiring multiple ultrasound images output by an ultrasound device scanning a target object from multiple angles; inputting the multiple ultrasound images into a weighted prediction network to obtain a weight map corresponding to each ultrasound image; and fusing the ultrasound images and their corresponding weight maps to obtain a target ultrasound image of the target object.

[0136] In one embodiment, when the processor executes the computer program, it further performs the following steps: weighted fusion of each ultrasound image and its corresponding weight map to obtain a weighted ultrasound image corresponding to each ultrasound image; and image fusion of all weighted ultrasound images to obtain a target ultrasound image.

[0137] In one embodiment, when the processor executes the computer program, it further performs the following steps: mapping the image matrix values ​​of multiple ultrasound image input mapping units to obtain the weight values ​​corresponding to each ultrasound image; and constructing a weight map by inputting and outputting each weight value to obtain the weight map corresponding to each ultrasound image.

[0138] In one embodiment, when the processor executes the computer program, it further performs the following steps: scanning the target object with an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images; and spatially registering all candidate ultrasound images to obtain multiple ultrasound images.

[0139] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring sample ultrasound images and label ultrasound images, and inputting the sample ultrasound images into an initial weight prediction network to obtain a predicted weight map; determining a first loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a first loss function; determining a second loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a second loss function; and adjusting the parameters of the initial weight prediction network based on the first loss value and the second loss value until the first loss value and the second loss value converge to obtain a trained weight prediction network.

[0140] In one embodiment, when the processor executes the computer program, it further performs the following steps: fusing the predicted weight map and the sample ultrasound image to obtain a sample fused image; calculating the pixel difference matrix between the sample fused image and the label ultrasound image based on a first loss function, and performing weighted calculation on the pixel difference matrix based on the edge feature map corresponding to the label ultrasound image to obtain a first loss value.

[0141] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: acquiring multiple ultrasound images output by an ultrasound device scanning a target object from multiple angles; inputting the multiple ultrasound images into a weighted prediction network to obtain a weight map corresponding to each ultrasound image; and fusing the ultrasound images and their corresponding weight maps to obtain a target ultrasound image of the target object.

[0142] In one embodiment, when the processor executes the computer program, it further performs the following steps: weighted fusion of each ultrasound image and its corresponding weight map to obtain a weighted ultrasound image corresponding to each ultrasound image; and image fusion of all weighted ultrasound images to obtain a target ultrasound image.

[0143] In one embodiment, when the processor executes the computer program, it further performs the following steps: mapping the image matrix values ​​of multiple ultrasound image input mapping units to obtain the weight values ​​corresponding to each ultrasound image; and constructing a weight map by inputting and outputting each weight value to obtain the weight map corresponding to each ultrasound image.

[0144] In one embodiment, when the processor executes the computer program, it further performs the following steps: scanning the target object with an ultrasound device according to a preset acquisition frequency and multiple preset acquisition angles to obtain multiple candidate ultrasound images; and spatially registering all candidate ultrasound images to obtain multiple ultrasound images.

[0145] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring sample ultrasound images and label ultrasound images, and inputting the sample ultrasound images into an initial weight prediction network to obtain a predicted weight map; determining a first loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a first loss function; determining a second loss value based on the predicted weight map, the sample ultrasound images, the label ultrasound images, and a second loss function; and adjusting the parameters of the initial weight prediction network based on the first loss value and the second loss value until the first loss value and the second loss value converge to obtain a trained weight prediction network.

[0146] In one embodiment, when the processor executes the computer program, it further performs the following steps: fusing the predicted weight map and the sample ultrasound image to obtain a sample fused image; calculating the pixel difference matrix between the sample fused image and the label ultrasound image based on a first loss function, and performing weighted calculation on the pixel difference matrix based on the edge feature map corresponding to the label ultrasound image to obtain a first loss value.

[0147] This application also provides a network training method, in an exemplary embodiment, such as... Figure 11 As shown, a network training method is provided, which can be applied to a network training device, including steps 1101 to 1104:

[0148] Step 1101: Obtain the training sample set.

[0149] During implementation, the network training device acquires a training sample set; the training sample set includes multiple sample ultrasound image groups and corresponding label ultrasound images for each sample ultrasound image group. Each sample ultrasound image group contains a second number of sample ultrasound images, and the label ultrasound images are generated based on a first number of ultrasound images, where the first number is greater than the second number. The label ultrasound images can be spatially composited based on the first number of ultrasound images.

[0150] For example, the second number of sample ultrasound images can be ultrasound images acquired at three angles: -15°, 0°, and +15°, in which case the second number is 3; the first number of ultrasound images can be ultrasound images acquired at five angles: -15°, -7.5°, 0°, 7.5°, and +15°, in which case the first number is 5.

[0151] Regarding the acquisition angle, there are certain requirements; it's an inclusion relationship. The acquisition angle range for the first number of ultrasound images is [-a, a], and the acquisition angle range for the second number of candidate sample ultrasound images is [-b, b], where b is less than or equal to a. Furthermore, the acquisition angle corresponding to the label ultrasound image includes the acquisition angle corresponding to the sample ultrasound images in the sample ultrasound image group. Continuing with the previous example: the five angles corresponding to the five ultrasound images used to generate the label ultrasound image include the three angles corresponding to the three sample ultrasound images in the sample ultrasound image group.

[0152] During the acquisition process, the network training device can obtain sample ultrasound image groups and corresponding label ultrasound images from the database; alternatively, the network training device can also receive sample ultrasound image groups and corresponding label ultrasound images from external devices.

[0153] The tagged ultrasound image can be an ultrasound image generated using existing spatial compositing techniques; furthermore, the number of ultrasound images used to generate the tagged ultrasound image is greater than the number of sample ultrasound images included in the sample ultrasound image group.

[0154] Step 1102: For each sample ultrasound image group and its corresponding label ultrasound image, input the sample image group into the initial weight prediction network to obtain the prediction weight map.

[0155] During implementation, for each sample ultrasound image group and its corresponding label ultrasound image, the network training device inputs the sample images into the initial weight prediction network. The encoder included in the initial weight prediction network performs mapping processing based on each sample ultrasound image to obtain the mapping result corresponding to each sample ultrasound image. The decoder included in the initial weight prediction network decodes the result to obtain the prediction weight map corresponding to each sample ultrasound image.

[0156] In addition, the encoder and decoder can be integrated into a mapping unit, and the mapping process can be performed by the mapping unit included in the initial weight prediction network to obtain the prediction weight map corresponding to each sample ultrasound image; or, the decoder and / or mapping unit can generate the prediction weight value corresponding to each sample ultrasound image, and construct a matrix of the prediction weight value by the output unit included in the initial weight prediction network to generate and output the prediction weight map corresponding to each sample ultrasound image.

[0157] Step 1103: The predicted weight map and the sample ultrasound image group are fused to obtain the sample fused image.

[0158] During implementation, the network training device performs weighted fusion based on the predicted weight map and the sample ultrasound images included in the sample ultrasound image group to obtain multiple weighted ultrasound images, and performs image fusion based on each weighted ultrasound image to obtain a sample fused image.

[0159] Step 1104: Based on the sample fusion image, the labeled ultrasound image, and the loss function, train the initial weight prediction network to obtain the trained weight prediction network.

[0160] During implementation, the network training device calculates the loss value of the sample fused image relative to the label ultrasound image based on the loss function. The parameters of the initial weight prediction network are adjusted according to the loss value. This operation is performed for each group of sample ultrasound images and its corresponding label ultrasound image until all samples are used, or until the loss value converges, resulting in the trained weight prediction network. During parameter adjustment, the parameters of the mapping unit of the initial weight prediction network, or the parameters of the encoder of the initial weight prediction network, can be adjusted.

[0161] It should be noted that the steps provided in this embodiment can be performed with reference to one or more of the above embodiments. When reading the one or more of the above embodiments, you can refer to the steps in this embodiment. Similarly, when reading this embodiment, you can also refer to the steps in one or more of the above embodiments.

[0162] This application also provides another ultrasound image processing method, in an exemplary embodiment, such as... Figure 12 As shown, another ultrasound image processing method is provided, which can be applied to ultrasound equipment, including steps 1201 to 1204:

[0163] Step 1201: Scan the target object according to the preset acquisition frequency and multiple preset acquisition angles to obtain multiple ultrasound images.

[0164] During implementation, the ultrasound equipment performs ultrasound scanning on the target object according to a preset acquisition frequency and multiple preset acquisition angles, obtaining multiple ultrasound images. The ultrasound image acquisition process can be performed as described in one or more of the above embodiments, and will not be repeated here.

[0165] Step 1202: Input each ultrasound image into the weight prediction network to obtain the weight map corresponding to each ultrasound image output by the weight prediction network.

[0166] During implementation, the ultrasound equipment inputs each ultrasound image into a weighted prediction network configured within the ultrasound equipment. The weighted prediction network then performs weighted graph prediction, resulting in a weighted graph corresponding to each ultrasound image output by the network. The prediction process of the weighted prediction network can be performed by referring to the steps described in one or more of the above embodiments, and will not be repeated here.

[0167] Step 1203: Weighted fusion of each ultrasound image and weight map to obtain multiple weighted ultrasound images.

[0168] During implementation, the ultrasound equipment performs weighted fusion of each ultrasound image and the corresponding weight map to obtain a weighted ultrasound image corresponding to each ultrasound image. In the weighted fusion process, the weighted fusion operation described in one or more of the above embodiments can be referred to, and will not be repeated here.

[0169] Step 1204: Perform image fusion on the weighted ultrasound images to obtain the target ultrasound image, and display the target ultrasound image on the screen.

[0170] During implementation, the ultrasound equipment fuses the weighted ultrasound images to obtain the target ultrasound image corresponding to the target object, and transmits the target ultrasound image to the screen for display. The fusion process can be performed with reference to the image fusion operations described in one or more of the above embodiments, and will not be repeated here.

[0171] It should be noted that the steps provided in this embodiment can be performed with reference to one or more of the above embodiments. When reading the one or more of the above embodiments, you can refer to the steps in this embodiment. Similarly, when reading this embodiment, you can also refer to the steps in one or more of the above embodiments.

[0172] This application provides an ultrasound image processing method. An ultrasound device acquires multiple ultrasound images from multiple angles when scanning a target object. These images are input into a weighted prediction network to obtain a weight map corresponding to each image. The weight map is used to reconstruct key anatomical details from the ultrasound images. The images and their corresponding weight maps are then fused to obtain a target ultrasound image of the target object, which is displayed on a screen. By predicting pixel-level weight maps, differentiated utilization of information from different angles is achieved. For each pixel location, the network determines the optimal angle based on local imaging quality and assigns it the highest weight. This preserves the clearest and most reliable tissue details during the fusion process, ensuring the reliability and accuracy of the fused target ultrasound image and improving the imaging effect of the ultrasound image.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, scanning information of target objects, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An ultrasound image processing method, characterized by, The method comprises: acquiring a plurality of ultrasound images output by an ultrasound device when the ultrasound device performs a plurality of angle scans on a target object; inputting the plurality of ultrasound images into a weight prediction network to obtain a weight map corresponding to each of the ultrasound images; fusing each of the ultrasound images and the corresponding weight map to obtain a target ultrasound image of the target object.

2. The method of claim 1, wherein, The fusing each of the ultrasound images and the corresponding weight map to obtain a target ultrasound image of the target object comprises: performing weighted fusion on each of the ultrasound images and the corresponding weight map to obtain a weighted ultrasound image corresponding to each of the ultrasound images; performing image fusion on all the weighted ultrasound images to obtain the target ultrasound image.

3. The method of claim 1, wherein, The weight prediction network comprises a mapping unit and an output unit, and the inputting the plurality of ultrasound images into the weight prediction network to obtain a weight map corresponding to each of the ultrasound images comprises: inputting the plurality of ultrasound images into the mapping unit to perform mapping on image matrix values to obtain a weight value corresponding to each of the ultrasound images; inputting each of the weight values into the output unit to construct a weight map to obtain a weight map corresponding to each of the ultrasound images.

4. The method of claim 1, wherein, The acquiring a plurality of ultrasound images output by an ultrasound device when the ultrasound device performs a plurality of angle scans on a target object comprises: scanning the target object using the ultrasound device according to a preset acquisition frequency and a plurality of preset acquisition angles to obtain a plurality of candidate ultrasound images; performing spatial registration on all the candidate ultrasound images to obtain the plurality of ultrasound images.

5. The method according to any one of claims 1 to 4, characterized in that, The training process of the weight prediction network comprises: acquiring a sample ultrasound image and a label ultrasound image, and inputting the sample ultrasound image into an initial weight prediction network to obtain a predicted weight map; determining a first loss value according to the predicted weight map, the sample ultrasound image, the label ultrasound image, and a first loss function; determining a second loss value according to the predicted weight map, the sample ultrasound image, the label ultrasound image, and a second loss function; adjusting parameters of the initial weight prediction network according to the first loss value and the second loss value until the first loss value and the second loss value converge, to obtain a trained weight prediction network.

6. The method of claim 5, wherein, The determining a first loss value according to the predicted weight map, the sample ultrasound image, the label ultrasound image, and a first loss function comprises: fusing the predicted weight map and the sample ultrasound image to obtain a sample fusion image; calculating a pixel difference matrix of the sample fusion image and the label ultrasound image based on the first loss function, and performing weighted calculation on the pixel difference matrix based on an edge feature map corresponding to the label ultrasound image to obtain the first loss value.

7. An ultrasound image processing apparatus, characterized by, The device comprises: an ultrasound image acquisition module configured to acquire a plurality of ultrasound images output by an ultrasound device when the ultrasound device performs a plurality of angle scans on a target object; a weight map acquisition module configured to input the plurality of ultrasound images into a weight prediction network to obtain a weight map corresponding to each of the ultrasound images; a fusion module configured to fuse each of the ultrasound images and the corresponding weight map to obtain a target ultrasound image of the target object.

8. An ultrasound device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.

9. A network training method, comprising: The method comprises: obtaining a training sample set, the training sample set comprising a plurality of sample ultrasound image groups and corresponding label ultrasound images of the sample ultrasound image groups, the sample ultrasound image groups containing a second number of sample ultrasound images, the label ultrasound images being generated based on a first number of ultrasound images, the first number being greater than the second number; for each of the sample ultrasound image groups and the corresponding label ultrasound images, inputting the sample ultrasound image groups into an initial weight prediction network to obtain a predicted weight map; fusing the predicted weight map and the sample ultrasound image groups to obtain a sample fused image; performing network training on the initial weight prediction network according to the sample fused image, the label ultrasound image and a loss function to obtain a trained weight prediction network.

10. An ultrasound image processing method, characterized by, The method is applied to an ultrasound device, and the method comprises: scanning a target object according to a preset acquisition frequency and a plurality of preset acquisition angles to obtain a plurality of ultrasound images; inputting each of the ultrasound images into a weight prediction network to obtain a weight map corresponding to each of the ultrasound images output by the weight prediction network; performing weighted fusion on each of the ultrasound images and the weight map to obtain a plurality of weighted ultrasound images; performing image fusion on each of the weighted ultrasound images to obtain a target ultrasound image, and displaying the target ultrasound image on a screen.

Images