Method, device and equipment for enhancing visualization of an ultrasound puncture needle

Abstract

The application discloses a method, device and equipment for enhancing the development of an ultrasonic puncture needle, to solve the problem of poor development effect of the ultrasonic image of the ultrasonic puncture needle in the related art. First, the ultrasonic image of the ultrasonic puncture needle under the current ultrasonic parameter is obtained, the ultrasonic image is taken as the input of the network model, and a plurality of quality parameter evaluation values for representing the needle body development quality of the ultrasonic puncture needle in the ultrasonic image are obtained. Then, the comparison result of the weighted result of the plurality of quality parameter evaluation values and the target quality value is judged. When the comparison result meets the set condition, the iteration adjustment of the ultrasonic parameter is stopped, and the puncture needle region segmentation image under the current ultrasonic parameter is obtained. If the comparison result of the weighted result and the target quality value does not meet the set condition, the iteration adjustment of the ultrasonic parameter is continued until the comparison result of the weighted result of the plurality of quality parameter evaluation values and the target quality value meets the set condition.

Description

Technical Field

[0001] This application relates to the field of medical ultrasound diagnostic technology, and in particular to methods, devices and equipment for enhancing the imaging of ultrasound puncture needles. Background Technology

[0002] Ultrasonic puncture surgery is a common surgical procedure in medicine. However, due to the specific surface properties of the puncture needle, the signal of the needle body in the ultrasound image is not obvious. Therefore, it is necessary to improve the imaging effect of the puncture needle body in puncture surgery, and the imaging of the puncture needle needs to be enhanced.

[0003] In related technologies, ultrasound puncture surgery mainly has two methods: (1) assisted puncture with the help of puncture frame and other guiding tools; (2) manual puncture. Each of these two methods has its own advantages and disadvantages: the first method reduces the flexibility of the puncture needle movement, but the advantage is that it does not require much puncture experience and is suitable for doctors with insufficient clinical experience; conversely, the second method requires doctors to have puncture surgery experience, but the advantage is that it has greater flexibility.

[0004] Considering the actual clinical process, manual puncture is a more suitable ultrasound puncture method. Currently, manual puncture methods mainly rely on the deflection control of the ultrasound emission beam to make the ultrasound emission beam in one frame perpendicular to the puncture needle, thereby improving the imaging effect of the puncture needle. However, since there are many needle-like structures in actual human tissues, even when the ultrasound emission beam is perpendicular to the puncture needle, the accuracy of needle detection may be limited, and the quality of needle imaging may be poor.

[0005] Therefore, the imaging effect of ultrasonic puncture needles in related technologies is poor. Summary of the Invention

[0006] The purpose of this application is to provide a method, apparatus, and device for enhancing the imaging of ultrasonic puncture needles, in order to solve the problem of poor imaging effect of ultrasonic images of ultrasonic puncture needles in related technologies.

[0007] In a first aspect, this application provides a method for enhancing the imaging of an ultrasonic puncture needle, the method comprising:

[0008] Iteratively adjust the ultrasound parameters and obtain the ultrasound image of the ultrasound puncture needle under the currently used ultrasound parameters;

[0009] The ultrasound image is used as input to a network model to obtain multiple quality parameter evaluation values ​​that characterize the imaging quality of the ultrasound puncture needle in the ultrasound image. The network model includes an image segmentation network and a classification network. The image segmentation network is used to detect the puncture needle in the ultrasound image to obtain a segmented image of the puncture needle region. The classification network is used to evaluate the imaging quality of the needle body in the segmented image of the puncture needle region.

[0010] The weighted result of the evaluation values ​​of multiple quality parameters is compared with the target quality value.

[0011] When the comparison result meets the set conditions, the iterative adjustment of the ultrasound parameters stops, and a segmentation image of the puncture needle region under the currently used ultrasound parameters is obtained; or...

[0012] If the comparison between the weighted result and the target quality value does not meet the set conditions, the ultrasound parameters are iteratively adjusted until the weighted result of the multiple quality parameter evaluation values ​​and the comparison result of the target quality value meet the set conditions.

[0013] In one possible implementation, iteratively adjusting the ultrasound parameters includes:

[0014] Based on the evaluation values ​​of multiple quality parameters under the ultrasound parameters used in the current application, a deep reinforcement learning network is used to adjust the ultrasound parameters used in the current application to obtain the adjusted ultrasound parameters.

[0015] The deep reinforcement learning network is trained using the Q-learning algorithm. In the Q-table used during the training of the deep reinforcement learning network using the Q-learning algorithm, the ultrasound parameters are used as actions in the Q-table, and multiple quality parameter evaluation values ​​are used as states in the Q-table. The adjusted ultrasound parameters are the ultrasound parameters corresponding to the maximum Q-value among the multiple quality parameter evaluation values ​​obtained in the current iteration in the Q-table.

[0016] In one possible implementation, the Q-table used in the process of training the deep reinforcement learning network using the Q-learning algorithm is obtained as follows:

[0017] Initialize the Q table;

[0018] During the i-th adjustment of the Q table, the i-th quality parameter evaluation value is selected from multiple sets of quality parameter evaluation values, and the i-th ultrasound parameter is selected from N sets of ultrasound parameters, where N is a positive integer; the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter is used as the input of the network model to obtain the quality parameter evaluation value under the i-th ultrasound parameter.

[0019] The reward item for the evaluation value of the i-th group of quality parameters is determined based on the evaluation value of the i-th group of quality parameters.

[0020] The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs to the deep reinforcement learning network to obtain the i-th group of quality parameter evaluation values ​​in the Q table as the state and the i-th group of ultrasound parameters as the Q value of the action.

[0021] In one possible implementation, the plurality of quality parameters includes at least two of the following: continuity of the ultrasonic puncture needle, contrast level of the ultrasonic puncture needle, brightness level of the ultrasonic puncture needle, or noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is a weighted sum of the plurality of quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0022] In one possible implementation, the deep reinforcement learning network is trained in the following manner;

[0023] During the process of adjusting the Q table, after obtaining the i-th group of quality parameter evaluation values ​​as the state and the N groups of ultrasound parameters as the actions, the first ultrasound parameter corresponding to the largest Q value among the N Q values ​​is obtained.

[0024] Based on the first quality parameter evaluation value output by the network model when the first ultrasound parameter is taken as input, and the reward item of the i-th group of quality parameter evaluation values, the Bellman algorithm is used to determine the Q target value corresponding to the first ultrasound parameter.

[0025] The evaluation value of the i-th group of quality parameters and the reward item of the evaluation value of the i-th group of quality parameters are used as a piece of empirical data, and the Q target value corresponding to the first ultrasound parameter is used as a label and stored in the empirical data pool.

[0026] n empirical data points are randomly selected from the empirical data pool as training datasets, where n is a positive integer. The training datasets are then input into the deep reinforcement learning network to train the network with the goal of outputting the corresponding Q target value.

[0027] For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q prediction value under the first ultrasound parameters output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated using gradient descent.

[0028] In one possible implementation, based on the plurality of quality parameter evaluation values, a deep reinforcement learning network is used to adjust the currently used ultrasound parameters to obtain adjusted ultrasound parameters, including:

[0029] The reward items for the multiple quality parameter evaluation values ​​are determined based on the current ultrasound parameters used.

[0030] The Q-values ​​under the multiple quality parameter evaluation values ​​and the reward terms of the multiple quality parameter evaluation values ​​are used as inputs to the deep reinforcement learning network to obtain the Q-values ​​under the multiple quality parameter evaluation values.

[0031] The ultrasound parameters corresponding to the Q value under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasound parameters.

[0032] In one possible implementation, the loss function value satisfies the condition shown in the following formula:

[0033]

[0034] Where L represents the loss function value, Q target Let Q be the target value, and Q(ss,aa) be the predicted Q value output by the deep reinforcement learning network.

[0035] Secondly, this application provides a contrast enhancement device for an ultrasonic puncture needle, the device comprising:

[0036] The ultrasound image acquisition module is configured to iteratively adjust ultrasound parameters and acquire ultrasound images of the ultrasound puncture needle under the currently used ultrasound parameters.

[0037] An ultrasound image processing module is configured to take the ultrasound image as input to a network model to obtain multiple quality parameter evaluation values ​​characterizing the imaging quality of the ultrasound puncture needle in the ultrasound image; the network model includes an image segmentation network and a classification network, wherein the image segmentation network is used to detect the puncture needle in the ultrasound image to obtain a segmented image of the puncture needle region; and the classification network is used to evaluate the imaging quality of the puncture needle region segmented image.

[0038] The ultrasound image evaluation module is configured to compare the weighted result of multiple quality parameter evaluation values ​​with the target quality value.

[0039] The puncture needle region segmentation image determination module is configured to stop iteratively adjusting the ultrasound parameters and obtain the puncture needle region segmentation image under the currently used ultrasound parameters when the comparison result meets the set conditions; or...

[0040] The ultrasound parameter optimization module is configured to iteratively adjust the ultrasound parameters if the comparison result between the weighted result and the target quality value does not meet the set conditions, until the weighted result of the obtained multiple quality parameter evaluation values ​​and the comparison result between the target quality value meet the set conditions.

[0041] In one possible implementation, the ultrasound parameters are iteratively adjusted, and the ultrasound parameter optimization module is configured as follows:

[0042] Based on the evaluation values ​​of multiple quality parameters under the ultrasound parameters used in the current application, a deep reinforcement learning network is used to adjust the ultrasound parameters used in the current application to obtain the adjusted ultrasound parameters.

[0043] The deep reinforcement learning network is trained using the Q-learning algorithm. In the Q-table used during the training of the deep reinforcement learning network using the Q-learning algorithm, the ultrasound parameters are used as actions in the Q-table, and multiple quality parameter evaluation values ​​are used as states in the Q-table. The adjusted ultrasound parameters are the ultrasound parameters corresponding to the maximum Q-value among the multiple quality parameter evaluation values ​​obtained in the current iteration in the Q-table.

[0044] In one possible implementation, the Q-table used by the ultrasound parameter optimization module in training the deep reinforcement learning network using the Q-learning algorithm is obtained as follows:

[0045] Initialize the Q table;

[0046] During the i-th adjustment of the Q table, the i-th quality parameter evaluation value is selected from multiple sets of quality parameter evaluation values, and the i-th ultrasound parameter is selected from N sets of ultrasound parameters, where N is a positive integer; the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter is used as the input of the network model to obtain the quality parameter evaluation value under the i-th ultrasound parameter.

[0047] The reward item for the evaluation value of the i-th group of quality parameters is determined based on the evaluation value of the i-th group of quality parameters.

[0048] The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs to the deep reinforcement learning network to obtain the i-th group of quality parameter evaluation values ​​in the Q table as the state and the i-th group of ultrasound parameters as the Q value of the action.

[0049] In one possible implementation, the plurality of quality parameters includes at least two of the following: continuity of the ultrasonic puncture needle, contrast level of the ultrasonic puncture needle, brightness level of the ultrasonic puncture needle, or noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is a weighted sum of the plurality of quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0050] In one possible implementation, the ultrasound parameter optimization module is configured to train the deep reinforcement learning network in the following manner;

[0051] During the process of adjusting the Q table, after obtaining the i-th group of quality parameter evaluation values ​​as the state and the N groups of ultrasound parameters as the actions, the first ultrasound parameter corresponding to the largest Q value among the N Q values ​​is obtained.

[0052] Based on the first quality parameter evaluation value output by the network model when the first ultrasound parameter is taken as input, and the reward item of the i-th group of quality parameter evaluation values, the Bellman algorithm is used to determine the Q target value corresponding to the first ultrasound parameter.

[0053] The evaluation value of the i-th group of quality parameters and the reward item of the evaluation value of the i-th group of quality parameters are used as a piece of empirical data, and the Q target value corresponding to the first ultrasound parameter is used as a label and stored in the empirical data pool.

[0054] n empirical data points are randomly selected from the empirical data pool as training datasets, where n is a positive integer. The training datasets are then input into the deep reinforcement learning network to train the network with the goal of outputting the corresponding Q target value.

[0055] For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q prediction value under the first ultrasound parameters output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated using gradient descent.

[0056] In one possible implementation, based on the plurality of quality parameter evaluation values, a deep reinforcement learning network is used to adjust the currently used ultrasound parameters to obtain adjusted ultrasound parameters. The ultrasound parameter optimization module is configured to:

[0057] The reward items for the multiple quality parameter evaluation values ​​are determined based on the current ultrasound parameters used.

[0058] The Q-values ​​under the multiple quality parameter evaluation values ​​and the reward terms of the multiple quality parameter evaluation values ​​are used as inputs to the deep reinforcement learning network to obtain the Q-values ​​under the multiple quality parameter evaluation values.

[0059] The ultrasound parameters corresponding to the Q value under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasound parameters.

[0060] In one possible implementation, the loss function value satisfies the condition shown in the following formula:

[0061]

[0062] Where L represents the loss function value, Q targetLet Q be the target value, and Q(ss,aa) be the predicted Q value output by the deep reinforcement learning network.

[0063] Thirdly, this application provides an electronic device, comprising:

[0064] Processor and memory;

[0065] The memory is used to store the processor-executable instructions;

[0066] The processor is configured to execute the instructions to implement the ultrasound puncture needle imaging enhancement method as provided in any of the first aspects of this application.

[0067] Fourthly, this application provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the ultrasound puncture needle imaging enhancement method as described in any of the first aspects of this application.

[0068] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the ultrasound puncture needle imaging enhancement method as described in any of the claims in the first aspect of this application.

[0069] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:

[0070] In this application, a neural network model is used to detect the needle body area of ​​the ultrasound puncture needle in the ultrasound image in real time and obtain quality parameter evaluation values. Then, a deep reinforcement learning network is used to continuously correct the ultrasound parameters, thereby optimizing the ultrasound parameters and achieving the best puncture needle imaging effect. This greatly improves the accuracy of ultrasound puncture, increases the efficiency of doctors' work, and enhances the patient's user experience.

[0071] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Based on common knowledge in the art, the above preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this application. Attached Figure Description

[0072] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 A schematic diagram illustrating an application scenario of the imaging enhancement method for ultrasonic puncture needles provided in this application embodiment;

[0074] Figure 2 A schematic diagram of the overall process of the ultrasound puncture needle imaging enhancement method provided in the embodiments of this application;

[0075] Figure 3 A flowchart illustrating the training Q-table provided in an embodiment of this application;

[0076] Figure 4 A schematic diagram illustrating the process of training a deep reinforcement learning network as provided in an embodiment of this application;

[0077] Figure 5 A schematic diagram of the imaging enhancement device for the ultrasonic puncture needle provided in the embodiments of this application;

[0078] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0080] Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0081] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0082] The following explains the technical terms and technologies involved in this application:

[0083] Reinforcement learning, also known as reward learning, evaluation learning, or reinforcement learning, is a machine learning method that lies between supervised and unsupervised learning. It describes and solves problems by describing how an agent learns strategies to maximize rewards or achieve specific goals through interaction with its environment. Reinforcement learning does not require any pre-provided data; instead, it learns and updates model parameters by receiving rewards (feedback) from the environment for actions. For example, receiving feedback from an ultrasound device adjusting a parameter, the quality of the resulting ultrasound image is scored, and the score serves as a reward or penalty for that action, thus adjusting the parameter.

[0084] The imaging enhancement method for the ultrasonic puncture needle provided in this application will be described below with reference to embodiments.

[0085] The inventive concept of this application can be summarized as follows: First, an ultrasound image of the ultrasound puncture needle under the currently used ultrasound parameters is acquired. This ultrasound image is used as input to a network model to obtain multiple quality parameter evaluation values ​​that characterize the imaging quality of the ultrasound puncture needle in the ultrasound image. Then, the weighted result of the multiple quality parameter evaluation values ​​is compared with the target quality value. When the comparison result meets the set conditions, the iterative adjustment of the ultrasound parameters is stopped, and a puncture needle region segmentation image under the currently used ultrasound parameters is obtained. If the comparison result of the weighted result and the target quality value does not meet the set conditions, the ultrasound parameters are iteratively adjusted until the comparison result of the weighted result of the multiple quality parameter evaluation values ​​and the target quality value meets the set conditions.

[0086] In summary, the ultrasound puncture needle imaging enhancement method provided in this application uses a neural network model to detect the needle body area of ​​the ultrasound puncture needle in the ultrasound image in real time and obtain quality parameter evaluation values. Then, it continuously corrects the ultrasound parameters through a deep reinforcement learning network, thereby optimizing the ultrasound parameters and achieving the best puncture needle imaging effect. This greatly improves the accuracy of ultrasound puncture, increases the efficiency of doctors' work, and enhances the patient's user experience.

[0087] After introducing the main inventive concepts of the embodiments of this application, the following is a brief description of the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0088] For ease of understanding, the imaging enhancement method of an ultrasonic puncture needle provided in this application will be described in detail below with reference to the accompanying drawings:

[0089] like Figure 1The diagram illustrates an application scenario of an ultrasound puncture needle imaging enhancement method provided in this embodiment of the application. The diagram includes: a network 10, a server 20, and a memory 30. The server 20 acquires ultrasound images via the network and processes these images using the method provided in this embodiment. If the imaging effect of the ultrasound image does not meet the requirements, the ultrasound parameters are iteratively processed until the imaging effect of the ultrasound image produced based on the iteratively processed ultrasound parameters meets the requirements.

[0090] The description in this application focuses on a single server only. However, those skilled in the art should understand that the illustrated network 10, server 20, and memory 30 are intended to illustrate the operation of the server and memory involved in the technical solutions of this application. The detailed description of a single server and memory is at least for ease of explanation and does not imply any limitation on the number, type, or location of servers. It should be noted that adding additional modules to or removing individual modules from the illustrated environment will not change the underlying concept of the exemplary embodiments of this application. Furthermore, although detailed descriptions are provided for ease of explanation... Figure 1 The diagram shows a bidirectional arrow from memory 30 to server 20, but those skilled in the art will understand that the sending and receiving of the aforementioned data also needs to be achieved through network 10.

[0091] It should be noted that the memory in the embodiments of this application can be, for example, a cache system, hard disk storage, memory storage, etc. Furthermore, the method for processing optically dispersed data proposed in this application is not only applicable to… Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of this application do not impose limitations. Figure 1 The functions that each device in the application scenario shown can achieve will be described in subsequent method embodiments, and will not be elaborated on here.

[0092] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, the method may include more or fewer operation steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application.

[0093] Based on the above description, this application provides a method for enhancing the imaging of an ultrasonic puncture needle, the overall process of which is as follows: Figure 2 As shown, it includes the following:

[0094] In step 201, the ultrasound parameters are iteratively adjusted, and an ultrasound image of the ultrasound puncture needle under the currently used ultrasound parameters is obtained.

[0095] In step 202, the ultrasound image is used as input to the network model to obtain evaluation values ​​of multiple quality parameters used to characterize the imaging quality of the ultrasound puncture needle in the ultrasound image. The network model includes an image segmentation network and a classification network. The image segmentation network is used to detect the puncture needle in the ultrasound image to obtain a segmented image of the puncture needle region. The classification network is used to evaluate the imaging quality of the puncture needle in the segmented image of the puncture needle region.

[0096] It should be noted that ultrasound parameters consist of multiple parameters; in this embodiment, a = [θ] is used. t d t ,re t ] represents ultrasound parameters, where θ t θ is the scanning angle of the ultrasonic emission line. t ∈[20, 21...50], d t d is the emission aperture of the ultrasonic emission line. t ∈[33...192], re t The receiving aperture of the ultrasonic receiving line is re. t ∈[2...192].

[0097] Furthermore, the embodiments of this application also employ Several quality parameters representing the quality of needle imaging are used for evaluation, where c t For the continuity of the ultrasonic puncture needle, c t ∈[1, 2, ... 100], σ t σ represents the contrast rating of the ultrasonic puncture needle. t ∈[1, 2, ... 100], This refers to the brightness level of the ultrasonic puncture needle. ρ t The noise level of the ultrasonic puncture needle, ρ t ∈[1,2,...100].

[0098] In one possible implementation, the above-described network models are all trained models. The embodiments of this application train the above-described network models using the following method:

[0099] For image segmentation networks, perform the following steps:

[0100] Obtain training samples, which include sample ultrasound images and ultrasound puncture needles in the labeled ultrasound images;

[0101] The sample ultrasound image is input into an image segmentation network for feature extraction to obtain a feature image. The feature image is then processed for ultrasound puncture needle detection and recognition to obtain the ultrasound puncture needle recognition result.

[0102] The obtained ultrasonic puncture needle identification results are compared with the labeled ultrasonic puncture needles to determine the loss function value. Based on the loss function value, the network parameters of the convolutional neural network model are updated.

[0103] It should be added that, before inputting the sample ultrasound images into the image segmentation network, this embodiment preprocesses the sample ultrasound images. Preprocessing includes cropping the original images and sequentially locating and labeling the puncture needle in each image sequence of the original medical image sequence group, thereby constructing the labeled training sample images required for training the image segmentation network. Furthermore, this application also expands and enhances the ultrasound images in the training set: by filling, cropping, and scaling the image sequences, placing the labeled regions at the center of the image, and adjusting brightness and / or contrast and / or grayscale parameters, image features are better extracted, new images are calculated and generated, and added to the training set.

[0104] For classification networks, perform the following steps:

[0105] Obtain training samples, which include the segmented image of the puncture needle region and multiple quality labels in the labeled segmented image of the puncture needle region.

[0106] The sample puncture needle region segmentation image is input into a classification network for feature extraction processing to obtain a feature image. The feature image is then subjected to quality assessment processing to obtain multiple quality parameter evaluation values ​​for the imaging quality of the ultrasonic puncture needle.

[0107] The obtained evaluation values ​​of multiple quality parameters are compared with the labeled multiple quality labels to determine the loss function value. Based on the loss function value, the network parameters of the classification network are updated.

[0108] In step 203, the weighted result of the evaluation values ​​of multiple quality parameters is compared with the target quality value.

[0109] In step 204, when the comparison result meets the set conditions, the iterative adjustment of ultrasound parameters is stopped, and the segmentation image of the puncture needle region under the ultrasound parameters used in the current iteration is obtained.

[0110] In step 205, if the comparison between the weighted result and the target quality value does not meet the set conditions, the ultrasound parameters are iteratively adjusted until the comparison between the weighted result of the multiple quality parameter evaluation values ​​and the target quality value meets the set conditions. In one possible implementation, this application evaluates ultrasound images on a percentage scale, determining that an ultrasound image with a score of 90 or higher is a good imaging effect. Moreover, the set condition is that the ratio of the weighted result of the multiple quality parameter evaluation values ​​to the target quality value is greater than a specified threshold. For example, if the target quality value is set to 90 and the specified threshold is set to 90%, if the weighted result of the multiple quality parameter evaluation values ​​is 80, it is determined that the comparison between the weighted result of the multiple quality parameter evaluation values ​​and the target quality value does not meet the set conditions, and the ultrasound parameters are iteratively adjusted. If the weighted result of the multiple quality parameter evaluation values ​​is 85, it is determined that the comparison between the weighted result of the multiple quality parameter evaluation values ​​and the target quality value meets the set conditions, the iterative adjustment of the ultrasound parameters is stopped, and the puncture needle region segmentation image under the currently used ultrasound parameters is obtained.

[0111] In related technologies, reinforcement learning methods can obtain the optimal strategy for the next action based on the current state and feedback reward. In this application, the deep reinforcement learning network can obtain the optimal emission line scanning angle and related parameters (i.e., ultrasound parameters) for the next step based on the current state (i.e., the evaluation values ​​of multiple quality parameters of the ultrasound puncture needle's imaging quality) and the current feedback reward, so that the imaging quality of the puncture needle reaches the optimal level.

[0112] Reinforcement learning networks use the Q-learning algorithm to estimate the optimal action for the next state. The training process of the Q-learning algorithm is the process of gradually updating the Q-values ​​of the Q-table, where the Q-value can be understood as quality, representing the optimal action in the current state S. t Take a certain action a t The maximum cumulative reward value R obtained in the next moment. t+1 In the following formula (1), Q(S) t a t () represents a function, and the value of this function is the Q-value. Different states s and actions a will each correspond to a Q-value, forming a multidimensional Q-table:

[0113] Q(S t a t ) = maxR t+1 (1)

[0114] In the following formula (2), π(s) represents the optimal strategy in the current state s, which is to find the next action a that maximizes the Q value of the Q table:

[0115] π(s)=argmax(Q(s,t)) (2)

[0116] The deep reinforcement learning network provided in this application uses the Bellman formula to calculate the Q-value of the Q-table:

[0117]

[0118] Where s is the current state, a is the action corresponding to the current state, Q(s, a) is the Q value after taking action a in the current state, and r is the reward for the current state. Let α be the state at the next moment, α be the optimal action corresponding to the state at the next moment, and γ be the reward discount factor, representing the weight of the influence of the Q value at the next moment on the current Q value after taking the optimal action α in the current state.

[0119] Based on the above description of reinforcement learning, in this embodiment, the Q-table used in the process of training the deep reinforcement learning network using the Q-learning algorithm to iteratively adjust the ultrasound parameters can be implemented as follows: Based on multiple quality parameter evaluation values ​​under the currently used ultrasound parameters, the deep reinforcement learning network adjusts the ultrasound parameters to obtain the adjusted ultrasound parameters. Here, the deep reinforcement learning network is trained using the Q-learning algorithm; in the Q-table used in the process of training the deep reinforcement learning network using the Q-learning algorithm, the ultrasound parameters are treated as actions in the Q-table, and the multiple quality parameter evaluation values ​​are treated as states in the Q-table; the adjusted ultrasound parameters are the ultrasound parameters corresponding to the maximum Q-value among the multiple quality parameter evaluation values ​​obtained in the current iteration in the Q-table.

[0120] In one possible implementation, the Q-table used in the process of training the deep reinforcement learning network using the Q-learning algorithm is obtained as follows: Figure 3 As shown, it includes the following steps:

[0121] In step 301, the Q table is initialized.

[0122] In step 302, during the i-th adjustment of the Q table, the i-th set of quality parameter evaluation values ​​is selected from multiple sets of quality parameter evaluation values, and the i-th set of ultrasound parameters is selected from N sets of ultrasound parameters, where N is a positive integer; the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter is used as the input of the network model to obtain the quality parameter evaluation value under the i-th set of ultrasound parameters.

[0123] In step 303, the reward item for the i-th group of quality parameter evaluation value is determined based on the i-th group of quality parameter evaluation value.

[0124] In step 304, the evaluation value of the i-th group of quality parameters and the reward item are used as inputs to the deep reinforcement learning network to obtain the evaluation value of the i-th group of quality parameters in the Q table as the state and the i-th group of ultrasound parameters as the Q value of the action.

[0125] In one possible implementation, the multiple quality parameters include at least two of the following: the continuity of the ultrasonic puncture needle, the contrast rating of the ultrasonic puncture needle, the brightness rating of the ultrasonic puncture needle, or the noise rating of the ultrasonic puncture needle. The reward for the i-th set of quality parameter evaluation values ​​is a weighted sum of the multiple quality parameter evaluation values ​​in the i-th set of quality parameter evaluation values.

[0126] The embodiment of this application uses the following formula (4) to determine the reward item for the evaluation value of the i-th group of quality parameters:

[0127]

[0128] Among them, c t For the continuity of the ultrasonic puncture needle, σ t This refers to the contrast rating of the ultrasonic puncture needle. The brightness level of the ultrasonic puncture needle, ρ t α represents the noise level of the ultrasonic puncture needle, and α, β, θ, and δ represent the weights corresponding to the quality parameter evaluation values.

[0129] It should be added that the weighted result of multiple quality parameter evaluation values ​​and the weighted sum of multiple quality parameter evaluation values ​​in the reward item may have the same or different weights, depending on the requirements.

[0130] In this application, the Q table uses ultrasound parameters as actions and multiple quality parameter evaluation values ​​as states. Since there is a large amount of data on states and actions, for example, the total number of states is 100. 4 The total number of actions is 192*160*31. In practical applications, the Q-table may be larger than this. Therefore, this embodiment cannot obtain the optimal action taken to maximize the Q-value of the Q-table under different states by looking up the table. If the states or actions are consecutive, the Q-learning algorithm cannot be used. Therefore, this embodiment uses a deep reinforcement learning network to fit the Q(s,a) table, which can help the Q-algorithm find the optimal action in the corresponding state, i.e., DQN (Deep Q Networks).

[0131] It should be noted that during an ultrasound puncture needle puncture, the difference between the previous state and the next state is very small and has a certain correlation. They are not independently identically distributed. If a traditional DQN network is used directly, overfitting will occur, and the trained network will lack generalization ability. Therefore, this embodiment uses an experience-based DQN network for training. By using a buffer to replay past experience information, past experience and current experience are mixed, reducing data correlation. Furthermore, experience replay also makes samples reusable, thereby improving learning efficiency.

[0132] In one possible implementation, embodiments of this application train a deep reinforcement learning network in the following manner, the process of which is as follows: Figure 4 As shown, it includes the following steps:

[0133] In step 401, during the process of adjusting the Q table, after obtaining the evaluation value of the i-th group of quality parameters as the state and the N groups of ultrasound parameters as the action, the first ultrasound parameter corresponding to the largest Q value among the N Q values ​​is obtained.

[0134] In step 402, based on the first quality parameter evaluation value output by the network model when the first ultrasound parameter is used as input, and the reward item of the i-th group of quality parameter evaluation values, the Bellman algorithm is used to determine the Q target value corresponding to the first ultrasound parameter.

[0135] In step 403, the evaluation value of the i-th group of quality parameters and the reward item of the evaluation value of the i-th group of quality parameters are used as a piece of empirical data, and the Q target value corresponding to the first ultrasound parameter is used as a label and stored in the empirical data pool.

[0136] In step 404, n empirical data points are randomly selected from the empirical data pool as the training dataset, where n is a positive integer. The training dataset is then input into the deep reinforcement learning network to train the network with the goal of outputting the corresponding Q target value.

[0137] In step 405, for each piece of empirical data, the loss function value is obtained based on the corresponding Q target value and the Q prediction value under the first ultrasound parameters output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated using gradient descent.

[0138] For example, the specific process of training a deep reinforcement learning network is as follows:

[0139] 1. Initialize the experience data pool, initialize the network parameters of the deep reinforcement learning network, such as the network parameter w of network Q(ss, aa, w), initialize the reward discount factor in the Bellman formula, initialize the Q objective value to 0, and use the experience data pool to store procedural experience data.

[0140] 2. During the adjustment of the Q table, after obtaining the evaluation value of the i-th group of quality parameters as the state and the N groups of ultrasound parameters as the action, obtain the first ultrasound parameter corresponding to the largest Q value among the N Q values.

[0141] 3. Determine the reward item for the i-th group of quality parameter evaluation value based on the evaluation value of the i-th group of quality parameters.

[0142] 4. Take the evaluation value of the i-th group of quality parameters and the reward item of the evaluation value of the i-th group of quality parameters as an empirical data point, and use the Q target value corresponding to the first ultrasound parameter as a label to store it in the empirical data pool.

[0143] It should be noted that, in the embodiments of this application, the evaluation values ​​of the i-th group of quality parameters, the evaluation values ​​of the (i+1)-th group of quality parameters, the i-th group of ultrasound parameters, and the reward item {s} can also be included. t a t r t s t+1 The process experience data is stored in the experience data pool. That is, the process experience data does not include the Q target value as a label. The Q target value is calculated later when the process experience data is used to train the deep reinforcement learning network.

[0144] 5. Randomly select n empirical data points {ss} from the empirical data pool. t ,aa t rr t ss t+1} is the training dataset, where n is a positive integer. The training dataset is input into the deep reinforcement learning network to train it with the output Q target value as the objective.

[0145] 6. For each piece of empirical data, a loss function value is obtained based on the Q-target value and the predicted value output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated using gradient descent. In one possible implementation, the loss function value satisfies the condition shown in the following formula:

[0146]

[0147] Where L represents the loss function value, Q target Let Q be the target value, and Q(ss,aa) be the predicted Q value of the deep reinforcement learning network output.

[0148] In one possible implementation, after obtaining a trained deep reinforcement learning network, the adjusted ultrasound parameters are obtained by using the deep reinforcement learning network to adjust the ultrasound parameters for the current application based on multiple quality parameter evaluation values. This can be implemented as follows:

[0149] The reward items for multiple quality parameter evaluation values ​​are determined based on the current ultrasound parameters used.

[0150] The Q-values ​​under multiple quality parameter evaluations and the reward terms of the multiple quality parameter evaluations are used as inputs to a deep reinforcement learning network to obtain the Q-values ​​under multiple quality parameter evaluations.

[0151] The ultrasound parameters corresponding to the Q value under multiple quality parameter evaluation values ​​are used as the adjusted ultrasound parameters.

[0152] In summary, the ultrasound puncture needle imaging enhancement method provided in this application uses a neural network model to detect the needle body area of ​​the ultrasound puncture needle in the ultrasound image in real time and obtain quality parameter evaluation values. Then, a deep reinforcement learning network is used to continuously correct the ultrasound parameters, thereby optimizing the ultrasound parameters and achieving the best imaging effect of the puncture needle. This greatly improves the accuracy of ultrasound puncture, increases the efficiency of doctors' work, and enhances the user experience for patients.

[0153] Based on the same inventive concept, this application also provides a imaging enhancement device for an ultrasonic puncture needle, such as... Figure 5 As shown, the device 500 includes:

[0154] The ultrasound image acquisition module 501 is configured to iteratively adjust ultrasound parameters and acquire ultrasound images of the ultrasound puncture needle under the currently used ultrasound parameters.

[0155] The ultrasound image processing module 502 is configured to use the ultrasound image as input to a network model to obtain multiple quality parameter evaluation values ​​characterizing the imaging quality of the ultrasound puncture needle in the ultrasound image; the network model includes an image segmentation network and a classification network, wherein the image segmentation network is used to detect the puncture needle in the ultrasound image to obtain a puncture needle region segmentation image; and the classification network is used to evaluate the imaging quality of the puncture needle region segmentation image.

[0156] The ultrasound image evaluation module 503 is configured to compare the weighted result of multiple quality parameter evaluation values ​​with the target quality value.

[0157] The puncture needle region segmentation image determination module 504 is configured to stop iteratively adjusting the ultrasound parameters and obtain a puncture needle region segmentation image under the currently used ultrasound parameters when the comparison result meets a set condition; or...

[0158] The ultrasound parameter optimization module 505 is configured to continue iteratively adjusting the ultrasound parameters if the comparison result between the weighted result and the target quality value does not meet the set conditions, until the weighted result of the obtained multiple quality parameter evaluation values ​​and the comparison result between the target quality value meet the set conditions.

[0159] In one possible implementation, the ultrasound parameters are iteratively adjusted, and the ultrasound parameter optimization module is configured as follows:

[0160] Based on the evaluation values ​​of multiple quality parameters under the ultrasound parameters used in the current application, a deep reinforcement learning network is used to adjust the ultrasound parameters used in the current application to obtain the adjusted ultrasound parameters.

[0161] The deep reinforcement learning network is trained using the Q-learning algorithm. In the Q-table used during the training of the deep reinforcement learning network using the Q-learning algorithm, the ultrasound parameters are used as actions in the Q-table, and multiple quality parameter evaluation values ​​are used as states in the Q-table. The adjusted ultrasound parameters are the ultrasound parameters corresponding to the maximum Q-value among the multiple quality parameter evaluation values ​​obtained in the current iteration in the Q-table.

[0162] In one possible implementation, the Q-table used by the ultrasound parameter optimization module in training the deep reinforcement learning network using the Q-learning algorithm is obtained as follows:

[0163] Initialize the Q table;

[0164] During the i-th adjustment of the Q table, the i-th quality parameter evaluation value is selected from multiple sets of quality parameter evaluation values, and the i-th ultrasound parameter is selected from N sets of ultrasound parameters, where N is a positive integer; the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter is used as the input of the network model to obtain the quality parameter evaluation value under the i-th ultrasound parameter.

[0165] The reward item for the evaluation value of the i-th group of quality parameters is determined based on the evaluation value of the i-th group of quality parameters.

[0166] The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs to the deep reinforcement learning network to obtain the i-th group of quality parameter evaluation values ​​in the Q table as the state and the i-th group of ultrasound parameters as the Q value of the action.

[0167] In one possible implementation, the plurality of quality parameters includes at least two of the following: continuity of the ultrasonic puncture needle, contrast level of the ultrasonic puncture needle, brightness level of the ultrasonic puncture needle, or noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is a weighted sum of the plurality of quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0168] In one possible implementation, the ultrasound parameter optimization module is configured to train the deep reinforcement learning network in the following manner;

[0169] During the process of adjusting the Q table, after obtaining the i-th group of quality parameter evaluation values ​​as the state and the N groups of ultrasound parameters as the actions, the first ultrasound parameter corresponding to the largest Q value among the N Q values ​​is obtained.

[0170] Based on the first quality parameter evaluation value output by the network model when the first ultrasound parameter is taken as input, and the reward item of the i-th group of quality parameter evaluation values, the Bellman algorithm is used to determine the Q target value corresponding to the first ultrasound parameter.

[0171] The evaluation value of the i-th group of quality parameters and the reward item of the evaluation value of the i-th group of quality parameters are used as a piece of empirical data, and the Q target value corresponding to the first ultrasound parameter is used as a label and stored in the empirical data pool.

[0172] n empirical data points are randomly selected from the empirical data pool as training datasets, where n is a positive integer. The training datasets are then input into the deep reinforcement learning network to train the network with the goal of outputting the corresponding Q target value.

[0173] For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q prediction value under the first ultrasound parameters output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated using gradient descent.

[0174] In one possible implementation, based on the plurality of quality parameter evaluation values, a deep reinforcement learning network is used to adjust the currently used ultrasound parameters to obtain adjusted ultrasound parameters. The ultrasound parameter optimization module is configured to:

[0175] The reward items for the multiple quality parameter evaluation values ​​are determined based on the current ultrasound parameters used.

[0176] The Q-values ​​under the multiple quality parameter evaluation values ​​and the reward terms of the multiple quality parameter evaluation values ​​are used as inputs to the deep reinforcement learning network to obtain the Q-values ​​under the multiple quality parameter evaluation values.

[0177] The ultrasound parameters corresponding to the Q value under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasound parameters. In one possible implementation, the loss function value satisfies the condition shown in the following formula:

[0178]

[0179] Where L represents the loss function value, Q target Let Q be the target value, and Q(ss,aa) be the predicted Q value output by the deep reinforcement learning network.

[0180] The following reference Figure 6 To describe an electronic device 130 according to this embodiment of the present application. Figure 6 The electronic device 130 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0181] like Figure 6 As shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).

[0182] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0183] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0184] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0185] Electronic device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 130, and / or with any device that enables electronic device 130 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, electronic device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in electronic device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0186] In an exemplary embodiment, this application also provides a computer-readable storage medium including instructions, such as a memory 132 including instructions, which can be executed by a processor 131 of an electronic device 130 to complete the ultrasound puncture needle imaging enhancement method described above. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0187] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor 131, implements the ultrasound puncture needle imaging enhancement method as provided in this application.

[0188] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for enhancing the imaging of an ultrasonic puncture needle, characterized in that, The method includes: Iteratively adjust the ultrasound parameters and obtain the ultrasound image of the ultrasound puncture needle under the currently used ultrasound parameters; The ultrasound image is used as input to a network model to obtain multiple quality parameter evaluation values ​​that characterize the imaging quality of the ultrasound puncture needle in the ultrasound image. The network model includes an image segmentation network and a classification network. The image segmentation network is used to detect the puncture needle in the ultrasound image to obtain a segmented image of the puncture needle region. The classification network is used to evaluate the imaging quality of the needle body in the segmented image of the puncture needle region. The weighted result of the evaluation values ​​of multiple quality parameters is compared with the target quality value. When the comparison result meets the set conditions, the iterative adjustment of the ultrasound parameters stops, and a segmentation image of the puncture needle region under the currently used ultrasound parameters is obtained; or... If the comparison between the weighted result and the target quality value does not meet the set conditions, the ultrasound parameters are iteratively adjusted until the comparison between the weighted result of the multiple quality parameter evaluation values ​​and the target quality value meets the set conditions. Iterative adjustment of the ultrasound parameters includes: Based on the evaluation values ​​of multiple quality parameters under the ultrasound parameters used in the current application, a deep reinforcement learning network is used to adjust the ultrasound parameters used in the current application to obtain the adjusted ultrasound parameters. The deep reinforcement learning network is trained using the Q-learning algorithm. In the Q-table used during the training of the deep reinforcement learning network using the Q-learning algorithm, the ultrasound parameters are used as actions in the Q-table, and multiple quality parameter evaluation values ​​are used as states in the Q-table. The adjusted ultrasound parameters are the ultrasound parameters corresponding to the maximum Q-value among the multiple quality parameter evaluation values ​​obtained in the current iteration in the Q-table.

2. The method according to claim 1, characterized in that, The Q-table used in training the deep reinforcement learning network using the Q-learning algorithm is obtained as follows: Initialize the Q table; During the i-th adjustment of the Q table, the i-th quality parameter evaluation value is selected from multiple sets of quality parameter evaluation values, and the i-th ultrasound parameter is selected from N sets of ultrasound parameters, where N is a positive integer; the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter is used as the input of the network model to obtain the quality parameter evaluation value under the i-th ultrasound parameter. The reward item for the evaluation value of the i-th group of quality parameters is determined based on the evaluation value of the i-th group of quality parameters. The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs to the deep reinforcement learning network to obtain the i-th group of quality parameter evaluation values ​​in the Q table as the state and the i-th group of ultrasound parameters as the Q value of the action.

3. The method according to claim 2, characterized in that, The multiple quality parameters include at least two of the following: continuity of the ultrasonic puncture needle, contrast level of the ultrasonic puncture needle, brightness level of the ultrasonic puncture needle, or noise level of the ultrasonic puncture needle. The reward for the i-th group of quality parameter evaluation values ​​is the weighted sum of the multiple quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

4. The method according to claim 2 or 3, characterized in that, The deep reinforcement learning network is trained in the following manner; During the process of adjusting the Q table, after obtaining the i-th group of quality parameter evaluation values ​​as the state and the N groups of ultrasound parameters as the actions, the first ultrasound parameter corresponding to the largest Q value among the N Q values ​​is obtained. Based on the first quality parameter evaluation value output by the network model when the first ultrasound parameter is taken as input, and the reward item of the i-th group of quality parameter evaluation values, the Bellman algorithm is used to determine the Q target value corresponding to the first ultrasound parameter. The evaluation value of the i-th group of quality parameters and the reward item of the evaluation value of the i-th group of quality parameters are used as a piece of empirical data, and the Q target value corresponding to the first ultrasound parameter is used as a label and stored in the empirical data pool. n empirical data points are randomly selected from the empirical data pool as training datasets, where n is a positive integer. The training datasets are then input into the deep reinforcement learning network to train the network with the goal of outputting the corresponding Q target value. For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q prediction value under the first ultrasound parameters output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated using gradient descent.

5. The method according to claim 4, characterized in that, Based on the multiple quality parameter evaluation values, a deep reinforcement learning network is used to adjust the ultrasound parameters for the current application, resulting in adjusted ultrasound parameters, including: The reward items for the multiple quality parameter evaluation values ​​are determined based on the current ultrasound parameters used. The Q-values ​​under the multiple quality parameter evaluation values ​​and the reward terms of the multiple quality parameter evaluation values ​​are used as inputs to the deep reinforcement learning network to obtain the Q-values ​​under the multiple quality parameter evaluation values. The ultrasound parameters corresponding to the Q value under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasound parameters.

6. The method according to claim 4, characterized in that, The loss function value satisfies the condition shown in the following formula: Where L represents the value of the loss function, Let Q be the target value. This is the Q-predicted value of the output of the deep reinforcement learning network.

7. A contrast-enhancing device for an ultrasonic puncture needle, characterized in that, The device includes: The ultrasound image acquisition module is configured to iteratively adjust ultrasound parameters and acquire ultrasound images of the ultrasound puncture needle under the currently used ultrasound parameters. An ultrasound image processing module is configured to take the ultrasound image as input to a network model to obtain multiple quality parameter evaluation values ​​characterizing the imaging quality of the ultrasound puncture needle in the ultrasound image; the network model includes an image segmentation network and a classification network, wherein the image segmentation network is used to detect the puncture needle in the ultrasound image to obtain a segmented image of the puncture needle region; and the classification network is used to evaluate the imaging quality of the puncture needle region segmented image. The ultrasound image evaluation module is configured to compare the weighted result of multiple quality parameter evaluation values ​​with the target quality value. The puncture needle region segmentation image determination module is configured to stop iteratively adjusting the ultrasound parameters and obtain the puncture needle region segmentation image under the currently used ultrasound parameters when the comparison result meets the set conditions. The ultrasound parameter optimization module is configured to continue iteratively adjusting the ultrasound parameters if the comparison result between the weighted result and the target quality value does not meet the set conditions, until the weighted result of the obtained multiple quality parameter evaluation values ​​and the comparison result between the target quality value meet the set conditions. The ultrasound parameters are iteratively adjusted, and the ultrasound parameter optimization module is configured as follows: Based on the evaluation values ​​of multiple quality parameters under the ultrasound parameters used in the current application, a deep reinforcement learning network is used to adjust the ultrasound parameters used in the current application to obtain the adjusted ultrasound parameters. The deep reinforcement learning network is trained using the Q-learning algorithm. In the Q-table used during the training of the deep reinforcement learning network using the Q-learning algorithm, the ultrasound parameters are used as actions in the Q-table, and multiple quality parameter evaluation values ​​are used as states in the Q-table. The adjusted ultrasound parameters are the ultrasound parameters corresponding to the maximum Q-value among the multiple quality parameter evaluation values ​​obtained in the current iteration in the Q-table.

8. An electronic device, characterized in that, include: Processor and memory; The memory is used to store the processor-executable instructions; The processor is configured to execute the instructions to implement the ultrasound puncture needle imaging enhancement method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the ultrasound puncture needle imaging enhancement method as described in any one of claims 1-6.

Images