Developing enhancement method, device and equipment for ultrasonic puncture needle

Abstract

The invention discloses a method, a device and equipment for enhancing development of an ultrasonic puncture needle, which are used for solving the problem of poor development effect of an ultrasonic image of the ultrasonic puncture needle in related technologies. The method comprises the following steps: firstly, acquiring an ultrasonic image of an ultrasonic puncture needle under ultrasonic parameters adopted at the current time, and taking the ultrasonic image as input of a network model to obtain a plurality of quality parameter evaluation values for representing the needle body development quality of the ultrasonic puncture needle in the ultrasonic image; judging a comparison result of a weighting result of the plurality of quality parameter evaluation values and a target quality value; when a comparison result meets a set condition, stopping iteratively adjusting the ultrasonic parameters, and obtaining a puncture needle region segmentation image under the ultrasonic parameters adopted at the current time; and if the comparison result of the weighting result and the target quality value does not meet the set condition, continuing to iteratively adjust the ultrasonic parameters until the comparison result of the weighting result of the plurality of obtained quality parameter evaluation values and the target quality value meets the set condition.

Description

Technical Field

[0001] The present application relates to the field of medical ultrasonic diagnosis technology, and in particular to a method, device and equipment for enhancing the development of an ultrasonic puncture needle. Background Art

[0002] Ultrasonic puncture surgery is a common surgical method in medical treatment. However, due to the specific surface properties of the puncture needle, the signal of the needle body in the ultrasound image is not obvious; therefore, in puncture surgery, it is very necessary to improve the imaging effect of the puncture needle body, and the imaging of the puncture needle needs to be enhanced.

[0003] In the related art, there are two main methods of ultrasonic puncture surgery: (1) assisted puncture with the help of guiding tools such as puncture holders; (2) manual puncture. These two methods have their own advantages and disadvantages: the first method reduces the flexibility of the puncture needle movement, but the advantage is that it does not require much puncture experience and is suitable for doctors with insufficient clinical experience; on the contrary, the second method requires doctors to have puncture surgery experience, but the advantage is that it has greater flexibility.

[0004] Comprehensively considering the actual clinical process, freehand puncture is a more appropriate method of ultrasonic puncture. At present, the freehand puncture method is mainly based on the deflection control method of the ultrasonic emission beam, so that the ultrasonic emission beam of one frame of the image is perpendicular to the puncture needle, which improves the imaging effect of the puncture needle body. However, since there are many structures similar to the needle body in the actual human tissue, even when the ultrasonic emission beam is perpendicular to the puncture needle, the accuracy of needle body detection may be limited and the needle body imaging quality is poor.

[0005] Therefore, the ultrasonic image development effect of the ultrasonic puncture needle in the related art is poor. Summary of the invention

[0006] The purpose of the present application is to provide a method, device and equipment for enhancing the development of an ultrasonic puncture needle, so as to solve the problem of poor development effect of ultrasonic images of ultrasonic puncture needles in the related art.

[0007] In a first aspect, the present application provides a method for enhancing the imaging of an ultrasonic puncture needle, the method comprising:

[0008] Iteratively adjusting ultrasound parameters, and acquiring an ultrasound image of the ultrasound puncture needle under the ultrasound parameters currently adopted;

[0009] The ultrasound image is used as an input of a network model to obtain a plurality of quality parameter evaluation values ​​for characterizing the needle body development quality of the ultrasound puncture needle in the ultrasound image; the network model comprises an image segmentation network and a classification network, the image segmentation network is used to perform puncture needle detection on the ultrasound image to obtain a puncture needle region segmentation image; the classification network is used to evaluate the needle body development quality of the puncture needle region segmentation image;

[0010] Determine the comparison result between the weighted results of the evaluation values ​​of multiple quality parameters and the target quality value;

[0011] When the comparison result satisfies the set condition, the iterative adjustment of the ultrasound parameters is stopped to obtain the puncture needle region segmentation image under the ultrasound parameters currently used; or,

[0012] If the comparison result between the weighted result and the target quality value does not satisfy the set condition, the ultrasonic parameter is adjusted iteratively until the comparison result between the weighted results of the obtained multiple quality parameter evaluation values ​​and the target quality value satisfies the set condition.

[0013] In a possible implementation, iteratively adjusting the ultrasound parameters includes:

[0014] According to the multiple quality parameter evaluation values ​​under the ultrasound parameters currently used, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters;

[0015] In which, the deep reinforcement learning network is trained using a Q learning algorithm; in the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm, the ultrasonic parameters are used as actions in the Q table, and multiple quality parameter evaluation values ​​are used as states in the Q table; the adjusted ultrasonic parameters are the ultrasonic parameters corresponding to the maximum Q value under the multiple quality parameter evaluation values ​​currently obtained in the Q table.

[0016] In a possible implementation, the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm is obtained in the following manner:

[0017] Initialize Q table;

[0018] In the process of adjusting the Q table for the i-th time, selecting the i-th group of quality parameter evaluation values ​​from multiple groups of quality parameter evaluation values ​​and selecting the i-th group of ultrasound parameters from N groups of ultrasound parameters, where N is a positive integer; using the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter as the input of the network model to obtain the quality parameter evaluation value under the i-th group of ultrasound parameters;

[0019] Determining a reward item for the i-th group of quality parameter evaluation values ​​according to the i-th group of quality parameter evaluation values;

[0020] The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs of a deep reinforcement learning network, and the i-th group of quality parameter evaluation values ​​in the Q table are obtained as states and the i-th group of ultrasound parameters are obtained as Q values ​​of actions.

[0021] In a possible embodiment, the multiple quality parameters include at least two of the continuity of the ultrasonic puncture needle, the contrast level of the ultrasonic puncture needle, the brightness level of the ultrasonic puncture needle, or the noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is the weighted sum of multiple quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0022] In a possible implementation, the deep reinforcement learning network is trained in the following manner:

[0023] In the process of adjusting the Q table, after obtaining the N Q values ​​in the case where the i-th group of quality parameter evaluation values ​​are used as states and the N groups of ultrasonic parameters are used as actions respectively, obtaining a first ultrasonic parameter corresponding to a maximum Q value among the N Q values;

[0024] Determining a Q target value corresponding to the first ultrasonic parameter by using a Bellman algorithm based on a first quality parameter evaluation value output by the network model when the first ultrasonic parameter is taken as input and a reward item of the i-th group of quality parameter evaluation values;

[0025] Taking the i-th group of quality parameter evaluation values ​​and the reward item of the i-th group of quality parameter evaluation values ​​as a piece of experience data, and taking the Q target value corresponding to the first ultrasound parameter as a label, and storing them in an experience data pool;

[0026] Randomly select n pieces of experience data from the experience data pool as a training data set, where n is a positive integer, and input the training data set into the deep reinforcement learning network to perform training with the goal of outputting a corresponding Q target value;

[0027] For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q estimate under the first ultrasonic parameter output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated by gradient descent.

[0028] In a possible implementation, according to the multiple quality parameter evaluation values, using a deep reinforcement learning network to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters includes:

[0029] Determine reward items for the multiple quality parameter evaluation values ​​according to the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters;

[0030] Using the multiple quality parameter evaluation values ​​and the reward items of the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters as inputs of the deep reinforcement learning network to obtain Q values ​​under the multiple quality parameter evaluation values;

[0031] The ultrasonic parameters corresponding to the Q values ​​under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasonic parameters.

[0032] In a possible implementation, the loss function value satisfies the condition shown in the following formula:

[0033]

[0034] Wherein, L represents the loss function value, Q target is the Q target value, and Q(ss,aa) is the Q estimate output by the deep reinforcement learning network.

[0035] In a second aspect, the present application provides a development enhancement device for an ultrasonic puncture needle, the device comprising:

[0036] An ultrasound image acquisition module is configured to iteratively adjust ultrasound parameters and acquire an ultrasound image of the ultrasound puncture needle under the ultrasound parameters currently adopted;

[0037] The ultrasonic image processing module is configured to use the ultrasonic image as an input of a network model to obtain a plurality of quality parameter evaluation values ​​for characterizing the needle body visualization quality of the ultrasonic puncture needle in the ultrasonic image; the network model includes an image segmentation network and a classification network, the image segmentation network is used to perform puncture needle detection on the ultrasonic image to obtain a puncture needle region segmentation image; the classification network is used to evaluate the needle body visualization quality of the puncture needle region segmentation image;

[0038] an ultrasound image evaluation module configured to determine a comparison result between a weighted result of a plurality of quality parameter evaluation values ​​and a target quality value;

[0039] The puncture needle region segmentation image determination module is configured to stop iteratively adjusting the ultrasound parameters and obtain the puncture needle region segmentation image under the ultrasound parameters currently used when the comparison result satisfies the set conditions; or

[0040] The ultrasonic parameter optimization module is configured to continue iteratively adjusting the ultrasonic parameters if the comparison result between the weighted result and the target quality value does not meet the set condition until the comparison result between the weighted results of the obtained multiple quality parameter evaluation values ​​and the target quality value meets the set condition.

[0041] In a possible implementation, the ultrasound parameters are iteratively adjusted, and the ultrasound parameter optimization module is configured to:

[0042] According to the multiple quality parameter evaluation values ​​under the ultrasound parameters currently used, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters;

[0043] In which, the deep reinforcement learning network is trained using a Q learning algorithm; in the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm, the ultrasonic parameters are used as actions in the Q table, and multiple quality parameter evaluation values ​​are used as states in the Q table; the adjusted ultrasonic parameters are the ultrasonic parameters corresponding to the maximum Q value under the multiple quality parameter evaluation values ​​currently obtained in the Q table.

[0044] In a possible implementation, the Q table used in the process of the ultrasound parameter optimization module using the Q learning algorithm to train the deep reinforcement learning network is obtained in the following manner:

[0045] Initialize Q table;

[0046] In the process of adjusting the Q table for the i-th time, selecting the i-th group of quality parameter evaluation values ​​from multiple groups of quality parameter evaluation values ​​and selecting the i-th group of ultrasound parameters from N groups of ultrasound parameters, where N is a positive integer; using the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter as the input of the network model to obtain the quality parameter evaluation value under the i-th group of ultrasound parameters;

[0047] Determining a reward item for the i-th group of quality parameter evaluation values ​​according to the i-th group of quality parameter evaluation values;

[0048] The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs of a deep reinforcement learning network, and the i-th group of quality parameter evaluation values ​​in the Q table are obtained as states and the i-th group of ultrasound parameters are obtained as Q values ​​of actions.

[0049] In a possible embodiment, the multiple quality parameters include at least two of the continuity of the ultrasonic puncture needle, the contrast level of the ultrasonic puncture needle, the brightness level of the ultrasonic puncture needle, or the noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is the weighted sum of multiple quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0050] In a possible implementation, the ultrasound parameter optimization module is configured to train the deep reinforcement learning network in the following manner;

[0051] In the process of adjusting the Q table, after obtaining the N Q values ​​in the case where the i-th group of quality parameter evaluation values ​​are used as states and the N groups of ultrasonic parameters are used as actions respectively, obtaining a first ultrasonic parameter corresponding to a maximum Q value among the N Q values;

[0052] Determining a Q target value corresponding to the first ultrasonic parameter by using a Bellman algorithm based on a first quality parameter evaluation value output by the network model when the first ultrasonic parameter is taken as input and a reward item of the i-th group of quality parameter evaluation values;

[0053] Taking the i-th group of quality parameter evaluation values ​​and the reward item of the i-th group of quality parameter evaluation values ​​as a piece of experience data, and taking the Q target value corresponding to the first ultrasound parameter as a label, and storing them in an experience data pool;

[0054] Randomly select n pieces of experience data from the experience data pool as a training data set, where n is a positive integer, and input the training data set into the deep reinforcement learning network to perform training with the goal of outputting a corresponding Q target value;

[0055] For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q estimate under the first ultrasonic parameter output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated by gradient descent.

[0056] In a possible implementation, according to the multiple quality parameter evaluation values, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters, and the ultrasound parameter optimization module is configured as follows:

[0057] Determine reward items for the multiple quality parameter evaluation values ​​according to the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters;

[0058] Using the multiple quality parameter evaluation values ​​and the reward items of the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters as inputs of the deep reinforcement learning network to obtain Q values ​​under the multiple quality parameter evaluation values;

[0059] The ultrasonic parameters corresponding to the Q values ​​under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasonic parameters.

[0060] In a possible implementation, the loss function value satisfies the condition shown in the following formula:

[0061]

[0062] Wherein, L represents the loss function value, Q targetis the Q target value, and Q(ss,aa) is the Q estimate output by the deep reinforcement learning network.

[0063] In a third aspect, the present application provides an electronic device, including:

[0064] Processor and memory;

[0065] The memory is used to store the processor executable instructions;

[0066] The processor is configured to execute the instructions to implement the ultrasound puncture needle visualization enhancement method as described in any one of the first aspects of the present application.

[0067] In a fourth aspect, the present application provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the ultrasonic puncture needle development enhancement method as described in any one of the items provided in the first aspect of the present application.

[0068] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for enhancing the development of an ultrasonic puncture needle as described in any one of the items provided in the first aspect of the present application.

[0069] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0070] In this application, a neural network model is used to detect the needle body area of ​​the ultrasonic puncture needle in the ultrasonic image in real time and obtain the quality parameter evaluation value, and then the ultrasonic parameters are continuously corrected through a deep reinforcement learning network to optimize the ultrasonic parameters and achieve the best puncture needle development effect, thereby greatly improving the accuracy of ultrasonic puncture, improving the doctor's work efficiency, and improving the patient's experience.

[0071] It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present application. Based on the common sense in the art, the above preferred conditions can be combined arbitrarily to obtain the preferred embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0073] Figure 1 A schematic diagram of an application scenario of the method for enhancing the development of an ultrasonic puncture needle provided in an embodiment of the present application;

[0074] Figure 2 A schematic diagram of the overall process of the ultrasound puncture needle development enhancement method provided in an embodiment of the present application;

[0075] Figure 3 A schematic diagram of a process for training a Q table provided in an embodiment of the present application;

[0076] Figure 4 A schematic diagram of a process for training a deep reinforcement learning network provided in an embodiment of the present application;

[0077] Figure 5 A schematic diagram of the structure of the imaging enhancement device of the ultrasonic puncture needle provided in an embodiment of the present application;

[0078] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0080] Furthermore, in the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0081] In the following, the terms "first", "second", and "first" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first", "second", or "first" may explicitly or implicitly include one or more of the features.

[0082] The following is an explanation of the professional terms and technologies involved in this application:

[0083] Reinforcement learning, also known as reinforcement learning, evaluation learning or enhanced learning, is a machine learning method that lies between supervised learning and unsupervised learning. Reinforcement learning is to describe and solve the problem of maximizing rewards or achieving specific goals through learning strategies in the process of interaction between the intelligent agent and the environment. Reinforcement learning does not require any data to be given in advance, but receives the rewards (feedback) of the environment for actions to obtain learning information and update model parameters. For example, when receiving the action of adjusting a parameter of the ultrasound device, the quality of the corresponding ultrasound image is scored, and the scoring result is used as the reward or penalty of the action to adjust the parameters.

[0084] The method for enhancing the development of an ultrasonic puncture needle provided in the present application will be introduced below in conjunction with embodiments.

[0085] The inventive concept of the present application can be summarized as follows: first, an ultrasonic image of an ultrasonic puncture needle under the ultrasonic parameters currently adopted is obtained, the ultrasonic image is used as the input of a network model, and a plurality of quality parameter evaluation values ​​for characterizing the needle body development quality of the ultrasonic puncture needle in the ultrasonic image are obtained, and then, a comparison result between a weighted result of the plurality of quality parameter evaluation values ​​and a target quality value is determined; when the comparison result meets a set condition, the iterative adjustment of the ultrasonic parameters is stopped to obtain a segmented image of the puncture needle region under the ultrasonic parameters currently adopted; if the comparison result between the weighted result and the target quality value does not meet the set condition, the iterative adjustment of the ultrasonic parameters is continued until the comparison result between the weighted result of the plurality of quality parameter evaluation values ​​obtained and the target quality value meets the set condition.

[0086] In summary, the ultrasonic puncture needle development enhancement method provided in the embodiment of the present application uses a neural network model to detect the needle body area of ​​the ultrasonic puncture needle in the ultrasonic image in real time and obtain a quality parameter evaluation value, and then continuously corrects the ultrasonic parameters through a deep reinforcement learning network, so as to optimize the ultrasonic parameters and achieve the best puncture needle development effect, greatly improving the accuracy of ultrasonic puncture, improving the doctor's work efficiency, and improving the patient's user experience.

[0087] After introducing the main inventive ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0088] For ease of understanding, the following is a detailed description of an ultrasonic puncture needle imaging enhancement method provided in an embodiment of the present application in conjunction with the accompanying drawings:

[0089] like Figure 1The figure is a schematic diagram of an application scenario of a method for enhancing the development of an ultrasonic puncture needle provided in an embodiment of the present application. The figure includes: a network 10, a server 20, and a memory 30. The server 20 obtains an ultrasonic image through the network, and the ultrasonic image can be processed by the method provided in the embodiment of the present application. If the development effect of the ultrasonic image does not meet the requirements, the ultrasonic parameters are iteratively processed until the development effect of the ultrasonic image produced according to the iterative ultrasonic parameters meets the requirements.

[0090] The description in this application only details a single server, but those skilled in the art should understand that the network 10, server 20 and memory 30 shown are intended to represent the operation of the server and memory involved in the technical solution of this application. The single server and memory are described in detail at least for the convenience of explanation, and do not imply any restrictions on the number, type or location of the server. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, it will not change the underlying concepts of the example embodiments of this application. In addition, although for the convenience of explanation, Figure 1 A bidirectional arrow from the storage 30 to the server 20 is shown in the figure, but those skilled in the art can understand that the sending and receiving of the above data also needs to be implemented through the network 10.

[0091] It should be noted that the memory in the embodiment of the present application may be, for example, a cache system, a hard disk storage, a memory storage, etc. In addition, the processing method of the spectral data proposed in the present application is not only applicable to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.

[0092] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application.

[0093] Based on the above description, the present application embodiment provides a method for enhancing the imaging of an ultrasonic puncture needle, and the overall process is as follows: Figure 2 As shown, including the following:

[0094] In step 201, the ultrasound parameters are iteratively adjusted, and an ultrasound image of the ultrasound puncture needle under the currently adopted ultrasound parameters is acquired.

[0095] In step 202, the ultrasound image is used as the input of the network model to obtain multiple quality parameter evaluation values ​​for characterizing the needle body development quality of the ultrasound puncture needle in the ultrasound image; the network model includes an image segmentation network and a classification network, the image segmentation network is used to detect the puncture needle in the ultrasound image and obtain a puncture needle area segmentation image; the classification network is used to evaluate the needle body development quality of the puncture needle area segmentation image.

[0096] It should be noted that the ultrasonic parameter is composed of multiple parameters. In the embodiment of the present application, a=[θ t , d t ,re t ] represents the ultrasonic parameters, where θ t is the scanning angle of the ultrasonic emission line, θ t ∈[20, 21...50], d t is the emission aperture of the ultrasonic emission line, d t ∈[33...192], re t is the receiving aperture of the ultrasound receiving line, re t ∈[2...192].

[0097] In addition, the embodiment of the present application also adopts Indicates multiple quality parameter evaluation values ​​of the needle body imaging quality, where c t is the continuity of the ultrasonic puncture needle, c t ∈[1, 2, ... 100], σ t is the contrast level of the ultrasound puncture needle, σ t ∈[1, 2, ..., 100], is the brightness level of the ultrasonic puncture needle, ρ t is the noise level of the ultrasonic puncture needle, ρ t ∈[1, 2, ...100].

[0098] In a possible implementation, the above network models are all trained models. The present application embodiment uses the following method to train the above network models:

[0099] For the image segmentation network, perform the following steps:

[0100] Acquire training samples, where the training samples include sample ultrasound images and ultrasound puncture needles in annotated ultrasound images;

[0101] The sample ultrasound image is input into the image segmentation network for feature extraction processing to obtain a feature image, and the feature image is subjected to ultrasonic puncture needle detection and recognition processing to obtain an ultrasonic puncture needle recognition result;

[0102] The obtained ultrasonic puncture needle recognition result is compared with the marked ultrasonic puncture needle to determine the loss function value, and based on the loss function value, the network parameters of the convolutional neural network model are updated.

[0103] It should be added that before the sample ultrasound image is input into the image segmentation network, the embodiment of the present application will pre-process the sample ultrasound image, and the pre-processing includes cropping the original image, positioning and annotating the puncture needle for each image sequence in the original medical image sequence group in turn, thereby forming the labeled training sample image required for the training image segmentation network. Moreover, the present application also expands and enhances the features of the ultrasound images in the training set: by filling, cropping, and scaling the image sequence, placing the annotated area at the center of the image, and adjusting the brightness and / or contrast and / or grayscale parameters, better extracting image features, and calculating and generating new images and adding them to the training set.

[0104] For the classification network, perform the following steps:

[0105] Acquire a training sample, where the training sample includes a sample puncture needle region segmentation image and a plurality of quality labels in the annotated puncture needle region segmentation image;

[0106] The sample puncture needle region segmentation image is input into the classification network for feature extraction processing to obtain a feature image, and the feature image is subjected to quality assessment processing to obtain multiple quality parameter assessment values ​​of the needle body development quality of the ultrasonic puncture needle;

[0107] The obtained multiple quality parameter evaluation values ​​are compared with the marked multiple quality labels to determine the loss function value, and based on the loss function value, the network parameters of the classification network are updated.

[0108] In step 203, the comparison result between the weighted result of the plurality of quality parameter evaluation values ​​and the target quality value is determined.

[0109] In step 204, when the comparison result meets the set conditions, the iterative adjustment of the ultrasound parameters is stopped to obtain a puncture needle region segmentation image under the currently adopted ultrasound parameters.

[0110] In step 205, if the comparison result of the weighted result and the target quality value does not meet the set condition, the ultrasonic parameters are adjusted iteratively until the comparison result of the weighted results of the obtained multiple quality parameter evaluation values ​​and the target quality value meets the set condition. In one possible embodiment, the present application evaluates the ultrasonic image in a percentage system, and determines that 90 points or more are ultrasonic images with better development effects, and the set condition is that the ratio of the weighted results of the multiple quality parameter evaluation values ​​to the target quality value is greater than a specified threshold. For example, the target quality value is set to 90, and the specified threshold is set to 90%. If the weighted results of the multiple quality parameter evaluation values ​​are 80, it is determined that the comparison result of the weighted results of the multiple quality parameter evaluation values ​​and the target quality value does not meet the set condition, and the ultrasonic parameters are continued to be iteratively adjusted; if the weighted results of the multiple quality parameter evaluation values ​​are 85, it is determined that the comparison result of the weighted results of the multiple quality parameter evaluation values ​​and the target quality value meets the set condition, and the ultrasonic parameters are stopped from being iteratively adjusted to obtain the puncture needle area segmentation image under the ultrasonic parameters currently used.

[0111] In the related art, the reinforcement learning method can obtain the optimal strategy for the next action based on the current state and feedback reward. In the present application, the deep reinforcement learning network can obtain the optimal emission line scanning angle and related parameters (i.e., ultrasound parameters) for the next step based on the current state (i.e., multiple quality parameter evaluation values ​​of the needle body development quality of the ultrasonic puncture needle) and the current feedback reward, so as to achieve the optimal development quality of the puncture needle.

[0112] The reinforcement learning network estimates the optimal action for the next state through the Q-leaming algorithm. The training process of the Q-learning algorithm is the process of gradually updating the Q value of the Q table, where the Q value can be understood as quality, which represents the optimal action for the next state S. t Take a certain action a t The maximum cumulative reward value R obtained at the next moment t+1 , as shown in the following formula (1): Q(S t , a t ) represents a function, the value of this function is the Q value, different states s and actions a will correspond to a Q value, forming a multi-dimensional Q table:

[0113] Q(S t , a t )=maxR t+1 (1)

[0114] In the following formula (2), π(s) represents the optimal strategy under the current state s, which is to find the next action a that can maximize the Q value of the Q table:

[0115] π(s)=argmax(Q(s,t)) (2)

[0116] The deep reinforcement learning network provided in this application uses the Bellman formula to calculate the Q value of the Q table:

[0117]

[0118] Among them, s is the current state, a is the action corresponding to the current state, Q(s, a) is the Q value after taking action a in the current state, and r is the reward of the current state. is the state at the next moment, á is the best action corresponding to the state at the next moment, and γ is the reward discount factor, which represents the weight of the influence of the Q value at the next moment on the current Q value after taking the best action a in the current state.

[0119] Based on the above description of reinforcement learning, in the embodiment of the present application, the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm, iteratively adjusting the ultrasonic parameters can be implemented as follows: according to the multiple quality parameter evaluation values ​​under the ultrasonic parameters currently adopted, the deep reinforcement learning network is used to adjust the ultrasonic parameters currently adopted to obtain the adjusted ultrasonic parameters. Among them, the deep reinforcement learning network is trained using the Q learning algorithm; in the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm, the ultrasonic parameters are used as actions in the Q table, and the multiple quality parameter evaluation values ​​are used as states in the Q table; the adjusted ultrasonic parameters are the ultrasonic parameters corresponding to the maximum Q value under the multiple quality parameter evaluation values ​​currently obtained in the Q table.

[0120] In a possible implementation, the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm is obtained as follows: Figure 3 As shown, the following steps are included:

[0121] In step 301, the Q table is initialized.

[0122] In step 302, during the i-th adjustment of the Q table, the i-th group of quality parameter evaluation values ​​is selected from multiple groups of quality parameter evaluation values ​​and the i-th group of ultrasound parameters is selected from N groups of ultrasound parameters, where N is a positive integer; an ultrasound image of an ultrasound puncture needle under the i-th ultrasound parameter is used as an input of a network model to obtain a quality parameter evaluation value under the i-th group of ultrasound parameters.

[0123] In step 303, a reward item for the i-th group of quality parameter evaluation values ​​is determined according to the i-th group of quality parameter evaluation values.

[0124] In step 304, the i-th group of quality parameter evaluation values ​​and reward items are used as inputs of the deep reinforcement learning network to obtain the i-th group of quality parameter evaluation values ​​as the state and the i-th group of ultrasound parameters as the Q value of the action in the Q table.

[0125] In one possible embodiment, the multiple quality parameters include at least two of the continuity of the ultrasonic puncture needle, the contrast level of the ultrasonic puncture needle, the brightness level of the ultrasonic puncture needle, or the noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is the weighted sum of the multiple quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0126] The embodiment of the present application uses the following formula (4) to determine the reward item of the i-th group of quality parameter evaluation values:

[0127]

[0128] Among them, c t is the continuity of the ultrasonic puncture needle, σ t is the contrast level of the ultrasound puncture needle, is the brightness level of the ultrasonic puncture needle, ρ t is the noise level of the ultrasonic puncture needle, and α, β, θ, and δ are the weights corresponding to the quality parameter evaluation values.

[0129] It should be added that the weighted result of multiple quality parameter evaluation values ​​and the weighted sum of multiple quality parameter evaluation values ​​in the reward item, the weights of the quality parameter evaluation values ​​may be the same or different, depending on the needs.

[0130] In this application, the Q table uses ultrasound parameters as actions and multiple quality parameter evaluation values ​​as states. Since there are a lot of state and action data, for example, the total number of states is 100 4 , the total number of actions is 192*160*31. In actual application, the Q table may be larger than this. Therefore, the embodiment of the present application cannot obtain the optimal action taken in different states to maximize the Q value of the Q table by looking up the table. If the state or action is continuous, the Q-learning algorithm cannot be used. Therefore, the embodiment of the present application fits the Q(s, a) table through a deep reinforcement learning network, which can help the Q algorithm find the optimal action in the corresponding state, namely DQN (Deep Q Networks).

[0131] It should be noted that during an ultrasonic puncture needle puncture, the difference between the previous state and the next state is very small and has a certain correlation. It is not independent and identically distributed. If the traditional DQN network is directly used, overfitting will occur, and the trained network lacks generalization ability. Therefore, the embodiment of the present application adopts a DQN network based on experience replay for training, using a buffer that replays past experience information to mix past experience with current experience, thereby reducing data correlation; and experience replay also makes samples reusable, thereby improving learning efficiency.

[0132] In a possible implementation, the present application embodiment trains the deep reinforcement learning network in the following manner, and the process is as follows: Figure 4 As shown, the following steps are included:

[0133] In step 401, in the process of adjusting the Q table, after obtaining N Q values ​​with the i-th group of quality parameter evaluation values ​​as states and N groups of ultrasound parameters as actions, the first ultrasound parameter corresponding to the maximum Q value among the N Q values ​​is obtained.

[0134] In step 402, based on the first quality parameter evaluation value output by the network model when the first ultrasound parameter is taken as input and the reward item of the i-th group of quality parameter evaluation values, the Q target value corresponding to the first ultrasound parameter is determined using the Bellman algorithm.

[0135] In step 403, the i-th group of quality parameter evaluation values ​​and the reward item of the i-th group of quality parameter evaluation values ​​are taken as an experience data, and the Q target value corresponding to the first ultrasound parameter is taken as a label and stored in the experience data pool.

[0136] In step 404, n pieces of experience data are randomly selected from the experience data pool as a training data set, where n is a positive integer, and the training data set is input into the deep reinforcement learning network for training with the goal of outputting the corresponding Q target value.

[0137] In step 405, for each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q estimate under the first ultrasonic parameter output by the deep reinforcement learning network, and based on the loss function value, the deep reinforcement learning network is updated by gradient descent.

[0138] For example, the specific process of training a deep reinforcement learning network is as follows:

[0139] 1. Initialize the experience data pool, initialize the network parameters of the deep reinforcement learning network, such as the network parameter w of the network Q(ss, aa, w), initialize the reward discount factor in the Bellman formula, initialize the Q target value to 0, and the experience data pool is used to store process experience data.

[0140] 2. In the process of adjusting the Q table, after obtaining N Q values ​​with the i-th group of quality parameter evaluation values ​​as states and N groups of ultrasonic parameters as actions, obtain the first ultrasonic parameter corresponding to the maximum Q value among the N Q values.

[0141] 3. Determine the reward item for the i-th group of quality parameter evaluation values ​​based on the i-th group of quality parameter evaluation values.

[0142] 4. The i-th group of quality parameter evaluation values ​​and the reward item of the i-th group of quality parameter evaluation values ​​are taken as an experience data, and the Q target value corresponding to the first ultrasonic parameter is taken as a label, and stored in the experience data pool.

[0143] It should be noted that the embodiment of the present application can also include the i-th group of quality parameter evaluation values, the i+1-th group of quality parameter evaluation values, the i-th group of ultrasound parameters, and the reward items {s t , a t , r t ,s t+1} is stored in the experience data pool as process experience data, that is, the process experience data does not include the Q target value as a label, and the Q target value is calculated later when the process experience data is used to train the deep reinforcement learning network.

[0144] 5. Randomly select n pieces of experience data {ss t , aa t ,rr t ,ss t+1} is used as the training data set, n is a positive integer, and the training data set is input into the deep reinforcement learning network for training with the output Q target value as the goal.

[0145] 6. For each piece of experience data, a loss function value is obtained based on the Q target value and the estimated value of the deep reinforcement learning network output, and based on the loss function value, the deep reinforcement learning network is updated by gradient descent. In one possible implementation, the loss function value satisfies the conditions shown in the following formula:

[0146]

[0147] Among them, L represents the loss function value, Q target is the Q target value, and Q(ss,aa) is the estimated Q output by the deep reinforcement learning network.

[0148] In a possible implementation, after obtaining a trained deep reinforcement learning network, the deep reinforcement learning network is used to adjust the currently used ultrasound parameters according to multiple quality parameter evaluation values ​​to obtain adjusted ultrasound parameters, which can be implemented as follows:

[0149] Determine reward items for multiple quality parameter evaluation values ​​according to multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters;

[0150] Using the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters and the reward items of the multiple quality parameter evaluation values ​​as inputs of the deep reinforcement learning network to obtain Q values ​​under the multiple quality parameter evaluation values;

[0151] The ultrasound parameters corresponding to the Q values ​​under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasound parameters.

[0152] To summarize, the ultrasonic puncture needle development enhancement method provided in the embodiment of the present application uses a neural network model to detect the needle body area of ​​the ultrasonic puncture needle in the ultrasonic image in real time and obtain a quality parameter evaluation value, and then continuously corrects the ultrasonic parameters through a deep reinforcement learning network, so as to optimize the ultrasonic parameters and achieve the best puncture needle development effect, greatly improving the accuracy of ultrasonic puncture, improving the doctor's work efficiency, and improving the patient's user experience.

[0153] Based on the same inventive concept, the present application also provides a development enhancement device for an ultrasonic puncture needle, such as Figure 5 As shown, the device 500 includes:

[0154] The ultrasound image acquisition module 501 is configured to iteratively adjust ultrasound parameters and acquire an ultrasound image of the ultrasound puncture needle under the ultrasound parameters currently used;

[0155] The ultrasound image processing module 502 is configured to use the ultrasound image as an input of a network model to obtain a plurality of quality parameter evaluation values ​​for characterizing the needle body visualization quality of the ultrasound puncture needle in the ultrasound image; the network model includes an image segmentation network and a classification network, the image segmentation network is used to perform puncture needle detection on the ultrasound image to obtain a puncture needle region segmentation image; the classification network is used to evaluate the needle body visualization quality of the puncture needle region segmentation image;

[0156] The ultrasound image evaluation module 503 is configured to determine a comparison result between a weighted result of a plurality of quality parameter evaluation values ​​and a target quality value;

[0157] The puncture needle region segmentation image determination module 504 is configured to stop iteratively adjusting the ultrasound parameters and obtain the puncture needle region segmentation image under the ultrasound parameters currently used when the comparison result satisfies the set conditions; or

[0158] The ultrasonic parameter optimization module 505 is configured to continue iteratively adjusting the ultrasonic parameters if the comparison result between the weighted result and the target quality value does not meet the set condition until the comparison result between the weighted results of the obtained multiple quality parameter evaluation values ​​and the target quality value meets the set condition.

[0159] In a possible implementation, the ultrasound parameters are iteratively adjusted, and the ultrasound parameter optimization module is configured to:

[0160] According to the multiple quality parameter evaluation values ​​under the ultrasound parameters currently used, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters;

[0161] In which, the deep reinforcement learning network is trained using a Q learning algorithm; in the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm, the ultrasonic parameters are used as actions in the Q table, and multiple quality parameter evaluation values ​​are used as states in the Q table; the adjusted ultrasonic parameters are the ultrasonic parameters corresponding to the maximum Q value under the multiple quality parameter evaluation values ​​currently obtained in the Q table.

[0162] In a possible implementation, the Q table used in the process of the ultrasound parameter optimization module using the Q learning algorithm to train the deep reinforcement learning network is obtained in the following manner:

[0163] Initialize Q table;

[0164] In the process of adjusting the Q table for the i-th time, selecting the i-th group of quality parameter evaluation values ​​from multiple groups of quality parameter evaluation values ​​and selecting the i-th group of ultrasound parameters from N groups of ultrasound parameters, where N is a positive integer; using the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter as the input of the network model to obtain the quality parameter evaluation value under the i-th group of ultrasound parameters;

[0165] Determining a reward item for the i-th group of quality parameter evaluation values ​​according to the i-th group of quality parameter evaluation values;

[0166] The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs of a deep reinforcement learning network, and the i-th group of quality parameter evaluation values ​​in the Q table are obtained as states and the i-th group of ultrasound parameters are obtained as Q values ​​of actions.

[0167] In a possible embodiment, the multiple quality parameters include at least two of the continuity of the ultrasonic puncture needle, the contrast level of the ultrasonic puncture needle, the brightness level of the ultrasonic puncture needle, or the noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is the weighted sum of multiple quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

[0168] In a possible implementation, the ultrasound parameter optimization module is configured to train the deep reinforcement learning network in the following manner;

[0169] In the process of adjusting the Q table, after obtaining the N Q values ​​in the case where the i-th group of quality parameter evaluation values ​​are used as states and the N groups of ultrasonic parameters are used as actions respectively, obtaining a first ultrasonic parameter corresponding to a maximum Q value among the N Q values;

[0170] Determining a Q target value corresponding to the first ultrasonic parameter by using a Bellman algorithm based on a first quality parameter evaluation value output by the network model when the first ultrasonic parameter is taken as input and a reward item of the i-th group of quality parameter evaluation values;

[0171] Taking the i-th group of quality parameter evaluation values ​​and the reward item of the i-th group of quality parameter evaluation values ​​as a piece of experience data, and taking the Q target value corresponding to the first ultrasound parameter as a label, and storing them in an experience data pool;

[0172] Randomly select n pieces of experience data from the experience data pool as a training data set, where n is a positive integer, and input the training data set into the deep reinforcement learning network to perform training with the goal of outputting a corresponding Q target value;

[0173] For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q estimate under the first ultrasonic parameter output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated by gradient descent.

[0174] In a possible implementation, according to the multiple quality parameter evaluation values, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters, and the ultrasound parameter optimization module is configured as follows:

[0175] Determine reward items for the multiple quality parameter evaluation values ​​according to the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters;

[0176] Using the multiple quality parameter evaluation values ​​and the reward items of the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters as inputs of the deep reinforcement learning network to obtain Q values ​​under the multiple quality parameter evaluation values;

[0177] The ultrasound parameter corresponding to the Q value under the multiple quality parameter evaluation values ​​is used as the adjusted ultrasound parameter. In a possible implementation, the loss function value satisfies the condition shown in the following formula:

[0178]

[0179] Wherein, L represents the loss function value, Q target is the Q target value, and Q(ss,aa) is the Q estimate output by the deep reinforcement learning network.

[0180] Refer to the following Figure 6 The electronic device 130 according to this embodiment of the present application is described. Figure 6 The electronic device 130 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0181] like Figure 6 As shown, the electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0182] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0183] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0184] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0185] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 136. As shown, the network adapter 136 communicates with other modules for the electronic device 130 via a bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0186] In an exemplary embodiment, the present application further provides a computer-readable storage medium including instructions, such as a memory 132 including instructions, and the instructions can be executed by a processor 131 of an electronic device 130 to complete the above-mentioned ultrasound puncture needle visualization enhancement method. Optionally, the computer-readable storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0187] In an exemplary embodiment, a computer program product is also provided, including a computer program, which, when executed by the processor 131, implements the method for enhancing the visualization of an ultrasonic puncture needle as provided in the present application.

[0188] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0189] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0192] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for enhancing the imaging of an ultrasonic puncture needle, characterized in that: The method comprises: Iteratively adjusting ultrasound parameters, and acquiring an ultrasound image of the ultrasound puncture needle under the ultrasound parameters currently adopted; The ultrasound image is used as an input of a network model to obtain a plurality of quality parameter evaluation values ​​for characterizing the needle body development quality of the ultrasound puncture needle in the ultrasound image; the network model comprises an image segmentation network and a classification network, the image segmentation network is used to perform puncture needle detection on the ultrasound image to obtain a puncture needle region segmentation image; the classification network is used to evaluate the needle body development quality of the puncture needle region segmentation image; Determine the comparison result between the weighted results of the evaluation values ​​of multiple quality parameters and the target quality value; When the comparison result satisfies the set condition, the iterative adjustment of the ultrasound parameters is stopped to obtain the puncture needle region segmentation image under the ultrasound parameters currently used; or, If the comparison result between the weighted result and the target quality value does not satisfy the set condition, the ultrasonic parameter is adjusted iteratively until the comparison result between the weighted results of the obtained multiple quality parameter evaluation values ​​and the target quality value satisfies the set condition.

2. The method according to claim 1, characterized in that Iteratively adjusting the ultrasound parameters, including: According to the multiple quality parameter evaluation values ​​under the ultrasound parameters currently used, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters; In which, the deep reinforcement learning network is trained using a Q learning algorithm; in the Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm, the ultrasonic parameters are used as actions in the Q table, and multiple quality parameter evaluation values ​​are used as states in the Q table; the adjusted ultrasonic parameters are the ultrasonic parameters corresponding to the maximum Q value under the multiple quality parameter evaluation values ​​currently obtained in the Q table.

3. The method according to claim 2, characterized in that The Q table used in the process of training the deep reinforcement learning network using the Q learning algorithm is obtained as follows: Initialize Q table; In the process of adjusting the Q table for the i-th time, selecting the i-th group of quality parameter evaluation values ​​from multiple groups of quality parameter evaluation values ​​and selecting the i-th group of ultrasound parameters from N groups of ultrasound parameters, where N is a positive integer; using the ultrasound image of the ultrasound puncture needle under the i-th ultrasound parameter as the input of the network model to obtain the quality parameter evaluation value under the i-th group of ultrasound parameters; Determining a reward item for the i-th group of quality parameter evaluation values ​​according to the i-th group of quality parameter evaluation values; The i-th group of quality parameter evaluation values ​​and the reward item are used as inputs of a deep reinforcement learning network, and the i-th group of quality parameter evaluation values ​​in the Q table are obtained as states and the i-th group of ultrasound parameters are obtained as Q values ​​of actions.

4. The method according to claim 3, characterized in that The multiple quality parameters include at least two of the continuity of the ultrasonic puncture needle, the contrast level of the ultrasonic puncture needle, the brightness level of the ultrasonic puncture needle, or the noise level of the ultrasonic puncture needle, and the reward item of the i-th group of quality parameter evaluation values ​​is the weighted sum of multiple quality parameter evaluation values ​​in the i-th group of quality parameter evaluation values.

5. The method according to claim 3 or 4, characterized in that: The deep reinforcement learning network is trained in the following manner: In the process of adjusting the Q table, after obtaining the N Q values ​​in the case where the i-th group of quality parameter evaluation values ​​are used as states and the N groups of ultrasonic parameters are used as actions respectively, obtaining a first ultrasonic parameter corresponding to a maximum Q value among the N Q values; Determining a Q target value corresponding to the first ultrasonic parameter by using a Bellman algorithm based on a first quality parameter evaluation value output by the network model when the first ultrasonic parameter is taken as input and a reward item of the i-th group of quality parameter evaluation values; Taking the i-th group of quality parameter evaluation values ​​and the reward item of the i-th group of quality parameter evaluation values ​​as a piece of experience data, and taking the Q target value corresponding to the first ultrasound parameter as a label, and storing them in an experience data pool; Randomly select n pieces of experience data from the experience data pool as a training data set, where n is a positive integer, and input the training data set into the deep reinforcement learning network to perform training with the goal of outputting a corresponding Q target value; For each piece of empirical data, a loss function value is obtained based on the corresponding Q target value and the Q estimate under the first ultrasonic parameter output by the deep reinforcement learning network. Based on the loss function value, the deep reinforcement learning network is updated by gradient descent.

6. The method according to claim 5, characterized in that According to the multiple quality parameter evaluation values, a deep reinforcement learning network is used to adjust the ultrasound parameters currently used to obtain adjusted ultrasound parameters, including: Determine reward items for the multiple quality parameter evaluation values ​​according to the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters; Using the multiple quality parameter evaluation values ​​and the reward items of the multiple quality parameter evaluation values ​​under the currently adopted ultrasound parameters as inputs of the deep reinforcement learning network to obtain Q values ​​under the multiple quality parameter evaluation values; The ultrasonic parameters corresponding to the Q values ​​under the multiple quality parameter evaluation values ​​are used as the adjusted ultrasonic parameters.

7. The method according to claim 5, characterized in that The loss function value satisfies the conditions shown in the following formula: Wherein, L represents the loss function value, Q target is the Q target value, and Q(ss,aa) is the Q estimate output by the deep reinforcement learning network.

8. A development enhancement device for an ultrasonic puncture needle, characterized in that: The device comprises: An ultrasound image acquisition module, configured to iteratively adjust ultrasound parameters and acquire an ultrasound image of the ultrasound puncture needle under the ultrasound parameters currently adopted; The ultrasonic image processing module is configured to use the ultrasonic image as an input of a network model to obtain a plurality of quality parameter evaluation values ​​for characterizing the needle body visualization quality of the ultrasonic puncture needle in the ultrasonic image; the network model includes an image segmentation network and a classification network, the image segmentation network is used to perform puncture needle detection on the ultrasonic image to obtain a puncture needle region segmentation image; the classification network is used to evaluate the needle body visualization quality of the puncture needle region segmentation image; an ultrasound image evaluation module configured to determine a comparison result between a weighted result of a plurality of quality parameter evaluation values ​​and a target quality value; The puncture needle region segmentation image determination module is configured to stop iteratively adjusting the ultrasound parameters and obtain the puncture needle region segmentation image under the ultrasound parameters currently used when the comparison result satisfies the set conditions; or The ultrasonic parameter optimization module is configured to continue iteratively adjusting the ultrasonic parameters if the comparison result between the weighted result and the target quality value does not meet the set condition until the comparison result between the weighted results of the obtained multiple quality parameter evaluation values ​​and the target quality value meets the set condition.

9. An electronic device, characterized in that: include: Processor and memory; The memory is used to store the processor executable instructions; The processor is configured to execute the instructions to implement the ultrasound puncture needle visualization enhancement method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method for enhancing the visualization of an ultrasonic puncture needle according to any one of claims 1 to 7.

Images