A method for visualization and early warning of acupuncture based on ultrasound imaging nerve block

Abstract

The disclosed embodiment provides a method for visualization and early warning of acupuncture for nerve block based on ultrasound imaging, which relates to the field of image processing technology. The method includes: obtaining a target ultrasound image and a nerve block site; obtaining a target image segmentation model according to the nerve block site, segmenting the target ultrasound image based on the model, and obtaining an image segmentation result of the target ultrasound image, including a nerve block area, an avoidance area, and a danger area; within a preset angle range and a preset horizontal distance range, trigonometric function calculations are performed based on the image segmentation result and the imaging depth of the target ultrasound image and the trajectory of the puncture needle to determine the relevant parameters, predict the target trajectory of the puncture needle in the target ultrasound image, and determine the positioning points in the target trajectory. The method can accurately identify the nerve block area, the avoidance area, and the danger area, and can also further analyze the segmentation results of the ultrasound image to accurately predict the trajectory of the puncture needle in the ultrasound image.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to Chinese patent application numbered 2024104130120, filed on April 7, 2024, and entitled “ULTRASONIC IMAGE PROCESSING METHOD AND SYSTEM”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to image processing technology, and in particular to a method for visualization and early warning of acupuncture for nerve block based on ultrasound imaging. Background Art

[0004] With the continuous development and improvement of ultrasound imaging technology, its application in the medical field is becoming more and more extensive. For example, in the field of nerve block, ultrasound images can display the anatomical structure of the nerve block area and provide intuitive and accurate image information, which helps to accurately locate and avoid dangerous areas and avoidance areas.

[0005] However, for pain nerves located in the trunk wall that transmit pain stimuli to the nerve center, it is difficult to directly display these nerves on ultrasound images due to the inherent characteristics of the perineurium surrounding them and the characteristics of the anatomical course of the nerves. Therefore, how to accurately identify ultrasound images has become an urgent problem to be solved. In addition, by analyzing the anatomical structure information in ultrasound images, a more accurate puncture needle trajectory prediction model can be established. Therefore, how to use ultrasound images to predict the trajectory of the puncture needle is also an important research direction.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0007] The disclosed embodiments provide a method for visualization and early warning of needle placement based on ultrasound imaging nerve block, so as to overcome the above problems or at least partially solve the above problems.

[0008] According to one aspect of the present disclosure, there is provided an ultrasound image processing method, comprising: obtaining a target ultrasound image and a nerve block site corresponding to the target ultrasound image; obtaining a target image segmentation model corresponding to the target ultrasound image according to the nerve block site corresponding to the target ultrasound image, performing segmentation processing on the target ultrasound image based on the target image segmentation model, and obtaining an image segmentation result of the target ultrasound image; the image segmentation result includes a nerve block area, an avoidance area, and a danger area of ​​the target ultrasound image; obtaining an imaging depth of the target ultrasound image and relevant parameters determined by the trajectory of a puncture needle; and determining relevant parameters of the trajectory of the puncture needle. It includes the height of a fixing device relative to a horizontal reference plane, the fixing device is used to fix the ultrasound probe, and the puncture needle is inserted into the fixing device; within a preset angle range between the puncture needle and the fixing device and a preset horizontal distance range between the puncture needle and the ultrasound probe, trigonometric function calculation is performed according to the nerve block area, the avoidance area, the danger area, the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to predict the target trajectory of the puncture needle in the target ultrasound image; a positioning point on the target trajectory is determined; the positioning point is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory.

[0009] In some embodiments of the present disclosure, the target image segmentation model is obtained by training according to the following process: obtaining an initial image segmentation model; the initial image segmentation model includes an encoder and a decoder; the encoder includes multiple converter blocks, and the multiple converter blocks are connected in series; obtaining a sample ultrasound image and a labeling result of the sample ultrasound image; the nerve block site corresponding to the sample ultrasound image is the same as the nerve block site corresponding to the target ultrasound image; inputting the sample ultrasound image into each converter block in the encoder for encoding processing, and obtaining the features of the sample ultrasound image at the resolution corresponding to each converter block; inputting the features of the sample ultrasound image at the resolution corresponding to each converter block into the decoder for decoding processing, and obtaining the predicted segmentation result of the sample ultrasound image; calculating the model loss according to the labeling result of the sample ultrasound image and the predicted segmentation result of the sample ultrasound image, adjusting the parameters of the encoder and the decoder according to the calculated model loss until the model converges, and determining that the model at the convergence is the target image segmentation model.

[0010] In some embodiments of the present disclosure, the converter block includes an overlapping module of a tile, a self-attention mechanism layer and a hybrid feedforward network; the sample ultrasound image is input into each converter block in the encoder for encoding processing to obtain the features of the sample ultrasound image at the resolution corresponding to each converter block, including: for each converter block, inputting the input data into the overlapping module of the tile for block processing to obtain initial encoding features; inputting the initial encoding features into the self-attention mechanism layer for processing to obtain features after self-attention weighted processing; inputting the features after self-attention weighted processing into the hybrid feedforward network for processing to obtain features at the resolution corresponding to the converter block; wherein, if the converter block is the first converter block among the multiple converter blocks, the input data is the sample ultrasound image; if the converter block is not the first converter block among the multiple converter blocks, the input data is the features output by the previous converter block.

[0011] In some embodiments of the present disclosure, the decoder includes a top-level multilayer perceptron and multiple multilayer perceptron layers; the multiple multilayer perceptron layers correspond to multiple converter blocks in the encoder, and each multilayer perceptron layer includes a basic multilayer perceptron and an upsampling layer; the features of the sample ultrasound image at the resolution corresponding to each converter block are input into the decoder for decoding processing to obtain the predicted segmentation result of the sample ultrasound image, including: for the features of the sample ultrasound image at the resolution corresponding to each converter block, processing is performed in the following manner to obtain the features output by each multilayer perceptron layer: determining a target converter block that outputs the features, and obtaining a target converter block corresponding to the target The target multi-layer perceptron layer corresponding to the standard converter block; input the features into the basic multi-layer perceptron in the target multi-layer perceptron layer, perform channel compression processing on the features according to the preset channel dimension, and obtain the features after channel compression processing; input the features after channel compression processing into the upsampling layer in the target multi-layer perceptron layer, perform upsampling processing on the features after channel compression processing according to the preset size, and obtain the features output by the target multi-layer perceptron layer; splice the features output by multiple multi-layer perceptron layers to obtain the spliced ​​features, input the spliced ​​features into the top-layer multi-layer perceptron, and output the predicted segmentation result of the sample ultrasound image.

[0012] In some embodiments of the present disclosure, within the preset angle range between the puncture needle and the fixture and the preset horizontal distance range between the puncture needle and the ultrasound probe, trigonometric function calculation is performed according to the nerve block area, the avoidance area, the danger zone, the height of the fixture relative to the horizontal reference plane, and the imaging depth of the target ultrasound image to predict the target trajectory of the puncture needle in the target ultrasound image, including: within the preset angle range between the puncture needle and the fixture, searching for the angle between the puncture needle and the fixture according to a unit angle step to obtain multiple optional angles; within the preset horizontal distance range between the puncture needle and the ultrasound probe, searching for the horizontal distance between the puncture needle and the ultrasound probe according to a unit distance step to obtain multiple optional horizontal distances; selecting the optional angles among the multiple optional angles and the optional horizontal distances among the multiple optional horizontal distances. The method comprises the steps of: combining the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to obtain a plurality of optional parameter combinations; for each optional parameter combination, performing trigonometric function calculation according to the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to determine the trajectory corresponding to the optional parameter combination; the trajectory corresponding to the optional parameter combination is the expected trajectory of the puncture needle in the target ultrasound image under the optional parameter combination; if the trajectory corresponding to the optional parameter combination passes through the nerve block area and does not pass through the avoidance area and the danger area, determining the trajectory corresponding to the optional parameter combination as the optional trajectory; obtaining all the optional trajectories after performing trajectory calculation on the plurality of optional reference combinations, determining the longest trajectory among all the optional trajectories that passes through the nerve block area as the target trajectory; determining the optional angle corresponding to the target trajectory as the target angle, and the optional horizontal distance corresponding to the target trajectory as the target horizontal distance.

[0013] In some embodiments of the present disclosure, the method further includes: obtaining, by a display device, the nerve block site corresponding to the target ultrasound image in response to a selection operation on the nerve block site corresponding to the target ultrasound image; and displaying, by the display device, the target ultrasound image, the image segmentation result of the target ultrasound image, the imaging depth of the target ultrasound image, the trajectory determination-related parameters, the target trajectory of the puncture needle in the target ultrasound image, and the positioning point.

[0014] In some embodiments of the present disclosure, the fixing device includes an infrared sensor, a slidable potentiometer and a rotary potentiometer; the infrared sensor is used to obtain the height of the fixing device relative to a horizontal reference plane; the slidable potentiometer is used to obtain the horizontal distance between the puncture needle and the ultrasound probe, and to adjust the horizontal distance between the puncture needle and the ultrasound probe to the target horizontal distance; the rotary potentiometer is used to obtain the angle between the puncture needle and the fixing device, and to adjust the angle between the puncture needle and the fixing device to the target angle.

[0015] According to another aspect of an embodiment of the present disclosure, there is provided an ultrasound image processing system, the system comprising an ultrasound probe, a server, a display device, a fixing device and a puncture needle, the fixing device being used to fix the ultrasound probe, and the puncture needle being inserted into the fixing device; wherein the ultrasound probe is used to obtain a target ultrasound image and an imaging depth of the target ultrasound image, and transmit the target ultrasound image to the server and the display device; the display device is used to obtain a nerve block site corresponding to the target ultrasound image in response to a selection operation on a nerve block site corresponding to the target ultrasound image, and transmit the nerve block site corresponding to the target ultrasound image to the server; the server is used to obtain a target image segmentation model corresponding to the target ultrasound image according to the nerve block site corresponding to the target ultrasound image, and perform segmentation processing on the target ultrasound image based on the target image segmentation model to obtain the target The image segmentation result corresponding to the ultrasound image; the image segmentation result includes the nerve block area, avoidance area and danger area of ​​the target ultrasound image; the fixture is used to obtain the trajectory determination related parameters of the puncture needle and transmit the trajectory determination related parameters of the puncture needle to the server; the trajectory determination related parameters of the puncture needle include the height of the fixture relative to the horizontal reference plane; the server is also used to perform trigonometric function calculations according to the nerve block area, the avoidance area, the danger area, the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image within a preset angle range between the puncture needle and the fixture and a preset horizontal distance range between the puncture needle and the ultrasound probe, to predict the target trajectory of the puncture needle in the target ultrasound image; determine the positioning point on the target trajectory; the positioning point is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory.

[0016] In some embodiments of the present disclosure, the server is further used to train and obtain the target image segmentation model according to the following process: obtain an initial image segmentation model; the initial image segmentation model includes an encoder and a decoder; the encoder includes multiple converter blocks, and the multiple converter blocks are connected in series; obtain a sample ultrasound image and a labeling result of the sample ultrasound image; the nerve block site corresponding to the sample ultrasound image is the same as the nerve block site corresponding to the target ultrasound image; input the sample ultrasound image into each converter block in the encoder for encoding processing, and obtain the features of the sample ultrasound image at the resolution corresponding to each converter block; input the features of the sample ultrasound image at the resolution corresponding to each converter block into the decoder for decoding processing, and obtain the predicted segmentation result of the sample ultrasound image; calculate the model loss according to the labeling result of the sample ultrasound image and the predicted segmentation result of the sample ultrasound image, adjust the parameters of the encoder and the decoder according to the calculated model loss, until the model converges, and determine that the model at the convergence is the target image segmentation model.

[0017] In some embodiments of the present disclosure, the converter block includes an overlapping module of the tile, a self-attention mechanism layer and a hybrid feedforward network; the server is also used to: for each converter block, input the input data into the overlapping module of the tile for block processing to obtain initial encoding features; input the initial encoding features into the self-attention mechanism layer for processing to obtain features after self-attention weighted processing; input the features after self-attention weighted processing into the hybrid feedforward network for processing to obtain features at the resolution corresponding to the converter block; wherein, if the converter block is the first converter block among the multiple converter blocks, the input data is the sample ultrasound image; if the converter block is not the first converter block among the multiple converter blocks, the input data is the features output by the previous converter block.

[0018] In some embodiments of the present disclosure, the decoder includes a top-level multilayer perceptron and multiple multilayer perceptron layers; the multiple multilayer perceptron layers correspond to multiple converter blocks in the encoder, and each multilayer perceptron layer includes a basic multilayer perceptron and an upsampling layer; the server is further used to: for the features of the sample ultrasound image at the resolution corresponding to each converter block, process in the following manner to obtain the features output by each multilayer perceptron layer: determine a target converter block that outputs the features, and obtain a target multilayer perceptron layer corresponding to the target converter block; input the features into the basic multilayer perceptron in the target multilayer perceptron layer, perform channel compression processing on the features according to a preset channel dimension, and obtain the features after channel compression processing; input the features after channel compression processing into the upsampling layer in the target multilayer perceptron layer, perform upsampling processing on the features after channel compression processing according to a preset size, and obtain the features output by the target multilayer perceptron layer; splice the features output by multiple multilayer perceptron layers to obtain the features after splicing processing, input the features after splicing processing into the top-level multilayer perceptron, and output the predicted segmentation result of the sample ultrasound image.

[0019] In some embodiments of the present disclosure, the server is further used to: search the angle between the puncture needle and the fixture according to the unit angle step within the preset angle range between the puncture needle and the fixture to obtain multiple optional angles; search the horizontal distance between the puncture needle and the ultrasound probe according to the unit distance step within the preset horizontal distance range between the puncture needle and the ultrasound probe to obtain multiple optional horizontal distances; combine the optional angles in the multiple optional angles and the optional horizontal distances in the multiple optional horizontal distances to obtain multiple optional parameter combinations; for each optional parameter combination, perform a search based on the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image. Perform trigonometric calculations to determine the trajectory corresponding to the optional parameter combination; the trajectory corresponding to the optional parameter combination is the expected trajectory of the puncture needle in the target ultrasound image under the optional parameter combination; if the trajectory corresponding to the optional parameter combination passes through the nerve block area and does not pass through the avoidance area and the danger area, determine the trajectory corresponding to the optional parameter combination as the optional trajectory; after performing trajectory calculations on the multiple optional reference combinations, obtain all optional trajectories, and determine the longest trajectory among all the optional trajectories that passes through the nerve block area as the target trajectory; determine the optional angle corresponding to the target trajectory as the target angle, and the optional horizontal distance corresponding to the target trajectory as the target horizontal distance.

[0020] In some embodiments of the present disclosure, the display device is also used to: display the target ultrasound image, the image segmentation result of the target ultrasound image, the imaging depth of the target ultrasound image, the trajectory determination related parameters, the target trajectory of the puncture needle in the target ultrasound image and the positioning point.

[0021] In some embodiments of the present disclosure, the fixing device includes an infrared sensor, a slidable potentiometer and a rotary potentiometer; the infrared sensor is used to obtain the height of the fixing device relative to a horizontal reference plane; the slidable potentiometer is used to obtain the horizontal distance between the puncture needle and the ultrasound probe, and to adjust the horizontal distance between the puncture needle and the ultrasound probe to the target horizontal distance; the rotary potentiometer is used to obtain the angle between the puncture needle and the fixing device, and to adjust the angle between the puncture needle and the fixing device to the target angle.

[0022] An embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the above-mentioned ultrasound image processing method.

[0023] An embodiment of the present disclosure provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned ultrasound image processing method.

[0024] An embodiment of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned ultrasound image processing method is implemented.

[0025] The method provided by the embodiment of the present disclosure can accurately segment the target ultrasound image by selecting the corresponding target image segmentation model according to the nerve block site, and can accurately identify the nerve block area, avoidance area and danger area in the target ultrasound image; the segmentation result of the target ultrasound image can be further analyzed, and based on the image segmentation result and combined with the imaging depth and related parameters of the target ultrasound image, the optimal trajectory of the puncture needle in the ultrasound image (i.e., the target trajectory) can be accurately predicted.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0028] Figure 1 is a flow chart of an ultrasonic image processing method according to an exemplary embodiment;

[0029] Figure 2 and Figure 3 is a schematic structural diagram of a fixing device according to an exemplary embodiment;

[0030] Figure 4 is a diagram of a training process of a target image segmentation model according to an exemplary embodiment;

[0031] Figure 5 is a structural schematic diagram of an image segmentation model according to an exemplary embodiment;

[0032] Figure 6 is a process diagram of predicting a target trajectory of a puncture needle in a target ultrasound image according to an exemplary embodiment;

[0033] Figure 7 is a schematic diagram showing a trigonometric function calculation according to an optional angle and an optional horizontal distance in an optional parameter combination, a height of a fixing device relative to a horizontal reference plane, and an imaging depth of a target ultrasound image according to an exemplary embodiment;

[0034] Figure 8 is a schematic diagram of positioning points on a target trajectory according to an exemplary embodiment;

[0035] Fig. 9 is a structural diagram of an ultrasonic image processing system according to an exemplary embodiment;

[0036] Fig.10 It is a schematic diagram of the structure of an electronic device suitable for implementing the exemplary embodiments of the present disclosure according to an exemplary embodiment. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0038] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0039] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in at least one hardware module or integrated circuit, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0041] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the term "plurality" refers to two or more; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0042] Figure 1 is a flow chart of an ultrasonic image processing method according to an exemplary embodiment. Figure 1 The method provided in the embodiment may be applied to an ultrasound image processing system, which may include an ultrasound probe, a server, a display device, a fixing device, and a puncture needle; or, Figure 1 The method provided in the embodiment may be specifically executed by a server in an ultrasonic image processing system.

[0043] The ultrasonic probe can be connected to the server via a dedicated cable or a wireless transmission module to transmit the acquired ultrasonic image. At the same time, the ultrasonic probe also synchronously transmits the ultrasonic image to the display device via a cable or wirelessly to display the ultrasonic image on the display device.

[0044] The server is the data processing center of the ultrasound image processing system. It can receive data from the ultrasound probe and display device and perform corresponding image processing and analysis tasks. The server is usually connected to the ultrasound probe and display device through a high-speed network interface to ensure fast data transmission. In addition, the server may also be equipped with a dedicated storage device for storing processed image data and related information.

[0045] The display device is connected to the server through a standard video interface (such as HDMI, DisplayPort, etc.) or wireless connection, and receives and displays ultrasound images, image segmentation results, parameters read by the fixture, and predicted trajectories in real time. In addition, some operations can be performed on the display device, such as selecting the nerve block site and triggering the predicted trajectory, and the relevant information can be transmitted to the server.

[0046] The fixing device may be a fixing clamp or a fixing frame for fixing the ultrasound probe. For example, the fixing device includes an adjustable bracket, and the ultrasound probe can be fixed on the bracket. Another example is that the fixing device includes an adjustable fixing clamp, and the ultrasound probe is clamped by the fixing clamp. The fixing device also includes a clamp or a guide rail for inserting the puncture needle. This clamp or guide rail has precise positioning and fixing functions, which can ensure that the puncture needle maintains a stable posture after insertion. The fixing device may include a sensor or a measuring device for obtaining the trajectory of the puncture needle to determine relevant parameters, and these parameters are transmitted to the server via wired or wireless means for analysis and processing.

[0047] The puncture needle can also be called a nerve block needle or a blocking needle. It is not directly connected to other devices, but is positioned and measured through a fixture. The fixture can obtain parameters such as the relative position and angle between the puncture needle and the ultrasound probe, and transmit these parameters to the server. The server predicts the trajectory of the puncture needle in the ultrasound image based on these parameters and the ultrasound image.

[0048] Reference Figure 1 , the ultrasound image processing method may include the following steps.

[0049] Step S110, acquiring a target ultrasound image and a nerve block site corresponding to the target ultrasound image.

[0050] In some embodiments of the present disclosure, a target ultrasound image is acquired through an ultrasound probe and the target ultrasound image is transmitted to a server, so that the server can acquire the target ultrasound image and subsequently perform image processing on the target ultrasound image.

[0051] In some embodiments of the present disclosure, the nerve block site corresponding to the target ultrasound image is obtained by the display device in response to the selection operation of the nerve block site corresponding to the target ultrasound image. After the ultrasound probe acquires the target ultrasound image, the target ultrasound image is transmitted to the display device so as to display the target ultrasound image through the display device. In addition, a selection control for the nerve block site is provided on the display device, and the nerve block site corresponding to the target ultrasound image is obtained by triggering the selection control, and the obtained nerve block site is transmitted to the server. In this way, the server can acquire the nerve block site corresponding to the target ultrasound image, and subsequently select the target image segmentation model according to the nerve block site.

[0052] Step S120, according to the nerve block site corresponding to the target ultrasound image, obtain a target image segmentation model corresponding to the target ultrasound image, and perform segmentation processing on the target ultrasound image based on the target image segmentation model to obtain an image segmentation result of the target ultrasound image; the image segmentation result includes a nerve block area, an avoidance area, and a danger area of ​​the target ultrasound image.

[0053] After obtaining the target ultrasound image and the nerve block site corresponding to the target ultrasound image, the server can select a corresponding target image segmentation model according to the nerve block site to perform segmentation processing to obtain an image segmentation result of the target ultrasound image.

[0054] The image segmentation results include the nerve block area, avoidance area, and danger area of ​​the target ultrasound image. The nerve block area refers to the specific area where the nerve block operation is required, that is, the area where the nerve conduction needs to be blocked by injecting drugs or other methods. The avoidance area refers to the area that cannot be penetrated during the nerve block operation. These areas may contain important tissue structures, such as bones, joints, or other parts that cannot be penetrated. The danger area refers to the area that needs to be avoided from being touched or damaged during the nerve block operation. These areas may contain important blood vessels, organs, or other structures that are easily punctured. In addition to the nerve block area, avoidance area, and danger area, the ultrasound image may also contain other areas. Although these areas may not be the direct focus of the nerve block operation, they still need to be considered when analyzing and interpreting the ultrasound image.

[0055] In some embodiments of the present disclosure, for different nerve block sites, image segmentation models corresponding to each nerve block site are pre-trained. For example, for the nine sites of thoracic paravertebral nerve block, intercostal nerve block, serratus anterior nerve block, pectoralis nerve block, transverse thoracic nerve block, erector spinae nerve block, quadratus lumborum nerve block, transverse abdominal nerve block and rectus sheath nerve block, the image segmentation models corresponding to each site are trained respectively. The model structures of the image segmentation models corresponding to different sites may be the same, and the specific model parameters may be different. In this way, by pre-training the image segmentation models corresponding to each nerve block site, customized optimization can be performed according to the characteristics of each site, thereby improving the segmentation accuracy and precision of the image segmentation model.

[0056] When performing image processing on a target ultrasound image, model parameters may be determined according to the nerve block site corresponding to the target ultrasound image, a target image segmentation model may be loaded according to the model parameters, and then the target image segmentation model may be used to perform segmentation processing on the target ultrasound image.

[0057] In some embodiments of the present disclosure, the model parameters of the target image segmentation model can be displayed through a display device, so that the specific parameters on which the model is based when performing image segmentation can be intuitively seen. In the case of poor model performance, the problem can be more easily located by viewing the model parameters, and targeted adjustments can be made to the model. The image segmentation results of the target ultrasound image can be displayed through a display device, so that the nerve block area, avoidance area, danger area and other areas in the target ultrasound image can be intuitively seen.

[0058] Step S130, obtaining the imaging depth of the target ultrasound image and the trajectory determination related parameters of the puncture needle; the trajectory determination related parameters of the puncture needle include the height of the fixing device relative to the horizontal reference plane; the fixing device is used to fix the ultrasound probe, and the puncture needle is inserted into the fixing device.

[0059] Among them, the imaging depth of the target ultrasound image refers to the maximum depth at which the ultrasound wave can maintain sufficient signal strength to form an effective image when propagating in the medium. Imaging depth is a key parameter in the ultrasound imaging process. In some embodiments of the present disclosure, the imaging depth of the target ultrasound image can be obtained by an ultrasound probe. Specifically, the ultrasound probe can calculate the imaging depth by emitting and receiving ultrasound waves, measuring their propagation time, and combining the propagation speed of the sound waves. After calculating the imaging depth of the target ultrasound image, the ultrasound probe can transmit the imaging depth to the server, so that the server can obtain the imaging depth and subsequently predict the trajectory in combination with the imaging depth.

[0060] In some embodiments of the present disclosure, the trajectory of the puncture needle can be obtained by a fixing device to determine relevant parameters. The fixing device includes an infrared sensor, a slidable potentiometer, and a rotary potentiometer. Specifically, the infrared sensor is used to obtain the height of the fixing device relative to the horizontal reference plane, that is, the position information of the fixing device in the vertical direction; the slidable potentiometer is used to obtain the horizontal distance between the puncture needle and the ultrasonic probe, that is, the horizontal distance between the puncture needle and the ultrasonic probe; the rotary potentiometer is used to obtain the angle between the puncture needle and the fixing device. It should be noted that the horizontal distance between the puncture needle and the ultrasonic probe obtained by the slidable potentiometer and the angle between the puncture needle and the fixing device obtained by the rotary potentiometer are considered to be initial parameters, and the parameters can be adjusted later according to the target horizontal distance and target angle corresponding to the predicted target trajectory.

[0061] Figure 2 and Figure 3 is a schematic diagram of a fixing device according to an exemplary embodiment. Figure 2 is a top view of the fixture. Figure 3 is a front view of the fixture. Figure 2 As shown, the fixing device is a fixing clamp structure, which may include a first clamping plate 210 , a second clamping plate 220 , an infrared sensor 230 , a slidable potentiometer 240 and a rotary potentiometer 250 .

[0062] The first clamping plate 210 and the second clamping plate 220 are the basic structures of the fixing device. The first clamping plate 210 and the second clamping plate 220 work together to clamp the ultrasonic probe or other related equipment, such as Figure 2 As shown, area 260 is where the ultrasound probe is placed. The infrared sensor 230 is disposed on the first clamping plate 210 and the second clamping plate 220 to obtain the height of the fixture relative to the horizontal reference plane. Figure 3 As shown, after the ultrasound probe is clamped by the first clamping plate 210 and the second clamping plate 220, the ultrasound probe is placed on a horizontal table, and the infrared sensor 230 can obtain the height d of the fixing device relative to the horizontal table. z The slidable potentiometer 240 is disposed on the first clamping plate 210 and the second clamping plate 220, and the slidable potentiometer 240 can slide on the first clamping plate 210 and the second clamping plate 220. The rotary potentiometer 250 is disposed on the slidable potentiometer 240, and the puncture needle can be inserted into the fixing device through the rotary potentiometer 250, such as Figure 3 As shown, the angle θ between the puncture needle and the fixing device can be obtained by rotating the potentiometer 250. Since the slidable potentiometer 240 is provided with the rotary potentiometer 250, and the puncture needle is inserted into the fixing device through the rotary potentiometer 250, the horizontal distance l between the puncture needle and the ultrasonic probe can be obtained by the slidable potentiometer 240 z ,like Figure 3 shown.

[0063] It should be noted that Figure 2 and Figure 3 It is only a schematic diagram of the fixing device. The design of the fixing device depends on the application scenario and specific requirements. In addition to the first clamping plate, the second clamping plate, the infrared sensor, the sliding potentiometer and the rotary potentiometer, it can also include a base or a bracket (for stabilizing the fixing device), an adjustment device (such as a telescopic rod or an adjustment knob) and accessories and connectors (such as cable ties, fixing straps, interface adapters, etc., for connecting and integrating with other devices or systems), etc., and the embodiments of the present disclosure are not limited to this.

[0064] In the disclosed embodiment, after the fixing device obtains the trajectory of the puncture needle and determines the relevant parameters, these relevant parameters can be transmitted to the server, so that the server can obtain these relevant parameters and subsequently predict the trajectory in combination with these relevant parameters.

[0065] In some embodiments of the present disclosure, a parameter reading control is provided on the display device, and a parameter reading instruction is generated by triggering the parameter reading control; the display device can send a parameter reading instruction to the ultrasound probe, so that the ultrasound probe obtains the imaging depth of the target ultrasound image, and then transmits the imaging depth to the display device, displays the imaging depth through the display device, and transmits the imaging depth to the server; and the display device can send a parameter reading instruction to the fixing device, so that the fixing device obtains the trajectory of the puncture needle to determine relevant parameters, transmits these relevant parameters to the display device, displays these relevant parameters through the display device, and transmits these relevant parameters to the server.

[0066] Step S140, within the preset angle range between the puncture needle and the fixture and the preset horizontal distance range between the puncture needle and the ultrasound probe, trigonometric function calculation is performed according to the nerve block area, the avoidance area, the danger area, the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image to predict the target trajectory of the puncture needle in the target ultrasound image.

[0067] The target trajectory refers to the predicted optimal trajectory of the puncture needle in the target ultrasound image. As mentioned above, the nerve block area refers to the specific area where the nerve block operation is required, the avoidance area refers to the area that cannot be penetrated during the nerve block operation, and the danger area refers to the area that needs to be avoided from being touched or damaged during the nerve block operation. These areas provide a clear reference for the prediction of the target path.

[0068] The angle between the puncture needle and the fixture and the horizontal distance between the puncture needle and the ultrasound probe are adjustable parameters, and the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image are fixed parameters. These parameters constitute the position and direction information of the puncture needle in three-dimensional space. The angle between the puncture needle and the fixture is searched within the preset angle range, and the horizontal distance between the puncture needle and the ultrasound probe is searched within the preset horizontal distance range. Combined with the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image, multiple parameter combinations are obtained. For each parameter combination, trigonometric function calculations are performed, and combined with the nerve block area, avoidance area and danger area in the image segmentation results, the target trajectory of the puncture needle in the target ultrasound image, that is, the optimal trajectory, can be predicted. This trajectory can accurately pass through the nerve block area and effectively avoid the avoidance area and danger area.

[0069] In some embodiments of the present disclosure, a trajectory calculation start control is provided on the display device. After the trajectory calculation start control is triggered, a trajectory calculation instruction can be generated and the trajectory calculation instruction is transmitted to the server. After receiving the trajectory calculation instruction, the server searches for the angle between the puncture needle and the fixture within a preset angle range based on the nerve block area, the avoidance area and the danger area, and searches for the horizontal distance between the puncture needle and the ultrasound probe within a preset horizontal distance range, and predicts the target trajectory of the puncture needle in the target ultrasound image in combination with the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image. In addition, the target trajectory of the puncture needle in the target ultrasound image can be displayed by the display device.

[0070] Step S150, determining a positioning point on the target trajectory; wherein the positioning point is used to indicate a position that the needle tip of the puncture needle can reach on the target trajectory.

[0071] The positioning point can also be called a reference point, a prompt point, or a warning point, and is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory. For example, when the needle tip of the puncture needle moves forward along the target trajectory, when the needle tip of the puncture needle coincides with the positioning point, a prompt mechanism will be triggered to inform the operator that the needle tip has reached the prompt position on the target trajectory and cannot move forward. In addition, the positioning point on the target trajectory can also be displayed on a display device.

[0072] The method provided by the embodiment of the present disclosure can accurately segment the target ultrasound image by selecting the corresponding target image segmentation model according to the nerve block site, and can accurately identify the nerve block area, avoidance area and danger area in the target ultrasound image; the segmentation result of the target ultrasound image can be further analyzed, and based on the image segmentation result and combined with the imaging depth and related parameters of the target ultrasound image, the optimal trajectory of the puncture needle in the ultrasound image (i.e., the target trajectory) can be accurately predicted.

[0073] Figure 4 FIG. 1 is a diagram showing a training process of a target image segmentation model according to an exemplary embodiment. Figure 4 ,The training process of the target image segmentation model can include the following steps.

[0074] Step S410, obtaining an initial image segmentation model; the initial image segmentation model includes an encoder and a decoder.

[0075] In the disclosed embodiment, the initial image segmentation model can be understood as a base model or a pre-trained model, which can be improved for the visual transformer. The visual transformer is a deep learning model for processing sequence data (such as images) and performing self-attention and self-encoding.

[0076] Specifically, the initial image segmentation model may include an encoder and a decoder. The encoder may include multiple transformer blocks, i.e., transformer blocks, which are connected in series, and each transformer block may include overlapping modules of tiles, self-attention mechanism layers, and hybrid feedforward networks. The decoder may include a top-level multilayer perceptron and multiple multilayer perceptron layers. Among them, the top-level multilayer perceptron is used for result prediction, the number of multilayer perceptron layers is the same as that of transformer blocks in the encoder, and multiple multilayer perceptron layers correspond to multiple transformer blocks. Each multilayer perceptron layer includes a basic multilayer perceptron and an upsampling layer.

[0077] Figure 5 FIG. 1 is a schematic diagram showing the structure of an image segmentation model according to an exemplary embodiment. Figure 5 , the image segmentation model includes an encoder 510 and a decoder 520. The encoder 510 includes a first converter block 511, a second converter block 512, a third converter block 513 and a fourth converter block 514, and these converter blocks are connected in series. Figure 5 As shown, the first converter block 511 includes a tile overlap module 5112, a self-attention mechanism layer 5112, and a hybrid feedforward network 5113. It should be noted that other converter blocks also include tile overlap modules, self-attention mechanism layers, and hybrid feedforward network components, which are not shown in the figure.

[0078] like Figure 5 As shown, the decoder 520 includes a first multi-layer perceptron layer 521, a second multi-layer perceptron layer 522, a third multi-layer perceptron layer 523, a fourth multi-layer perceptron layer 524, and a top multi-layer perceptron 525. Among them, the first converter block 511 corresponds to the first multi-layer perceptron layer 521, the second converter block 512 corresponds to the second multi-layer perceptron layer 522, the third converter block 513 corresponds to the third multi-layer perceptron layer 523, and the fourth converter block 514 corresponds to the fourth multi-layer perceptron layer 524. Figure 5 It can also be seen that the first multi-layer perceptron layer 521 may include a basic multi-layer perceptron 5211 and an upsampling layer 5212. It should be noted that other multi-layer perceptron layers also include a basic multi-layer perceptron and an upsampling layer, which are not shown in the figure.

[0079] Step S420, obtaining a sample ultrasound image and a labeling result of the sample ultrasound image; the nerve block site corresponding to the sample ultrasound image is the same as the nerve block site corresponding to the target ultrasound image.

[0080] Ultrasound images related to nerve block were collected, including the ultrasound images corresponding to the nine parts of paravertebral nerve block, intercostal nerve block, serratus anterior nerve block, pectoralis nerve block, transverse thoracic nerve block, erector spinae nerve block, quadratus lumborum nerve block, transverse abdominal nerve block and rectus sheath nerve block. The various areas in the collected ultrasound images were annotated using annotation software, that is, the nerve block area, avoidance area and danger area in the ultrasound image were outlined to obtain the annotation results of the ultrasound image.

[0081] For example, two clinicians with at least 5 years of experience in ultrasound-guided truncal nerve block analgesia use annotation software to annotate each area in the collected ultrasound images. When the annotations of the two doctors encounter disputes, a senior clinician with at least 10 years of experience in ultrasound-guided truncal nerve block analgesia is introduced to make final confirmation and classification, and their different areas are classified as nerve block areas, avoidance areas or danger areas. Specifically, for the thoracic paravertebral nerve block site, the nerve block area in the ultrasound image may include the paravertebral space (the target nerve to be blocked is in this space), the avoidance area in the ultrasound image may include the thoracic vertebrae and the transverse process of the thoracic vertebrae, and the danger area in the ultrasound image may include the intercostal arteries and veins near the paravertebral space, the nerves in the intervertebral foramen, the pleura, and the lungs; for the ultrasound-guided intercostal nerve block site, the nerve block area in the ultrasound image may include the fascia between the internal intercostal muscle and the innermost intercostal muscle, the avoidance area in the ultrasound image may include the ribs, and the danger area in the ultrasound image may include the pleura, the lungs, the subcostal blood vessels, and the intercostal blood vessel nerve bundles.

[0082] As described above, for different nerve block sites, image segmentation models corresponding to each nerve block site are pre-trained. Therefore, sample ultrasound images can be obtained from the collected ultrasound images according to the nerve block site corresponding to the target ultrasound image, and the target image segmentation model can be subsequently trained using the sample ultrasound images.

[0083] After obtaining the sample ultrasound images, they can be randomly divided into training set, validation set and test set in a certain ratio (such as 6:2:2). Among them, the main purpose of the training set is to train the machine learning model and minimize the prediction error by adjusting the parameters of the model; the main function of the validation set is to perform performance evaluation and further parameter adjustment during the model training process, that is, after the model is initially trained on the training set, the validation set will be used to evaluate the performance of the model; the test set is mainly used to evaluate the final performance of the model on completely unseen data, which is invisible during the model development and training process and is only used in the final evaluation stage. In addition, data enhancement operations such as rotation, flipping, cropping, and color gamut conversion can be performed on the sample ultrasound images to improve the generalization ability of the model.

[0084] Step S430: Input the sample ultrasonic image to each converter block in the encoder for encoding processing to obtain features of the sample ultrasonic image at resolutions corresponding to each converter block.

[0085] Specifically, the sample ultrasonic image is sequentially input into each converter block in the encoder for encoding processing. After passing through each converter block, the feature resolution of the sample ultrasonic image is reconstructed to half of the previous size, and the features of the sample ultrasonic image at the resolution corresponding to each converter block are obtained.

[0086] by Figure 5 For example, the resolution of the sample ultrasound image is H×W×3, where H is the height of the sample ultrasound image, W is the width of the sample ultrasound image, and 3 is the number of channels. Considering that the encoder 510 includes 4 converter blocks, the features output by the 4 converter blocks are subsequently spliced, and one quarter of the sample ultrasound image can be input to the first converter block 511 for encoding processing to obtain the features of the sample ultrasound image at the resolution corresponding to the first converter block 511. Among them, C1 is the number of channels after encoding. Then, The input is sent to the second converter block 512 for encoding processing to obtain the characteristics of the sample ultrasound image at the resolution corresponding to the second converter block 512 Among them, C2 is the number of channels after encoding. The input is sent to the third converter block 513 for encoding processing to obtain the characteristics of the sample ultrasound image at the resolution corresponding to the third converter block 513 Among them, C3 is the number of channels after encoding. The input is sent to the fourth converter block 514 for encoding processing to obtain the characteristics of the sample ultrasound image at the resolution corresponding to the fourth converter block 514 Among them, C4 is the number of channels after encoding processing.

[0087] In some embodiments of the present disclosure, the sample ultrasound image is input to each converter block in the encoder for encoding processing to obtain the features of the sample ultrasound image at the resolution corresponding to each converter block, which may include: for each converter block, inputting the input data to the overlapping module of the block for block processing and merging operations to obtain the initial encoding features; inputting the initial encoding features to the self-attention mechanism layer for processing to obtain the features after self-attention weighted processing; inputting the features after self-attention weighted processing to the hybrid feedforward network for processing to obtain the features at the resolution corresponding to the converter block. And, if the converter block is the first converter block among multiple converter blocks, the input data is the sample ultrasound image; if the converter block is not the first converter block among multiple converter blocks, the input data is the features output by the previous converter block.

[0088] The multiple converter blocks in the encoder are connected in series. When the converter block is used for encoding, if the converter block is the first converter block, the input data is a sample ultrasound image. Figure 5 As an example, the input data of the first converter block 511 is a sample ultrasound image. If the converter block is not the first converter block, the input data is the data output by the previous converter block. Figure 5 For example, the input data of the second converter block 512, the third converter block 513 and the fourth converter block are all features output by the previous converter block.

[0089] Each converter block includes a tile overlap module, a self-attention mechanism layer, and a hybrid feedforward network. The tile overlap module processes the input data in blocks through a convolution kernel. In convolution processing, the convolution kernel slides on the input image data and performs a dot product operation with a local area of ​​the image each time. This local area can be regarded as a tile. Specifically, three variables K, S, and P are defined; K is used to represent the size of the convolution kernel, which determines the size of the image block processed each time. For example, K = 3 means that a 3×3 convolution kernel will process a 3×3 pixel block on the input image each time; S is used to represent the step size. During the convolution process, the step size S is used to control the distance that the convolution kernel slides on the image. For example, S = 2 means that the convolution kernel moves two pixels each time on the input image; P is used to represent the padding size. For example, P = 1 means that a pixel of padding is added around the boundary of each block. P can be used to increase the continuity of edge analysis.

[0090] by Figure 5 For example, the first converter block 511, the second converter block 512, the third converter block 513 and the fourth converter block 514 all include overlapping modules of the image blocks. For the overlapping modules of the image blocks in the first converter block 511, the convolution kernel size K=7, the step size S=4 and the padding P=3 are set, and after the convolution operation, a feature map of 32 channels will be output, that is, C1 is 32. For the overlapping modules of the image blocks in the second converter block 512, the convolution kernel size K=3, the step size S=2 and the padding P=1 are set, and after the convolution operation, a feature map of 64 channels will be output respectively, that is, C2 is 64. For the overlapping modules of the image blocks in the third converter block 513, the convolution kernel size K=3, the step size S=2 and the padding P=1 are set, and after the convolution operation, a feature map of 128 channels will be output respectively, that is, C3 is 128. For the overlapping module of the image blocks in the fourth converter block 514, the convolution kernel size K=3, the step size S=2 and the padding P=1 are set. After the convolution operation, 256 channel feature maps are output respectively, that is, C4 is 256.

[0091] The initial coding features output from the tile overlap and merge module are input into the self-attention mechanism layer. In the self-attention mechanism layer, the correlation or attention weight between each position in the initial coding features and all other positions is calculated. The initial coding features are weighted according to the calculated attention weights to obtain the features after self-attention weighting.

[0092] The speed bottleneck of the visual Transformer is mainly concentrated in the high-complexity calculation of the self-attention layer. In the calculation process of the multi-head self-attention mechanism of the original visual Transformer, the query vector Q, key vector K, and value vector V of each head have the same dimension N×C, where N=H×W reflects the data length, and the calculation complexity is O(N 2 ), and the complexity increases with the increase of image size. In the disclosed embodiment, the data length can be reduced by equal proportion R (i.e., the scaling factor of each converter block), that is, the complexity is reduced to

[0093]

[0094]

[0095] The Reshape operation is a commonly used technique in deep learning and data processing, which is used to change the shape without changing its data. Formula (1) means reconstructing K into a sequence MLP(Cin, Cout) represents a multi-layer perceptron that takes as input features of the Cin dimension and outputs features of the Cout dimension. is a reconstructed sequence, which is used as the input of this multi-layer perceptron. Formula (2) represents The output K is transformed through a multi-layer perceptron.

[0096] The hybrid feedforward network can be a 3×3 convolution and a multi-layer perceptron mixed into each feedforward network. The features after self-attention weighting are input into the hybrid feedforward network, and the input features are further converted and extracted to output the features of the sample ultrasound image at the resolution corresponding to the converter block. The specific formula is as follows.

[0097] xout=MLP(GELU(Conv3(MLP(xm))))+Xi n (3)

[0098] Among them, x in is the feature after self-attention weighting, x out It is the feature of the sample ultrasound image at the resolution corresponding to the converter block, MLP() indicates processing by a multi-layer perceptron, Conv3() indicates convolution processing by a 3×3 convolution kernel, and GELU() indicates processing by a nonlinear activation function process. In the disclosed embodiment, the hybrid feedforward network is a 3×3 convolution and a multi-layer perceptron mixed into each feedforward network, which can reduce the number of parameters and improve computational efficiency.

[0099] Step S440 , inputting the features of the sample ultrasonic image at the resolution corresponding to each converter block into the decoder for decoding processing to obtain the predicted segmentation result of the sample ultrasonic image.

[0100] In some embodiments of the present disclosure, the features of the sample ultrasound image at the resolution corresponding to each converter block are input into the decoder for decoding processing to obtain the predicted segmentation result of the sample ultrasound image, which may include: processing the features of the sample ultrasound image at the resolution corresponding to each converter block to obtain the features output by each multi-layer perceptron layer; splicing the features output by multiple multi-layer perceptron layers to obtain the spliced ​​features, inputting the spliced ​​features into the top-level multi-layer perceptron, and outputting the predicted segmentation result of the sample ultrasound image.

[0101] Among them, the features of the sample ultrasound images at the resolution corresponding to each converter block can be processed in the following manner to obtain the features of the output of each multi-layer perceptron layer: determine the target converter block of the output features to obtain the target multi-layer perceptron layer corresponding to the target converter block; input the features into the basic multi-layer perceptron in the target multi-layer perceptron layer, perform channel compression processing on the features according to the preset channel dimension, and obtain the features after channel compression processing; input the features after channel compression processing into the upsampling layer in the target multi-layer perceptron layer, upsample the features after channel compression processing according to the preset size, and obtain the features output by the target multi-layer perceptron layer.

[0102] The decoder includes a top-level multi-layer perceptron and multiple multi-layer perceptron layers, and the multiple multi-layer perceptron layers correspond to the multiple converter blocks in the encoder, and each multi-layer perceptron layer includes a basic multi-layer perceptron and an upsampling layer. For the features of the sample ultrasound image at the resolution corresponding to each converter block, it can be clear which converter block outputs the features at the resolution, and the multi-layer perceptron layer corresponding to the converter block is determined as the target multi-layer perceptron layer, and then the basic perceptron layer in the target multi-layer perceptron layer performs channel compression processing on the features at the resolution to compress the features at the resolution into features with the same number of channels; then the upsampling layer in the target multi-layer perceptron layer performs upsampling processing on the features after channel compression processing to obtain features of the same size.

[0103] by Figure 5 Taking an example for explanation, when processing the feature F1 at the resolution corresponding to the first converter block 511, the target multi-layer perceptron layer is determined to be the first multi-layer perceptron layer 521 corresponding to the first converter block 511. The feature F1 is input to the basic multi-layer perceptron 5211 in the first multi-layer perceptron layer 521, and the basic multi-layer perceptron 5211 performs channel compression processing on the feature F1 according to the preset channel dimension to obtain the feature after channel compression processing. The specific formula is as follows.

[0104]

[0105] Where i represents the i-th converter block, and the feature F i represents the features at the resolution corresponding to the i-th transformer block, Represents the feature F i After the compression processing, MLP (C in ,C out ) represents a multi-layer perceptron with input C in Dimension and output C out The characteristics of the dimension, C iis the number of channels of the feature at the resolution corresponding to the i-th converter block, and C is the preset channel dimension, that is, the channel dimension after compression. Through the above formula (4), the feature F1 is compressed according to the preset channel dimension C to obtain the feature with channel number C.

[0106] Accordingly, according to the above formula (4), the basic multi-layer perceptron in the second multi-layer perceptron layer 522 performs channel compression processing on the feature F2 (the number of channels is C2) at the resolution corresponding to the second converter block 512 according to the preset channel dimension C to obtain the feature F2 with the number of channels C. Through the basic multi-layer perceptron in the third multi-layer perceptron layer 523, the feature F3 (the number of channels is C3) at the resolution corresponding to the third converter block 513 is subjected to channel compression processing according to the preset channel dimension C to obtain the feature F3 with the number of channels C Through the basic multi-layer perceptron in the fourth multi-layer perceptron layer 524, the feature F4 (the number of channels is C4) at the resolution corresponding to the fourth converter block 514 is subjected to channel compression processing according to the preset channel dimension C to obtain the feature F4 with the number of channels C

[0107] Continue with Figure 5 Take the example to illustrate the characteristics of the basic multi-layer perceptron 5211 output After that, it is input into the upsampling layer 5212 in the first multi-layer perceptron layer 521, and the feature is quantized according to the preset size by the upsampling layer 5212. Upsampling is performed to obtain the features output by the first multi-layer perceptron layer 521. The specific formula is as follows.

[0108]

[0109] Where i represents the i-th converter block, and the feature F i represents the features at the resolution corresponding to the i-th transformer block, Represents the feature F i After compression, the features represents the features output by the i-th MLP layer (i.e., the MLP layer corresponding to the i-th transformer block), Indicates upsampling to Size. According to the above formula (5), according to the preset size Features Perform upsampling to obtain Size characteristics That is, the features output by the first multi-layer perceptron layer.

[0110] Accordingly, according to the above formula (5), through the upsampling layer in the second multi-layer perceptron layer 522, according to the preset size The features of the basic multi-layer perceptron output in the second multi-layer perceptron layer 522 Perform upsampling to obtain Size characteristics Through the upsampling layer in the third multi-layer perceptron layer 523, according to the preset size The features of the basic multi-layer perceptron output in the third multi-layer perceptron layer 523 Perform upsampling to obtain Size characteristics Through the upsampling layer in the fourth multi-layer perceptron layer 524, according to the preset size The features of the basic multi-layer perceptron output in the fourth multi-layer perceptron layer 524 Perform upsampling to obtain Size characteristics

[0111] The number of channels of the features output by each multi-layer perceptron layer is C, and the size is The features output by each multi-layer perceptron layer can be spliced ​​to obtain the spliced ​​features. After obtaining the spliced ​​features, the spliced ​​features are input into the top-layer multi-layer perceptron for processing, and a multi-target segmentation mask M is output, that is, the predicted segmentation result of the sample ultrasound image. The specific formula is as follows.

[0112]

[0113] M=MLP(C,N cls )(F) (7)

[0114] Among them, F represents the feature after splicing, Concat() represents splicing, M is the multi-target segmentation mask, N cls The features output by each multi-layer perceptron layer are concatenated, and the number of feature channels after concatenation becomes 4C. Then a multi-layer perceptron is used to fuse the 4C channel dimensions to become the number of target segmentations. Each channel is the result of different target segmentations, and the feature size after concatenation is still the preset size. That is one quarter of the original size of the image. Therefore, after splicing the features output by each multi-layer perceptron layer, linear interpolation is performed to finally obtain the segmentation result of the original size image.

[0115] Step S450, calculating the model loss according to the labeling results of the sample ultrasound image and the predicted segmentation results of the sample ultrasound image, adjusting the parameters of the encoder and the decoder according to the calculated model loss until the model converges, and determining the converged model as the target image segmentation model.

[0116] Tversky Loss is a loss function used in image segmentation tasks, which can effectively measure the similarity between the predicted results and the true labels. In the disclosed embodiment, Tversky Loss can be used as a loss function to perform gradient updates on the model. Specifically, the annotation results (true labels) of the sample ultrasound images are compared with the segmentation results predicted by the model, and the loss value is calculated using Tversky Loss. This loss value reflects the difference between the model prediction and the actual situation. According to the calculated loss value, the gradient of the model parameters is calculated using the back propagation algorithm, and the parameters of the encoder and decoder are updated accordingly. This process aims to minimize the loss function so that the prediction results of the model are closer to the true labels. Through multiple iterative training, the model gradually converges, that is, the loss value gradually decreases and tends to stabilize. When the model reaches a convergence state, it is determined as the target image segmentation model.

[0117] It should be noted that the sample ultrasound images used in the above training model are the training set. After the training is completed and the target image segmentation model is obtained, the validation set can be used to tune the parameters of the target image segmentation model, and the test set can be used to evaluate the performance of the target image segmentation model. Among them, the evaluation indicators of the model include but are not limited to Dice coefficient, Intersection over Union (IOU), Panoptic Quality (PQ), pixel accuracy (PA), etc.

[0118] The disclosed embodiment improves the visual Transformer to obtain an initial image segmentation model. On the basis of an encoder and a decoder, the features of the input ultrasound image at different resolutions are obtained through multiple transformer blocks in the encoder to meet the needs of segmentation targets of different sizes. The edge segmentation performance is optimized by adding overlapping and merging modules of the tiles. In addition, in order to meet the high-frequency frames of the segmentation, the calculation method of the attention mechanism and the structure of the decoder are optimized to reduce the complexity of the calculation.

[0119] Figure 6 The present invention is a process diagram of predicting a target trajectory of a puncture needle in a target ultrasound image according to an exemplary embodiment. Figure 6The embodiment is a specific implementation of predicting the target trajectory of the puncture needle in the target ultrasound image by performing trigonometric calculations based on the nerve block area, avoidance area, danger area, the height of the fixture relative to the horizontal reference plane, and the imaging depth of the target ultrasound image within the preset angle range between the puncture needle and the fixture and the preset horizontal distance range between the puncture needle and the ultrasound probe. Figure 6 ,The prediction process of the target trajectory can include the following steps.

[0120] Step S610: Searching for the angle between the puncture needle and the fixing device within a preset angle range according to a unit angle step to obtain a plurality of optional angles.

[0121] In the disclosed embodiment, the angle range between the puncture needle and the fixing device can be preset to be 0 to 90 degrees, and within this angle range, a search is performed according to the unit angle step to obtain multiple optional angles. For example, if the unit step is set to 0.1 degrees, the search is performed from 0 to 90 degrees, and the optional angles obtained are 0 degrees, 0.1 degrees, 0.2 degrees ... 89.8 degrees, 89.9 degrees and 90 degrees.

[0122] In the disclosed embodiment, constraints for predicting the optimal trajectory can also be set: the angle between the puncture needle and the ultrasound probe is less than 45 degrees; it is necessary to avoid the non-passable area, that is, avoid the nerve block avoidance area; the minimum value of the angle between the puncture needle and the ultrasound probe is 15 degrees, and the angle is as small as possible under this condition. Based on the above constraints, it is determined that the angle range between the puncture needle and the fixture can be greater than 45 degrees and less than or equal to 75 degrees. Assuming that the set unit angle size is 0.1 degrees, a search is performed in the angle range of 45 degrees to 75 degrees with a size of 0.1 degrees to obtain an optional angle. That is, the optional angle is between 45 degrees and 75 degrees, and the difference between two adjacent optional angles is 0.1 degrees. Specifically, the optional angles are 45.1 degrees, 45.2 degrees, 45.3 degrees...74.9 degrees, 75 degrees.

[0123] Step S620: Search the horizontal distance between the puncture needle and the ultrasound probe according to the unit distance step within the preset horizontal distance range between the puncture needle and the ultrasound probe to obtain a plurality of optional horizontal distances.

[0124] In the embodiment of the present disclosure, the horizontal distance range between the puncture needle and the ultrasound probe can be preset, and within this horizontal distance range, a search is performed according to the unit distance step to obtain multiple optional horizontal distances. For example, the horizontal distance range between the puncture needle and the ultrasound probe is set to 10 cm to 20 cm, and the unit distance step is 0.2 cm. Then, the search is performed within the distance range of 10 cm to 20 cm, and the optional horizontal distances obtained are 10 cm, 10.2 cm, 10.4 cm...19.6 cm, 19.8 cm and 20 cm.

[0125] It should be noted that in the embodiments of the present disclosure, the angle range between the puncture needle and the fixing device, the horizontal distance range between the puncture needle and the ultrasound probe, the unit angle step and the unit distance step can be set according to actual needs or experience, and there is no limitation on this.

[0126] Step S630: Combining the optional angles among the multiple optional angles and the optional horizontal distances among the multiple optional horizontal distances to obtain multiple optional parameter combinations.

[0127] After obtaining multiple optional angles and multiple optional horizontal distances, these optional angles and optional horizontal distances can be arranged and combined to obtain multiple optional parameter combinations. For example, if there are m optional angles and n optional horizontal distances, m*n optional parameter combinations can be obtained.

[0128] Step S640, for each optional parameter combination, trigonometric function calculation is performed according to the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to determine the trajectory corresponding to the optional parameter combination; the trajectory corresponding to the optional parameter combination is the expected trajectory of the puncture needle in the target ultrasound image under the optional parameter combination; if the trajectory corresponding to the optional parameter combination passes through the nerve block area and does not pass through the avoidance area and the danger area, the trajectory corresponding to the optional parameter combination is determined to be the optional trajectory.

[0129] For each optional parameter combination, trigonometric function calculation is performed on the optional angle and optional horizontal distance in the optional reference combination, the height of the fixture relative to the horizontal reference plane, and the imaging depth of the target ultrasound image to obtain the trajectory corresponding to the optional parameter combination, that is, the estimated trajectory of the puncture needle in the target ultrasound image when the angle between the puncture needle and the fixture is the optional angle and the horizontal distance between the puncture needle and the ultrasound probe is the optional horizontal distance. Then, it is determined whether the trajectory passes through the nerve block area and does not pass through the nerve block avoidance area. If so, the trajectory corresponding to the optional parameter combination is determined to be an optional trajectory. Otherwise, it is confirmed that the trajectory is not an optional trajectory and the trajectory is discarded.

[0130] Figure 7FIG. 1 is a schematic diagram showing a trigonometric function calculation according to an optional angle and an optional horizontal distance in an optional parameter combination, a height of a fixture relative to a horizontal reference plane, and an imaging depth of a target ultrasound image according to an exemplary embodiment. Figure 7 As shown, the initial offset Δd when the puncture needle enters the ultrasound image is obtained through trigonometric calculation. The specific calculation formula is as follows.

[0131]

[0132] Wherein, θ is the optional angle, i.e., the angle between the puncture needle and the fixing device; d z is the height of the fixture relative to the horizontal reference plane; l z is the optional horizontal distance; dc is the imaging depth of the target ultrasound image. According to formula (8), if Then the puncture needle enters the ultrasound image from the left end of the ultrasound image, otherwise, the puncture needle enters the ultrasound image from the top of the ultrasound image. After obtaining the initial offset Δd, the entry point of the puncture needle in the ultrasound image can be located according to the initial offset Δd, and then the trajectory corresponding to the optional angle can be obtained according to the entry point and the optional angle θ, and then it can be determined whether it is an optional trajectory.

[0133] Reference Figure 7 After obtaining the initial offset Δd, the entry point 730 (i.e., the initial point) of the puncture needle in the ultrasound image can be located according to the initial offset Δd, and then the position of the entry point 730 can be extended along the direction of the optional angle θ to obtain the trajectory d corresponding to the optional angle θ. a , that is, the estimated trajectory d of the puncture needle in the target ultrasound image when the angle between the puncture needle and the fixture is the optional angle θ a Then, determine the expected trajectory d a The predicted trajectory d is confirmed by passing through the nerve block region 710 in the ultrasound image and avoiding the nerve block avoidance region 720. a Optional track.

[0134] Step S650, after performing trajectory calculation on multiple optional reference combinations, all optional trajectories are obtained, and the longest trajectory passing through the nerve blocking area among all optional trajectories is determined as the target trajectory.

[0135] Step S660: Determine the optional angle corresponding to the target trajectory as the target angle, and the optional horizontal distance corresponding to the target trajectory as the target horizontal distance.

[0136] According to the above step S640, each optional parameter combination is analyzed to obtain all optional trajectories, and the longest trajectory passing through the nerve block area is selected from all optional trajectories as the target trajectory. Figure 7 Take the optional trajectory d as an example. a , the length of the nerve block area is d s , judge d s Is it the longest path through the nerve block area among all the optional paths? If so, determine the optional path d a is the optimal trajectory, i.e. the target trajectory.

[0137] In the disclosed embodiment, the horizontal distance between the puncture needle and the ultrasound probe can also be fixed, and the angle range between the fixed puncture needle and the fixing device can be set, and all optional angles can be obtained by searching within the angle range according to the unit angle step. For each optional angle, the trajectory corresponding to the optional angle is determined based on trigonometric function operations. If the trajectory passes through the nerve block area and does not pass through the nerve block avoidance area, the trajectory is determined to be an optional trajectory. After analyzing all optional angles, all optional trajectories are obtained. Then, the longest trajectory passing through the nerve block area among all optional trajectories is determined as the target trajectory, and the optional angle corresponding to the target trajectory is determined as the target angle.

[0138] In the disclosed embodiment, the angle between the puncture needle and the fixing device can also be fixed, the horizontal distance range between the puncture needle and the ultrasound probe can be set, and all optional horizontal distances can be obtained by searching within the distance range according to the unit distance step. For each optional horizontal distance, the trajectory corresponding to the optional horizontal distance is determined based on trigonometric function operations. If the trajectory passes through the nerve block area and does not pass through the nerve block avoidance area, the trajectory is determined to be an optional trajectory. After analyzing all optional horizontal distances, all optional trajectories are obtained. Then, the longest trajectory passing through the nerve block area among all optional trajectories is determined as the target trajectory, and the optional horizontal distance corresponding to the target trajectory is determined as the target horizontal distance.

[0139] After determining the target trajectory, the optional angle corresponding to the target trajectory can be determined as the target angle, and the angle between the puncture needle and the fixing device can be adjusted to the target angle by adjusting the rotary potentiometer; and the optional horizontal distance corresponding to the target trajectory can be determined as the target horizontal distance, and the horizontal distance between the puncture needle and the ultrasound probe can be adjusted to the target horizontal distance by adjusting the slidable potentiometer. During the parameter adjustment process, the rotary potentiometer can read the currently adjusted angle, the slidable potentiometer can read the currently adjusted horizontal distance, and the read parameters are transmitted to the display device to display the currently adjusted angle and the currently adjusted horizontal distance through the display device, so that the reference adjustment personnel can understand the current adjustment status according to the parameters displayed by the display device, so that the rotary potentiometer and the slidable potentiometer can be adjusted more accurately to ensure that each step of adjustment is close to the target parameters (i.e., the target angle and the target horizontal distance).

[0140] In the disclosed embodiment, after obtaining the target trajectory, a positioning point on the target trajectory can also be determined. The positioning point is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory. Specifically, the target trajectory is the predicted optimal trajectory of the puncture needle in the target ultrasound image, and the trajectory passing through the nerve block area in the target trajectory is obtained. A certain proportional position on the trajectory from the entrance of the nerve block area or from the exit of the nerve block area is set as a positioning point to ensure that when the needle tip of the puncture needle reaches the positioning point, a prompt mechanism can be triggered to prompt that the needle tip of the puncture needle has reached its reachable position and should not continue to move forward.

[0141] Figure 8 is a schematic diagram of positioning points on a target trajectory according to an exemplary embodiment. Figure 8 In the optional trajectory d a is the target trajectory, the target trajectory d a The trajectory passing through the nerve block area 710 is d s , entry point 810 is the target trajectory d a After the entrance position of the nerve block area, the exit point 810 is the target trajectory d a Passing the exit position of the nerve block area, set the trajectory d s A certain percentage position of the upper distance from the entrance position or the exit position of the nerve block area is used as the positioning point 830. For example, the trajectory d is set s The position of 0.8 ratio of the upper distance to the entrance position of the nerve block area is taken as the positioning point 830, that is, the trajectory distance between the entry point 810 and the positioning point 830 accounts for the trajectory d s of 0.8.

[0142] The ultrasound image processing method provided by the embodiment of the present disclosure can perform segmentation processing on the target ultrasound image based on a pre-trained target image segmentation model to obtain the image segmentation result of the target ultrasound image, and display the obtained target ultrasound image and the image segmentation result of the target ultrasound image through a display device; based on the pre-summarized needle avoidance area and conditions and the image segmentation result of the target ultrasound image, the relevant trigonometric function calculations can be performed according to the imaging depth of the target ultrasound probe, the angle between the puncture needle and the fixing device, the height of the fixing device relative to the horizontal reference plane, and the horizontal distance between the puncture needle and the ultrasound probe to determine the target trajectory, that is, the optimal needle trajectory recommended to the operator, and display the optimal needle trajectory, the feasible needle angle (that is, the target angle) and the target horizontal distance of the target ultrasound image through a display device, so that the operator can intuitively observe the optimal needle trajectory and the feasible needle angle (that is, the target angle) of the optimal needle trajectory and the target horizontal distance through the display device, and then refer to them when performing trunk nerve block. In addition, the entry point (i.e., initial point) and positioning point (i.e., reference point, prompt point, warning point) on the optimal needle trajectory can also be displayed through the display device, so that the operator can visually observe the entry point and positioning point through the display device, and then when performing trunk nerve block, when the puncture needle initially reaches the block area and reaches the risk warning point, an early warning prompt is issued to remind the operator to avoid excessive puncture of the needle tip to injure deep organs or blood vessels and other important tissues and organs. It can be seen that the ultrasound image processing method provided in the embodiment of the present disclosure can also be regarded as a needle visualization and early warning method based on ultrasound image nerve block.

[0143] The method provided by the embodiment of the present disclosure can further analyze the segmentation result of the target ultrasound image, and based on the image segmentation result, and in combination with the imaging depth of the target ultrasound image and the angle between the puncture needle and the fixture, the height of the fixture relative to the horizontal reference plane, and the horizontal distance between the puncture needle and the ultrasound probe, accurately predict the optimal trajectory of the puncture needle in the ultrasound image (i.e., the target trajectory), and can also set a positioning point on the trajectory to ensure that when the needle tip of the puncture needle reaches the positioning point, a prompt mechanism can be triggered to prompt that the needle tip of the puncture needle has reached its reachable position and should not move forward.

[0144] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0145] Fig. 9 FIG. 1 is a structural diagram of an ultrasonic image processing system according to an exemplary embodiment. Fig. 9As shown, the system 900 includes an ultrasound probe 910, a server 920, a display device 930, a fixing device 940, and a puncture needle 950. The fixing device 940 is used to fix the ultrasound probe 910, and the puncture needle 950 is inserted into the fixing device 940.

[0146] The ultrasonic probe 910 is used to obtain the target ultrasonic image and the imaging depth of the target ultrasonic image, and transmit the target ultrasonic image to the server and the display device. The display device 930 is used to respond to the selection operation of the nerve block site corresponding to the target ultrasonic image, obtain the nerve block site corresponding to the target ultrasonic image, and transmit the nerve block site corresponding to the target ultrasonic image to the server. The server 920 is used to obtain the target image segmentation model corresponding to the target ultrasonic image according to the nerve block site corresponding to the target ultrasonic image, and perform segmentation processing on the target ultrasonic image based on the target image segmentation model to obtain the image segmentation result corresponding to the target ultrasonic image; the image segmentation result includes the nerve block area, avoidance area and danger area of ​​the target ultrasonic image. The fixing device 940 is used to obtain the trajectory determination related parameters of the puncture needle, and transmit the trajectory determination related parameters of the puncture needle to the server; the trajectory determination related parameters of the puncture needle include the height of the fixing device relative to the horizontal reference plane. Server 920 is also used to perform trigonometric function calculations within a preset angle range between the puncture needle and the fixture and a preset horizontal distance range between the puncture needle and the ultrasound probe based on the nerve block area, the avoidance area, the danger zone, the height of the fixture relative to the horizontal reference plane, and the imaging depth of the target ultrasound image to predict the target trajectory of the puncture needle in the target ultrasound image; determine the positioning point on the target trajectory; the positioning point is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory.

[0147] In some embodiments of the present disclosure, the server 920 is also used to train and obtain a target image segmentation model according to the following process: obtain an initial image segmentation model; the initial image segmentation model includes an encoder and a decoder; the encoder includes multiple converter blocks, and the multiple converter blocks are connected in series; obtain a sample ultrasound image and a labeling result of the sample ultrasound image; the nerve block site corresponding to the sample ultrasound image is the same as the nerve block site corresponding to the target ultrasound image; the sample ultrasound image is input into each converter block in the encoder for encoding processing, and the features of the sample ultrasound image at the resolution corresponding to each converter block are obtained; the features of the sample ultrasound image at the resolution corresponding to each converter block are input into the decoder for decoding processing, and the predicted segmentation result of the sample ultrasound image is obtained; according to the labeling result of the sample ultrasound image and the predicted segmentation result of the sample ultrasound image, the model loss is calculated, and the parameters of the encoder and the decoder are adjusted according to the calculated model loss until the model converges, and the model at the time of convergence is determined to be the target image segmentation model.

[0148] In some embodiments of the present disclosure, the converter block includes an overlapping module of the tile, a self-attention mechanism layer and a hybrid feedforward network; the server 920 is also used to: for each converter block, input the input data to the overlapping module of the tile for block processing to obtain initial encoding features; input the initial encoding features to the self-attention mechanism layer for processing to obtain features after self-attention weighted processing; input the features after self-attention weighted processing to the hybrid feedforward network for processing to obtain features at the resolution corresponding to the converter block; wherein, if the converter block is the first converter block among multiple converter blocks, the input data is a sample ultrasound image; if the converter block is not the first converter block among multiple converter blocks, the input data is the features output by the previous converter block.

[0149] In some embodiments of the present disclosure, the decoder includes a top-level multilayer perceptron and multiple multilayer perceptron layers; the multiple multilayer perceptron layers correspond to multiple converter blocks in the encoder, and each multilayer perceptron layer includes a basic multilayer perceptron and an upsampling layer; the server 920 is also used to: for the features of the sample ultrasound image at the resolution corresponding to each converter block, process them in the following manner to obtain the features output by each multilayer perceptron layer: determine the target converter block for outputting the features, and obtain the target multilayer perceptron layer corresponding to the target converter block; input the features into the basic multilayer perceptron in the target multilayer perceptron layer, perform channel compression processing on the features according to a preset channel dimension, and obtain the features after channel compression processing; input the features after channel compression processing into the upsampling layer in the target multilayer perceptron layer, perform upsampling processing on the features after channel compression processing according to a preset size, and obtain the features output by the target multilayer perceptron layer; splice the features output by multiple multilayer perceptron layers to obtain the features after splicing processing, and input the features after splicing processing into the top-level multilayer perceptron to output the predicted segmentation result of the sample ultrasound image.

[0150] In some embodiments of the present disclosure, the server 920 is further used to: search for the angle between the puncture needle and the fixture according to the unit angle step within the preset angle range between the puncture needle and the fixture to obtain multiple optional angles; search for the horizontal distance between the puncture needle and the ultrasound probe according to the unit distance step within the preset horizontal distance range between the puncture needle and the ultrasound probe to obtain multiple optional horizontal distances; combine the optional angles in the multiple optional angles and the optional horizontal distances in the multiple optional horizontal distances to obtain multiple optional parameter combinations; for each optional parameter combination, according to the height of the fixture relative to the horizontal reference plane and the target A trigonometric function calculation is performed on the imaging depth of the target ultrasound image to determine the trajectory corresponding to the optional parameter combination; the trajectory corresponding to the optional parameter combination is the expected trajectory of the puncture needle in the target ultrasound image under the optional parameter combination; if the trajectory corresponding to the optional parameter combination passes through the nerve block area and does not pass through the avoidance area and the danger area, the trajectory corresponding to the optional parameter combination is determined to be the optional trajectory; after performing trajectory calculation on multiple optional reference combinations, all optional trajectories are obtained, and the longest trajectory passing through the nerve block area among all optional trajectories is determined to be the target trajectory; the optional angle corresponding to the target trajectory is determined to be the target angle, and the optional horizontal distance corresponding to the target trajectory is determined to be the target horizontal distance.

[0151] In some embodiments of the present disclosure, the display device 930 is also used to: display the target ultrasound image, the image segmentation result of the target ultrasound image, the imaging depth of the target ultrasound image, trajectory determination related parameters, the target trajectory and positioning point of the puncture needle in the target ultrasound image.

[0152] In some embodiments of the present disclosure, the fixing device 940 includes an infrared sensor, a slidable potentiometer and a rotary potentiometer; the infrared sensor is used to obtain the height of the fixing device relative to the horizontal reference plane; the slidable potentiometer is used to obtain the horizontal distance between the puncture needle and the ultrasound probe, and adjust the horizontal distance between the puncture needle and the ultrasound probe to the target horizontal distance; the rotary potentiometer is used to obtain the angle between the puncture needle and the fixing device, and adjust the angle between the puncture needle and the fixing device to the target angle.

[0153] Regarding the system in the above embodiment, the specific manner in which each part performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0154] Fig.10 is a schematic diagram of the structure of an electronic device suitable for implementing an exemplary embodiment of the present disclosure according to an exemplary embodiment. It should be noted that: Fig.10 The electronic device 1000 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Fig.10As shown, the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 may include but are not limited to: the at least one processing unit 1010, the at least one storage unit 1020, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

[0155] The storage unit stores program codes, which can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1010 can perform the following steps: Figure 1 In step S110, a target ultrasound image and a nerve block site corresponding to the target ultrasound image are obtained; in step S120, a target image segmentation model corresponding to the target ultrasound image is obtained according to the nerve block site corresponding to the target ultrasound image, and the target ultrasound image is segmented based on the target image segmentation model to obtain an image segmentation result of the target ultrasound image, wherein the image segmentation result includes a nerve block area, an avoidance area, and a danger area of ​​the target ultrasound image; in step S130, an imaging depth of the target ultrasound image and related parameters for determining the trajectory of the puncture needle are obtained, wherein the related parameters for determining the trajectory of the puncture needle include a fixed The device is relative to the horizontal reference plane, the fixing device is used to fix the ultrasound probe, and the puncture needle is inserted into the fixing device; step S140, within the preset angle range between the puncture needle and the fixing device and the preset horizontal distance range between the puncture needle and the ultrasound probe, trigonometric function calculation is performed according to the nerve block area, the avoidance area, the danger area, the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to predict the target trajectory of the puncture needle in the target ultrasound image; step S150, determining the positioning point on the target trajectory, wherein the positioning point is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory.

[0156] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 10201 and / or a cache storage unit 10202, and may further include a read-only storage unit (ROM) 10203. The storage unit 1020 may also include a program / utility 10204 having a set (at least one) of program modules 10205, such program modules 10205 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment.

[0157] Bus 1030 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0158] The electronic device 1000 may also communicate with one or more external devices 1100 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1050. Furthermore, the electronic device 1000 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1040. As shown, the network adapter 1040 communicates with other modules of the electronic device 1000 via a bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0159] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0160] The program product for implementing the above method according to the embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0161] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0162] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0163] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0164] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0165] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0166] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0167] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0168] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0169] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An ultrasonic image processing method, characterized in that: include: Acquiring a target ultrasound image and a nerve block site corresponding to the target ultrasound image; According to the nerve block site corresponding to the target ultrasound image, a target image segmentation model corresponding to the target ultrasound image is obtained, and the target ultrasound image is segmented based on the target image segmentation model to obtain an image segmentation result of the target ultrasound image; the image segmentation result includes a nerve block area, an avoidance area, and a danger area of ​​the target ultrasound image; Acquiring imaging depth of the target ultrasound image and trajectory determination related parameters of the puncture needle; the trajectory determination related parameters of the puncture needle include a height of a fixture relative to a horizontal reference plane, the fixture being used to fix the ultrasound probe, and the puncture needle being inserted into the fixture; Within a preset angle range between the puncture needle and the fixture, the angle between the puncture needle and the fixture is searched according to a unit angle step to obtain a plurality of optional angles; within a preset horizontal distance range between the puncture needle and the ultrasound probe, the horizontal distance between the puncture needle and the ultrasound probe is searched according to a unit distance step to obtain a plurality of optional horizontal distances; the optional angles in the plurality of optional angles and the optional horizontal distances in the plurality of optional horizontal distances are combined to obtain a plurality of optional parameter combinations; For each optional parameter combination, a trigonometric function calculation is performed according to the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image to determine a trajectory corresponding to the optional parameter combination; the trajectory corresponding to the optional parameter combination is the expected trajectory of the puncture needle in the target ultrasound image under the optional parameter combination; if the trajectory corresponding to the optional parameter combination passes through the nerve block area and does not pass through the avoidance area and the danger area, the trajectory corresponding to the optional parameter combination is determined to be an optional trajectory; After performing trajectory calculation on the plurality of optional parameter combinations, all optional trajectories are obtained, and the longest trajectory among all the optional trajectories passing through the nerve blocking area is determined as the target trajectory; Determine that an optional angle corresponding to the target trajectory is a target angle, and an optional horizontal distance corresponding to the target trajectory is a target horizontal distance; Determine a positioning point on the target trajectory; the positioning point is used to indicate a position that the needle tip of the puncture needle can reach on the target trajectory; Wherein, for each optional parameter combination, performing trigonometric function calculation according to the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to determine the trajectory corresponding to the optional parameter combination includes: The initial offset Δd when the puncture needle enters the ultrasound image is obtained by trigonometric calculation, and the calculation formula is as follows: Wherein, θ is the optional angle size in the optional parameter combination; d z is the height of the fixture relative to the horizontal reference plane; l z is the optional horizontal distance in the optional parameter combination; d c is the imaging depth of the target ultrasound image; The entry point of the puncture needle in the target ultrasound image is located according to the initial offset Δd, and the trajectory d corresponding to the optional parameter combination is obtained by extending the position of the entry point along the direction of the optional angle θ. a .

2. The method according to claim 1, characterized in that The target image segmentation model is trained according to the following process: Acquire an initial image segmentation model; the initial image segmentation model includes an encoder and a decoder; the encoder includes a plurality of converter blocks, and the plurality of converter blocks are connected in series; Acquire a sample ultrasound image and a labeling result of the sample ultrasound image; the nerve block site corresponding to the sample ultrasound image is the same as the nerve block site corresponding to the target ultrasound image; Inputting the sample ultrasound image into each converter block in the encoder for encoding processing to obtain features of the sample ultrasound image at resolutions corresponding to each converter block; Inputting the features of the sample ultrasound image at the resolution corresponding to each converter block into the decoder for decoding processing to obtain a predicted segmentation result of the sample ultrasound image; According to the labeling results of the sample ultrasound image and the predicted segmentation results of the sample ultrasound image, the model loss is calculated, and the parameters of the encoder and the decoder are adjusted according to the calculated model loss until the model converges, and the model at the time of convergence is determined to be the target image segmentation model.

3. The method according to claim 2, characterized in that The transformer block includes an overlapping module of tiles, a self-attention mechanism layer, and a hybrid feed-forward network; The step of inputting the sample ultrasound image into each converter block in the encoder for encoding processing to obtain features of the sample ultrasound image at resolutions corresponding to each converter block includes: For each converter block, inputting the input data into the overlapping module of the block for block processing to obtain initial coding features; inputting the initial coding features into the self-attention mechanism layer for processing to obtain features after self-attention weighted processing; inputting the features after self-attention weighted processing into the hybrid feedforward network for processing to obtain features at the resolution corresponding to the converter block; Wherein, if the converter block is the first converter block among the multiple converter blocks, the input data is the sample ultrasound image; if the converter block is not the first converter block among the multiple converter blocks, the input data is the characteristics output by the previous converter block.

4. The method according to claim 2, characterized in that: The decoder includes a top multilayer perceptron and a plurality of multilayer perceptron layers; the plurality of multilayer perceptron layers correspond to a plurality of converter blocks in the encoder, and each multilayer perceptron layer includes a base multilayer perceptron and an upsampling layer; The step of inputting the features of the sample ultrasound image at the resolution corresponding to each converter block into the decoder for decoding processing to obtain the predicted segmentation result of the sample ultrasound image comprises: The features of the sample ultrasound image at the resolution corresponding to each converter block are processed in the following manner to obtain the features output by each multi-layer perceptron layer: Determine a target converter block that outputs the feature, and obtain a target multi-layer perceptron layer corresponding to the target converter block; input the feature into a basic multi-layer perceptron in the target multi-layer perceptron layer, perform channel compression processing on the feature according to a preset channel dimension, and obtain the feature after the channel compression processing; input the feature after the channel compression processing into an upsampling layer in the target multi-layer perceptron layer, perform upsampling processing on the feature after the channel compression processing according to a preset size, and obtain the feature output by the target multi-layer perceptron layer; The features output by multiple multi-layer perceptron layers are spliced ​​to obtain spliced ​​features, and the spliced ​​features are input into the top-layer multi-layer perceptron to output the predicted segmentation result of the sample ultrasound image.

5. The method according to claim 1, characterized in that The method further comprises: Obtaining the nerve block site corresponding to the target ultrasound image by a display device in response to a selection operation of the nerve block site corresponding to the target ultrasound image; The target ultrasound image, the image segmentation result of the target ultrasound image, the imaging depth of the target ultrasound image, the trajectory determination related parameters, the target trajectory of the puncture needle in the target ultrasound image and the positioning point are displayed by the display device.

6. The method according to claim 1, characterized in that The fixing device includes an infrared sensor, a slidable potentiometer and a rotary potentiometer; the infrared sensor is used to obtain the height of the fixing device relative to a horizontal reference plane; the slidable potentiometer is used to obtain the horizontal distance between the puncture needle and the ultrasonic probe, and to adjust the horizontal distance between the puncture needle and the ultrasonic probe to the target horizontal distance; the rotary potentiometer is used to obtain the angle between the puncture needle and the fixing device, and to adjust the angle between the puncture needle and the fixing device to the target angle.

7. An ultrasonic image processing system, characterized in that: The system comprises an ultrasound probe, a server, a display device, a fixing device and a puncture needle, wherein the fixing device is used to fix the ultrasound probe, and the puncture needle is inserted into the fixing device; wherein, The ultrasonic probe is used to obtain a target ultrasonic image and an imaging depth of the target ultrasonic image, and transmit the target ultrasonic image to the server and the display device; The display device is used for obtaining the nerve block site corresponding to the target ultrasound image in response to a selection operation of the nerve block site corresponding to the target ultrasound image, and transmitting the nerve block site corresponding to the target ultrasound image to the server; The server is used to obtain a target image segmentation model corresponding to the target ultrasound image according to the nerve block site corresponding to the target ultrasound image, and perform segmentation processing on the target ultrasound image based on the target image segmentation model to obtain an image segmentation result corresponding to the target ultrasound image; the image segmentation result includes a nerve block area, an avoidance area, and a danger area of ​​the target ultrasound image; The fixing device is used to obtain the trajectory determination related parameters of the puncture needle and transmit the trajectory determination related parameters of the puncture needle to the server; the trajectory determination related parameters of the puncture needle include the height of the fixing device relative to the horizontal reference plane; The server is further used to search the angle between the puncture needle and the fixture according to a unit angle step within a preset angle range between the puncture needle and the fixture to obtain multiple optional angles; search the horizontal distance between the puncture needle and the ultrasound probe according to a unit distance step within a preset horizontal distance range between the puncture needle and the ultrasound probe to obtain multiple optional horizontal distances; combine the optional angles in the multiple optional angles and the optional horizontal distances in the multiple optional horizontal distances to obtain multiple optional parameter combinations; for each optional parameter combination, perform trigonometric function calculation according to the height of the fixture relative to the horizontal reference plane and the imaging depth of the target ultrasound image to determine the trajectory corresponding to the optional parameter combination; the optional parameters The trajectory corresponding to the combination is the expected trajectory of the puncture needle in the target ultrasound image under the optional parameter combination; if the trajectory corresponding to the optional parameter combination passes through the nerve block area and does not pass through the avoidance area and the danger area, the trajectory corresponding to the optional parameter combination is determined to be the optional trajectory; after performing trajectory calculation on the multiple optional parameter combinations, all optional trajectories are obtained, and the longest trajectory passing through the nerve block area among all the optional trajectories is determined to be the target trajectory; the optional angle corresponding to the target trajectory is determined to be the target angle, and the optional horizontal distance corresponding to the target trajectory is determined to be the target horizontal distance; the positioning point on the target trajectory is determined; the positioning point is used to indicate the position that the needle tip of the puncture needle can reach on the target trajectory; Wherein, for each optional parameter combination, performing trigonometric function calculation according to the height of the fixing device relative to the horizontal reference plane and the imaging depth of the target ultrasound image to determine the trajectory corresponding to the optional parameter combination includes: The initial offset Δd when the puncture needle enters the ultrasound image is obtained by trigonometric calculation, and the calculation formula is as follows: Wherein, θ is the optional angle size in the optional parameter combination; d z is the height of the fixture relative to the horizontal reference plane; l z is the optional horizontal distance in the optional parameter combination; d c is the imaging depth of the target ultrasound image; The entry point of the puncture needle in the target ultrasound image is located according to the initial offset Δd, and the trajectory d corresponding to the optional parameter combination is obtained by extending the position of the entry point along the direction of the optional angle θ. a .

8. The system according to claim 7, characterized in that The server is also used to train and obtain the target image segmentation model according to the following process: Acquire an initial image segmentation model; the initial image segmentation model includes an encoder and a decoder; the encoder includes a plurality of converter blocks, and the plurality of converter blocks are connected in series; Acquire a sample ultrasound image and a labeling result of the sample ultrasound image; the nerve block site corresponding to the sample ultrasound image is the same as the nerve block site corresponding to the target ultrasound image; Inputting the sample ultrasound image into each converter block in the encoder for encoding processing to obtain features of the sample ultrasound image at resolutions corresponding to each converter block; Inputting the features of the sample ultrasound image at the resolution corresponding to each converter block into the decoder for decoding processing to obtain a predicted segmentation result of the sample ultrasound image; According to the labeling results of the sample ultrasound image and the predicted segmentation results of the sample ultrasound image, the model loss is calculated, and the parameters of the encoder and the decoder are adjusted according to the calculated model loss until the model converges, and the model at the time of convergence is determined to be the target image segmentation model.

Images