Sacral plexus sciatic nerve ultrasonic image recognition and puncture guide method and device

Abstract

The invention discloses a sacral plexus and sciatic nerve ultrasound image recognition and puncture guide method and device, and the method comprises the steps: inputting a sacral plexus and sciatic nerve ultrasound image into a target image classification model, and carrying out the classification of the sacral plexus and sciatic nerve ultrasound image, and obtaining the image types of the sacral plexus and sciatic nerve ultrasound image; determining a region category set based on the image categories of the sacral plexus and sciatic nerve ultrasound images; inputting the sacral plexus and sciatic nerve ultrasonic image into a target image segmentation model and carrying out image segmentation on the sacral plexus and sciatic nerve ultrasonic image to obtain a plurality of tissue segmentation regions and region categories of the plurality of tissue segmentation regions on the sacral plexus and sciatic nerve ultrasonic image; and sending prompt information based on the plurality of tissue segmentation regions and the region categories of the plurality of tissue segmentation regions. According to the method, the accuracy of sacral plexus and sciatic nerve ultrasonic image recognition and puncture guiding can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method and device for ultrasonic image recognition and puncture guidance of the sacral plexus and sciatic nerve. Background Art

[0002] The sacral plexus is composed of the lumbosacral trunk and the anterior branches of all sacral and coccygeal nerves. Traditional techniques for sacral plexus and sciatic nerve block mainly rely on surface landmarks and nerve stimulators, and doctors perform blind probing operations by combining clinical experience with patient responses, resulting in a relatively high failure rate. In recent years, the application of ultrasound technology in the medical field has become increasingly widespread, especially showing great potential in nerve block and pain treatment. Although compared with traditional techniques, the ultrasound-guided nerve block technique has made progress, there are still many challenges in practical applications. First, the sacral plexus and sciatic nerve are located deep, and the surrounding anatomical structures are relatively complex, making it difficult to identify. Second, there are differences in the ultrasound display effects of the sacral plexus and sciatic nerve in different patients. Factors such as obesity, advanced age, muscle atrophy, or fat infiltration may lead to a decrease in the contrast between tissues in the ultrasound image, and the boundaries between tissues are not clearly shown. Finally, the resolution of the ultrasound device and the scanning technique of the operator will also affect the clarity of the imaging, thus affecting the accuracy of ultrasonic image recognition and puncture guidance of the sacral plexus and sciatic nerve. Summary of the Invention

[0003] The embodiments of this application provide a method and device for ultrasonic image recognition and puncture guidance of the sacral plexus and sciatic nerve, which can improve the accuracy of ultrasonic image recognition and puncture guidance of the sacral plexus and sciatic nerve.

[0004] In the first aspect, the method for ultrasonic image recognition and puncture guidance of the sacral plexus and sciatic nerve provided by this application includes:

[0005] Obtain ultrasonic scanning videos of multiple different sections of an ultrasonic device;

[0006] Perform frame splitting on the ultrasonic scanning videos to obtain a plurality of consecutive single-frame ultrasonic images;

[0007] Obtain the labeled ultrasonic images obtained by labeling the single-frame ultrasonic images and the image categories of the labeled ultrasonic images. Among them, the single-frame ultrasonic images obtained from the ultrasonic scanning videos of different sections belong to different image categories, and the labeled ultrasonic images include labeled areas of puncturable categories, target categories, and avoidance categories;

[0008] Train a preset classification model based on the image categories of multiple labeled ultrasonic images to obtain a target image classification model;

[0009] Train a preset segmentation model based on multiple labeled ultrasonic images to obtain a target image segmentation model;

[0010] Input the ultrasound images of the sacral plexus and sciatic nerve into the target image classification model to classify the ultrasound images of the sacral plexus and sciatic nerve, and obtain the image categories of the ultrasound images of the sacral plexus and sciatic nerve;

[0011] Determine the set of region categories in the ultrasound images of the sacral plexus and sciatic nerve based on the image categories of the ultrasound images of the sacral plexus and sciatic nerve;

[0012] Input the ultrasound images of the sacral plexus and sciatic nerve into the target image segmentation model to perform image segmentation on the ultrasound images of the sacral plexus and sciatic nerve, and obtain multiple tissue segmentation regions on the ultrasound images of the sacral plexus and sciatic nerve and the region categories of the multiple tissue segmentation regions. Among them, the region categories of the tissue segmentation regions include the puncturable category, the target category, and the avoidance category, and the region categories of each of the tissue segmentation regions belong to the set of region categories;

[0013] Send a prompt message based on the multiple tissue segmentation regions and the region categories of the multiple tissue segmentation regions, and the prompt message is used for puncture guidance.

[0014] Optionally, when the image category of the ultrasound image of the sacral plexus and sciatic nerve is para-sacral sciatic nerve block, the set of region categories in the ultrasound image of the sacral plexus and sciatic nerve includes the sciatic nerve category, the sacrum category, the ilium category, the gluteus maximus category, and the piriformis category; the tissue segmentation regions belonging to the puncturable category on the ultrasound image of the sacral plexus and sciatic nerve include the gluteus maximus region and the piriformis region, the tissue segmentation regions belonging to the target category on the ultrasound image of the sacral plexus and sciatic nerve include the sacral plexus and sciatic nerve regions, and the tissue segmentation regions belonging to the avoidance category on the ultrasound image of the sacral plexus and sciatic nerve include the sciatic nerve region, the sacrum region, and the ilium region;

[0015] When the image category of the ultrasound image of the sacral plexus and sciatic nerve is para-sacral sacral plexus nerve block, the set of region categories in its ultrasound image includes the gluteus maximus category, the piriformis category, the sacral plexus nerve category, the sacrum category, and the ilium category; the tissue segmentation regions belonging to the avoidance category on the ultrasound image of the sacral plexus and sciatic nerve include the sacral plexus nerve region, the sacrum region, and the ilium region.

[0016] Optionally, the training of the preset segmentation model based on the multiple labeled ultrasound images to obtain the target image segmentation model includes:

[0017] Input the labeled ultrasound images into the preset segmentation model to obtain the predicted pixel categories of each pixel point on the labeled ultrasound images, where the predicted pixel categories are the puncturable category, the target category, and the avoidance category;

[0018] Determine a predicted region corresponding to the labeled region on the labeled ultrasound image based on the predicted pixel classes of each pixel point on the labeled ultrasound image, where the predicted pixel classes within the same predicted region belong to the same class;

[0019] Obtain the intersection region and the union region between the labeled region and the corresponding predicted region;

[0020] Determine a first loss based on the ratio of the intersection region to the union region;

[0021] Use the cross - entropy loss function to determine a second loss based on the labeled pixel classes and the predicted pixel classes of each pixel point on the labeled ultrasound image, where the labeled pixel classes within the labeled regions of the same class belong to the same class;

[0022] Determine the total loss based on the first loss and the second loss;

[0023] Iteratively update the preset segmentation model until the total loss is less than a preset loss value to obtain the target image segmentation model.

[0024] Optionally, the determining the total loss based on the first loss and the second loss includes:

[0025] Perform a weighted sum of the first loss and the second loss based on a first preset weight coefficient and a second preset weight coefficient to obtain the total loss, where the sum of the first preset weight coefficient and the second preset weight coefficient is 1.

[0026] Optionally, the labeled ultrasound image and the sacral plexus and sciatic nerve ultrasound image include a puncture needle segmentation region, and the sending a prompt message based on the multiple tissue segmentation regions and the region classes of the multiple tissue segmentation regions includes:

[0027] Refine the puncture needle segmentation region to obtain a needle - refined line segment;

[0028] Perform a straight - line fitting on the needle - refined line segment to obtain a first fitted straight - line segment;

[0029] Obtain a plurality of first needle points at equally - spaced intervals on the needle - refined line segment;

[0030] Calculate the perpendicular distances of the plurality of first needle points from the first fitted straight - line segment respectively and sum them to obtain the total distance;

[0031] If the total distance is greater than a first preset distance value, determine that the puncture needle segmentation region contains the needle tip region, and send a prompt message based on the multiple tissue segmentation regions, the region classes of the multiple tissue segmentation regions, and the puncture needle segmentation region.

[0032] Optionally, before determining whether the puncture needle segmentation region contains the needle tip region, it includes:

[0033] Obtain the segmentation confidence of each segmented area, where each of the segmented areas includes a puncture needle segmented area and a plurality of tissue segmented areas;

[0034] Determine the overall confidence based on the segmentation confidence of each segmented area;

[0035] If the overall confidence is higher than the preset confidence, determine whether the puncture needle segmented area contains the needle tip area; if the overall confidence is not higher than the preset confidence, issue an instruction to adjust the posture of the ultrasound device, and re-obtain the ultrasound image of the sacral plexus and sciatic nerve taken.

[0036] Optionally, the issuing of the prompt information based on the plurality of tissue segmented areas, the area categories of the plurality of tissue segmented areas, and the puncture needle segmented area includes:

[0037] Generate a puncture direction line of the puncture needle on the ultrasound image of the sacral plexus and sciatic nerve based on the puncture needle segmented area;

[0038] Determine whether the puncture direction line passes through the tissue segmented area of the puncturable category;

[0039] Determine whether the puncture direction line passes through the area of the target category. If the puncture direction line does not pass through, adjust it so that the puncture direction line passes through the area of the target category. At the same time, according to whether the puncture direction line passes through the tissue segmented area of the ilium category or the sacrum category, if it passes through, adjust it so that the puncture direction line does not pass through the tissue segmented area of the ilium category or the sacrum. After that, issue an instruction to continue puncturing in the current puncture direction; at the same time, according to the distance between the puncture needle tip position and the target category, if the distance between the puncture needle tip position and the target category is less than the preset value, give a warning prompt.

[0040] In a second aspect, the sacral plexus and sciatic nerve ultrasound image recognition and puncture guidance device provided by the present application includes:

[0041] A first acquisition module for acquiring ultrasound scan videos of multiple different sections of an ultrasound device;

[0042] A frame processing module for performing frame processing on the ultrasound scan video to obtain a plurality of consecutive single-frame ultrasound images;

[0043] A second acquisition module for acquiring the labeled ultrasound image obtained by labeling the single-frame ultrasound image and the image category of the labeled ultrasound image. Among them, the single-frame ultrasound images obtained from the ultrasound scan videos of different sections belong to different image categories. The labeled ultrasound image includes a labeled area of the puncturable category, a labeled area of the target category, and a labeled area of the avoidance category;

[0044] The first training module is configured to train a preset classification model based on the image categories of multiple annotated ultrasound images to obtain a target image classification model;

[0045] The second training module is configured to train a preset segmentation model based on multiple annotated ultrasound images to obtain a target image segmentation model;

[0046] The classification module is configured to input the ultrasound image of the sacral plexus and sciatic nerve into the target image classification model to classify the ultrasound image of the sacral plexus and sciatic nerve, and obtain the image category of the ultrasound image of the sacral plexus and sciatic nerve;

[0047] The determination module is configured to determine a set of regional categories in the ultrasound image of the sacral plexus and sciatic nerve based on the image category of the ultrasound image of the sacral plexus and sciatic nerve;

[0048] The segmentation module is configured to input the ultrasound image of the sacral plexus and sciatic nerve into the target image segmentation model to perform image segmentation on the ultrasound image of the sacral plexus and sciatic nerve, and obtain multiple tissue segmentation regions on the ultrasound image of the sacral plexus and sciatic nerve and the regional categories of the multiple tissue segmentation regions, wherein the regional categories of the tissue segmentation regions include a puncturable category, a target category, and an avoidance category, and the regional categories of each tissue segmentation region belong to the set of regional categories;

[0049] The prompt module is configured to send a prompt message based on the multiple tissue segmentation regions and the regional categories of the multiple tissue segmentation regions, and the prompt message is used for puncture guidance.

[0050] In a third aspect, an electronic device provided by the present application includes a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program in the memory to implement the steps in the method for identifying and puncture guiding the ultrasound image of the sacral plexus and sciatic nerve provided by the present application.

[0051] In a fourth aspect, a computer-readable storage medium provided by the present application stores multiple instructions, and the instructions are suitable for being loaded by a processor to implement the steps in the method for identifying and puncture guiding the ultrasound image of the sacral plexus and sciatic nerve provided by the present application.

[0052] In a fifth aspect, a computer program product provided by the present application includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps in the method for identifying and puncture guiding the ultrasound image of the sacral plexus and sciatic nerve provided by the present application are implemented.

[0053] In this application, compared with the related art, ultrasonic scanning videos of multiple different sections of an ultrasonic device are obtained; the ultrasonic scanning videos are frame-processed to obtain a plurality of consecutive single-frame ultrasonic images; labeled ultrasonic images obtained by labeling the single-frame ultrasonic images and the image categories of the labeled ultrasonic images are obtained. Among them, the single-frame ultrasonic images obtained from the ultrasonic scanning videos of different sections belong to different image categories. The labeled ultrasonic images include labeled areas of the puncturable category, target category, and avoidance category; a preset classification model is trained based on the image categories of the multiple labeled ultrasonic images to obtain a target image classification model; a preset segmentation model is trained based on the multiple labeled ultrasonic images to obtain a target image segmentation model; the sacral plexus and sciatic nerve ultrasonic image is input into the target image classification model to classify the sacral plexus and sciatic nerve ultrasonic image, and the image category of the sacral plexus and sciatic nerve ultrasonic image is obtained; a set of regional categories in the sacral plexus and sciatic nerve ultrasonic image is determined based on the image category of the sacral plexus and sciatic nerve ultrasonic image; the sacral plexus and sciatic nerve ultrasonic image is input into the target image segmentation model to perform image segmentation on the sacral plexus and sciatic nerve ultrasonic image, and multiple tissue segmentation regions on the sacral plexus and sciatic nerve ultrasonic image and the regional categories of the multiple tissue segmentation regions are obtained. Among them, the regional categories of the tissue segmentation regions include the puncturable category, target category, and avoidance category, and the regional categories of each tissue segmentation region belong to the set of regional categories; a prompt message is sent based on the multiple tissue segmentation regions and the regional categories of the multiple tissue segmentation regions, and the prompt message is used for puncture guidance. This application can improve the accuracy of identifying the sacral plexus and sciatic nerve ultrasonic image and guiding puncture. Description of the Drawings

[0054] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0055] Figure 1 It is a schematic diagram of the scenario of the sacral plexus and sciatic nerve ultrasonic image recognition and puncture guidance system provided by the embodiment of the present application;

[0056] Figure 2 It is a schematic flowchart of an embodiment of the method for recognizing the sacral plexus and sciatic nerve ultrasonic image and guiding puncture provided by the embodiment of the present application;

[0057] Figure 3 It is a schematic diagram of the sacral plexus and sciatic nerve ultrasonic image with the image category of sacral paravertebral sacral plexus nerve block provided by the embodiment of the present application;

[0058] Figure 4It is a schematic diagram of the adjusted direction of the puncture needle in the ultrasound image of the sacral plexus and sciatic nerve with the image category of para-sacral sacral plexus nerve block provided by an embodiment of the present application;

[0059] Figure 5 It is a schematic diagram of the ultrasound image of the sacral plexus and sciatic nerve with the image category of para-sacral sciatic nerve block provided by an embodiment of the present application;

[0060] Figure 6 It is a schematic diagram of the structure of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device provided by an embodiment of the present application;

[0061] Figure 7 It is a schematic diagram of the structure of the electronic device provided by an embodiment of the present application. Detailed implementation manners

[0062] It should be noted that the principle of the present application is illustrated by being implemented in a suitable computing environment. The following description is based on the specific embodiments of the present application illustrated, and it should not be regarded as limiting other specific embodiments of the present application not detailed herein.

[0063] In the following description of the present application, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0064] In the following description of the present application, the terms "first / second / third" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0066] In order to improve the effect of sacral plexus sciatic nerve ultrasound image recognition and puncture guidance, an embodiment of the present application provides a sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method, a sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device, an electronic device, a computer-readable storage medium, and a computer program product. Among them, the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method can be executed by the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device, or by an electronic device integrated with the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device.

[0067] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0068] Please refer to Figure 1 , the present application also provides a sacral plexus sciatic nerve ultrasound image recognition and puncture guidance system. As Figure 1 shown, the electronic device 100 of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance system, and the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device provided by the present application is integrated in the electronic device 100.

[0069] Among them, the electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile electronic devices equipped with a processor such as smartphones, tablets, palmtop computers, laptop computers, smart speakers, or fixed electronic devices equipped with a processor such as desktop computers, televisions, servers, and industrial devices.

[0070] In addition, as Figure 1 shown, the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance system may further include a memory 200 for storing original data, intermediate data, and result data.

[0071] In the embodiments of the present application, the memory 200 can be a cloud memory. Cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that combines a large number of different types of storage devices (storage devices are also called storage nodes) in the network through functions such as cluster applications, grid technology, and distributed file systems, and collaborates through application software or application interfaces to jointly provide data storage and business access functions to the outside.

[0072] Currently, the storage method of the storage system is as follows: Create a logical volume. When creating a logical volume, physical storage space is allocated for each logical volume. This physical storage space may be a certain storage device or the disks of several storage devices. The client stores data on a certain logical volume, that is, stores the data on the file system. The file system divides the data into many parts, and each part is an object. The object not only contains the data but also contains additional information such as data identifiers (ID, ID entity). The file system writes each object into the physical storage space of the logical volume respectively, and the file system will record the storage location information of each object. Thus, when the client requests to access the data, the file system can enable the client to access the data according to the storage location information of each object.

[0073] The process of the storage system allocating physical storage space for the logical volume is specifically as follows: According to the capacity estimation of the objects stored in the logical volume (this estimation often has a large margin relative to the capacity of the actually stored objects) and the group of the redundant array of independent disks (RAID, Redundant Array of Independent Disk), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space for the logical volume.

[0074] It should be noted that Figure 1 The scene schematic diagram of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance system shown is only an example. The sacral plexus sciatic nerve ultrasound image recognition and puncture guidance system and the scene described in the embodiments of the present application are for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance system and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0075] The following will be described in detail respectively. It should be noted that the serial numbers of the following embodiments do not limit the preferred order of the embodiments.

[0076] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an embodiment of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method provided by the embodiments of the present application. As Figure 2 shown, the flow of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method provided by the present application is as follows:

[0077] 201. Obtain ultrasound scanning videos of multiple different sections of the ultrasound device.

[0078] In the embodiments of the present application, the ultrasound device can be operated and moved by a robot or a medical worker. During the movement of the ultrasound device, ultrasound scanning videos of different sections are collected. Among them, the single-frame ultrasound images obtained from the ultrasound scanning videos of different sections belong to different image categories, and the different image categories can be para-sacral sciatic nerve block, intertrochanteric sciatic nerve block, and para-sacral sacral plexus nerve block.

[0079] 202. Perform frame splitting on the ultrasound scanning video to obtain a plurality of consecutive single-frame ultrasound images.

[0080] 203. Obtain the labeled ultrasound image obtained by labeling the single-frame ultrasound image and the image category of the labeled ultrasound image.

[0081] Among them, the single-frame ultrasound images obtained from the ultrasound scanning videos of different sections belong to different image categories. The labeled ultrasound image includes a labeled area of the puncturable category, a labeled area of the target category, and a labeled area of the avoidance category.

[0082] Among them, when the image category of the labeled ultrasound image is para-sacral sciatic nerve block, the labeled area belonging to the puncturable category on the labeled ultrasound image includes the gluteus maximus area (Gluteus Maximus, GMAX) and the piriformis area (Piriformis). The labeled area belonging to the avoidance category on the labeled ultrasound image includes the sacrum area (Sacrum), the ischial tuberosity area (IT), and the ilium area (Ilium). The labeled area belonging to the target category on the labeled ultrasound image includes the sciatic nerve area (SN).

[0083] When the image category of the labeled ultrasound image is intertrochanteric sciatic nerve block, the labeled area belonging to the puncturable category on the labeled ultrasound image includes the quadratus femoris area (QFM) and the gluteus maximus area (Gluteus Maximus, GMAX). The labeled area of the target category includes the sciatic nerve area (SN), and the non-puncturable labeled area includes the ischial tuberosity (IT) and the greater trochanter of the femur area (GT).

[0084] When the image category of the labeled ultrasound image is para-sacral sacral plexus / sciatic nerve block, the labeled area belonging to the puncturable category on the labeled ultrasound image includes the gluteus maximus area (Gluteus maximus, GMAX) and the piriformis area (Piriformis). The labeled area of the target category includes the sacral plexus nerve area (sacral plexus, SP) or the sciatic nerve area (SN), and the non-puncturable labeled area includes the sacrum area (Sacrum) and the ilium area (Ilium).

[0085] 204. Train a preset classification model based on the image categories of multiple labeled ultrasound images to obtain a target image classification model.

[0086] Specifically, the target image classification model and the preset classification model are YOLOv8 models. The preset classification model is trained based on multiple annotated ultrasound images and the image categories of the annotated ultrasound images using the cross-entropy loss function to obtain the target image classification model.

[0087] Build a YOLOv8 classification model. The network structure of YOLOv8 mainly consists of the following three major parts:

[0088] Backbone: A series of convolutional and deconvolutional layers are used to extract features. At the same time, residual connections and bottleneck structures are also used to reduce the size of the network and improve performance.

[0089] Neck: The Neck part adopts the Feature Pyramid Network (FPN) structure and uses multi-scale feature fusion technology to fuse feature maps from different stages of the Backbone in order to better capture information about targets of different scales.

[0090] Head: Responsible for the final object detection and classification tasks, including a detection head and a classification head. The detection head contains a series of convolutional and deconvolutional layers for generating classification results; the classification head uses global average pooling to classify each feature map. The marked data is input into the classification network for training, and the trained model is used for the site recognition task to output the image of the site to which the current image belongs, guiding the doctor to quickly and accurately reach the best puncture position. Among them, the loss function used in the classification model is the cross-entropy loss:

[0091]

[0092] Among them, M represents the number of classification categories; y ic represents the true category of sample i; p ic represents the predicted probability of sample i.

[0093] 205. Train a preset segmentation model based on multiple annotated ultrasound images to obtain a target image segmentation model.

[0094] In the embodiments of this application, the target image segmentation model can be a trained UNet++ segmentation model. Specifically, build a UNet++ segmentation model. The UNet++ model is an encoder-decoder architecture. The encoder uses a context-aware feature encoder with convolutional blocks and residual blocks to extract multi-scale feature maps, that is, shallow image features; the decoder uses a feature decoder with convolution and deconvolution to adjust the size of the multi-scale feature maps, that is, deep image features, to achieve end-to-end segmentation; the marked data is input into the segmentation network for training, and the contour lines of the boundaries of each tissue are output.

[0095] Specifically, data collection is carried out first. Data collection includes scanning videos of three different sections: para-sacral sciatic nerve block, intertrochanteric sciatic nerve block, and para-sacral sacral plexus nerve block. Then, the videos are frame-processed to obtain continuous single-frame ultrasound images. Then, data marking is carried out. The single-frame ultrasound images are classified by a nerve block expert according to the ultrasound image video information and the scanning position to mark the site information to be recognized. On the ultrasound image of the ideal puncture interface, the boundaries of the anatomical structures to be recognized are manually marked to display the areas of the corresponding tissues, highlighting the target areas while also highlighting the dangerous areas.

[0096] In the embodiment of the present application, a preset segmentation model is trained based on multiple annotated ultrasound images to obtain a target image segmentation model, including:

[0097] (1) Input the annotated ultrasound image into the preset segmentation model to obtain the predicted pixel categories of each pixel point on the annotated ultrasound image.

[0098] In the embodiment of the present application, the predicted pixel categories are puncturable categories, target categories, or avoidance categories. Further, when the image category of the annotated ultrasound image is para-sacral sciatic nerve block, the predicted pixel categories of each pixel point on the annotated ultrasound image include gluteus maximus category, piriformis category, sciatic nerve category, sacrum category, and iliac region category, which can be set according to specific situations.

[0099] (2) Determine the predicted region corresponding to the annotated region on the annotated ultrasound image based on the predicted pixel categories of each pixel point on the annotated ultrasound image, where the predicted pixel categories within the same predicted region belong to the same category.

[0100] The pixel categories in the annotated region and the corresponding predicted region belong to the same category.

[0101] For example, the pixel points in the gluteus maximus region (Gluteus maximus, GMAX) are all of the gluteus maximus category.

[0102] (3) Obtain the intersection region and the sum of the region areas between the annotated region and the corresponding predicted region.

[0103] (4) Determine the first loss based on the ratio of the intersection region to the sum of the region areas.

[0104] In the embodiment of the present application, the first loss L dice is calculated as follows:

[0105]

[0106] where X represents the predicted region composed of the pixel set of the prediction result, and Y represents the annotated region of the pixel set of the label.

[0107] (5) Determine a second loss based on the labeled pixel categories and the predicted pixel categories of each pixel point on the labeled ultrasound image by using a cross-entropy loss function, wherein the labeled pixel categories in the labeled regions of the same category belong to the same category.

[0108] Wherein, the labeled pixel category of each pixel point is determined according to the category of each labeled region.

[0109] Specifically, the second loss L BCE is calculated as follows:

[0110]

[0111] Wherein, N represents the total number of pixel points, y i represents the true label of the pixel point, and p i represents the prediction result of the pixel point.

[0112] (6) Determine the total loss based on the first loss and the second loss.

[0113] In a specific embodiment, add the first loss and the second loss to obtain the total loss.

[0114] In another specific embodiment, determine the total loss based on the first loss and the second loss; when the total loss is less than a preset loss value after iteratively updating the preset segmentation model, obtain the target image segmentation model.

[0115] (7) When the total loss is less than a preset loss value after iteratively updating the preset segmentation model, obtain the target image segmentation model.

[0116] Wherein, the preset loss value can be set according to specific situations.

[0117] 206. Input the ultrasound image of the sacral plexus and the sciatic nerve into the target image classification model to classify the ultrasound image of the sacral plexus and the sciatic nerve, and obtain the image category of the ultrasound image of the sacral plexus and the sciatic nerve.

[0118] In the embodiments of the present application, the ultrasound device can be operated and moved by a robot or a medical worker. The ultrasound device collects the ultrasound image of the sacral plexus and the sciatic nerve during the movement.

[0119] The image category of the ultrasound image of the sacral plexus and the sciatic nerve can be sciatic nerve block beside the sacrum, sciatic nerve block between the trochanters, or sacral plexus nerve block beside the sacrum.

[0120] 207. Determine the set of region categories in the ultrasound image of the sacral plexus and the sciatic nerve based on the image category of the ultrasound image of the sacral plexus and the sciatic nerve.

[0121] Input the ultrasound images of the sacral plexus and sciatic nerve into the target image classification model to classify the ultrasound images of the sacral plexus and sciatic nerve, obtain the image categories of the ultrasound images of the sacral plexus and sciatic nerve, and determine the set of regional categories in the ultrasound images of the sacral plexus and sciatic nerve based on the image categories of the ultrasound images of the sacral plexus and sciatic nerve. Different image categories correspond to different sets of regional categories.

[0122] Specifically, the ultrasound images of the sacral plexus and sciatic nerve can be images of the sciatic nerve block beside the sacrum, the sciatic nerve block between the trochanters, or the sacral plexus nerve block beside the sacrum. The images of these three different image categories are obtained by splitting the scanning videos of three different cross-sections of the sciatic nerve block beside the sacrum, the sciatic nerve block between the trochanters, or the sacral plexus nerve block beside the sacrum into single-frame videos. The single-frame ultrasound images obtained from the scanning videos of different cross-sections belong to different image categories. For different image categories, there are corresponding different sets of regional categories, and the set of regional categories includes multiple regional categories.

[0123] As Figure 3 and Figure 4 shown, when the image category of the ultrasound image of the sacral plexus and sciatic nerve is the sacral plexus nerve block beside the sacrum, the set of regional categories in the ultrasound image of the sacral plexus and sciatic nerve includes the gluteus maximus category, the piriformis category, the sacral plexus nerve category, the sacrum category, and the ilium category; the tissue segmentation regions belonging to the puncturable category on the ultrasound image of the sacral plexus and sciatic nerve include the gluteus maximus region (Gluteus maximus, GMAX) and the piriformis region (Piriformis), and the tissue segmentation regions belonging to the avoidance category on the ultrasound image of the sacral plexus and sciatic nerve include the sacral plexus nerve region (sacral plexus, SP), the sacrum region (Sacrum), and the ilium region (Ilium). The sacrum region (Sacrum) and the ilium region (Ilium) are arranged at intervals, and the gluteus maximus region (Gluteus maximus, GMAX), the piriformis region (Piriformis), and the sacral plexus nerve region (sacral plexus, SP) are arranged in sequence between the sacrum region (Sacrum) and the ilium region (Ilium). The arrangement directions of the gluteus maximus region (Gluteus maximus, GMAX), the piriformis region (Piriformis), and the sacral plexus nerve region (sacral plexus, SP) are inclined to the arrangement direction of the sacrum region (Sacrum) and the ilium region (Ilium).

[0124] As Figure 5As shown, when the image category of the sacral plexus and sciatic nerve ultrasound image is para-sacral sciatic nerve block, the set of regional categories in the sacral plexus and sciatic nerve ultrasound image includes sciatic nerve category, sacrum category, ilium category, gluteus maximus category, and piriformis category; the tissue segmentation regions belonging to the puncturable category on the sacral plexus and sciatic nerve ultrasound image include the gluteus maximus region (Gluteus maximus, GMAX) and the piriformis region (Piriformis), and the tissue segmentation regions belonging to the avoidance category on the sacral plexus and sciatic nerve ultrasound image include the sciatic nerve region (SN), the sacrum region (Sacrum), and the ilium region (Ilium). The sacrum region (Sacrum) and the ilium region (Ilium) are arranged at intervals, and the gluteus maximus region (Gluteus maximus, GMAX), the piriformis region (Piriformis), and the sciatic nerve region (SN) are arranged in sequence between the sacrum region (Sacrum) and the ilium region (Ilium). The arrangement directions of the gluteus maximus region (Gluteus maximus, GMAX), the piriformis region (Piriformis), and the sciatic nerve region (SN) are inclined to the arrangement direction of the sacrum region (Sacrum) and the ilium region (Ilium).

[0125] 208. Input the sacral plexus and sciatic nerve ultrasound image into the target image segmentation model to perform image segmentation on the sacral plexus and sciatic nerve ultrasound image, and obtain multiple tissue segmentation regions and the regional categories of the multiple tissue segmentation regions on the sacral plexus and sciatic nerve ultrasound image.

[0126] The regional categories of the tissue segmentation regions include the puncturable category, the target category, and the avoidance category. Among them, the regional categories of each tissue segmentation region belong to the set of regional categories.

[0127] When the image category of the sacral plexus and sciatic nerve ultrasound image is para-sacral sciatic nerve block, the regional categories of the tissue segmentation regions are sciatic nerve category, sacrum category, ilium category, gluteus maximus category, and piriformis category.

[0128] When the image category of the sacral plexus and sciatic nerve ultrasound image is para-sacral sacral plexus nerve block, the regional categories of the tissue segmentation regions are gluteus maximus category, piriformis category, sacral plexus nerve category, sacrum category, and ilium category.

[0129] Specifically, input the ultrasound images of the sacral plexus and sciatic nerve into the target image segmentation model to perform image segmentation on the ultrasound images of the sacral plexus and sciatic nerve, and obtain multiple tissue segmentation regions, a puncture needle segmentation region, and the region categories of the multiple tissue segmentation regions on the ultrasound images of the sacral plexus and sciatic nerve, including: when the region categories of the multiple tissue segmentation regions output by the target image segmentation model are not completely the same as the region categories in the region category set, a prompt message indicating that complete observation is not possible is sent to prompt the user to move the ultrasound device to re-take the ultrasound images of the sacral plexus and sciatic nerve until the region categories of the multiple tissue segmentation regions output by the target image segmentation model are completely the same as the region categories in the region category set. When the region categories of the multiple tissue segmentation regions output by the target image segmentation model are completely the same as the region categories in the region category set, output the multiple tissue segmentation regions, the puncture needle segmentation region, and the region categories of the multiple tissue segmentation regions on the ultrasound images of the sacral plexus and sciatic nerve.

[0130] 209. Send a prompt message based on the multiple tissue segmentation regions and the region categories of the multiple tissue segmentation regions.

[0131] The prompt message is used for puncture guidance.

[0132] In the embodiments of the present application, the labeled ultrasound image and the ultrasound image of the sacral plexus and sciatic nerve include a puncture needle segmentation region, and a prompt message is sent based on the multiple tissue segmentation regions and the region categories of the multiple tissue segmentation regions. The prompt message is used for puncture guidance, including:

[0133] (1) Use the ZhangSuen thinning algorithm to thin the puncture needle segmentation region to obtain a needle-thinned line segment.

[0134] (2) Perform linear fitting on the needle-thinned line segment to obtain a first fitted line segment.

[0135] Specifically, the least squares method can be used to perform linear fitting on the needle-thinned line segment to obtain a first fitted line segment.

[0136] (3) Obtain multiple first needle points at equal intervals on the needle-thinned line segment.

[0137] Among them, the multiple first needle points include the two endpoints of the needle-thinned line segment.

[0138] (4) Calculate the perpendicular distances of the multiple first needle points from the first fitted line segment respectively and sum them to obtain the total distance.

[0139] (5) If the total distance is greater than the first preset distance value, it is determined that the puncture needle segmentation region contains the needle tip region, and a prompt message is sent based on the multiple tissue segmentation regions, the region categories of the multiple tissue segmentation regions, and the puncture needle segmentation region.

[0140] Among them, the first preset distance value is set according to specific circumstances.

[0141] If the total distance is greater than the first preset distance value, it indicates that the needle-thinned line segment has a large difference from the straight line, indicating that the puncture needle segmentation area contains the needle tip area and the next prompt can be given.

[0142] In the embodiment of the present application, before determining whether the puncture needle segmentation area contains the needle tip area, it includes:

[0143] (1) Obtain the segmentation confidence of each segmentation area, where each segmentation area includes the puncture needle segmentation area and multiple tissue segmentation areas.

[0144] Specifically, the segmentation confidence of the segmentation area is the sum of the probabilities that the pixel categories of each pixel point in the segmentation area are predicted correctly.

[0145] (2) Determine the overall confidence based on the segmentation confidence of each segmentation area.

[0146] Specifically, determine the weight coefficient of each segmentation area based on the area size of each segmentation area. Weight-sum the segmentation confidence of each segmentation area based on the weight coefficient of each segmentation area to obtain the overall confidence. Among them, the larger the area of the segmentation area, the smaller the weight coefficient of the segmentation area.

[0147] (3) If the overall confidence is higher than the preset confidence, determine whether the puncture needle segmentation area contains the needle tip area; if the overall confidence is not higher than the preset confidence, issue an instruction to adjust the posture of the ultrasonic device, and re-obtain the ultrasonic image of the sacral plexus and sciatic nerve taken.

[0148] In a specific embodiment, generate a puncture direction line of the puncture needle based on the puncture needle segmentation area on the ultrasonic image of the sacral plexus and sciatic nerve; determine whether the puncture direction line passes through the area of the target category, if it does not pass through, adjust it so that the puncture direction line passes through the area of the target category, and at the same time, according to whether the puncture direction line passes through the tissue segmentation area of the ilium category or the sacrum category, if it passes through, adjust it so that the puncture direction line does not pass through the tissue segmentation area of the ilium or the sacrum, and then issue an instruction to continue puncturing in the current puncture direction; at the same time, according to the distance between the puncture needle tip position and the target category, if the distance is less than the preset value, give a warning prompt.

[0149] In another specific embodiment, issue a prompt message based on multiple tissue segmentation areas, the area categories of multiple tissue segmentation areas, and the puncture needle segmentation area, including:

[0150] (1) Generate a puncture direction line of the puncture needle based on the puncture needle segmentation area on the ultrasonic image of the sacral plexus and sciatic nerve.

[0151] In a specific embodiment, the first fitted straight line segment is determined as the puncture direction line of the puncture needle.

[0152] In another specific embodiment, based on the puncture needle segmentation region, a puncture direction line of the puncture needle is generated on the ultrasound image of the sacral plexus and the sciatic nerve, including: drawing a normal line through the midpoint of the first fitted straight line segment to divide the puncture needle segmentation region into two sub-segmentation regions. The line segment of the needle thinning line segment located in the sub-segmentation region with the larger area is used as the first line segment. Linear fitting is performed on the first line segment to obtain a second fitted straight line segment. The straight line where the second fitted straight line segment is located is determined as the puncture direction line.

[0153] Since the needle tip is generally triangular, removing the lines in the sub-segmentation region with the smaller area where the needle tip is located and only retaining the first line segment for fitting can improve the accuracy of the puncture direction line.

[0154] (2) Determine whether the puncture direction line passes through the tissue segmentation region of the puncturable category.

[0155] Specifically, if the extension line of the puncture direction line passes through the tissue segmentation region of the puncturable category, then the puncture direction line passes through the tissue segmentation region of the puncturable category.

[0156] (3) If the puncture direction line passes through the tissue segmentation region of the puncturable category, then determine whether the puncture direction line passes through the tissue segmentation region of the avoidance category.

[0157] Specifically, if the extension line of the puncture direction line passes through the tissue segmentation region of the avoidance category, then the puncture direction line passes through the tissue segmentation region of the avoidance category.

[0158] If the puncture direction line passes through the tissue segmentation region of the puncturable category, then determine whether the puncture direction line passes through the tissue segmentation region of the avoidance category. If the puncture direction line does not pass through the tissue segmentation region of the puncturable category, an alarm signal for stopping puncture is sent, enabling the user to adjust the direction and puncture again.

[0159] (4) If the puncture direction line does not pass through the tissue segmentation region of the avoidance category, then an instruction to continue puncturing in the current puncture direction is sent.

[0160] If the puncture direction line does not pass through the tissue segmentation region of the avoidance category, indicating that puncture can continue, then an instruction to continue puncturing in the current puncture direction is sent.

[0161] Further, if the puncture direction line does not pass through the tissue segmentation region of the avoidance category, then an instruction to continue puncturing in the current puncture direction is sent, including:

[0162] (1) If the puncture direction line does not pass through the tissue segmentation region of the avoidance category, then calculate the minimum distance between the needle tip region and the tissue segmentation region of the puncturable category.

[0163] (2) If the minimum distance between the needle tip area and the tissue segmentation area of the puncturable category is not less than the second preset distance value, an instruction to continue puncturing in the current puncturing direction is issued.

[0164] If the minimum distance between the needle tip area and the tissue segmentation area of the puncturable category is not less than the second preset distance value, an instruction to continue puncturing in the current puncturing direction is issued. If the minimum distance between the needle tip area and the tissue segmentation area of the puncturable category is less than the second preset distance value, an alarm signal to stop puncturing is issued.

[0165] In a specific embodiment, the end point of the needle refinement line segment located in the sub-segmentation area with a smaller area is determined as the needle tip point. If the minimum distance between the needle tip point and the tissue segmentation area of the puncturable category is not less than the second preset distance value, an instruction to continue puncturing in the current puncturing direction is issued. If the minimum distance between the needle tip point and the tissue segmentation area of the puncturable category is less than the second preset distance value, an alarm signal to stop puncturing is issued.

[0166] To facilitate better implementation of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method provided by the embodiments of the present application, the embodiments of the present application further provide a sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device based on the above sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method. The meanings of the nouns are the same as those in the above sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method. For specific implementation details, please refer to the description in the above method embodiments.

[0167] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device provided by the embodiments of the present application. The sacral plexus sciatic nerve ultrasound image recognition and puncture guidance device may include:

[0168] The first acquisition module 701 is used to acquire ultrasound scanning videos of multiple different sections of the ultrasound device;

[0169] The frame splitting processing module 702 is used to perform frame splitting processing on the ultrasound scanning video to obtain a continuous plurality of single-frame ultrasound images;

[0170] The second acquisition module 703 is used to acquire the labeled ultrasound image obtained by labeling the single-frame ultrasound image and the image category of the labeled ultrasound image. Among them, the single-frame ultrasound images obtained from the ultrasound scanning videos of different sections belong to different image categories. The labeled ultrasound image includes a labeled area of the puncturable category and a labeled area of the avoidance category;

[0171] The first training module 704 is configured to train a preset classification model based on multiple annotated ultrasound images and the image categories of the annotated ultrasound images to obtain a target image classification model;

[0172] The second training module 705 is configured to train a preset segmentation model based on multiple annotated ultrasound images to obtain a target image segmentation model;

[0173] The classification module 706 is configured to input the ultrasound image of the sacral plexus and the sciatic nerve into the target image classification model to classify the ultrasound image of the sacral plexus and the sciatic nerve, so as to obtain the image category of the ultrasound image of the sacral plexus and the sciatic nerve;

[0174] The determination module 707 is configured to determine a set of regional categories in the ultrasound image of the sacral plexus and the sciatic nerve based on the image category of the ultrasound image of the sacral plexus and the sciatic nerve;

[0175] The segmentation module 708 is configured to input the ultrasound image of the sacral plexus and the sciatic nerve into the target image segmentation model to perform image segmentation on the ultrasound image of the sacral plexus and the sciatic nerve, so as to obtain multiple tissue segmentation regions on the ultrasound image of the sacral plexus and the sciatic nerve and the regional categories of the multiple tissue segmentation regions. The regional categories of the tissue segmentation regions include a puncturable category, a target category, and an avoidance category. Among them, the regional categories of each tissue segmentation region belong to the set of regional categories;

[0176] The prompting module 709 is configured to issue a prompting message based on the multiple tissue segmentation regions and the regional categories of the multiple tissue segmentation regions, and the prompting message is used for puncture guidance.

[0177] For the specific implementation of each of the above modules, reference may be made to the previous embodiments, which will not be elaborated herein.

[0178] An embodiment of the present application further provides an electronic device, including a memory and a processor. The processor is configured to execute the steps in the method for identifying and puncture guiding the ultrasound image of the sacral plexus and the sciatic nerve provided in this embodiment by calling a computer program stored in the memory.

[0179] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of the electronic device provided in an embodiment of the present application.

[0180] The electronic device may include a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, an input unit 104, and other components. Those skilled in the art can understand that the structural diagram of the electronic device shown in the figure does not constitute a limitation on the electronic device, and it may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Among them:

[0181] The processor 101 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 102, and by invoking the data stored in the memory 102, it performs various functions of the electronic device and processes data. Optionally, the processor 101 may include one or more processing cores; optionally, the processor 101 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 101 either.

[0182] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store the data created according to the use of the electronic device. In addition, the memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. Correspondingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0183] The electronic device also includes a power supply 103 that powers each component. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 103 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0184] The electronic device may further include an input unit 104, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0185] Although not shown, the electronic device may also include a display unit 105, an image acquisition component, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 101 in the electronic device will load the executable code corresponding to one or more computer programs into the memory 102 according to the following instructions, and the processor 101 will execute the steps in the sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method provided in this application, such as:

[0186] Obtain ultrasonic scanning videos of multiple different sections of an ultrasonic device; perform frame division on the ultrasonic scanning videos to obtain multiple consecutive single-frame ultrasonic images; obtain the annotated ultrasonic images obtained by annotating the single-frame ultrasonic images and the image categories of the annotated ultrasonic images. Among them, the single-frame ultrasonic images obtained from the ultrasonic scanning videos of different sections belong to different image categories, and the annotated ultrasonic images include annotated regions of the puncturable category, target category, and avoidance category; train a preset classification model based on the image categories of multiple annotated ultrasonic images to obtain a target image classification model; train a preset segmentation model based on multiple annotated ultrasonic images to obtain a target image segmentation model; input the sacral plexus and sciatic nerve ultrasonic images into the target image classification model to classify the sacral plexus and sciatic nerve ultrasonic images, and obtain the image categories of the sacral plexus and sciatic nerve ultrasonic images; determine the set of region categories in the sacral plexus and sciatic nerve ultrasonic images based on the image categories of the sacral plexus and sciatic nerve ultrasonic images; input the sacral plexus and sciatic nerve ultrasonic images into the target image segmentation model to perform image segmentation on the sacral plexus and sciatic nerve ultrasonic images, and obtain multiple tissue segmentation regions on the sacral plexus and sciatic nerve ultrasonic images and the region categories of the multiple tissue segmentation regions. Among them, the region categories of the tissue segmentation regions include the puncturable category, target category, and avoidance category, and the region categories of each tissue segmentation region belong to the set of region categories; send a prompt message based on the multiple tissue segmentation regions and the region categories of the multiple tissue segmentation regions, and the prompt message is used for puncture guidance.

[0187] It should be noted that the electronic device provided in the embodiments of the present application and the method for identifying and guiding puncture of the sacral plexus and sciatic nerve ultrasonic images in the above embodiments belong to the same concept. The specific implementation process is detailed in the above relevant embodiments and will not be repeated here.

[0188] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program stored thereon is executed by the processor of the electronic device provided in the embodiments of the present application, the processor of the electronic device executes the steps in the method for identifying and guiding puncture of the sacral plexus and sciatic nerve ultrasonic images provided by the present application. Among them, the storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0189] The present application also provides a computer program product or computer program. The computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes various optional implementation manners of the above method for identifying and guiding puncture of the sacral plexus and sciatic nerve ultrasonic images.

[0190] The above has introduced in detail a method and device for sacral plexus sciatic nerve ultrasound image recognition and puncture guidance provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

[0191] It should be noted that when the above embodiments of the present application are applied to specific products or technologies, involving relevant data of users, user permission or consent is required, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A method for sacral plexus sciatic nerve ultrasound image recognition and puncture guidance, characterized in that, Including: Obtaining ultrasonic scanning videos of multiple different cross-sections of an ultrasonic device; Performing frame division processing on the ultrasonic scanning videos to obtain a plurality of consecutive single-frame ultrasonic images; Obtaining the annotated ultrasonic images obtained by annotating the single-frame ultrasonic images and the image categories of the annotated ultrasonic images. Among them, the single-frame ultrasonic images obtained from the ultrasonic scanning videos of different cross-sections belong to different image categories. The annotated ultrasonic images include annotated areas of puncturable categories, target categories, and avoidance categories; Training a preset classification model based on the image categories of multiple annotated ultrasonic images to obtain a target image classification model; Training a preset segmentation model based on multiple annotated ultrasonic images to obtain a target image segmentation model; Inputting the ultrasonic image of the sacral plexus and sciatic nerve into the target image classification model to classify the ultrasonic image of the sacral plexus and sciatic nerve, and obtaining the image category of the ultrasonic image of the sacral plexus and sciatic nerve; Determining a set of regional categories in the ultrasonic image of the sacral plexus and sciatic nerve based on the image category of the ultrasonic image of the sacral plexus and sciatic nerve; Inputting the ultrasonic image of the sacral plexus and sciatic nerve into the target image segmentation model to perform image segmentation on the ultrasonic image of the sacral plexus and sciatic nerve, and obtaining a plurality of tissue segmentation regions on the ultrasonic image of the sacral plexus and sciatic nerve and the regional categories of the plurality of tissue segmentation regions. Among them, the regional categories of the tissue segmentation regions include puncturable categories, target categories, and avoidance categories, and the regional categories of each tissue segmentation region belong to the set of regional categories; Sending a prompt message based on the plurality of tissue segmentation regions and the regional categories of the plurality of tissue segmentation regions, where the prompt message is used for puncture guidance.

2. The method for identifying and guiding puncture of the ultrasonic image of the sacral plexus and sciatic nerve according to claim 1, wherein When the image category of the ultrasonic image of the sacral plexus and sciatic nerve is sciatic nerve block beside the sacrum, the set of regional categories in the ultrasonic image of the sacral plexus and sciatic nerve includes sciatic nerve category, sacrum category, ilium category, gluteus maximus category, and piriformis category; the tissue segmentation regions belonging to the puncturable category on the ultrasonic image of the sacral plexus and sciatic nerve include the gluteus maximus region and the piriformis region, the tissue segmentation regions belonging to the target category on the ultrasonic image of the sacral plexus and sciatic nerve include the sacral plexus and sciatic nerve region, and the tissue segmentation regions belonging to the avoidance category on the ultrasonic image of the sacral plexus and sciatic nerve include the sciatic nerve region, the sacrum region, and the ilium region; When the image category of the ultrasonic image of the sacral plexus and sciatic nerve is sacral plexus nerve block beside the sacrum, the set of regional categories in its ultrasonic image includes gluteus maximus category, piriformis category, sacral plexus nerve category, sacrum category, and ilium category; the tissue segmentation regions belonging to the avoidance category on the ultrasonic image of the sacral plexus and sciatic nerve include the sacral plexus nerve region, the sacrum region, and the ilium region.

3. The method for identifying and puncture guiding of sacral plexus sciatic nerve ultrasound images according to claim 2, characterized in that, The training of the preset segmentation model based on multiple annotated ultrasonic images to obtain a target image segmentation model includes: Input the labeled ultrasound image into a preset segmentation model to obtain the predicted pixel categories of each pixel point on the labeled ultrasound image, where the predicted pixel categories are puncturable categories, target categories, or avoidance categories; Determine the predicted region corresponding to the labeled region on the labeled ultrasound image based on the predicted pixel categories of each pixel point on the labeled ultrasound image, where the predicted pixel categories within the same predicted region belong to the same category; Obtain the intersection region and the union region between the labeled region and the corresponding predicted region; Determine the first loss based on the ratio of the intersection region to the union region; Use the cross-entropy loss function to determine the second loss based on the labeled pixel categories and the predicted pixel categories of each pixel point on the labeled ultrasound image, where the labeled pixel categories within the labeled region of the same category belong to the same category; Determine the total loss based on the first loss and the second loss; Iteratively update the preset segmentation model until the total loss is less than a preset loss value to obtain the target image segmentation model.

4. The sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method according to claim 3, characterized in that, The determining the total loss based on the first loss and the second loss includes: Performing weighted summation on the first loss and the second loss based on a first preset weight coefficient and a second preset weight coefficient to obtain the total loss, where the sum of the first preset weight coefficient and the second preset weight coefficient is 1.

5. The method for identifying sacral plexus sciatic nerve ultrasound images and puncture guidance according to claim 4, wherein The labeled ultrasound image and the sacral plexus and sciatic nerve ultrasound image include a puncture needle segmentation region, and the sending a prompt message based on the multiple tissue segmentation regions and the region categories of the multiple tissue segmentation regions includes: Refine the puncture needle segmentation region to obtain a needle-refined line segment; Perform linear fitting on the needle-refined line segment to obtain a first fitted line segment; Obtain multiple first needle points at equal-spacing intervals on the needle-refined line segment; Calculate the perpendicular distances of the multiple first needle points from the first fitted line segment respectively and sum them to obtain the total distance; If the total distance is greater than a first preset distance value, determine that the puncture needle segmentation region contains a needle tip region, and send a prompt message based on the multiple tissue segmentation regions, the region categories of the multiple tissue segmentation regions, and the puncture needle segmentation region.

6. The sacral plexus sciatic nerve ultrasound image recognition and puncture guidance method according to claim 5, characterized in that, Before determining whether the puncture needle segmentation region contains a needle tip region, it includes: Obtain the segmentation confidence of each segmentation region, where each of the segmentation regions includes a puncture needle segmentation region and multiple tissue segmentation regions; Determine the overall confidence based on the segmentation confidence of each segmentation region; If the overall confidence is higher than a preset confidence, determine whether the puncture needle segmentation region contains a needle tip region; if the overall confidence is not higher than the preset confidence, send an instruction to adjust the posture of the ultrasound device, and re-obtain the captured sacral plexus and sciatic nerve ultrasound image.

7. The method for identifying and puncture guiding of sacral plexus sciatic nerve ultrasound images according to claim 6, characterized in that The sending a prompt message based on the multiple tissue segmentation regions, the region categories of the multiple tissue segmentation regions, and the puncture needle segmentation region includes: Generate a puncture direction line of the puncture needle on the sacral plexus and sciatic nerve ultrasound image based on the puncture needle segmentation region; Determine whether the puncture direction line passes through the area of the target category. If it does not pass through, adjust it so that the puncture direction line passes through the area of the target category. At the same time, according to whether the puncture direction line passes through the tissue segmentation area of the ilium category or the sacrum category, if it passes through, adjust it so that the puncture direction line does not pass through the tissue segmentation area of the ilium category or the sacrum. Then issue an instruction to continue puncturing in the current puncture direction; at the same time, according to the distance between the puncture needle tip position and the target category, if the distance between the puncture needle tip position and the target category is less than the preset value, give a warning prompt.

8. An ultrasound image recognition and puncture guidance device for the sciatic nerve of the sacral plexus, characterized in that, Including: A first acquisition module for acquiring ultrasound scan videos of multiple different cross-sections of an ultrasound device; A frame processing module for performing frame processing on the ultrasound scan video to obtain a series of consecutive single-frame ultrasound images; A second acquisition module for acquiring the labeled ultrasound images obtained by labeling the single-frame ultrasound images and the image categories of the labeled ultrasound images. Among them, the single-frame ultrasound images obtained from the ultrasound scan videos of different cross-sections belong to different image categories. The labeled ultrasound images include labeled areas of puncturable categories, target categories, and avoidance categories; A first training module for training a preset classification model based on the image categories of multiple labeled ultrasound images to obtain a target image classification model; A second training module for training a preset segmentation model based on multiple labeled ultrasound images to obtain a target image segmentation model; A classification module for inputting the ultrasound images of the sacral plexus and sciatic nerve into the target image classification model to classify the ultrasound images of the sacral plexus and sciatic nerve and obtain the image categories of the ultrasound images of the sacral plexus and sciatic nerve; A determination module for determining the set of regional categories in the ultrasound images of the sacral plexus and sciatic nerve based on the image categories of the ultrasound images of the sacral plexus and sciatic nerve; A segmentation module for inputting the ultrasound images of the sacral plexus and sciatic nerve into the target image segmentation model to perform image segmentation on the ultrasound images of the sacral plexus and sciatic nerve, and obtain multiple tissue segmentation areas and the regional categories of the multiple tissue segmentation areas on the ultrasound images of the sacral plexus and sciatic nerve. Among them, the regional categories of the tissue segmentation areas include puncturable categories, target categories, and avoidance categories, and the regional categories of each tissue segmentation area belong to the set of regional categories; A prompt module for issuing a prompt message based on multiple tissue segmentation areas and the regional categories of the multiple tissue segmentation areas, and the prompt message is used for puncture guidance.

9. An electronic device, characterized in that, Including a memory and a processor. The memory stores a computer program, and the processor is used to run the computer program in the memory to execute the steps in the method for identifying and guiding the puncture of the ultrasound images of the sacral plexus and sciatic nerve according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by the processor to execute the steps in the method for identifying and guiding the puncture of the ultrasound images of the sacral plexus and sciatic nerve according to any one of claims 1 to 7.

Images