Ultrasonic interventional guidance devices, media and electronic equipment

Abstract

The present invention provides an ultrasonic interventional guidance device, a medium and an electronic device. The ultrasonic interventional guidance device includes: an ultrasonic image acquisition module for acquiring ultrasonic images in real time; an image processing module for processing one or more frames of target images in the ultrasonic image using a deep learning model to identify interventional targets, interventional obstacles and / or interventional instruments in the target image; an interventional path planning module for generating an interventional path according to the processing results of the deep learning model. The ultrasonic interventional guidance device can assist medical personnel in completing the planning of ultrasonic interventional paths, reduce the requirements for medical personnel's seniority and experience in ultrasonic interventional surgery, and is conducive to improving the efficiency of patients undergoing interventional surgery and alleviating the problem of tight medical resources.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an ultrasonic intervention guidance device, a medium and an electronic device. Background Art

[0002] Ultrasound detection is an important part of clinical imaging examinations and an important basis for clinical diagnosis and the formulation of reasonable treatment plans. With the continuous development of ultrasound technology, it has been widely used in the diagnosis of clinical diseases. As a branch of modern ultrasound medicine, ultrasound intervention technology is a new technology developed on the basis of ultrasound imaging to further meet the needs of clinical diagnosis and treatment. The main feature of ultrasound intervention is to complete various puncture biopsies, X-ray angiography, aspiration, intubation, injection treatment and other operations under the monitoring or guidance of real-time ultrasound, thereby avoiding certain surgical operations and achieving the same effect as surgical operations.

[0003] Ultrasound interventional treatment requires that the interventional device accurately avoids obstacles such as organs and blood vessels in the human body and reaches the target location. In actual applications, the planning of the interventional path depends on the experience and subjective judgment of medical staff, and can only be completed by senior and experienced ultrasound doctors, which greatly reduces the efficiency of patients receiving interventional treatment and aggravates the shortage of medical resources. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide an ultrasonic intervention guidance device, a medium and an electronic device for solving the above-mentioned problems existing in the prior art.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present invention provides an ultrasonic interventional guidance device, a medium and an electronic device, wherein the ultrasonic interventional guidance device comprises: an ultrasonic image acquisition module, used to acquire ultrasonic images in real time; an image processing module, used to process one or more frames of target images in the ultrasonic image using a deep learning model to identify interventional targets, interventional obstacles and / or interventional instruments in the target image; and an interventional path planning module, used to generate an interventional path according to the processing results of the deep learning model.

[0006] In an embodiment of the first aspect, the ultrasound intervention guidance device further includes a display module, and the display module is used to display the ultrasound image and the intervention path.

[0007] In an embodiment of the first aspect, the ultrasonic interventional guidance device also includes a judgment module, and the judgment module is used to judge whether the intervention path passes through an intervention obstacle based on the processing results of the deep learning model. When the intervention path passes through an intervention obstacle, the display module displays the intervention path in a first display mode, and when the intervention path does not pass through an intervention obstacle, the display module displays the intervention path in a second display mode.

[0008] In an embodiment of the first aspect, the ultrasound intervention guidance device is applied to an electronic device, and the electronic device is connected to an ultrasound scanning device via a high-definition video output port to acquire the ultrasound image from the ultrasound scanning device in real time.

[0009] In an embodiment of the first aspect, the training method of the deep learning model includes: constructing a deep learning initial model; obtaining training data, the training data including ultrasound images marked with interventional targets, interventional obstacles and / or interventional instruments; using the training data to train the deep learning initial model to obtain the deep learning model; and testing the trained deep learning model.

[0010] In an embodiment of the first aspect, the interventional path planning module includes: a first endpoint acquisition unit, used to acquire a point on the interventional target as a first endpoint; a second endpoint acquisition unit, used to acquire a point on the interventional instrument as a second endpoint; and an interventional path planning unit, used to generate the interventional path based on the first endpoint, the second endpoint and the interventional obstacles in the target image.

[0011] In an embodiment of the first aspect, the intervention path planning module is further used to update the intervention path in real time according to the real-time position of the intervention target, intervention obstacle and / or intervention instrument in the target image.

[0012] In an embodiment of the first aspect, the intervention target includes a nodule, the intervention obstacle includes a tissue and / or an organ, and / or the intervention instrument includes a puncture needle.

[0013] The second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an ultrasound intervention guidance method based on deep learning, wherein the ultrasound intervention guidance method comprises: acquiring ultrasound images in real time; processing one or more frames of target images in the ultrasound images using a deep learning model to identify intervention targets, intervention obstacles and / or intervention instruments in the target images; and generating an intervention path based on the processing results of the deep learning model.

[0014] The third aspect of the present invention provides an electronic device, comprising: a memory storing a computer program; a processor communicating with the memory, and executing an ultrasound intervention guidance method based on deep learning when calling the computer program, wherein the ultrasound intervention guidance method comprises: acquiring ultrasound images in real time; processing one or more frames of target images in the ultrasound images using a deep learning model to identify intervention targets, intervention obstacles and / or intervention instruments in the target images; and generating an intervention path based on the processing results of the deep learning model.

[0015] As described above, the ultrasound intervention guidance device described in one or more embodiments of the present invention has the following beneficial effects:

[0016] The ultrasonic intervention guidance device includes an ultrasonic image acquisition module, an image processing module and an intervention path planning module, wherein the ultrasonic image acquisition module is used to acquire ultrasonic images in real time, the image processing module can use a deep learning model to process one or more frames of target images in the ultrasonic image to identify the intervention target, intervention obstacle and / or intervention instrument in the target image, and the intervention path planning module can automatically generate an intervention path according to the identification result. Medical personnel can perform ultrasonic intervention surgery according to the intervention path generated by the intervention path planning module. Therefore, the ultrasonic intervention guidance device can assist medical personnel in completing the planning of ultrasonic intervention paths, reduce the requirements for medical personnel's seniority and experience in ultrasonic intervention surgery, and help improve the efficiency of patients undergoing intervention surgery and alleviate the problem of tight medical resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Shown is a schematic structural diagram of the ultrasound interventional guidance device of the present invention in a specific embodiment.

[0018] Figure 2 Shown is a flow chart of a training method for a deep learning model of the ultrasound interventional guidance device in a specific embodiment of the present invention.

[0019] Figure 3 Shown is a structural schematic diagram of an intervention path planning module in a specific embodiment of the ultrasound intervention guidance device of the present invention.

[0020] Figure 4 Shown is a flow chart of the ultrasound intervention guidance method according to a specific embodiment of the present invention.

[0021] Figure 5 Shown is a schematic structural diagram of the electronic device of the present invention in a specific embodiment.

[0022] Component number description

[0023] 1 Ultrasound interventional guidance device

[0024] 11 Ultrasound image acquisition module

[0025] 12 Image Processing Module

[0026] 13 Intervention Path Planning Module

[0027] 131 First endpoint acquisition unit

[0028] 132 Second endpoint acquisition unit

[0029] 133 Intervention Pathway Planning Unit

[0030] 500 Electronic equipment

[0031] 510 Memory

[0032] 520 processor

[0033] 530 Display

[0034] Steps S21 to S24

[0035] Steps S41 to S43 DETAILED DESCRIPTION

[0036] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0037] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show the components related to the present invention rather than the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in actual implementation may be changed at will, and the component layout type may also be more complicated. In addition, in this article, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0038] Ultrasonic interventional treatment requires that the interventional instrument accurately avoids obstacles such as organs and blood vessels in the human body and reaches the intervention target. In practical applications, the planning of the interventional path depends on the experience and subjective judgment of medical personnel, and can only be completed by senior and experienced ultrasound doctors. This greatly reduces the efficiency of patients receiving interventional treatment and aggravates the shortage of medical resources. To address this problem, an ultrasonic interventional guidance device based on deep learning is provided in one embodiment of the present invention. For details, please refer to Figure 1 The ultrasonic intervention guidance device 1 in this embodiment includes an ultrasonic image acquisition module 11 , an image processing module 12 and an intervention path planning module 13 .

[0039] The ultrasonic image acquisition module 11 is used to acquire ultrasonic images in real time, wherein the ultrasonic images include one or more frames of ultrasonic images. The ultrasonic image acquisition module 11 is, for example, a module with an ultrasonic image acquisition function. In this case, the ultrasonic image acquisition module 11 uses an ultrasonic probe to acquire an ultrasonic image of a specific part of the target object in real time. The ultrasonic image acquisition module 11 can also be a module that is connected to an ultrasonic scanning device in communication. In this case, the ultrasonic image acquisition module 11 acquires the ultrasonic image of the specific part of the target object acquired by the ultrasonic scanning device in real time through a communication connection with the ultrasonic scanning device.

[0040] The image processing module 12 is connected to the ultrasound image acquisition module 11, and is used to process one or more frames of target images in the ultrasound image using a deep learning model to identify interventional targets, interventional obstacles and / or interventional instruments in the target image. Wherein, the target image is one or more frames of ultrasound images in the ultrasound image, and the target image includes at least one or more of interventional targets, interventional obstacles and interventional instruments. The interventional target refers to the implementation target of ultrasound interventional surgery, including but not limited to nodules in the target object's body, such as thyroid nodules. The interventional obstacle refers to an object in the target object's body that hinders the implementation of intervention, including but not limited to tissues and / or organs in the target object's body. The interventional instrument refers to a medical device used to perform interventional surgery, including but not limited to a puncture needle.

[0041] The intervention path planning module 13 is connected to the image processing module 12 and is used to generate an intervention path according to the processing result of the deep learning model. The deep learning model is, for example, a U-Net deep learning image segmentation model.

[0042] According to the above description, the ultrasonic intervention guidance device 1 described in this embodiment includes an ultrasonic image acquisition module 11, an image processing module 12 and an intervention path planning module 13, wherein the ultrasonic image acquisition module 11 is used to acquire ultrasonic images in real time, the image processing module 12 is used to process one or more frames of target images in the ultrasonic image using a deep learning model so as to identify the intervention target, intervention obstacle and / or intervention instrument in the target image, and the intervention path planning module 13 can automatically generate an intervention path according to the recognition result. Medical personnel can perform ultrasonic intervention surgery according to the intervention path generated by the intervention path planning module 13. Therefore, the ultrasonic intervention guidance device 1 can assist medical personnel in completing the planning of ultrasonic intervention paths, reduce the requirements for medical personnel's seniority and experience in ultrasonic intervention surgery, and is conducive to improving the efficiency of patients undergoing intervention surgery and alleviating the problem of tight medical resources.

[0043] In one embodiment of the present invention, the image processing module 12 is used to process each frame of ultrasound image acquired in real time by the ultrasound image acquisition module 11 in real time, so as to identify the intervention target, intervention obstacle and / or intervention instrument contained in each frame of ultrasound image. The intervention path planning module 13 is used to generate the intervention path based on the real-time processing results of each frame of ultrasound image and / or update the intervention path in real time.

[0044] According to the above description, the image processing module 12 in this embodiment can process each frame of the ultrasound image in real time, so as to identify the real-time position and / or shape of the intervention target, intervention obstacle and / or intervention instrument. The intervention path planning module 13 can update the intervention path in real time according to the real-time position and / or shape of the intervention target, intervention obstacle and / or intervention instrument in the ultrasound image. Therefore, when medical personnel use intervention instruments to perform ultrasound intervention surgery, the ultrasound intervention guidance device 1 can obtain the current intervention path in real time according to the current position of the intervention instrument, so as to better assist medical personnel in performing intervention surgery.

[0045] In one embodiment of the present invention, the ultrasound intervention guidance device further includes a display module, which is connected to the intervention path planning module and the ultrasound image acquisition module and is used to display the ultrasound image and the intervention path.

[0046] Optionally, the ultrasonic interventional guidance device also includes a judgment module, which is used to judge whether the intervention path passes through an intervention obstacle based on the processing results of the deep learning model, wherein the intervention path passes through an intervention obstacle means that the intervention path is tangent to or intersects with the intervention obstacle.

[0047] When the intervention path passes through an intervention obstacle, the display module displays the intervention path in a first display mode, for example, displays the intervention path in a first color. When the intervention path does not pass through an intervention obstacle, the display module displays the intervention path in a second display mode, for example, displays the intervention path in a second color.

[0048] Optionally, the ultrasonic intervention guidance device further comprises an early warning module. When the intervention path passes through an intervention obstacle, the early warning module is used to generate an early warning signal to prompt medical personnel to correct the intervention path and / or adjust the position, angle, posture, etc. of the intervention instrument.

[0049] In one embodiment of the present invention, the ultrasound intervention guidance device is applied to an electronic device, and the electronic device is connected to an ultrasound scanning device via a high-definition video output port to acquire the ultrasound image from the ultrasound scanning device in real time.

[0050] See also Figure 2 In one embodiment of the present invention, the training method of the deep learning model includes:

[0051] S21, build the initial deep learning model.

[0052] S22, obtaining training data, wherein the training data includes ultrasound images annotated with intervention targets, intervention obstacles and / or intervention instruments. The annotation of ultrasound images in the training data may be achieved by manual annotation or the like.

[0053] S23: Use the training data to train the deep learning initial model to obtain the deep learning model.

[0054] S24, testing the trained deep learning model.

[0055] Optionally, since ultrasound images may involve the privacy of patients, it is difficult to obtain a sufficient number of ultrasound images to train the deep learning initial model in practical applications. To address this issue, the training method of the deep learning model in this embodiment also includes: augmenting the ultrasound images in the training data. Among them, the method of augmenting the ultrasound images includes but is not limited to image flipping, image translation, image scaling, image miscutting, image cropping, etc. By augmenting the ultrasound images in the training data, a sufficient amount of training data can be obtained, thereby achieving sufficient training of the deep learning initial model, so as to obtain a high-performance deep learning model.

[0056] Optionally, the loss function used in step S23 in this embodiment may be: Wherein, A represents the prediction box in the ultrasound image, B represents the target box in the ultrasound image, (d A ,d B ) represents the Euclidean distance between the center points of the prediction box A and the target box B, d dia Indicates the diagonal length of the minimum enclosing rectangle of the prediction box A and the target box B.

[0057] See also Figure 3 In one embodiment of the present invention, the intervention path planning module 13 includes a first endpoint acquisition unit 131 , a second endpoint acquisition unit 132 and an intervention path planning unit 133 .

[0058] The first endpoint acquisition unit 131 is connected to the image processing module 12 and is used to acquire a point on the intervention target as a first endpoint. The first endpoint is, for example, the center point of the intervention target.

[0059] The second endpoint acquisition unit 132 is connected to the image processing module 12, and is used to acquire a point on the interventional instrument as the second endpoint, and the second endpoint is, for example, the point on the interventional instrument closest to the interventional target, or the center point of the interventional instrument. In particular, when the interventional instrument is a puncture needle, the second endpoint is the needle tip of the puncture needle.

[0060] The intervention path planning unit 133 is connected to the first endpoint acquisition unit 131 and the second endpoint acquisition unit 132, and is used to generate the intervention path according to the first endpoint, the second endpoint and the intervention obstacle in the target image.

[0061] Specifically, when there is no intervention obstacle on the line connecting the first endpoint and the second endpoint, the intervention path planning unit 133 obtains the line connecting the first endpoint and the second endpoint as the intervention path.

[0062] When there is an intervention obstacle on the line connecting the first endpoint and the second endpoint, the intervention path planning unit 133 prompts the medical staff to adjust the intervention position, angle and / or posture of the intervention instrument so that there is no intervention obstacle on the line connecting the first endpoint and the second endpoint, and obtains the line between the first endpoint and the adjusted second endpoint as the intervention path.

[0063] Optionally, when there is no interventional instrument in the target image or the interventional instrument in the target image has not yet entered the body of the target object, the interventional path planning unit 133 generates the interventional path according to the first endpoint and the position and shape of the interventional obstacle in the target image. For example, the interventional path planning unit 133 can generate a line segment, one endpoint of which is the first endpoint, and the other endpoint is located on the skin surface of the target object, and the line segment does not pass through any interventional obstacle. The line segment is the interventional path, and one endpoint of the line segment located on the skin surface of the target object is the intervention point of the interventional instrument.

[0064] In one embodiment of the present invention, the intervention path planning module is further used to update the intervention path in real time according to the real-time positions of the intervention target, intervention obstacle and / or intervention instrument in the target image.

[0065] In one embodiment of the present invention, the ultrasonic intervention guidance device includes an ultrasonic image acquisition module, an image processing module, an intervention path planning module and a display module.

[0066] The ultrasonic image acquisition module is connected to the ultrasonic scanning device via a high-definition video output port, and is used to acquire the ultrasonic image collected by the ultrasonic scanning device in real time.

[0067] The image processing module is connected to the ultrasound image acquisition module, and is used to use a deep learning model to perform real-time processing on each frame of the ultrasound image to obtain the interventional targets, interventional obstacles and / or interventional instruments in each frame of the ultrasound image.

[0068] The intervention path planning module is used to generate an intervention path according to the intervention target, intervention obstacle and / or intervention instrument in each frame of ultrasound image and to update the intervention path in real time.

[0069] Optionally, for any frame of the ultrasound image, the image intervention path planning module calculates the center point of the intervention target as the first endpoint according to the segmentation result of the intervention target, and obtains the endpoint of the intervention instrument as the second endpoint according to the segmentation result of the intervention instrument. With the second endpoint as the starting point and the first endpoint as the end point, combined with the intervention obstacles such as organs and blood vessels in the ultrasound image of the frame, the intervention path planning module obtains the intervention path of the intervention instrument through the AI ​​algorithm.

[0070] Optionally, for any frame of the ultrasound image, if there is an intervention obstacle in the frame of the ultrasound image, a method for the intervention path planning module to obtain the intervention path includes:

[0071] S11, obtaining a point on the intervention target in the frame of ultrasound image as a first endpoint, and obtaining an endpoint of the intervention instrument in the frame of ultrasound image as a second endpoint.

[0072] S12, obtaining a rectangular area including the first endpoint and the second endpoint in the frame of ultrasound image as a search area, and dividing the search area into multiple grids, obtaining the grids where the first endpoint, the second endpoint and the intervening obstacle are located, putting the second endpoint as the starting point into an open list, and taking the first endpoint as the target point.

[0073] S13, searching for an adjacent grid of the grid where the starting point is located, and calculating a distance between the adjacent grid and the first endpoint.

[0074] S14, taking the points corresponding to the grid as the states in the state space, and the distances between the points as the costs, and using the A* algorithm to calculate the cost estimate from the initial state to the target state.

[0075] S15, compare the cost estimation values ​​corresponding to each path from the starting point to the target point, and determine in turn whether the cost estimation value corresponding to each path from the starting point to the target point is the minimum value: if so, put the starting point into the closed list, and at the same time take the grid point adjacent to the grid where the starting point is located as the new starting point, and put the new starting point into the open list; if not, put the adjacent grid points corresponding to the path from the starting point to the target point into the closed list, and reselect the next path from the starting point to the target point.

[0076] S16, judging whether the new starting point is the target point: if so, obtaining the intervention path according to the current search result, and completing the planning of the intervention path; if not, returning to step 13.

[0077] The display module is used to display the intervention path, the ultrasound image, and the intervention target, intervention obstacle, and / or intervention instrument in real time. In addition, when the intervention path passes through an intervention obstacle, the display module displays the intervention path in a first display mode, and when the intervention path does not pass through an intervention obstacle, the display module displays the intervention path in a second display mode.

[0078] For example, taking thyroid ultrasound puncture surgery as an example, the image processing module can identify the puncture needle, organs and nodules in each frame of ultrasound image. The display module marks normal organ tissues with a red frame to enhance the warning effect. When the puncture needle contacts and enters the ultrasound image, the intervention path planning module automatically generates an intervention path based on the identification result of the puncture needle. If the intervention path does not pass through normal organ tissues, it is marked with a blue dotted line. Conversely, if the intervention path passes through normal organ tissues, it is marked with a red dotted line.

[0079] Based on the above description of the ultrasound intervention guidance device, the present invention also provides an ultrasound intervention guidance method based on deep learning. Figure 4 In one embodiment of the present invention, the ultrasound intervention guidance method includes:

[0080] S41, acquiring ultrasound images in real time.

[0081] S42, using a deep learning model to process one or more frames of target images in the ultrasound image to identify interventional targets, interventional obstacles and / or interventional instruments in the target image.

[0082] S43, generating an intervention path according to the processing result of the deep learning model.

[0083] It should be noted that the above steps S41 to S43 are Figure 1 The corresponding modules in the ultrasonic intervention guidance device 1 are shown in one-to-one correspondence, and in order to save space in the specification, they will not be described in detail here.

[0084] Based on the above description of the ultrasound intervention guidance method, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement Figure 4 The ultrasound-guided interventional method is shown.

[0085] Based on the above description of the ultrasound intervention guidance method, the present invention also provides an electronic device. Figure 5 In one embodiment of the present invention, the electronic device 500 includes a memory 510 and a processor 520. The memory 510 stores a computer program, and the processor 520 is connected to the memory 510 for executing the computer program when the computer program is called. Figure 4 The ultrasound-guided interventional method is shown.

[0086] Optionally, the electronic device 500 in this embodiment may further include a display 530, and the display 530 is communicatively connected with the memory 510 and the processor 520, and is used to display a related GUI interaction interface of the ultrasound intervention guidance method.

[0087] In summary, the ultrasonic intervention guidance device includes an ultrasonic image acquisition module, an image processing module and an intervention path planning module, wherein the ultrasonic image acquisition module is used to acquire ultrasonic images in real time, the image processing module can use a deep learning model to process one or more frames of target images in the ultrasonic image to identify the intervention target, intervention obstacle and / or intervention instrument in the target image, and the intervention path planning module can automatically generate an intervention path according to the identification result. Medical personnel can perform ultrasonic intervention surgery according to the intervention path generated by the intervention path planning module. Therefore, the ultrasonic intervention guidance device can assist medical personnel in completing the planning of ultrasonic intervention paths, reduce the requirements for medical personnel's seniority and experience in ultrasonic intervention surgery, and is conducive to improving the efficiency of patients receiving intervention surgery and alleviating the problem of tight medical resources. Therefore, the present invention effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0088] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. An ultrasound intervention guidance device based on deep learning, characterized in that: The ultrasonic intervention guidance device comprises: An ultrasonic image acquisition module, used for acquiring ultrasonic images in real time; An image processing module is used to process one or more target images in the ultrasound image using a deep learning model to identify interventional targets, interventional obstacles and interventional instruments in the target image; wherein the training method of the deep learning model includes: constructing a deep learning initial model; obtaining training data, wherein the training data includes ultrasound images annotated with interventional targets, interventional obstacles and interventional instruments; performing augmentation processing on the ultrasound images in the training data, wherein the augmentation processing includes image flipping, image translation, image scaling, image misalignment and image cropping; and training the deep learning initial model using the training data to obtain the deep learning model, wherein the loss function used in the training is A represents the prediction box in the ultrasound image, B represents the target box in the ultrasound image, (d A ,d B ) represents the Euclidean distance between the center points of the prediction box A and the target box B, d dia Representing the diagonal length of the minimum circumscribed rectangle of the prediction box A and the target box B; testing the trained deep learning model; An intervention path planning module, used to generate an intervention path according to the processing result of the deep learning model; Wherein, for any frame of the ultrasound image, if there is an intervention obstacle in the frame of the ultrasound image, the intervention path planning module generates the intervention path including: S11, acquiring a point on the intervention target in the frame of ultrasound image as a first endpoint, and acquiring an endpoint of the intervention instrument in the frame of ultrasound image as a second endpoint; S12, obtaining a rectangular area including the first endpoint and the second endpoint in the frame of ultrasound image as a search area, and dividing the search area into a plurality of grids, obtaining the grids where the first endpoint, the second endpoint, and the intervention obstacle are located, putting the second endpoint as a starting point into an open list, and taking the first endpoint as a target point; S13, searching for an adjacent grid of the grid where the starting point is located, and calculating a distance between the adjacent grid and the first endpoint; S14, taking the points corresponding to the grid as the states in the state space and the distances between the points as the costs, and using the A* algorithm to calculate the cost estimate from the starting point to the target point; S15, comparing the cost estimation values ​​corresponding to the paths from the starting point to the target point, and determining in turn whether the cost estimation values ​​corresponding to the paths from the starting point to the target point are minimum values; wherein, for any path, if the cost estimation value corresponding to the path is the minimum value, the starting point is put into a closed list, and the grid point adjacent to the grid where the starting point is located is taken as a new starting point, and the new starting point is put into an open list; if not, the adjacent grid points corresponding to the path are put into a closed list; S16, determining whether the new starting point is the target point: if so, obtaining the intervention path according to the current search result, otherwise returning to step S13; The ultrasonic intervention guidance device further comprises an early warning module, which is used to generate an early warning signal to prompt medical personnel to correct the intervention path and adjust the position, angle and posture of the intervention instrument when the intervention path passes through the intervention obstacle; The ultrasonic intervention guidance device further includes a display module, and the display module is used to display the ultrasonic image and the intervention path; The ultrasonic intervention guidance device also includes a judgment module, which is used to judge whether the intervention path passes through an intervention obstacle based on the processing results of the deep learning model. When the intervention path passes through the intervention obstacle, the display module displays the intervention path in a first display mode. When the intervention path does not pass through the intervention obstacle, the display module displays the intervention path in a second display mode, wherein the first display mode and the second display mode are different.

2. The ultrasonic intervention guidance device according to claim 1, characterized in that: The ultrasonic intervention guidance device is applied to an electronic device, and the electronic device is connected to an ultrasonic scanning device via a high-definition video output port to obtain the ultrasonic image from the ultrasonic scanning device in real time.

3. The ultrasonic intervention guidance device according to claim 1, characterized in that: The intervention path planning module includes: A first endpoint acquisition unit, configured to acquire a point on the intervention target as a first endpoint; A second endpoint acquisition unit, used to acquire a point on the interventional instrument as a second endpoint; An intervention path planning unit is used to generate the intervention path according to the first endpoint, the second endpoint and the intervention obstacle in the target image.

4. The ultrasonic interventional guidance device according to claim 1, characterized in that: The intervention path planning module is also used to update the intervention path in real time according to the real-time positions of the intervention target, intervention obstacle and intervention instrument in the target image.

5. The ultrasonic intervention guidance device according to claim 1, characterized in that: The intervention target includes nodules, the intervention obstacle includes tissues and organs, and the intervention instrument includes a puncture needle.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an ultrasound intervention guidance method based on deep learning is implemented, wherein the ultrasound intervention guidance method comprises: Acquire ultrasound images in real time; A deep learning model is used to process one or more target images in the ultrasound image to identify interventional targets, interventional obstacles and interventional instruments in the target image. The training method of the deep learning model includes: constructing a deep learning initial model; obtaining training data, wherein the training data includes ultrasound images annotated with interventional targets, interventional obstacles and interventional instruments; performing augmentation processing on the ultrasound images in the training data, wherein the augmentation processing includes image flipping, image translation, image scaling, image misalignment and image cropping; and training the deep learning initial model using the training data to obtain the deep learning model. The loss function used in the training is A represents the prediction box in the ultrasound image, B represents the target box in the ultrasound image, (d A ,d B ) represents the Euclidean distance between the center points of the prediction box A and the target box B, d dia Representing the diagonal length of the minimum circumscribed rectangle of the prediction box A and the target box B; testing the trained deep learning model; generating an intervention path according to the processing result of the deep learning model; Wherein, for any frame of the ultrasound image, if there is an intervention obstacle in the frame of the ultrasound image, generating the intervention path includes: S11, acquiring a point on the intervention target in the frame of ultrasound image as a first endpoint, and acquiring an endpoint of the intervention instrument in the frame of ultrasound image as a second endpoint; S12, obtaining a rectangular area including the first endpoint and the second endpoint in the frame of ultrasound image as a search area, and dividing the search area into a plurality of grids, obtaining the grids where the first endpoint, the second endpoint, and the intervention obstacle are located, putting the second endpoint as a starting point into an open list, and taking the first endpoint as a target point; S13, searching for an adjacent grid of the grid where the starting point is located, and calculating a distance between the adjacent grid and the first endpoint; S14, taking the points corresponding to the grid as the states in the state space and the distances between the points as the costs, and using the A* algorithm to calculate the cost estimate from the starting point to the target point; S15, comparing the cost estimation values ​​corresponding to the paths from the starting point to the target point, and determining in turn whether the cost estimation values ​​corresponding to the paths from the starting point to the target point are minimum values; wherein, for any path, if the cost estimation value corresponding to the path is the minimum value, the starting point is put into a closed list, and the grid point adjacent to the grid where the starting point is located is taken as a new starting point, and the new starting point is put into an open list; if not, the adjacent grid points corresponding to the path are put into a closed list; S16, determining whether the new starting point is the target point: if so, obtaining the intervention path according to the current search result, otherwise returning to step S13; The ultrasonic intervention guidance method further includes: when the intervention path passes through the intervention obstacle, generating a warning signal to prompt medical personnel to correct the intervention path and adjust the position, angle, and posture of the intervention instrument; The ultrasound intervention guidance method further comprises displaying the ultrasound image and the intervention path; The ultrasound intervention guidance method also includes: judging whether the intervention path passes through an intervention obstacle based on the processing results of the deep learning model; when the intervention path passes through the intervention obstacle, displaying the intervention path in a first display mode; when the intervention path does not pass through the intervention obstacle, displaying the intervention path in a second display mode, wherein the first display mode and the second display mode are different.

7. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory, and executes an ultrasound intervention guidance method based on deep learning when calling the computer program, wherein the ultrasound intervention guidance method includes: Acquire ultrasound images in real time; A deep learning model is used to process one or more target images in the ultrasound image to identify interventional targets, interventional obstacles and interventional instruments in the target image. The training method of the deep learning model includes: constructing a deep learning initial model; obtaining training data, wherein the training data includes ultrasound images annotated with interventional targets, interventional obstacles and interventional instruments; performing augmentation processing on the ultrasound images in the training data, wherein the augmentation processing includes image flipping, image translation, image scaling, image misalignment and image cropping; and training the deep learning initial model using the training data to obtain the deep learning model. The loss function used in the training is A represents the prediction box in the ultrasound image, B represents the target box in the ultrasound image, (d A ,d B ) represents the Euclidean distance between the center points of the prediction box A and the target box B, d dia Representing the diagonal length of the minimum circumscribed rectangle of the prediction box A and the target box B; testing the trained deep learning model; generating an intervention path according to the processing result of the deep learning model; Wherein, for any frame of the ultrasound image, if there is an intervention obstacle in the frame of the ultrasound image, generating the intervention path includes: S11, acquiring a point on the intervention target in the frame of ultrasound image as a first endpoint, and acquiring an endpoint of the intervention instrument in the frame of ultrasound image as a second endpoint; S12, obtaining a rectangular area including the first endpoint and the second endpoint in the frame of ultrasound image as a search area, and dividing the search area into a plurality of grids, obtaining the grids where the first endpoint, the second endpoint, and the intervention obstacle are located, putting the second endpoint as a starting point into an open list, and taking the first endpoint as a target point; S13, searching for an adjacent grid of the grid where the starting point is located, and calculating a distance between the adjacent grid and the first endpoint; S14, taking the points corresponding to the grid as the states in the state space and the distances between the points as the costs, and using the A* algorithm to calculate the cost estimate from the starting point to the target point; S15, comparing the cost estimation values ​​corresponding to the paths from the starting point to the target point, and determining in turn whether the cost estimation values ​​corresponding to the paths from the starting point to the target point are minimum values; wherein, for any path, if the cost estimation value corresponding to the path is the minimum value, the starting point is put into a closed list, and the grid point adjacent to the grid where the starting point is located is taken as a new starting point, and the new starting point is put into an open list; if not, the adjacent grid points corresponding to the path are put into a closed list; S16, determining whether the new starting point is the target point: if so, obtaining the intervention path according to the current search result, otherwise returning to step S13; The ultrasonic intervention guidance method further includes: when the intervention path passes through the intervention obstacle, generating a warning signal to prompt medical personnel to correct the intervention path and adjust the position, angle, and posture of the intervention instrument; The ultrasound intervention guidance method further comprises displaying the ultrasound image and the intervention path; The ultrasound intervention guidance method also includes: judging whether the intervention path passes through an intervention obstacle based on the processing results of the deep learning model; when the intervention path passes through the intervention obstacle, displaying the intervention path in a first display mode; when the intervention path does not pass through the intervention obstacle, displaying the intervention path in a second display mode, wherein the first display mode and the second display mode are different.

Images