An ultrasound-based fascial interplane block image recognition aid and method

Abstract

Disclosed is an ultrasound-based fascial plane block image recognition auxiliary device and method, which comprises a processor, a memory coupled with the processor, and instructions stored in the memory and executable by the processor to cause the device to: identify a drug administration area in an ultrasound image; divide the drug administration area into multiple levels of identification zones based on the width of the drug administration area; identify a needle body in the ultrasound image; and when the distance between one end of the needle body and the boundary of the drug administration area is less than a first distance threshold, determine whether to display a virtual needle body corresponding to the needle body on the needle body based on the speed of the needle body and the level of the identification zone that the needle body is about to enter, in different strategies. The present application significantly improves the accuracy, safety and operation guiding efficiency of fascial plane block surgery.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image recognition, in particular to an ultrasound-based fascial plane block image recognition auxiliary device and method. BACKGROUND

[0002] With the deepening of the concept of precision and comfort medical treatment, regional anesthesia is increasingly widely used in perioperative multimodal analgesia and enhanced recovery after surgery (ERAS). Fascial plane block technology guided by ultrasound is a new regional block technology emerging in recent years, and its clinical significance is increasingly prominent. This technology injects local anesthetics between specific fascial layers, uses the diffusion of the drug in the fascial plane to block the nerves running in it, and thus achieves analgesia of a specific region.

[0003] The existing ultrasound-based fascial plane block image recognition auxiliary device is insufficient in prompting medical personnel, and the decision of the needle position and the needle speed by medical personnel depends on the experience of the medical personnel. This mode is not conducive to the standardization of medical treatment, and at the same time, it increases the difficulty of medical personnel training. SUMMARY

[0004] The present application provides an ultrasound-based fascial plane block image recognition auxiliary device and method. The present application intelligently identifies the drug administration area in the ultrasound image and divides it into different levels of identification zones according to its width, and then dynamically and differentially decides whether to display a virtual needle body and prompt information in combination with the real-time identified needle body position and movement speed. The device can also display simulated drug diffusion images containing different expected needle body angles to assist decision-making when the drug diffusion boundary approaches the target area. This technical solution realizes hierarchical warning according to the risk level of the needle tip approaching the target area, effectively guides the operator to control the needle body movement, and assists in adjusting the needle tip angle through simulation prediction, thereby significantly improving the precision, safety and operation guidance efficiency of fascial plane block surgery.

[0005] The present application provides an ultrasound-based fascial plane block image recognition auxiliary device, comprising:

[0006] a processor;

[0007] a memory coupled to the processor; and

[0008] instructions stored in the memory and executable by the processor to cause the device to:

[0009] identify a drug administration area in an ultrasound image;

[0010] divide the drug administration area into a plurality of identification zones according to the width of the drug administration area;

[0011] identifying the needle in the ultrasound image; and

[0012] when the distance between the one end of the needle and the boundary of the drug diffusion region is less than the first distance threshold, determining whether to display the virtual needle corresponding to the needle on the needle based on the speed of the needle and the level of the identified zone that the needle is about to enter, with different strategies.

[0013] In one preferred embodiment, the multiple levels of identified zones include a first level of identified zones, a second level of identified zones, and a third level of identified zones.

[0014] wherein the width of the first level of identified zones is less than the width of the second level of identified zones, and the width of the second level of identified zones is less than the width of the third level of identified zones.

[0015] In one preferred embodiment, determining whether to display the virtual needle corresponding to the needle on the needle based on the speed of the needle and the level of the identified zone that the needle is about to enter, with different strategies, includes:

[0016] determining the level of the identified zone that the needle is about to enter based on the position of the needle;

[0017] if it is determined that the needle is about to enter the first level of identified zones, displaying the virtual needle corresponding to the needle on the needle and displaying a first prompt information, wherein the first prompt information prompts the user to stop moving the needle.

[0018] In one preferred embodiment, determining whether to display the virtual needle corresponding to the needle on the needle based on the speed of the needle and the level of the identified zone that the needle is about to enter, with different strategies, further includes:

[0019] if it is determined that the needle is about to enter the second level of identified zones, determining whether the virtual needle corresponding to the needle needs to be displayed on the needle based on the speed of the needle and the size of the second level of identified zones in the direction of the speed of the needle;

[0020] if it is determined that the virtual needle corresponding to the needle needs to be displayed, displaying the virtual needle corresponding to the needle on the needle and displaying a second prompt information, wherein the second prompt information prompts the user to reduce the speed of the needle.

[0021] if it is determined that the needle is about to enter the third level of identified zones, not displaying the virtual needle corresponding to the needle.

[0022] In one preferred embodiment, the instructions are further executable by the processor to cause the apparatus to:

[0023] identify the boundary of the drug diffusion region in the ultrasound image;

[0024] When the distance between the boundary of the simulated drug diffusion region and the boundary of the drug administration region is less than the second distance threshold, display a plurality of simulated drug diffusion region images on a predetermined region of the display device.

[0025] In one preferred embodiment, the plurality of simulated drug diffusion region images comprises a first simulated drug diffusion region image and a second simulated drug diffusion region image;

[0026] wherein the first simulated drug diffusion region image comprises a virtual needle image representing the current needle angle, a virtual needle image representing a first intended needle angle, and a first simulated drug diffusion region boundary, the first simulated drug diffusion region boundary being associated with the first intended needle angle;

[0027] the second simulated drug diffusion region image comprises a virtual needle image representing the current needle angle, a virtual needle image representing a second intended needle angle, and a second simulated drug diffusion region boundary, the second simulated drug diffusion region boundary being associated with the second intended needle angle, wherein the first intended needle angle is different from the second intended needle angle.

[0028] The present application also provides an ultrasound-based fascial plane block image recognition method, comprising the following steps:

[0029] identifying a drug administration region in the ultrasound image;

[0030] dividing the drug administration region into a plurality of levels of marker zones based on the width of the drug administration region;

[0031] identifying a needle in the ultrasound image; and

[0032] when the distance between one end of the needle and the boundary of the drug administration region is less than a first distance threshold, determining whether to display a virtual needle corresponding to the needle on the needle based on the speed of the needle and the level of the marker zone that the needle is about to enter, in different strategies.

[0033] In one preferred embodiment, the plurality of levels of marker zones comprises a first level of marker zones, a second level of marker zones, and a third level of marker zones;

[0034] wherein the width of the first level of marker zones is less than the width of the second level of marker zones, and the width of the second level of marker zones is less than the width of the third level of marker zones.

[0035] In one preferred embodiment, determining whether to display a virtual needle corresponding to the needle on the needle based on the speed of the needle and the level of the marker zone that the needle is about to enter, in different strategies comprises:

[0036] determining the level of the marker zone that the needle is about to enter based on the position of the needle.

[0037] If it is determined that the needle is about to enter the identification zone of the first level, a virtual needle corresponding to the needle is displayed on the needle and a first prompt information is displayed, wherein the first prompt information prompts the user to stop moving the needle.

[0038] In a preferred embodiment, determining whether to display the virtual needle corresponding to the needle on the needle based on the speed of the needle and the level of the identification zone that the needle is about to enter in different strategies further comprises:

[0039] If it is determined that the needle is about to enter the identification zone of the second level, whether to display the virtual needle corresponding to the needle on the needle is determined based on the speed of the needle and the size of the identification zone of the second level in the direction of the speed of the needle;

[0040] If it is determined that the virtual needle corresponding to the needle needs to be displayed, the virtual needle corresponding to the needle is displayed on the needle and a second prompt information is displayed, wherein the second prompt information prompts the user to reduce the speed of the needle;

[0041] If it is determined that the needle is about to enter the identification zone of the third level, the virtual needle corresponding to the needle is not displayed.

[0042] Compared with the prior art, the present application has the following advantages:

[0043] The present application intelligently identifies the drug administration area in the ultrasound image and divides it into identification zones of different levels according to its width, and then dynamically and differentially decides whether to display the virtual needle and the prompt information in combination with the real-time identified needle position and movement speed. The device can also display simulated drug diffusion images containing different expected needle angles when the drug diffusion boundary approaches the target area to assist decision-making. This technical solution realizes hierarchical warning according to the danger level of the needle tip approaching the target area, effectively guides the operator to control the needle movement, and assists in adjusting the needle angle through simulation prediction, thereby significantly improving the precision, safety and operation guiding efficiency of the intermuscular plane block surgery. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is an example of an existing ultrasound image for intermuscular plane block.

[0045] Figure 2 is another example of an existing ultrasound image for intermuscular plane block.

[0046] Figure 3 is a schematic diagram of multiple possible positions of the drug administration needle entering the drug administration area.

[0047] Figure 4 is a method flowchart of an embodiment of the present application.

[0048] Figure 5 A schematic diagram showing the identification of the administration region divided into multiple levels.

[0049] Figure 6 A schematic diagram showing the determination of the level of the identification region into which the needle body is about to enter.

[0050] Figure 7 A schematic diagram showing the display of a virtual needle body and the display of first prompt information according to an embodiment of the present application.

[0051] Figure 8 A schematic diagram showing the determination of whether a virtual needle body needs to be displayed according to an embodiment of the present application.

[0052] Figure 9 A schematic diagram showing the display of multiple simulated drug diffusion region images according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] The specific embodiments of the present application are described below in detail with reference to the accompanying drawings, but the scope of protection of the present application is not limited by the specific embodiments.

[0054] Figure 1 is an example of an existing ultrasound image for fascial plane block. In the existing system, a classic anatomical atlas and an image database of fascial plane block under ultrasound guidance in actual operation can be stored in the computer. In the actual scanning process, when the system identifies the target anatomical structure matching the database, the corresponding target plane information, such as “latissimus dorsi muscle” and “anterior serratus muscle”, can be automatically prompted in the ultrasound image display screen. Such prompt information can assist the operator to quickly and accurately identify the key structure. After determining the target anatomical image (such as the anterior serratus muscle plane), the system can further visually display the anatomical hierarchical structure corresponding to the shallow and deep layers, and clearly identify the boundaries of each layer on the image through a white dashed line. In Figure 1 , when the needle body of the administration needle passes through the shallow layer site, the system can automatically identify that the needle body has moved to the administration region, at which time the existing system can automatically switch to Figure 2 . As shown in Figure 2 , instead of displaying various anatomical structures in Figure 2 , the administration region is framed with a white dashed line, and the existing system identifies the administration region in the following way: the system first needs to locate the potential fascial space based on the identified anatomical structure (such as the surface of the anterior serratus muscle, the intercostal muscle, and the rib acoustic shadow). Then, combined with the preset anatomical knowledge base and the drug diffusion model, a polygonal region that fits the fascial plane is dynamically generated as the recommended administration region. The boundary of the region can be automatically adjusted according to the muscle contour.

[0055] The following is described in combination with Figure 3to illustrate the problems existing in the prior systems. In Figure 3 In FIG. 2B, the possible angles and possible positions of several injection needles entering the injection region are schematically indicated by arrows. Since the movement of the injection needles is controlled by the medical staff, the trajectory of the needle body has certain randomness, thus the injection needle can enter the injection region at the position indicated by arrow 200, from Figure 3 As can be seen from the dashed box in FIG. 2B, the size of the injection region at arrow 200 is very small, no matter the angle of the injection needle, as long as the injection needle moves slightly, the injection needle is very easy to cross the boundary of the injection region, thus entering other anatomical tissues, in some cases, the injection needle entering other anatomical tissues will cause additional harm to the patient. Similarly, if the injection needle enters the injection region at the position indicated by arrow 205 with the angle shown by arrow 205, in the direction of arrow 205, the size of the injection region is also small. At this time, if the medical staff is not timely, the injection needle is also very easy to cross the boundary of the injection region, thus entering other anatomical tissues. Considering that in the ultrasound image, the image of the needle body of the injection needle is very blurred (for example as shown in FIG. 2A), it is difficult for the medical staff to accurately control the movement of the injection needle, thus the injection needle is very easy to enter other anatomical tissues, which will cause additional harm to the patient. Figure 2As shown in FIG. 1, the probability of the medical staff lacking sufficient reaction time will greatly increase. In the prior art, there is no method for prompting the medical staff to enter the position or angle of the administration needle into the administration area, and a possible means for solving the technical problem at present can be referred to, for example, the prior art CN109276296A, which can enhance the image of the puncture needle (for example, enhance the contrast, brightness, sharpness, etc. of the puncture needle image in the ultrasound image) by means of image processing. Since the prior art can enhance the image of the puncture needle, the prior art can be used to prompt the position or angle of the administration needle into the administration area. However, as can be seen from the drawings of the prior art, the prior art has limited optimization capability for the effect on the puncture needle. Even if the prior art is used, the puncture needle is almost invisible in the ultrasound image. It should be noted that since the prior art is based on image processing of the actual image of the puncture needle to enhance the image of the puncture needle, the enhancement capability of the prior art for the effect on the puncture needle is very limited due to the low clarity of the original actual image of the puncture needle. This defect is caused by the inherent logic of the technical solution of the prior art, and there is almost no effective means to overcome this defect without changing the technical solution of the prior art. In addition, the prior art requires real-time image enhancement of the puncture needle, that is, the prior art continuously uses the puncture needle image after enhancement to replace the real puncture needle image, which will cause a medical ethics problem. In today's medical system, the medical staff is ultimately responsible for the medical results. If the medical staff is deprived of the right to view the original image data, the medical staff cannot be responsible for their medical behavior. Specifically, the puncture needle image after enhancement is not the real puncture needle image, and there must be some difference between the two. If the puncture needle image is continuously used to replace the real puncture needle image, it is equivalent to depriving the medical staff of the right to view the original image data. Therefore, using the prior art to solve the technical problem mentioned above will cause a new problem. The method proposed by the present application aims to solve the foregoing technical problem.

[0056] Embodiment 1

[0057] Figure 4 is a method flowchart of an embodiment of the present application. As shown in the figure, the method of the present application comprises the following steps:

[0058] Step 1: identifying the administration area in the ultrasound image; the method for identifying the administration area is the prior art, and the identification result can be referred to Figure 2 , the relevant identification process has been briefly described above, and the present application will not be described again;

[0059] Step 2: dividing the administration area into multiple levels of identification zones based on the width of the administration area;

[0060] Step 3: Identify the needle in the ultrasound image; in one example, the example code for identifying the needle in the ultrasound image is as follows:

[0061] import cv2

[0062] import numpy as np

[0063] def identify_needle_in_ultrasound(ultrasound_image):

[0064] """

[0065] Identify the needle in the ultrasound image

[0066] Parameters:

[0067] ultrasound_image: Input ultrasound image

[0068] Returns:

[0069] needle_lines: Detected needle line segments

[0070] needle_tip: Needle tip position coordinates

[0071] """

[0072] # Image preprocessing

[0073] # 1. Convert to grayscale

[0074] gray = cv2.cvtColor(ultrasound_image, cv2.COLOR_BGR2GRAY)

[0075] # 2. Apply Gaussian blur for noise reduction

[0076] blurred = cv2.GaussianBlur(gray, (5, 5), 0)

[0077] # 3. Enhance contrast - use CLAHE

[0078] clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))

[0079] enhanced = clahe.apply(blurred)

[0080] # 4. Edge detection - combine Canny and Sobel

[0081] edges = cv2.Canny(enhanced, 50, 150, apertureSize=3)

[0082] # 5. Hough Transform for Linear Detection - Targeting the Linear Characteristics of Needles

[0083] lines = cv2.HoughLinesP(edges,

[0084] rho=1,

[0085] theta=np.pi / 180,

[0086] threshold=30,

[0087] minLineLength=50, # Minimum line segment length

[0088] maxLineGap=10) # Maximum line segment gap

[0089] needle_lines = []

[0090] needle_tip = None

[0091] if lines is not None:

[0092] # Filter possible needle body segments

[0093] for line in lines:

[0094] x1, y1, x2, y2 = line[0]

[0095] length = np.sqrt((x2-x1)**2 + (y2-y1)**2)

[0096] angle = np.abs(np.arctan2(y2-y1, x2-x1) * 180 / np.pi)

[0097] # Filtering needle features based on length and angle

[0098] if length>40 and (angle<20 or angle>160):

[0099] needle_lines.append(line[0])

[0100] # Determine needle tip position (assuming it's the farthest endpoint)

[0101] if needle_lines:

[0102] endpoints = []

[0103] for line in needle_lines:

[0104] endpoints.extend([(line[0], line[1]), (line[2], line[3])])

[0105] # Find the point farthest from the image center as the needle tip

[0106] center = (ultrasound_image.shape[1] / / 2, ultrasound_image.shape[0] / / 2)

[0107] needle_tip = max(endpoints,

[0108] key=lambda pt: np.sqrt((pt[0]-center[0])**2 + (pt[1]-center[1])**2))

[0109] return needle_lines, needle_tip

[0110] # Usage example

[0111] if __name__ == "__main__":

[0112] # Read the ultrasound image

[0113] ultrasound_img = cv2.imread("ultrasound_image.jpg")

[0114] # Identify the needle

[0115] detected_lines, tip_position = identify_needle_in_ultrasound(ultrasound_img)

[0116] # Visualize the results

[0117] result_img = ultrasound_img.copy()

[0118] for line in detected_lines:

[0119] cv2.line(result_img, (line[0], line[1]), (line[2], line[3]),(0, 255, 0), 2)

[0120] if tip_position:

[0121] cv2.circle(result_img, tip_position, 8, (0, 0, 255), -1)

[0122] cv2.imshow("Needle Detection", result_img)

[0123] cv2.waitKey(0)

[0124] cv2.destroyAllWindows()”。

[0125] The skilled in the art should understand that the above code is only an exemplary code, any code capable of achieving the purpose of identifying the needle body is applicable to the present application, in addition, the method of identifying the needle body in the ultrasound image also belongs to the prior art (for example, CN109276296A provides a method of identifying the needle body), additionally, a professional software engineer can also be commissioned to write program code to achieve the identification of the needle body;

[0126] Step 4: when the distance between one end of the needle body and the boundary of the drug delivery area is less than the first distance threshold value, based on the speed of the needle body and the level of the identified area that the needle body will enter, different strategies are used to determine whether to display a virtual needle body corresponding to the needle body on the needle body. In one example, one end of the needle body refers to the end of the needle body that pierces the human body; in one example, the first distance threshold value can be set in real time based on the experience of medical personnel, for example, before the operation, the medical personnel can set the first distance threshold value in advance according to their own ability in the operation interface of the computer, as a general principle, the more experienced the medical personnel, the lower the first distance threshold value can be set, on the contrary, the less experienced the medical personnel, the higher the first distance threshold value can be set, it can be understood that the larger the first distance threshold value, the earlier the virtual needle body is displayed in the ultrasound image (in the case of determining to display the virtual needle body), the smaller the first distance threshold value, the later the virtual needle body is displayed in the ultrasound image. In the present application, the speed of the needle body should be understood as a vector, that is, the speed of the needle body can represent the absolute value of the speed of the needle body and the moving direction of the needle body.

[0127] The identification areas of the plurality of levels include an identification area of a first level, an identification area of a second level, and an identification area of a third level; in one example, see Figure 5 In Figure 5 , the gray area represents the administration area, and the boundary of the administration area is represented by the white dotted line; in Figure 5 , the area to the left of the line segment 505 can be an identification area of a first level, the area between the line segments 505-510 can be an identification area of a second level, the area between the line segments 510-515 can be an identification area of a third level, the area between the line segments 515-520 can be an identification area of a second level, and the area to the right of the line segment 520 can be an identification area of a first level; in one example, the level of the identification area to which the administration area belongs can be determined based on the width of the administration area in the vertical direction, for example, it can be set that the part of the administration area whose width in the vertical direction is less than a distance threshold x belongs to an identification area of a first level, the part of the administration area whose width in the vertical direction is greater than or equal to the distance threshold x and less than a distance threshold y belongs to an identification area of a second level, and the part of the administration area whose width in the vertical direction is greater than or equal to the distance threshold y belongs to an identification area of a third level; it can be understood that the distance threshold x and the distance threshold y should be set by each hospital using the system of the present application according to the actual situation of each hospital, therefore, the present application cannot give the specific values of the distance threshold x and the distance threshold y, as a principle, the greater the distance threshold x, the greater the probability of displaying the virtual needle body, and the smaller the distance threshold x, the smaller the probability of displaying the virtual needle body;

[0128] In the above embodiment, the width of the identification area of the first level is less than the width of the identification area of the second level, and the width of the identification area of the second level is less than the width of the identification area of the third level.

[0129] Embodiment 2

[0130] In embodiment 2, based on the speed of the needle body and the level of the identification area into which the needle body is about to enter, whether to display the virtual needle body corresponding to the needle body on the needle body is determined by different strategies, which includes:

[0131] The level of the identification area into which the needle body is about to enter is determined based on the position of the needle body; in one example, the end of the needle body in the ultrasound image can be recognized by using the code provided in embodiment 1, since the distance between the end of the needle body and the boundary of the administration area is already very small, the part of the administration area pointed by the end of the needle body is the part of the administration area to be penetrated by the needle body; and the level of the identification area corresponding to the part of the administration area to be penetrated by the needle body is the level of the identification area into which the needle body is about to enter; one example of this step can be seen in Figure 6 , the area to the left of the line segment 505 can be an identification area of a first level, the area between the line segments 505-510 can be an identification area of a second level, the area between the line segments 510-515 can be an identification area of a third level, the area between the line segments 515-520 can be an identification area of a second level, and the area to the right of the line segment 520 can be an identification area of a first level; in one example, the level of the identification area to which the administration area belongs can be determined based on the width of the administration area in the vertical direction, for example, it can be set that the part of the administration area whose width in the vertical direction is less than a distance threshold x belongs to an identification area of a first level, the part of the administration area whose width in the vertical direction is greater than or equal to the distance threshold x and less than a distance threshold y belongs to an identification area of a second level, and the part of the administration area whose width in the vertical direction is greater than or equal to the distance threshold y belongs to an identification area of a third level; it can be understood that the distance threshold x and the distance threshold y should be set by each hospital using the system of the present application according to the actual situation of each hospital, therefore, the present application cannot give the specific values of the distance threshold x and the distance threshold y, as a principle, the greater the distance threshold x, the greater the probability of displaying the virtual needle body, and the smaller the distance threshold x, the smaller the probability of displaying the virtual needle body; Figure 6In the figure, the end of the needle body 605 is close to the identification area of the first level, so it can be determined that the needle body 605 will enter the identification area of the first level, the ends of the needle body 610 and the needle body 615 are close to the identification area of the second level, so it can be determined that the needle body 610 and the needle body 615 will enter the identification area of the second level, and the end of the needle body 620 is close to the identification area of the third level, so it can be determined that the needle body 620 will enter the identification area of the third level.

[0132] If it is determined that the needle body will enter the identification area of the first level, a virtual needle body corresponding to the needle body is displayed on the needle body and first prompt information is displayed, wherein the first prompt information prompts the user to stop moving the needle body. In one example, the identification area of the first level is similar to the part of the injection area indicated by the arrow 200 in Figure 3 , as previously described, the size of the part of the injection area is very small, and no matter the angle of the needle body, as long as the needle body moves slightly, the needle body is very easy to cross the boundary of the injection area and thus enter other anatomical tissues. Therefore, in the present application, once it is determined that the needle body will enter the identification area of the first level, a virtual needle body corresponding to the needle body should be displayed on the needle body and first prompt information should be displayed to prompt the medical staff to re-needle to adjust the position of the needle body into the injection area; in the present application, displaying a virtual needle body on the needle body means that the image of the virtual needle body is overlaid on the actual needle body image. Examples of displaying a virtual needle body corresponding to the needle body on the needle body and displaying first prompt information can be seen in Figure 7 . By comparison Figure 7 and Figure 6 It can be seen that the present application allows the virtual needle body 705 to be directly overlaid on the real needle body image (for example, the needle body 605 in Figure 6 , and in the present application, the virtual needle body only needs to have the most basic correspondence with the needle body, that is, the virtual needle body only needs to be able to generally represent the size and position of the actual needle body. In other words, the present application does not require the use of a method similar to CN109276296A to generate an enhanced needle body image based on the actual image of the needle body to generate a virtual needle body, and therefore, the present application can significantly show the virtual needle body through various means, which greatly increases the prompting effect on the user. For example, in Figure 7 , the virtual needle body 705 is displayed using a bright red color, and the width and length of the virtual needle body are significantly larger than the real needle body image, in addition, a flashing effect can also be added to the virtual needle body in order to further enhance the prompting effect. In the example of Figure 7 , the first prompt information is “please adjust the position of the needle body”, which prompts the user to re-needle, so as to make the needle body enter the injection area from other parts of the injection area.

[0133] Embodiment 3

[0134] In Example 3, determining whether to display a virtual needle corresponding to the needle body using different strategies based on the needle speed and the level of the marker area the needle body is about to enter also includes:

[0135] If it is determined that the needle is about to enter the second-level marking area, then based on the needle's speed and the size of the second-level marking area in the direction of the needle's speed, it is determined whether a virtual needle corresponding to the needle needs to be displayed on the needle; the following uses... Figure 8 Let's further introduce Example 3. As mentioned above... Figure 3 As described, if the injection needle enters the drug delivery area at the location indicated by arrow 205 and at the angle shown by arrow 205, the size of the drug delivery area is smaller in the direction of arrow 205. In this case, if the medical staff does not react in time, the injection needle can easily cross the boundary of the drug delivery area and enter other anatomical tissues. Conversely, if the injection needle enters the drug delivery area at the location indicated by arrow 210 and at the angle shown by arrow 210, the size of the drug delivery area is larger in the direction of arrow 210. In this case, as long as the speed of the injection needle is not too high, the needle can remain within the drug delivery area for a sufficient amount of time, thus allowing the medical staff enough reaction time. In this situation, it is almost impossible for the injection needle to cross the boundary of the drug delivery area. Figure 3 If the needle enters the drug delivery area at position 805 and angle 805, the size of the second-level marking area in the direction of needle velocity is represented by line segment 815. Based on the length of line segment 815 and the absolute value of needle velocity, the time it takes for the needle to cross the drug delivery area can be calculated. Since the length of line segment 815 is relatively small, the calculated time is relatively small. If this time is less than the reaction time of medical personnel (the specific reaction time depends on hospital regulations and the experience of medical personnel; no specific value is given in this invention), it can be determined that a virtual needle corresponding to the needle needs to be displayed on the needle body. Conversely, if the needle enters the drug delivery area at position 810 and angle 810, the size of the second-level marking area in the direction of needle velocity is represented by line segment 820. Based on the length of line segment 820 and the absolute value of needle velocity, the time it takes for the needle to cross the drug delivery area can be calculated. Since the length of line segment 820 is relatively small, the calculated time is relatively small. If this time is greater than the reaction time of medical personnel, it can be determined that a virtual needle corresponding to the needle body does not need to be displayed on the needle body.

[0136] If it is determined that the virtual needle corresponding to the needle is needed to be displayed, the virtual needle corresponding to the needle is displayed on the needle and a second prompt information is displayed, wherein the second prompt information prompts the user to reduce the speed of the needle; an example of the second prompt information can be “reduce the needle insertion speed”, after seeing the prompt and the virtual needle, the user can reduce the needle insertion speed and adjust the angle of the needle, if the angle of the needle is adjusted to be similar to the angle of the needle 810, the display of the virtual needle and the second prompt information can be stopped at this time;

[0137] If it is determined that the needle is about to enter the third level of the identification area, the virtual needle corresponding to the needle is not displayed. In combination with the arrow 215 in the drug delivery area in Figure 3 , Figure 3 The size of the drug delivery area at the arrow 215 in the drug delivery area in

[0138] Embodiment 4

[0139] After the needle is inserted into the drug delivery area by the medical staff, the system will control the drug pump to deliver the drug, and the current medical guidelines stipulate that the drug diffusion range should not exceed the boundary of the drug delivery area. It can be understood that when the drug diffuses in the irregular drug delivery area, the drug in one area often approaches the boundary of the drug delivery area, while the drug in another area is still far away from the boundary of the drug delivery area, at this time, the medical staff needs to adjust the angle of the needle, but the existing technology lacks prompt for the medical staff on how to adjust the angle of the needle, and the method proposed in embodiment 4 aims to solve this problem.

[0140] In embodiment 4, the method of the present application further comprises:

[0141] identifying the boundary of the drug diffusion area in the ultrasound image; in one example, an example code for identifying the boundary of the drug diffusion area in the ultrasound image is as follows:

[0142] “import cv2

[0143] import numpy as np

[0144] from sklearn.cluster import KMeans

[0145] import matplotlib.pyplot as plt

[0146] def identify_drug_diffusion_boundary(ultrasound_image, roi_mask=None):

[0147] """

[0148] Identify the boundary of the drug diffusion area in an ultrasound image

[0149] Parameters:

[0150] ultrasound_image: Input ultrasound image (BGR format)

[0151] roi_mask: Region of interest mask (optional)

[0152] Returns:

[0153] diffusion_boundary: Set of boundary points of the drug diffusion area

[0154] diffusion_area: Area of the diffusion region

[0155] processed_image: Processed image with boundary markers

[0156] """

[0157] # Convert to grayscale

[0158] gray = cv2.cvtColor(ultrasound_image, cv2.COLOR_BGR2GRAY)

[0159] # Apply ROI mask if provided

[0160] if roi_mask is not None:

[0161] gray = cv2.bitwise_and(gray, gray, mask=roi_mask)

[0162] # 1. Image enhancement - Increase contrast between diffusion area and surrounding tissue

[0163] clahe = cv2.createCLAHE(clipLimit=3.0, tileGridSize=(8, 8))

[0164] enhanced = clahe.apply(gray)

[0165] # 2. High-pass filter to enhance edges

[0166] blurred = cv2.GaussianBlur(enhanced, (7, 7), 0)

[0167] # 3. Diffusion area identification based on texture features

[0168] # Enhance texture features using Local Binary Patterns (LBP)

[0169] lbp = local_binary_pattern(blurred, 8, 1, method='uniform')

[0170] # 4. Multi-threshold segmentation - Identify possible diffusion areas

[0171] # Use adaptive thresholding

[0172] adaptive_thresh = cv2.adaptiveThreshold(

[0173] blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,

[0174] cv2.THRESH_BINARY, 11, 2 )

[0176] # 5. Morphological operations to optimize boundaries

[0177] kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))

[0178] opened = cv2.morphologyEx(adaptive_thresh, cv2.MORPH_OPEN,kernel)

[0179] closed = cv2.morphologyEx(opened, cv2.MORPH_CLOSE, kernel)

[0180] # 6. Find contours - Identify diffusion region boundaries

[0181] contours, _ = cv2.findContours(

[0182] image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE )

[0184] # 7. Filter most likely drug diffusion regions

[0185] diffusion_contours = []

[0186] for contour in contours:

[0187] area = cv2.contourArea(contour)

[0188] # Filter based on area, shape, etc.

[0189] if area>100: # Minimum area threshold

[0190] # Calculate convexity defects, etc.

[0191] hull = cv2.convexHull(contour)

[0192] hull_area = cv2.contourArea(hull)

[0193] solidity = area / hull_area if hull_area>0 else 0

[0194] # Drug diffusion regions usually have high convexity and certain area

[0195] if solidity>0.7:

[0196] diffusion_contours.append(contour)

[0197] # 8. Merge adjacent diffusion regions

[0198] if len(diffusion_contours)>1:

[0199] # Create a merge mask

[0200] merged_mask = np.zeros_like(closed)

[0201] cv2.drawContours(merged_mask, diffusion_contours, -1, 255, -1)

[0202] # Extract the boundary of the merged region again

[0203] merged_contours, _ = cv2.findContours(

[0204] merged_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE )

[0206] if merged_contours:

[0207] diffusion_contours = merged_contours

[0208] # 9. Calculate total area and main boundary

[0209] diffusion_area = 0

[0210] main_boundary = None

[0211] max_area = 0

[0212] for contour in diffusion_contours:

[0213] area = cv2.contourArea(contour)

[0214] diffusion_area += area

[0215] if area>max_area:

[0216] max_area = area

[0217] main_boundary = contour

[0218] # 10. Visualize the results

[0219] processed_image = ultrasound_image.copy()

[0220] if main_boundary is not None:

[0221] # Draw the boundary

[0222] cv2.drawContours(

[0223] processed_image, [main_boundary], -1,

[0224] (0, 255, 0), 2 # Green boundary )

[0226] # Calculate the convex hull of the boundary points to make the boundary smoother

[0227] hull = cv2.convexHull(main_boundary)

[0228] cv2.drawContours(

[0229] processed_image, [hull], -1,

[0230] (255, 0, 0), 1 # Blue convex hull boundary )

[0232] # Mark the center of the region

[0233] M = cv2.moments(main_boundary)

[0234] if M["m00"]!= 0:

[0235] cx = int(M["m10"] / M["m00"])

[0236] cy = int(M["m01"] / M["m00"])

[0237] cv2.circle(processed_image, (cx, cy), 5, (0, 0, 255), -1)

[0238] return main_boundary, diffusion_area, processed_image

[0239] def local_binary_pattern(image, points, radius, method):

[0240] """Computes the Local Binary Pattern (LBP) of an image"""

[0241] lbp = np.zeros_like(image)

[0242] for i in range(radius, image.shape[0]-radius):

[0243] for j in range(radius, image.shape[1]-radius):

[0244] center = image[i, j]

[0245] binary = ''

[0246] for p in range(points):

[0247] angle = 2 * np.pi * p / points

[0248] x = j + radius * np.cos(angle)

[0249] y = i - radius * np.sin(angle)

[0250] x = int(round(x))

[0251] y = int(round(y))

[0252] binary += '1' if image[y, x]>= center else '0'

[0253] lbp[i, j]= int(binary, 2)

[0254] return lbp

[0255] # Usage example

[0256] if __name__ == "__main__":

[0257] # Read ultrasound image

[0258] ultrasound_img = cv2.imread("ultrasound_diffusion.jpg")

[0259] identify drug diffusion region boundary

[0260] boundary, area, result_img = identify_drug_diffusion_boundary(ultrasound_img)

[0261] print(f"Detected drug diffusion region area: {area} pixels")

[0262] # Display the result

[0263] cv2.imshow("Drug Diffusion Boundary Detection", result_img)

[0264] cv2.waitKey(0)

[0265] cv2.destroyAllWindows()

[0266] # Optionally: Save the result

[0267] cv2.imwrite("diffusion_detection_result.jpg", result_img)”

[0268] It should be understood by those skilled in the art that the above code is only an exemplary code, any code that can achieve the purpose of identifying the drug diffusion region is suitable for the present application, and in addition, a professional software engineer can be commissioned to write program code to achieve the identification of the drug diffusion region.

[0269] When the distance between the boundary of the drug diffusion region and the boundary of the drug administration region is less than a second distance threshold value, a plurality of simulated drug diffusion region images are displayed on a predetermined area of the display device. In one example, the second distance threshold value can be set based on the experience of medical personnel on site. It can be understood that the larger the second distance threshold value is set, the earlier the simulated drug diffusion region image is displayed, and vice versa. The second distance threshold value is set smaller, the later the simulated drug diffusion region image is displayed. It should be understood that the drug administration region is generally irregular in shape. The distance between the boundary of the drug diffusion region and the boundary of the drug administration region referred to by the present application refers to the minimum distance between the boundary of the drug diffusion region and the boundary of the drug administration region.

[0270] Further, the plurality of simulated drug diffusion region images include a first simulated drug diffusion region image and a second simulated drug diffusion region image.

[0271] The first simulated drug diffusion region image includes a virtual needle image representing the current needle angle, a virtual needle image representing the first expected needle angle, and a first simulated drug diffusion region boundary associated with the first expected needle angle.

[0272] The second simulated drug diffusion region image includes a virtual needle image representing the current needle angle, a virtual needle image representing the second expected needle angle, and a second simulated drug diffusion region boundary associated with the second expected needle angle, wherein the first expected needle angle is different from the second expected needle angle. For examples of displaying a plurality of simulated drug diffusion region images, please refer to Figure 9 , in Figure 9 , the left image is an actual ultrasound image, the green region in the image represents the drug delivery region, and the needle 905 represents the actual needle in the actual ultrasound image; and on the right side of Figure 9 , the first simulated drug diffusion region image ( Figure 9 , the upper right image) and the second simulated drug diffusion region image ( Figure 9 , the lower right image) are schematically shown on the right side of a display device (such as a computer display), it can be seen that the virtual needle image 910 representing the current needle angle in the first simulated drug diffusion region image is consistent with the position and angle of the needle 905 representing the actual needle on the left side, and the virtual needle image 915 representing the first expected needle angle is used to show the user the position and angle of the needle after the user turns the needle clockwise, from Figure 9 the first simulated drug diffusion region image of Figure 9 , it can be seen that if the user turns the needle clockwise, the drug diffusion region ( Figure 9 , the green region in the upper right image of Figure 9 , will be well filled with the drug delivery region; the virtual needle image 920 representing the current needle angle in the second simulated drug diffusion region image is consistent with the position and angle of the needle 905 representing the actual needle on the left side, and the virtual needle image 925 representing the second expected needle angle is used to show the user the position and angle of the needle after the user turns the needle counterclockwise, from Figure 9 the second simulated drug diffusion region image of , it can be seen that if the user turns the needle counterclockwise, the drug diffusion region will exceed the drug delivery region; therefore, the scheme of embodiment 4 can better prompt the user how to turn the needle.

[0273] Embodiment 5

[0274] An ultrasound-based fascial plane block image recognition assistance device is provided, comprising: a processor; a memory coupled with the processor; and instructions stored in the memory and executable by the processor to cause the device to:

[0275] identify a dosing region in an ultrasound image;

[0276] divide the dosing region into a plurality of levels of marker zones based on a width of the dosing region;

[0277] identify a needle body in the ultrasound image; and

[0278] when a distance between an end of the needle body and a boundary of the dosing region is less than a first distance threshold, determine whether to display a virtual needle body corresponding to the needle body on the needle body based on a speed of the needle body and a level of a marker zone that the needle body is about to enter, with different strategies.

[0279] Further, the plurality of levels of marker zones includes a first level of marker zones, a second level of marker zones, and a third level of marker zones.

[0280] The width of the first level of marker zones is less than the width of the second level of marker zones, and the width of the second level of marker zones is less than the width of the third level of marker zones.

[0281] Further, determining whether to display the virtual needle body on the needle body based on the speed of the needle body and the level of the marker zone that the needle body is about to enter, with different strategies, includes:

[0282] determining the level of the marker zone that the needle body is about to enter based on a position of the needle body;

[0283] if it is determined that the needle body is about to enter the first level of marker zones, displaying the virtual needle body corresponding to the needle body on the needle body and displaying first prompt information, wherein the first prompt information prompts a user to stop moving the needle body.

[0284] Further, determining whether to display the virtual needle body on the needle body based on the speed of the needle body and the level of the marker zone that the needle body is about to enter, with different strategies, further includes:

[0285] if it is determined that the needle body is about to enter the second level of marker zones, determining whether the virtual needle body corresponding to the needle body needs to be displayed on the needle body based on the speed of the needle body and a size of the second level of marker zones in a direction of the speed of the needle body;

[0286] If it is determined that the virtual needle corresponding to the needle needs to be displayed, the virtual needle corresponding to the needle is displayed on the needle and second prompt information is displayed, wherein the second prompt information prompts the user to reduce the speed of the needle;

[0287] If it is determined that the needle is about to enter the third level of the identification area, the virtual needle corresponding to the needle is not displayed.

[0288] Further, the instructions are further executable by the processor to cause the apparatus to:

[0289] identify a boundary of the drug diffusion region in the ultrasound image;

[0290] When the distance between the boundary of the drug diffusion region and the boundary of the administration region is less than the second distance threshold, a plurality of simulated drug diffusion region images are displayed on a predetermined region of the display device.

[0291] Further, the plurality of simulated drug diffusion region images include a first simulated drug diffusion region image and a second simulated drug diffusion region image;

[0292] The first simulated drug diffusion region image includes a virtual needle image representing a current needle angle, a virtual needle image representing a first expected needle angle, and a first simulated drug diffusion region boundary, the first simulated drug diffusion region boundary being associated with the first expected needle angle.

[0293] The second simulated drug diffusion region image includes a virtual needle image representing a current needle angle, a virtual needle image representing a second expected needle angle, and a second simulated drug diffusion region boundary, the second simulated drug diffusion region boundary being associated with the second expected needle angle, wherein the first expected needle angle is different from the second expected needle angle.

[0294] The above description is merely a specific implementation of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the scope of protection of the claims.

Claims

1. An ultrasound-based fascial inter-muscular plane block image recognition aid, comprising: Comprising: a processor; a memory coupled with the processor; and instructions stored in the memory and executable by the processor to cause the apparatus to: identify a drug administration region in an ultrasound image; divide the drug administration region into a plurality of levels of identification zones based on a width of the drug administration region; identify a needle body in the ultrasound image; and determine whether to display a virtual needle body corresponding to the needle body on the needle body based on a speed of the needle body and a level of an identification zone that the needle body is about to enter, when a distance between one end of the needle body and a boundary of the drug administration region is less than a first distance threshold. The plurality of levels of identification zones comprises a first level of identification zones, a second level of identification zones, and a third level of identification zones; 2. The apparatus of claim 1, wherein, wherein a width of the first level of identification zones is less than a width of the second level of identification zones, and the width of the second level of identification zones is less than a width of the third level of identification zones. Determining whether to display the virtual needle body on the needle body based on the speed of the needle body and the level of the identification zone that the needle body is about to enter comprises:

3. The apparatus of claim 2, wherein, determining the level of the identification zone that the needle body is about to enter based on a position of the needle body; if it is determined that the needle body is about to enter the first level of identification zones, displaying the virtual needle body corresponding to the needle body on the needle body and displaying first prompt information, wherein the first prompt information prompts a user to stop moving the needle body. Determining whether to display the virtual needle body on the needle body based on the speed of the needle body and the level of the identification zone that the needle body is about to enter further comprises:

4. The apparatus of claim 3, wherein, if it is determined that the needle body is about to enter the second level of identification zones, determining whether the virtual needle body corresponding to the needle body needs to be displayed on the needle body based on the speed of the needle body and a size of the second level of identification zones in a direction of the speed of the needle body; if it is determined that the virtual needle body needs to be displayed, displaying the virtual needle body corresponding to the needle body on the needle body and displaying second prompt information, wherein the second prompt information prompts the user to reduce the speed of the needle body; if it is determined that the needle body is about to enter the third level of identification zones, not displaying the virtual needle body corresponding to the needle body. The instructions are further executable by the processor to cause the apparatus to:

5. The apparatus of claim 1, wherein, identify a boundary of a drug diffusion region in an ultrasound image; display a plurality of simulated drug diffusion region images on a predetermined area of a display apparatus when a distance between the boundary of the drug diffusion region and a boundary of the drug administration region is less than a second distance threshold. The plurality of simulated drug diffusion region images comprises a first simulated drug diffusion region image and a second simulated drug diffusion region image; 6. The apparatus of claim 5, wherein, ​ The first simulated drug diffusion region image includes a virtual needle image representing a current needle angle, a virtual needle image representing a first intended needle angle, and a first simulated drug diffusion region boundary associated with the first intended needle angle. The second simulated drug diffusion region image includes a virtual needle image representing a current needle angle, a virtual needle image representing a second intended needle angle, and a second simulated drug diffusion region boundary associated with the second intended needle angle, wherein the first intended needle angle is different from the second intended needle angle.

Images