Method and device for automatically punching C mode of liver portal vein for ultrasonic robot

Abstract

The invention discloses a method and a device for automatically making a C mode for a liver portal vein of an ultrasonic robot, and the method comprises the steps: controlling a probe at the tail end of a mechanical arm of the ultrasonic robot to scan the portal vein according to a preset path, adding scanning pose data into a pose queue, and calculating the score of a scanned ultrasonic image; when the scores of the plurality of continuous scanning ultrasonic images reach a first preset score threshold value, taking the tail of the pose queue as a current target pose, and deleting the tail of the pose queue; controlling the probe to move towards the current target pose, and determining whether the probe reaches the optimal probe pose according to the score of the backtracking ultrasonic image: if so, obtaining the optimal ultrasonic image; if the probe moves to the current target pose and still does not reach the optimal probe pose, the tail of the pose queue is taken again to serve as the next target pose, the probe is controlled to move until the pose queue is empty, and the probe is controlled to continue scanning the portal vein according to the preset path. According to the method, the optimal probe pose is determined through path backtracking, and a high-quality ultrasonic image is obtained.

Description

Technical Field

[0001] The invention relates to a method and a device for automatically marking a C-mode of a liver portal vein for an ultrasonic robot. Background Art

[0002] In ultrasound medicine, C-mode (Color-Mode) color Doppler imaging is an imaging technology with great advantages. It can help evaluate hemodynamic characteristics by displaying the direction and velocity of blood flow in real time. C-mode technology encodes blood flow velocity and direction into color information to intuitively display the flow of blood in blood vessels. It has become an important tool for the examination of complex organs such as the liver. Especially in liver ultrasound examination, due to the complex structure of the liver and the changeable blood flow characteristics, the use of C-mode imaging can dynamically observe the blood flow distribution and structure inside the liver, providing doctors with more detailed liver diagnostic information, thereby helping to diagnose a variety of pathological conditions including cirrhosis and vascular lesions. C-mode imaging not only helps to reveal the overall condition of blood flow, but also assists doctors in discovering subtle blood flow abnormalities, providing key support for the judgment and treatment of liver lesions. Traditional C-mode imaging usually relies on manual operation. Ultrasound physicians need to manually adjust the parameters of the ultrasound equipment and the probe posture to obtain the best section to ensure that the imaging effect meets clinical needs. Summary of the invention

[0003] In order to obtain the optimal probe posture and optimal ultrasound image for performing C-mode, an embodiment of the present invention provides a method and device for automatically performing C-mode on the liver portal vein of an ultrasound robot.

[0004] In a first aspect, an embodiment of the present invention provides a method for automatically setting a C-mode for a liver portal vein of an ultrasonic robot, which may include:

[0005] Controlling the probe at the end of the ultrasonic robot mechanical arm to scan the portal vein along a preset path, collecting the scanning posture data of the probe and the corresponding scanning ultrasound image in real time, adding the scanning posture data to a posture queue, and calculating the score of the scanning ultrasound image; the scanning posture data in the posture queue are arranged in sequence;

[0006] When the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted;

[0007] Control the probe to move toward the current target posture, collect the probe's backtracking posture data and backtracking ultrasound images in real time, and determine whether the probe has reached the optimal posture according to the score of the backtracking ultrasound images:

[0008] If yes, controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe position;

[0009] If the probe moves to the current target posture and still has not determined to have reached the optimal probe posture, the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move to the next target posture until the posture queue is empty, and the process of controlling the probe to scan the portal vein along the preset path is continued until the optimal probe posture and the optimal ultrasound image are obtained.

[0010] In one or some optional implementations of the embodiment of the present application, controlling the probe to move toward the current target posture, collecting the backtracking posture data and the backtracking ultrasound image of the probe in real time, and determining whether the optimal probe posture is reached according to the score of the backtracking ultrasound image, includes:

[0011] Control the probe to move toward the current target posture, collect the backtracking posture data and the backtracking ultrasound image of the probe in real time, add the backtracking posture data to the backtracking queue, calculate the score of the backtracking ultrasound image, and determine whether the scores of multiple consecutive backtracking ultrasound images all reach a second preset score threshold:

[0012] If so, the latest backtracking posture data is determined as the optimal probe posture;

[0013] If not, continue to control the probe to move toward the current target posture until it is determined that the probe moves to the current target posture.

[0014] In one or some optional implementations of the embodiment of the present application, the movement of the probe to the current target posture is determined by:

[0015] According to the position distance and angle distance between the latest backtracking posture data and the current target posture, the current end distance and the current end angle distance are used;

[0016] If the current-to-final distance is less than a first distance threshold, and the current-to-final angle distance is less than a first angle threshold, it is determined that the probe has reached the current target posture.

[0017] In one or some optional implementations of the embodiments of the present application, further comprising:

[0018] Calculate the position distance and angle distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and the current backtracking angle distance;

[0019] If the current return distance is less than the second distance threshold, and the current return angle distance is less than the second angle threshold, the process of moving the probe to the current target posture is terminated, and the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move to the next target posture.

[0020] In one or some optional implementations of the embodiment of the present application, the method further includes: when it is determined that the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold, recording the position of the probe at that time as the backtracking initial position;

[0021] After taking the scanned posture data at the end of the posture queue as the current target posture, the method further includes:

[0022] Calculate the distance between the current target position and the initial position of the backtracking as the start-end distance;

[0023] If the start-end distance is greater than the third distance threshold, the process of controlling the probe to scan the portal vein along the preset path continues.

[0024] In one or some optional implementations of the embodiment of the present application, controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture includes:

[0025] When acquiring the optimal ultrasound image, the robotic arm is controlled to translate and / or rotate according to a preset rule.

[0026] In a second aspect, an embodiment of the present invention provides a device for automatically setting a C-mode for the portal vein of a liver of an ultrasonic robot, which may include:

[0027] The first scanning module is used to control the probe at the end of the ultrasonic robot mechanical arm to scan the portal vein according to a preset path, collect the scanning posture data of the probe and the corresponding scanning ultrasound image in real time, add the scanning posture data to the posture queue, and calculate the score of the scanning ultrasound image; the scanning posture data in the posture queue are arranged in sequence;

[0028] A first backtracking module, configured to take the scanned posture data at the end of the posture queue as the current target posture, and delete the current target posture in the posture queue when the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold;

[0029] A first judgment module is used to control the probe to move toward the current target posture, collect the back-tracking posture data and the back-tracking ultrasound image of the probe in real time, and determine whether the optimal probe posture is reached according to the score of the back-tracking ultrasound image: if so, execute the first determination module; if not, continue to control the probe to move toward the current target posture;

[0030] A first determination module is used to control the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture when it is determined that the probe has reached the optimal probe posture;

[0031] The second backtracking module is used to re-take the scanning posture data at the end of the posture queue as the next target posture if the probe has not yet reached the optimal probe posture when it moves to the current target posture, and control the probe to move to the next target posture until the posture queue is empty, and then continue to control the probe to scan the portal vein along the preset path until the optimal probe posture and the optimal ultrasound image are obtained.

[0032] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the method for automatically marking the C-mode of the liver portal vein for an ultrasonic robot as described above.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for automatically marking the C-mode of the liver portal vein for an ultrasonic robot as described above.

[0034] In a fifth aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory, and when the processor executes the computer program, it implements the method for automatically marking the C-mode of the liver portal vein for an ultrasonic robot as described above.

[0035] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0036] The embodiment of the present invention provides a method for automatically playing C mode of the portal vein of the liver for an ultrasonic robot. The method scans the area where the portal vein is located according to a preset path, collects ultrasonic images in real time and calculates scores. When the scores of multiple consecutive scanned ultrasonic images reach a first preset score threshold, the backtracking search phase is entered, and the scanning posture data in the posture queue are taken in reverse order as the current target posture, and the probe is controlled to move to the current target posture. According to the scores of the backtracking ultrasonic images collected during the movement, the optimal probe posture is determined. This method enables the ultrasonic robot to control the probe to scan the liver area, and collect postures and ultrasonic images in real time, and perform backtracking search and dynamic judgment according to the scores of the ultrasonic images, so that the probe reaches the vicinity of the optimal probe posture, and finally accurately determines the optimal probe posture for playing C mode. This process does not require human intervention, ensures that high-quality ultrasonic images are obtained quickly and accurately, and improves the blood flow filling degree of the C mode. At the same time, this method is highly flexible and can be adjusted according to the needs of different patients, providing accurate and reliable data support for clinical diagnosis.

[0037] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 A schematic flow chart of a method for automatically setting a C-mode for the liver portal vein of an ultrasonic robot provided in an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of an ultrasonic robot provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the minimum circumscribed rectangle of the portal vein outline provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a tool coordinate system at the end of a robotic arm provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of the structure of a device for automatically setting the C-mode for the liver portal vein of an ultrasonic robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] The inventors found that in the prior art, when using ultrasonic robots to perform autonomous liver scanning, the liver portal vein is thin, which makes it difficult to scan, is greatly affected by breathing, and has problems such as insufficient blood flow in the C mode due to differences in the inclination angle of the blood vessels. As a result, the image quality is difficult to meet clinical needs and it is difficult to obtain the optimal section of the portal vein. Based on this, the inventors made the present invention after further research and development, providing a method and device for automatically scanning the liver portal vein in the C mode for ultrasonic robots.

[0047] Embodiment 1

[0048] In the first embodiment of the present invention, a method for automatically marking the C-mode of the portal vein of the liver for an ultrasonic robot is provided, referring to Figure 1 As shown, the method may include the following steps S101-S105:

[0049] S101: Control the probe at the end of the ultrasonic robot arm to scan the portal vein along a preset path, collect the scanning posture data of the probe and the corresponding scanning ultrasound image in real time, add the scanning posture data to the posture queue, and calculate the score of the scanning ultrasound image. The scanning posture data in the posture queue are arranged in sequence.

[0050] S102: When the scores of a plurality of consecutive scanned ultrasound images reach a first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted.

[0051] S103: Control the probe to move toward the current target posture, collect the probe's retrospective posture data and retrospective ultrasound image in real time, and determine whether the optimal probe posture is reached according to the score of the retrospective ultrasound image: if so, execute step S104; if not, continue to control the probe to move toward the current target posture until it is determined that the probe has moved to the current target posture, and execute step S105.

[0052] S104: Control the probe to acquire an optimal ultrasound image of the portal vein at an optimal probe position.

[0053] S105: If the probe has not yet reached the optimal probe posture after moving to the current target posture, the scanning posture data at the end of the posture queue is taken as the next target posture, and the above step S103 is re-executed to control the probe to move to the next target posture until the posture queue is empty, and the process of controlling the probe to scan the portal vein along the preset path is continued until the optimal probe posture and the optimal ultrasound image are obtained.

[0054] The embodiment of the present invention provides a method for automatically playing C mode of the portal vein of the liver for an ultrasonic robot. The method scans the area where the portal vein is located according to a preset path, collects ultrasonic images in real time and calculates scores. When the scores of multiple consecutive scanned ultrasonic images reach a first preset score threshold, the backtracking search phase is entered, and the scanning posture data in the posture queue are taken in reverse order as the current target posture, and the probe is controlled to move to the current target posture. According to the scores of the backtracking ultrasonic images collected during the movement, the optimal probe posture is determined. This method enables the ultrasonic robot to control the probe to scan the liver area, and collect postures and ultrasonic images in real time, and perform backtracking search and dynamic judgment according to the scores of the ultrasonic images, so that the probe reaches the vicinity of the optimal probe posture, and finally accurately determines the optimal probe posture for playing C mode. This process does not require human intervention, ensures that high-quality ultrasonic images are obtained quickly and accurately, and improves the blood flow filling degree of the C mode. At the same time, this method is highly flexible and can be adjusted according to the needs of different patients, providing accurate and reliable data support for clinical diagnosis.

[0055] In the embodiment of the present application, the schematic diagram of the ultrasonic robot is as follows Figure 2 As shown, a probe is connected to the end of the robot arm, and the XYZ at the probe is a schematic diagram of the direction of the tool coordinate system at the end of the robot arm, and the XYZ axis of the robot arm base is a schematic diagram of the direction of the base coordinate system, which is used to indicate the three-dimensional spatial position of the probe at the end of the robot arm.

[0056] In the above step S101, the probe at the end of the ultrasonic robot arm is controlled to scan the portal vein according to a preset path, and the scanning posture data of the probe and the corresponding scanning ultrasound image are collected in real time, the scanning posture data is added to the posture queue, and the score of the scanning ultrasound image is calculated. Specifically, the following steps S1011-S1012 are included:

[0057] S1011: Control the probe at the end of the ultrasonic robot arm to scan the portal vein along a preset path, collect scanning posture data and scanning ultrasound images in real time, and add the scanning posture data to a posture queue.

[0058] Specifically, it can be that the probe at the end of the ultrasonic robot arm is controlled to scan the portal vein along a preset path, and the position P, posture R and scanning ultrasound image of the probe are collected in real time, wherein the position P and posture R of the probe constitute the scanning posture data, which are inserted into the end of the posture queue posList, that is, posList[i] = {P i , R i}, i represents the index number.

[0059] Before the scan begins, a pose queue posList needs to be established in advance and the length of the pose queue needs to be set. It should be noted that the length of the pose queue is fixed and can be set to 50 for example. In the pose queue, data is added from the end of the queue in the order of acquisition. When the maximum length of the queue is reached, the data at the head of the queue will be automatically deleted to ensure that the pose queue always retains only the data of the most recent period of time.

[0060] In the embodiment of the present application, while collecting the position P and posture R of the probe in real time, the contact force F between the probe and the skin and the probe velocity V can also be collected and stored in the position queue together with the position P and posture R of the probe, that is, posList[i] = {P i , R i , F i , V i}, i represents the index number, which can be provided to the force control algorithm to control the longitudinal displacement of the probe in subsequent scanning.

[0061] S1012: Calculate the score of the scanned ultrasound image. Specifically, it includes the following steps S10121-S10126:

[0062] S10121: Use an image segmentation network to segment the scanned ultrasound image to obtain a contour of the portal vein in the scanned ultrasound image.

[0063] Those skilled in the art can select a suitable neural network for pre-training based on the detailed description of the prior art to obtain an image segmentation network. The training process may specifically include:

[0064] In the first step, ultrasound images of the portal vein are collected and the outlines of the portal vein in the ultrasound images are marked respectively, and the portal vein outline dataset is obtained after preprocessing.

[0065] The second step is to select a suitable neural network model as the initial image segmentation network, such as U-Net model, SegNet model, etc.

[0066] The third step is to divide the portal vein contour dataset into training set and test set.

[0067] The fourth step is to define the loss function (such as cross entropy loss function, mean square error loss function, etc.), optimization algorithm (such as Adam, SGD, etc.), etc.

[0068] The fifth step is to train the selected neural network model using the training set of the portal vein contour dataset to obtain a trained image segmentation model.

[0069] Repeat the training process of the above image segmentation model until the preset conditions are met, stop training, and obtain the image segmentation network. The preset conditions can be set to reach a fixed number of iterations, the accuracy reaches a threshold, the accuracy does not change within the preset number of iterations, etc. No specific limitation is made here.

[0070] S10122: Determine the minimum circumscribed rectangle of the outline of the portal vein, and obtain the outline area, outline length, and outline inclination angle based on the minimum circumscribed rectangle.

[0071] Specifically, the minimum circumscribed rectangle can be determined according to the outline of the portal vein, and the contour area, contour length and contour inclination angle can be obtained according to the minimum circumscribed rectangle. Figure 3 As shown in the figure, the red box is the minimum circumscribed rectangle, h is the outline width, w is the outline length, and angle is the outline inclination angle.

[0072] In a specific embodiment, the above step S10122 can be implemented based on OpenCV (Open Source Computer Vision Library). Specifically, the minimum bounding rectangle of the portal vein contour data is determined by the function cv2.minAreaRect(), the length, width and tilt angle of the minimum bounding rectangle are obtained, the length and width of the minimum bounding rectangle are multiplied to obtain the contour area, the length of the minimum bounding rectangle is used as the contour length, and the tilt angle of the minimum bounding rectangle is used as the contour tilt angle.

[0073] S10123: Calculate the area fraction based on the contour area.

[0074] Specifically, the actual physical area of ​​the portal vein may be calculated based on the following formula 1:

[0075] pV r_Area =phy rDis 2 ×pV area Formula 1

[0076] Where, pV r_Area is the actual physical area of ​​the portal vein, phy rDis is the actual physical distance per pixel, pV area The actual physical distance per unit pixel is determined according to the depth of the ultrasound image. For example, when the depth of the ultrasound image is 20 mm, the actual physical distance per unit pixel is 0.2643 mm.

[0077] The preset expected width and expected length are obtained, and the expected area of ​​the portal vein is calculated based on the following formula 2:

[0078] pV d_Area =pV d_len ×pV d_w Formula 2

[0079] Where, pV d_Area is the expected area of ​​the portal vein, pV d_len is the expected length, pV d_w is the desired width.

[0080] The ratio of the actual physical area of ​​the portal vein to the expected area was calculated based on the following formula 3:

[0081] pV area_R =pV r_Area / pV d_Area Formula 3

[0082] Where, pV area_R is the ratio of the actual physical area of ​​the portal vein to the expected area, pV r_Area is the actual physical area of ​​the portal vein, pV d_w is the expected area of ​​the portal vein.

[0083] The preset maximum ratio threshold and minimum ratio threshold are obtained, and the area score is obtained based on the following formula 4:

[0084]

[0085] Where, pV area_S is the area fraction, pV area_R is the ratio of the actual physical area of ​​the portal vein to the expected area, δ is the proportionality coefficient, min st is the minimum ratio threshold, max st is the maximum ratio threshold. The proportional coefficient δ can be set to 1.3 for example, and the minimum ratio threshold min st It can be set to 0.2 for example, and the maximum ratio threshold max st It can be set to 1.0 by way of example.

[0086] S10124: Calculate a length score based on the contour length.

[0087] Specifically, the actual physical length of the portal vein may be calculated based on the following formula 5:

[0088] pV rLen =pV len ×phy rDis Formula 5

[0089] Where, pV rLen is the actual physical length of the portal vein, pht rDis is the actual physical distance per pixel, pVlen The actual physical distance per unit pixel is determined according to the depth of the ultrasound image. For example, when the depth of the ultrasound image is 20 mm, the actual physical distance per unit pixel is 0.2643 mm.

[0090] Based on the actual physical length of the portal vein, a threshold interval (10n, 10(n+1)) is set, where n=0, 1, 2, 3, 4, and the length score is obtained based on the following formula 6:

[0091]

[0092] Where, pV len_S is the length fraction, pV rLen is the actual physical length of the portal vein.

[0093] S10125: Calculate a tilt angle score based on the contour tilt angle.

[0094] Specifically, the preset expected tilt angle and fluctuation range may be obtained, and the initial tilt angle score may be calculated based on the following formula 7:

[0095]

[0096] Where, pV init_angle_S is the initial tilt angle fraction, pV Angle is the profile inclination angle, θ d is the desired tilt angle, θ ran is the fluctuation range. Among them, the expected tilt angle θ d It can be set to 60° for example. It is considered that when the inclination angle of the portal vein is around 60°, it is the best C-mode situation. The fluctuation range θ d It can be set to 30° by way of example.

[0097] According to the initial tilt angle score, the tilt angle score is determined based on the following formula 8:

[0098]

[0099] Where, pV init_angle_S is the initial tilt angle fraction, pV angle_S is the tilt angle fraction.

[0100] S10126: Perform weighted summation of the area score, length score and tilt angle score to obtain a score of the scanned ultrasound image.

[0101] Specifically, the score of the scanned ultrasound image may be obtained by weighted summing the area score, the length score and the inclination angle score based on the following formula 9:

[0102] pVs =a×pV area_S +b×pV len_S +c×pV angle_S Formula 9

[0103] Where, pV s is the fraction of the scanned ultrasound image, pV area_S is the area fraction, pV len_S is the length fraction, pV angle_S is the tilt angle fraction, a, b, c are proportional parameters, and the sum of the proportional parameters must be 1. The proportional parameters a, b, c can be exemplarily set to 0.2, 0.4, and 0.4, respectively.

[0104] In the embodiment of the present application, in the above step S101, the area, length and inclination angle characteristics of the portal vein are comprehensively evaluated through image segmentation and geometric calculation methods, and its actual physical size and inclination angle are accurately calculated. By setting a reasonable score threshold and weighted calculation, the quality of the scanned ultrasound image is accurately evaluated, providing a judgment basis for the subsequent optimal probe posture search.

[0105] In the above step S102, when the scores of the consecutive multiple scanned ultrasound images reach the first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted. Specifically, it can be achieved by the following method:

[0106] When the scores of a preset number of consecutive scanned ultrasound images reach a first preset score threshold, it is considered that the probe is near the optimal probe posture at this time, and the trajectory backtracking search phase is entered. No more data is added to the posture queue, and the position of the probe at this time is recorded as the backtracking initial position. The preset number can be set to 2 for example, and the first preset score threshold s t It can be set to 1.5 by way of example.

[0107] Take the scanned pose data at the end of the pose queue as the current target pose, as the target pose for the next backtracking search of the probe, and delete the current target pose in the pose queue, that is, delete the end element of the pose queue.

[0108] In a specific embodiment, the function back() can be used to obtain the scanned pose data at the end of the pose queue posList, that is, the code Pos tar =posList.back(),Pos tar That is the current target pose. Use the pop_back() function to delete the tail element in the pose queue, that is, execute the code posList.pop_back().

[0109] In the embodiment of the present application, after taking the scanned posture data at the end of the posture queue as the current target posture, step S106 needs to be executed to determine whether the distance between the current target posture and the initial position of the backtracking is less than the third distance threshold. If the distance between the current target posture and the initial position of the backtracking is greater than the third distance threshold, it is considered that the current probe posture is not near the optimal probe posture, and the above step S101 is continued to control the probe to scan the portal vein according to the new preset path. Step S106 specifically includes the following steps S1061-S1062:

[0110] S1061: Calculate the distance between the current target position and the initial position of the backtracking as the start-end distance.

[0111] Specifically, the distance between the current target posture and the initial position of the backtracking may be calculated based on the following formula 10 as the start-end distance:

[0112] d init_tar =sqrt((P init (0)-P tar (0)) 2 +(P init (1)-P tar (1)) 2 ) Formula 10

[0113] Where, d init_tar is the start-end distance, P tar (0) and P tar (1) represents the position vector P in the current target pose tar The coordinates on the XY axis, P init (0) and P init (1) indicates backtracking to the initial position P init The coordinates on the X and Y axes respectively, and sqrt represents square root calculation.

[0114] S1062: If the start-end distance is greater than the third distance threshold, continue to execute the above step S101 to control the probe to scan the portal vein along the new preset path.

[0115] The third distance threshold may be exemplarily set to 0.015 m.

[0116] In the above step S103, the probe is controlled to move toward the current target posture, and the retrospective posture data and retrospective ultrasound images of the probe are collected in real time. It is determined whether the optimal probe posture is reached according to the score of the retrospective ultrasound image: if so, step S104 is executed; if not, the probe is continued to be controlled to move toward the current target posture until it is determined that the probe moves to the current target posture, and step S105 is executed. Specifically, the following steps S1031-S1033 are included:

[0117] S1031: Control the probe to move toward the current target posture, collect the backtracking posture data and backtracking ultrasound image of the probe in real time, add the backtracking posture data to the backtracking queue, and calculate the score of the backtracking ultrasound image to determine whether the scores of multiple consecutive backtracking ultrasound images have reached the second preset score threshold: if so, execute step S1032; if not, execute step S1033.

[0118] Specifically, the probe may be controlled to move toward the current target posture, and the position P, posture R, and back-tracking ultrasound image of the probe may be collected in real time, wherein the position P and posture R of the probe are combined into back-tracking posture data and inserted into the tail of the back-tracking queue backList, i.e., backList[i] = {P i , R i}, i represents the index number.

[0119] Before the probe moves to the current target posture, a backlist queue needs to be established in advance and the length of the backlist queue needs to be set. It should be noted that the length of the backlist queue is fixed and can be set to 10 for example. In the backlist queue, data is added from the end of the queue in the order of acquisition. When the maximum length of the queue is reached, the data at the head of the queue will be automatically deleted, and the backlist posture data in the most recent period will be retained.

[0120] The score of the latest retrospective ultrasound image in the retrospective queue is calculated in real time, and the number of times the score of the retrospective ultrasound image is continuously greater than the second preset score threshold is recorded based on the following formula 11:

[0121]

[0122] Where count is the number of times the score of the retrospective ultrasound image is continuously greater than the second preset score threshold, pV s_back is the score of the retrospective ultrasound image, is the second preset score threshold. The second preset score threshold can be set to the ratio of the first preset score threshold, that is, Where β is the proportionality coefficient, and the first preset score threshold s t It can be exemplarily set to 1.5, and the proportionality coefficient β can be exemplarily set to 0.95.

[0123] The method of calculating the score of the retrospective ultrasound image is the same as the method of calculating the score of the scan ultrasound image in the above step S101, and will not be repeated here.

[0124] Determine whether the number of times that the score count of the retrospective ultrasound image is continuously greater than the second preset score threshold is greater than the preset number threshold count min :

[0125] If so, it is considered that the current probe position is the optimal probe posture, that is, the latest backtracking posture data is the optimal probe posture, and step S1032 is executed.

[0126] If not, continue to control the probe to move toward the current target posture and execute step S1033.

[0127] S1032: Determine the latest backtracking posture data as the optimal probe posture, and execute step S104.

[0128] S1033: Continue to control the probe to move toward the current target posture until it is determined that the probe moves to the current target posture.

[0129] Among them, the movement of the probe to the current target posture is determined through the following steps S10331-S10332.

[0130] S10331: The position distance and angle distance between the latest backtracking posture data and the current target posture are used as the current end distance and current end angle distance.

[0131] Specifically, the position distance from the latest backtracking pose data to the current target pose can be calculated based on the following formula 12 as the current end distance:

[0132] d cur_tar =sqrt((P cur (0)-P tar (0)) 2 +(P cur (1)-P tar (1)) 2 ) Formula 12

[0133] Where, d cur_tar is the end-to-end distance, P tar (0) and P tar (1) represents the position vector P in the current target pose tar The coordinates on the XY axis, P cur (0) and P cur (1) represents the position P in the latest backtracking pose data cur The coordinates on the X and Y axes respectively, and sqrt represents square root calculation.

[0134] Get the posture data in the current target posture, recorded as the target posture matrix R tar , obtain the latest posture data in the backtracking posture data, recorded as the current posture matrix R cur . The target posture matrix R tar and the current posture matrix R cur Convert to attitude quaternion and get the target attitude quaternion Q tar and the current attitude quaternion Qcur The method of converting the attitude matrix into the attitude quaternion is a prior art and will not be described in detail here.

[0135] According to the target attitude quaternion Q tar and the current attitude quaternion Q cur , the attitude change quaternion is calculated based on the following formula 13:

[0136] dQ×Q cur Q tar *={dw, dx, dy, dz} Formula 13

[0137] Where dQ is the attitude change quaternion, Q tar * represents the target attitude quaternion Q tar The conjugate quaternion of , dw, dx, dy and dz represent the four elements of the attitude change quaternion dQ respectively.

[0138] The angular distance from the latest backtracking pose data to the current target pose is calculated based on the following formula 14 as the current end angular distance:

[0139]

[0140] Where, d a_cur_tar is the final angle distance, dw, dx, dy and dz represent the four elements of the attitude change quaternion dQ respectively.

[0141] S10332: If the current final distance is less than the first distance threshold, and the current final angle distance is less than the first angle threshold, it is determined that the probe has reached the current target posture.

[0142] Specifically, it can be that if the current end distance d cur_tar Less than the first distance threshold d max , and the final angle distance d a _ cur_tar Less than the first angle threshold d a_max , it means that the probe has reached the current target position. Among them, the first distance threshold d max It can be set to 0.001m for example, and the first angle threshold d a_max It can be set to 0.005 rad as an example.

[0143] In the embodiment of the present application, during the process of the probe moving toward the current target posture, step S107 needs to be executed to determine whether the robot arm stops moving. If it stops moving, the movement process is terminated, the next target posture is obtained as the current probe posture, and the above step S103 is re-executed to control the movement of the probe toward the current target posture. Step S107 specifically includes the following steps S1071-S1072:

[0144] S1071: Calculate the position distance and angle distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and current backtracking angle distance.

[0145] Specifically, it can be that the method of calculating the position distance and angle distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and the current backtracking angle distance is consistent with the method of calculating the current end distance and the current end angle distance in the above step S10331, which will not be repeated here.

[0146] S1072: If the current return distance is less than the second distance threshold, and the current return angle distance is less than the second angle threshold, the process of moving the probe to the current target posture is terminated, and the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the above step S103 is executed to control the process of moving the probe to the next target posture.

[0147] Specifically, it can be that if the current distance d cur_b Less than the second distance threshold d min , and the final angle distance d a_cur_b Less than the second angle threshold d a_min , it means that the robot arm stops moving due to force protection and other reasons. At this time, the robot arm will not be able to reach the current target posture, so the process will be terminated, and the scan posture data at the end of the posture queue will be taken as the next target posture, and the above step S103 will be executed to control the probe to move to the next target posture. Among them, the second distance threshold d min It can be set as 0.0002m, and the second angle threshold d a_min It can be set to 0.001 rad as an example.

[0148] In the above step S104, the probe is controlled to acquire an optimal ultrasound image of the portal vein at the optimal probe position. Specifically, based on the optimal probe position obtained in the above step S103, the ultrasound robot is controlled to call the interface to control the ultrasound device to enter the C mode. However, during the execution of the C mode, due to the influence of factors such as human breathing, the ultrasound image will still change, resulting in an unsatisfactory effect of the C mode. Therefore, the method may also include, when acquiring the optimal ultrasound image, controlling the robotic arm to translate and / or rotate according to preset rules.

[0149] Among them, the preset rules are manually set according to the movement laws of the portal vein during human breathing, including the movement strategy of the robotic arm's translation or rotation, and the method of calculating the specific translation or rotation movement amount.

[0150] In a specific embodiment, the motion strategy of the preset rule is to control the robot arm to translate along the X-axis of the tool coordinate system and rotate around the Z-axis of the tool coordinate system. Figure 4 As shown in the figure, x t ,y t and z t They represent the X, Y, and Z axes of the tool coordinate system at the end of the robot arm.

[0151] The specific method for calculating the translation and rotation motion includes: calculating the motion of the robot arm along the X-axis of the tool coordinate system at the next moment based on the following formula 15:

[0152] Δp xtcpi =d xi *p xstep Formula 15

[0153] In the formula, Δp xtcpi is the amount of translation of the robot arm along the X-axis of the tool coordinate system at the next moment, d xi is the current movement direction of the robot arm, the value is 1 or -1, p xstep is the step length of the robot arm moving along the X-axis of the tool coordinate system. xstep It can be set to 0.0003 m by way of example.

[0154] The amount of rotation of the robot arm around the Z axis of the tool coordinate system at the next moment is calculated based on the following formula 16:

[0155] Δr ztcpi =d rzi *p xstep *μ Formula 16

[0156] In the formula, Δr ztcpi is the amount of rotation of the robot arm around the Z axis of the tool coordinate system at the next moment, d rzi is the current movement direction of the robot arm rotating around the Z axis of the tool coordinate system, and its value is 1 or -1, and μ is the proportional coefficient. The proportional coefficient μ can be set to 5 as an example.

[0157] Among them, when the C-mode is started, the movement direction of the robot arm is d xi The initial setting is 1. When the robot moves the total distance p along the X-axis of the tool coordinate system xtcpi Greater than the preset maximum translation distance p along the X axis of the tool coordinate system xmax When , it starts to move in the opposite direction, changing the movement direction of the robot arm to d xi The initial setting is -1, that is, when p xtcpi >p xmax When xi =-d xiAmong them, the total distance the robot moves along the X-axis of the tool coordinate system is The preset maximum translation distance p along the X axis of the tool coordinate system xmax It can be set as 0.004m by way of example. The direction of motion d of the robot arm rotating around the Z axis of the tool coordinate system is rzi Equal to the movement direction of the robot arm d xi .

[0158] Therefore, the translation change of the robot arm in the tool coordinate system at the next moment is determined as (Δp xtcpi , 0, Δp ztcpi ), the attitude change is (0, 0, Δr ztcpi ). Among them, Δp ztcpi It is the displacement output by the force control algorithm. The force control algorithm is a prior art and will not be described in detail here.

[0159] In the embodiment of the present application, the above-mentioned step S104 dynamically adapts to the changes in the ultrasound image caused by factors such as breathing by synchronously controlling the translation and rotation of the probe when executing in C mode, thereby improving the stability and clarity of the ultrasound image, ensuring better imaging effect of the portal vein section, and obtaining the optimal ultrasound image.

[0160] In the above step S105, if the probe has not yet reached the optimal probe posture after moving to the current target posture, the scanning posture data at the end of the posture queue is taken as the next target posture, and the above step S103 is re-executed to control the probe to move to the next target posture until the posture queue is empty, and then the above step S101 is continued to control the probe to scan the portal vein along the new preset path until the optimal probe posture and the optimal ultrasound image are obtained.

[0161] In the embodiment of the present application, the backtracking search phase described in the above steps S103 and S105 performs detailed scanning near the optimal probe posture, dynamically adjusts the probe position and posture, and gradually optimizes and accurately locates the optimal imaging position, which not only improves the image quality, but also significantly reduces unnecessary repeated scanning, improves imaging efficiency, adapts to complex anatomical structures, increases the robustness of positioning, reduces errors, and has flexible real-time response capabilities, ensuring that the system can quickly find the imaging position that meets clinical needs and obtain higher quality C-mode imaging results.

[0162] Embodiment 2

[0163] Based on the same inventive concept, the embodiment of the present invention also provides a device for automatically opening a C-mode of the liver portal vein of an ultrasonic robot, referring to Figure 5 As shown, the device comprises:

[0164] The first scanning module 101 is used to control the probe at the end of the ultrasonic robot arm to scan the portal vein according to a preset path, collect the scanning posture data of the probe and the corresponding scanning ultrasound image in real time, add the scanning posture data to the posture queue, and calculate the score of the scanning ultrasound image; the scanning posture data in the posture queue are arranged in sequence;

[0165] A first backtracking module 102, configured to take the scanned posture data at the end of the posture queue as the current target posture, and delete the current target posture in the posture queue when the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold;

[0166] The first judgment module 103 is used to control the probe to move toward the current target posture, collect the back-tracking posture data and the back-tracking ultrasound image of the probe in real time, and determine whether the optimal probe posture is reached according to the score of the back-tracking ultrasound image: if so, execute the first determination module; if not, continue to control the probe to move toward the current target posture;

[0167] A first determination module 104 is configured to control the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture when it is determined that the probe has reached the optimal probe posture;

[0168] The second backtracking module 105 is used to re-take the scanning posture data at the end of the posture queue as the next target posture if the probe has not yet reached the optimal probe posture when it moves to the current target posture, and control the probe to move to the next target posture until the posture queue is empty, and then continue to control the probe to scan the portal vein along the preset path until the optimal probe posture and the optimal ultrasound image are obtained.

[0169] Embodiment 3

[0170] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the method for automatically marking the C-mode of the liver portal vein for an ultrasonic robot as described in the above-mentioned embodiment 1 is implemented.

[0171] Embodiment 4

[0172] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for automatically marking the C-mode of the liver portal vein for an ultrasonic robot as described in the above embodiment 1.

[0173] Embodiment 5

[0174] Based on the same inventive concept, an embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the method for automatically marking the C-mode of the liver portal vein for an ultrasonic robot as described in the above embodiment 1.

[0175] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0176] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatically performing C-mode on the portal vein of the liver using an ultrasound robot, characterized in that: include: Controlling the probe at the end of the ultrasonic robot mechanical arm to scan the portal vein along a preset path, collecting the scanning posture data of the probe and the corresponding scanning ultrasound image in real time, adding the scanning posture data to a posture queue, and calculating the score of the scanning ultrasound image; the scanning posture data in the posture queue are arranged in sequence; When the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted; Control the probe to move toward the current target posture, collect the probe's backtracking posture data and backtracking ultrasound images in real time, and determine whether the probe has reached the optimal posture according to the score of the backtracking ultrasound images: If yes, controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe position; If the probe moves to the current target posture and still has not determined to have reached the optimal probe posture, the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move to the next target posture until the posture queue is empty, and the process of controlling the probe to scan the portal vein along the preset path is continued until the optimal probe posture and the optimal ultrasound image are obtained.

2. The method according to claim 1, characterized in that The controlling the probe to move toward the current target posture, collecting the backtracking posture data and the backtracking ultrasound image of the probe in real time, and determining whether the optimal probe posture is reached according to the score of the backtracking ultrasound image, includes: Control the probe to move toward the current target posture, collect the backtracking posture data and the backtracking ultrasound image of the probe in real time, add the backtracking posture data to the backtracking queue, calculate the score of the backtracking ultrasound image, and determine whether the scores of multiple consecutive backtracking ultrasound images all reach a second preset score threshold: If so, the latest backtracking posture data is determined as the optimal probe posture; If not, continue to control the probe to move toward the current target posture until it is determined that the probe moves to the current target posture.

3. The method according to claim 2, characterized in that The probe is moved to the current target position by: According to the position distance and angle distance between the latest backtracking posture data and the current target posture, the current end distance and the current end angle distance are used; If the current-to-final distance is less than a first distance threshold, and the current-to-final angle distance is less than a first angle threshold, it is determined that the probe has reached the current target posture.

4. The method according to claim 3, characterized in that Also includes: Calculate the position distance and angle distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and the current backtracking angle distance; If the current return distance is less than the second distance threshold, and the current return angle distance is less than the second angle threshold, the process of moving the probe to the current target posture is terminated, and the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move to the next target posture.

5. The method according to claim 1, characterized in that Also includes: When it is determined that the scores of the continuous plurality of scanned ultrasound images reach a first preset score threshold, recording the position of the probe at that time as the initial position of the backtracking; After taking the scanned posture data at the end of the posture queue as the current target posture, the method further includes: Calculate the distance between the current target position and the initial position of the backtracking as the start-end distance; If the start-end distance is greater than the third distance threshold, the process of controlling the probe to scan the portal vein along the preset path continues.

6. The method according to claim 1, characterized in that The controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe position comprises: When acquiring the optimal ultrasound image, the robotic arm is controlled to translate and / or rotate according to a preset rule.

7. A device for automatically setting the C-mode of the liver portal vein for an ultrasonic robot, characterized in that: include: The first scanning module is used to control the probe at the end of the ultrasonic robot mechanical arm to scan the portal vein according to a preset path, collect the scanning posture data of the probe and the corresponding scanning ultrasound image in real time, add the scanning posture data to the posture queue, and calculate the score of the scanning ultrasound image; the scanning posture data in the posture queue are arranged in sequence; A first backtracking module, configured to take the scanned posture data at the end of the posture queue as the current target posture, and delete the current target posture in the posture queue when the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold; A first judgment module is used to control the probe to move toward the current target posture, collect the back-tracking posture data and the back-tracking ultrasound image of the probe in real time, and determine whether the optimal probe posture is reached according to the score of the back-tracking ultrasound image: if so, execute the first determination module; if not, continue to control the probe to move toward the current target posture; A first determination module is used to control the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture when it is determined that the probe has reached the optimal probe posture; The second backtracking module is used to re-take the scanning posture data at the end of the posture queue as the next target posture if the probe has not yet reached the optimal probe posture when it moves to the current target posture, and control the probe to move to the next target posture until the posture queue is empty, and then continue to control the probe to scan the portal vein along the preset path until the optimal probe posture and the optimal ultrasound image are obtained.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the method for automatically setting the C-mode of the liver portal vein for an ultrasonic robot as described in any one of claims 1 to 6 is implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for automatically setting the C-mode of the liver portal vein for an ultrasonic robot as described in any one of claims 1 to 6 is implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method for automatically opening the C-mode of the liver portal vein for an ultrasonic robot as described in any one of claims 1-6.

Images