Method and device for automatically marking C-mode on the portal vein of the liver for an ultrasound robot

Abstract

The present invention discloses a method and device for automatically performing a C-mode on the portal vein of the liver for an ultrasonic robot. The method comprises: controlling the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, adding the scanned posture data to a posture queue, and calculating the score of the scanned ultrasound image; when the scores of multiple consecutive scanned ultrasound images reach a first preset score threshold, taking the end of the posture queue as the current target posture, and deleting the end of the posture queue; controlling the probe to move toward the current target posture, and determining whether the optimal probe posture has been reached based on the scores of the backtracked ultrasound images: if so, obtaining the optimal ultrasound image; if the probe still has not reached the optimal probe posture after moving to the current target posture, retaking the end of the posture queue as the next target posture, and controlling the probe to move until the posture queue is empty, and then controlling the probe to continue scanning the portal vein along the preset path. The method determines the optimal probe posture by backtracking the path, thereby obtaining a high-quality ultrasound image.

Description

Technical Field

[0001] The present invention relates to a method and device for automatically marking a C-mode on the liver portal vein of an ultrasonic robot. Background Art

[0002] In ultrasound medicine, C-mode (Color-Mode) color Doppler imaging is a highly advantageous imaging technique that can help assess hemodynamic characteristics by displaying blood flow direction and velocity in real time. By encoding blood flow velocity and direction as color information, C-mode technology intuitively displays blood flow within blood vessels and has become a vital tool for examining complex organs such as the liver. In particular, in liver ultrasound examinations, due to the liver's complex structure and variable blood flow characteristics, C-mode imaging allows for dynamic observation of blood flow distribution and structure within the liver, providing physicians with more detailed liver diagnostic information, aiding in the diagnosis of various pathological conditions, including cirrhosis and vascular lesions. C-mode imaging not only reveals the overall state of blood flow but also helps physicians detect subtle blood flow abnormalities, providing critical support for the diagnosis and treatment of liver lesions. Traditional C-mode imaging often relies on manual operation, requiring ultrasound physicians to manually adjust ultrasound equipment parameters and probe positioning to achieve the optimal view and ensure imaging quality meets clinical needs. Summary of the Invention

[0003] In order to obtain the optimal probe posture and optimal ultrasound image for performing C-mode, an embodiment of the present invention provides a method and device for automatically performing C-mode on the liver portal vein of an ultrasound robot.

[0004] In a first aspect, an embodiment of the present invention provides a method for automatically performing a C-mode on the portal vein of the liver for an ultrasound robot, which may include:

[0005] Controlling the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collecting the probe's scanning posture data and corresponding scanning ultrasound images in real time, adding the scanning posture data to a posture queue, and calculating a score for the scanning ultrasound image; the scanning posture data in the posture queue are arranged sequentially;

[0006] When the scores of the consecutive scanned ultrasound images reach a first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted;

[0007] Control the probe to move toward the current target posture, collect the probe's backtracking posture data and backtracking ultrasound images in real time, and determine whether the optimal probe posture is reached based on the score of the backtracking ultrasound image:

[0008] If so, controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe position;

[0009] If the probe moves to the current target posture and still has not reached the optimal probe posture, the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move toward the next target posture until the posture queue is empty, and the process of controlling the probe to scan the portal vein along the preset path is continued until the optimal probe posture and the optimal ultrasound image are obtained.

[0010] In one or some optional implementations of the embodiment of the present application, controlling the probe to move toward the current target posture, collecting backtracking posture data and backtracking ultrasound images of the probe in real time, and determining whether the optimal probe posture is reached according to a score of the backtracking ultrasound image, includes:

[0011] Control the probe to move toward the current target posture, collect backtracking posture data and backtracking ultrasound images of the probe in real time, add the backtracking posture data to a backtracking queue, calculate the score of the backtracking ultrasound image, and determine whether the scores of multiple consecutive backtracking ultrasound images all reach a second preset score threshold:

[0012] If so, the latest backtracking pose data is determined as the optimal probe pose;

[0013] If not, continue to control the probe to move toward the current target posture until it is determined that the probe moves to the current target posture.

[0014] In one or some optional implementations of the embodiment of the present application, the movement of the probe to the current target posture is determined by:

[0015] The position distance and angular distance between the latest backtracking posture data and the current target posture are used as the current end distance and the current end angular distance;

[0016] If the current-to-final distance is less than a first distance threshold, and the current-to-final angle distance is less than a first angle threshold, it is determined that the probe has reached the current target posture.

[0017] One or some optional implementations of the embodiments of the present application may further include:

[0018] Calculate the position distance and angular distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and the current backtracking angular distance;

[0019] If the current return distance is less than the second distance threshold and the current return angle distance is less than the second angle threshold, the process of moving the probe to the current target posture is terminated, and the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move to the next target posture.

[0020] In one or some optional implementations of the embodiment of the present application, the method further includes: when it is determined that the scores of the plurality of consecutive scanned ultrasound images reach a first preset score threshold, recording the position of the probe at that time as a backtracking initial position;

[0021] After taking the scanned pose data at the end of the pose queue as the current target pose, the method further includes:

[0022] Calculating the distance between the current target position and the initial position of the backtracking as the start-end distance;

[0023] If the start-end distance is greater than the third distance threshold, the process of controlling the probe to scan the portal vein along the preset path continues.

[0024] In one or some optional implementations of the embodiment of the present application, controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture includes:

[0025] When acquiring the optimal ultrasound image, the robotic arm is controlled to translate and / or rotate according to preset rules.

[0026] In a second aspect, an embodiment of the present invention provides a device for automatically performing C-mode on the portal vein of the liver of an ultrasound robot, which may include:

[0027] a first scanning module, configured to control the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collect scanning posture data of the probe and corresponding scanning ultrasound images in real time, add the scanning posture data to a posture queue, and calculate a score for the scanning ultrasound image; the scanning posture data in the posture queue are arranged sequentially;

[0028] a first backtracking module, configured to take the scanned posture data at the end of the posture queue as the current target posture, and delete the current target posture from the posture queue when the scores of the consecutive scanned ultrasound images reach a first preset score threshold;

[0029] a first judgment module, configured to control the probe to move toward the current target posture, collect the probe's retrospective posture data and retrospective ultrasound images in real time, and determine whether the optimal probe posture is reached based on the score of the retrospective ultrasound image; if so, execute the first determination module; if not, continue to control the probe to move toward the current target posture;

[0030] A first determining module is configured to, when it is determined that the probe has reached an optimal probe posture, control the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture;

[0031] The second backtracking module is used to re-take the scanning posture data at the end of the posture queue as the next target posture if the probe has not yet reached the optimal probe posture when it moves to the current target posture, and control the probe to move toward the next target posture until the posture queue is empty, and then continue to control the probe to scan the portal vein along the preset path until the optimal probe posture and the optimal ultrasound image are obtained.

[0032] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the above-mentioned method for automatically marking the liver portal vein in C-mode for an ultrasound robot.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the above-mentioned method for automatic C-mode marking of the liver portal vein for an ultrasound robot.

[0034] In a fifth aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory, wherein when the processor executes the computer program, the method for automatically marking the liver portal vein in C-mode for an ultrasonic robot as described above is implemented.

[0035] The beneficial effects of the above technical solutions provided by the embodiments of the present invention include at least:

[0036] An embodiment of the present invention provides a method for automatically performing C-mode on the portal vein of the liver using an ultrasound robot. This method scans the portal vein region along a preset path, acquires ultrasound images in real time, and calculates scores. When the scores of multiple consecutive scanned ultrasound images reach a first preset score threshold, a backtracking search phase begins. Scanned pose data from a pose queue is sequentially retrieved in reverse order as the current target pose, and the probe is controlled to move to the current target pose. The optimal probe pose is determined based on the scores of the backtracking ultrasound images acquired during the movement. This method enables the ultrasound robot to control the probe to scan the liver region and acquire poses and ultrasound images in real time. Based on the ultrasound image scores, a backtracking search is performed and dynamic judgment is made to ensure that the probe reaches the vicinity of the optimal probe pose. Finally, a precise micro-search is performed to determine the optimal probe pose for C-mode. This process requires no human intervention, ensuring rapid and accurate acquisition of high-quality ultrasound images and improving C-mode blood flow filling. Furthermore, this method is highly flexible and can be adjusted to meet the needs of different patients, providing accurate and reliable data support for clinical diagnosis.

[0037] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 A schematic flow chart of a method for automatically performing C-mode on the liver portal vein using an ultrasound robot according to an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of an ultrasonic robot provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the minimum circumscribed rectangle of the portal vein outline provided in an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a tool coordinate system at the end of a robotic arm provided by an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the structure of the device for automatically performing C-mode on the liver portal vein of an ultrasonic robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] The inventors discovered that prior art approaches to autonomous liver scanning using ultrasound robots present challenges due to the thinness of the liver's portal vein, making it difficult to scan, significant respiration-induced interference, and insufficient C-mode blood flow due to differences in vascular inclination. These issues result in image quality that fails to meet clinical requirements and makes it difficult to obtain optimal sections of the portal vein. Based on this, the inventors conducted further research and developed the present invention, which provides a method and apparatus for automated C-mode scanning of the liver's portal vein using an ultrasound robot.

[0047] Example 1

[0048] The first embodiment of the present invention provides a method for automatically marking the C-mode of the portal vein of the liver for an ultrasound robot, referring to Figure 1 As shown, the method may include the following steps S101-S105:

[0049] S101: Control the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collect the probe's scanning posture data and corresponding scanning ultrasound images in real time, add the scanning posture data to a posture queue, and calculate the score of the scanning ultrasound image. The scanning posture data in the posture queue are arranged sequentially.

[0050] S102: When the scores of a plurality of consecutive scanned ultrasound images reach a first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted.

[0051] S103: Control the probe to move toward the current target posture, collect the probe's retrospective posture data and retrospective ultrasound image in real time, and determine whether the optimal probe posture is reached based on the score of the retrospective ultrasound image: if so, execute step S104; if not, continue to control the probe to move toward the current target posture until it is determined that the probe has moved to the current target posture, and execute step S105.

[0052] S104: Control the probe to acquire an optimal ultrasound image of the portal vein at an optimal probe position.

[0053] S105: If the probe moves to the current target posture and still has not reached the optimal probe posture, the scanning posture data at the end of the posture queue is taken as the next target posture, and the above step S103 is re-executed to control the probe to move to the next target posture until the posture queue is empty, and the process of controlling the probe to scan the portal vein along the preset path is continued until the optimal probe posture and the optimal ultrasound image are obtained.

[0054] An embodiment of the present invention provides a method for automatically performing C-mode on the portal vein of the liver using an ultrasound robot. This method scans the portal vein region along a preset path, acquires ultrasound images in real time, and calculates scores. When the scores of multiple consecutive scanned ultrasound images reach a first preset score threshold, a backtracking search phase begins. Scanned pose data from a pose queue is sequentially retrieved in reverse order as the current target pose, and the probe is controlled to move to the current target pose. The optimal probe pose is determined based on the scores of the backtracking ultrasound images acquired during the movement. This method enables the ultrasound robot to control the probe to scan the liver region and acquire poses and ultrasound images in real time. Based on the ultrasound image scores, a backtracking search is performed and dynamic judgment is made to ensure that the probe reaches the vicinity of the optimal probe pose. Finally, a precise micro-search is performed to determine the optimal probe pose for C-mode. This process requires no human intervention, ensuring rapid and accurate acquisition of high-quality ultrasound images and improving C-mode blood flow filling. Furthermore, this method is highly flexible and can be adjusted to meet the needs of different patients, providing accurate and reliable data support for clinical diagnosis.

[0055] In the embodiment of the present application, the ultrasonic robot schematic diagram is as follows Figure 2 As shown, a probe is connected to the end of the robotic arm. The XYZ at the probe is a schematic diagram of the direction of the tool coordinate system at the end of the robotic arm, and the XYZ axis of the robotic arm base is a schematic diagram of the direction of the base coordinate system, which is used to indicate the three-dimensional spatial position of the probe at the end of the robotic arm.

[0056] In step S101 above, the probe at the end of the ultrasonic robot's mechanical arm is controlled to scan the portal vein along a preset path, and the probe's scanning posture data and corresponding scanning ultrasound image are collected in real time. The scanning posture data is added to the posture queue, and the score of the scanning ultrasound image is calculated. Specifically, the following steps S1011-S1012 are included:

[0057] S1011: Control the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collect scanning posture data and scanning ultrasound images in real time, and add the scanning posture data to a posture queue.

[0058] Specifically, it can be to control the probe at the end of the ultrasonic robot arm to scan the portal vein according to the preset path, and collect the probe's position P, posture R and scanning ultrasound image in real time, wherein the probe's position P and posture R constitute the scanning posture data, which is inserted into the end of the posture queue posList, that is, posList[i] = {P i , R i}, i represents the index number.

[0059] Before scanning begins, you need to create a pose queue, posList, and set its length. Note that the length of the pose queue is fixed, typically set to 50. Data is added to the pose queue from the end in the order it was acquired. When the maximum queue length is reached, data from the head of the queue is automatically deleted, ensuring that the pose queue only retains the most recent data.

[0060] In the embodiment of the present application, while collecting the position P and posture R of the probe in real time, the contact force F between the probe and the skin and the probe velocity V can also be collected and stored in the posture queue together with the position P and posture R of the probe, that is, posList[i] = {P i , R i , F i , V i}, i represents the index number, which can be provided to the force control algorithm to control the longitudinal displacement of the probe in subsequent scans.

[0061] S1012: Calculate the score of the scanned ultrasound image. Specifically, it includes the following steps S10121-S10126:

[0062] S10121: Use an image segmentation network to segment the scanned ultrasound image to obtain a contour of the portal vein in the scanned ultrasound image.

[0063] Those skilled in the art can select a suitable neural network for pre-training based on the detailed description of the prior art to obtain an image segmentation network. The training process may specifically include:

[0064] In the first step, ultrasound images of the portal vein are collected and the contours of the portal vein in the ultrasound images are marked respectively. After preprocessing, a portal vein contour dataset is obtained.

[0065] The second step is to select a suitable neural network model as the initial image segmentation network, such as the U-Net model, SegNet model, etc.

[0066] In the third step, the portal vein contour dataset is divided into training set and test set.

[0067] The fourth step is to define the loss function (such as cross entropy loss function, mean square error loss function, etc.), optimization algorithm (such as Adam, SGD, etc.), etc.

[0068] In the fifth step, the selected neural network model is trained using the training set of the portal vein contour dataset to obtain a trained image segmentation model.

[0069] The training process of the image segmentation model is repeated until the preset conditions are met, and then training is stopped to obtain the image segmentation network. The preset conditions may be set to, for example, reaching a fixed number of iterations, reaching a threshold accuracy, or maintaining no change in accuracy within a preset number of iterations. These conditions are not specifically limited here.

[0070] S10122: Determine the minimum circumscribed rectangle of the portal vein contour, and obtain the contour area, contour length, and contour inclination angle based on the minimum circumscribed rectangle.

[0071] Specifically, the minimum circumscribed rectangle can be determined based on the outline of the portal vein, and the contour area, contour length and contour inclination angle can be obtained based on the minimum circumscribed rectangle. Figure 3 As shown in the figure, the red box is the minimum circumscribed rectangle, h is the outline width, w is the outline length, and angle is the outline inclination angle.

[0072] In one specific embodiment, step S10122 can be implemented based on OpenCV (Open Source Computer Vision Library). Specifically, the minimum bounding rectangle of the portal vein contour data can be determined using the function cv2.minAreaRect() to obtain the length, width, and tilt angle of the minimum bounding rectangle. The length and width of the minimum bounding rectangle are multiplied to obtain the contour area. The length of the minimum bounding rectangle is used as the contour length, and the tilt angle of the minimum bounding rectangle is used as the contour tilt angle.

[0073] S10123: Calculate the area fraction based on the contour area.

[0074] Specifically, the actual physical area of ​​the portal vein may be calculated based on the following formula 1:

[0075] pV r_Area =phy rDis 2 ×pV area Formula 1

[0076] Where, pV r_Area is the actual physical area of ​​the portal vein, phy rDis is the actual physical distance per pixel, pV area The actual physical distance per unit pixel is determined by the depth of the ultrasound image. For example, if the depth of the ultrasound image is 20 mm, the actual physical distance per unit pixel is 0.2643 mm.

[0077] The preset expected width and expected length are obtained, and the expected area of ​​the portal vein is calculated based on the following formula 2:

[0078] pV d_Area =pV d_len ×pV d_w Formula 2

[0079] Where, pV d_Area is the expected area of ​​the portal vein, pV d_len is the expected length, pV d_w is the desired width.

[0080] The ratio of the actual physical area of ​​the portal vein to the expected area was calculated based on the following formula 3:

[0081] pV area_R =pV r_Area / pV d_Area Formula 3

[0082] Where, pV area_R is the ratio of the actual physical area of ​​the portal vein to the expected area, pV r_Area is the actual physical area of ​​the portal vein, pV d_w is the expected area of ​​the portal vein.

[0083] Obtain the preset maximum ratio threshold and minimum ratio threshold, and calculate the area score based on the following formula 4:

[0084]

[0085] Where, pV area_S is the area fraction, pV area_R is the ratio of the actual physical area of ​​the portal vein to the expected area, δ is the proportional coefficient, min st is the minimum ratio threshold, max st The ratio coefficient δ can be set to 1.3, and the minimum ratio threshold min st It can be set to 0.2 for example, and the maximum ratio threshold max st It can be set to 1.0 for example.

[0086] S10124: Calculate a length score based on the contour length.

[0087] Specifically, the actual physical length of the portal vein may be calculated based on the following formula 5:

[0088] pV rLen =pV len ×phy rDis Formula 5

[0089] Where, pV rLen is the actual physical length of the portal vein, pht rDis is the actual physical distance per pixel, pVlen The actual physical distance per unit pixel is determined by the depth of the ultrasound image. For example, if the depth of the ultrasound image is 20 mm, the actual physical distance per unit pixel is 0.2643 mm.

[0090] Based on the actual physical length of the portal vein, a threshold interval (10n, 10(n+1)) is set, where n = 0, 1, 2, 3, 4, and the length score is obtained based on the following formula 6:

[0091]

[0092] Where, pV len_S is the length fraction, pV rLen is the actual physical length of the portal vein.

[0093] S10125: Calculate an inclination angle score based on the contour inclination angle.

[0094] Specifically, a preset expected tilt angle and fluctuation range may be obtained, and an initial tilt angle score may be calculated based on the following formula 7:

[0095]

[0096] Where, pV init_angle_S is the initial tilt angle fraction, pV Angle is the profile tilt angle, θ d is the desired tilt angle, θ ran is the fluctuation range. Among them, the expected tilt angle θ d It can be set to 60° for example. It is considered that when the inclination angle of the portal vein is around 60°, it is the best time to play the C mode. The fluctuation range θ d It can be set to 30° as an example.

[0097] Based on the initial tilt angle score, the tilt angle score is determined based on the following formula 8:

[0098]

[0099] Where, pV init_angle_S is the initial tilt angle fraction, pV angle_S is the tilt angle fraction.

[0100] S10126: Perform weighted summation of the area score, length score, and tilt angle score to obtain a score of the scanned ultrasound image.

[0101] Specifically, the score of the scanned ultrasound image may be calculated by weighted summing the area score, the length score, and the tilt angle score based on the following formula 9:

[0102] pVs =a×pV area_S +b×pV len_S +c×pV angle_S Formula 9

[0103] Where, pV s is the fraction of the scanned ultrasound image, pV area_S is the area fraction, pV len_S is the length fraction, pV angle_S is the tilt angle fraction, a, b, and c are proportional parameters, and the sum of the proportional parameters must be 1. The proportional parameters a, b, and c can be set to 0.2, 0.4, and 0.4, respectively.

[0104] In the embodiment of the present application, in the above step S101, the area, length and inclination angle characteristics of the portal vein are comprehensively evaluated through image segmentation and geometric calculation methods, and its actual physical size and inclination angle are accurately calculated. By setting a reasonable score threshold and weighted calculation, the quality of the scanned ultrasound image is accurately evaluated, providing a posture evaluation benchmark for the subsequent optimal probe posture search.

[0105] In the above step S102, when the scores of the consecutive multiple scanned ultrasound images reach the first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted. Specifically, it can be achieved by the following method:

[0106] When the scores of a preset number of consecutive scanned ultrasound images reach the first preset score threshold, it is considered that the probe is near the optimal probe posture and the trajectory backtracking search phase begins. No more data is added to the posture queue and the position of the probe at this time is recorded as the backtracking initial position. The preset number can be set to 2 for example, and the first preset score threshold s t It can be set to 1.5 as an example.

[0107] The scanned pose data at the end of the pose queue is taken as the current target pose, which is used as the target pose for the next backtracking search of the probe, and the current target pose in the pose queue is deleted, that is, the tail element of the pose queue is deleted.

[0108] In one embodiment, the function back() can be used to get the scanned pose data at the end of the pose queue posList, that is, execute the code Pos tar =posList.back(), Pos tar That is the current target pose. Use the pop_back() function to delete the tail element in the pose queue, that is, execute the code posList.pop_back().

[0109] In the embodiment of the present application, after taking the scanned posture data at the end of the posture queue as the current target posture, step S106 needs to be executed to determine whether the distance between the current target posture and the initial position of the backtracking is less than a third distance threshold. If the distance between the current target posture and the initial position of the backtracking is greater than the third distance threshold, it is considered that the current probe posture is not near the optimal probe posture, and the above step S101 is continued to control the probe to scan the portal vein according to the new preset path. Step S106 specifically includes the following steps S1061-S1062:

[0110] S1061: Calculate the distance between the current target position and the initial position of the backtracking as the start-end distance.

[0111] Specifically, the distance between the current target position and the initial position of the backtracking can be calculated based on the following formula 10 as the start-end distance:

[0112] d init_tar =sqrt((P init (0)-P tar (0)) 2 +(P init (1)-P tar (1) 2 ) Formula 10

[0113] Where, d init_tar is the start-end distance, P tar (0) and P tar (1) represents the position vector P in the current target pose tar The coordinates on the XY axis, P init (0) and P init (1) indicates the backtracking initial position P init The coordinates on the X and Y axes respectively, and sqrt represents square root calculation.

[0114] S1062: If the start-end distance is greater than the third distance threshold, continue executing the above step S101 to control the probe to scan the portal vein along the new preset path.

[0115] The third distance threshold may be exemplarily set to 0.015 m.

[0116] In the above step S103, the probe is controlled to move toward the current target posture, and the probe's retrospective posture data and retrospective ultrasound images are collected in real time. If the optimal probe posture is reached, step S104 is executed based on the score of the retrospective ultrasound image. If not, the probe is controlled to move toward the current target posture until it is determined that the probe has moved to the current target posture, and step S105 is executed. Specifically, the following steps S1031-S1033 are included:

[0117] S1031: Control the probe to move toward the current target posture, collect the probe's retrospective posture data and retrospective ultrasound image in real time, add the retrospective posture data to the retrospective queue, and calculate the score of the retrospective ultrasound image to determine whether the scores of multiple consecutive retrospective ultrasound images reach the second preset score threshold: if so, execute step S1032; if not, execute step S1033.

[0118] Specifically, the probe can be controlled to move toward the current target posture, and the probe position P, posture R and back-tracking ultrasound image can be collected in real time, wherein the probe position P and posture R form the back-tracking posture data and are inserted into the tail of the back-tracking queue backList, i.e., backList[i] = {P i , R i}, i represents the index number.

[0119] Before the probe begins moving toward the current target pose, a backlist queue (backList) must be created and its length set. Note that the backlist length is fixed, typically set to 10. Data is added to the backlist from the end in the order it was acquired. When the maximum queue length is reached, the data at the head of the queue is automatically deleted, retaining the most recent backlist pose data.

[0120] The score of the latest retrospective ultrasound image in the retrospective queue is calculated in real time, and the number of times the score of the retrospective ultrasound image is continuously greater than the second preset score threshold is recorded based on the following formula 11:

[0121]

[0122] Where count is the number of times the score of the retrospective ultrasound image is continuously greater than the second preset score threshold, pV s_back is the score of the retrospective ultrasound image, The second preset score threshold can be set to the ratio of the first preset score threshold, that is, Where β is the proportional coefficient, the first preset score threshold s t It can be exemplarily set to 1.5, and the proportional coefficient β can be exemplarily set to 0.95.

[0123] The method for calculating the score of the retrospective ultrasound image is the same as the method for calculating the score of the scan ultrasound image in the above step S101, and will not be repeated here.

[0124] Determine whether the number of times the score count of the retrospective ultrasound image is continuously greater than the second preset score threshold is greater than the preset number threshold count min :

[0125] If so, the current probe position is considered to be the optimal probe posture, that is, the latest backtracking posture data is the optimal probe posture, and step S1032 is executed.

[0126] If not, continue to control the probe to move toward the current target posture and execute step S1033.

[0127] S1032: Determine the latest backtracking posture data as the optimal probe posture and execute step S104.

[0128] S1033: Continue controlling the probe to move toward the current target posture until the probe is determined to have moved to the current target posture.

[0129] Among them, the probe is moved to the current target posture through the following steps S10331-S10332.

[0130] S10331: The position distance and angle distance between the latest backtracking pose data and the current target pose are used as the current end distance and current end angle distance.

[0131] Specifically, the position distance from the latest backtracking pose data to the current target pose can be calculated based on the following formula 12 as the current end distance:

[0132] d cur_tar =sqrt((P cur (0)-P tar (0)) 2 +(P cur (1)-P tar (1) 2 ) Formula 12

[0133] Where, d cur_tar is the current end distance, P tar (0) and P tar (1) represents the position vector P in the current target pose tar The coordinates on the XY axis, P cur (0) and P cur (1) represents the position P in the latest backtracking pose data cur The coordinates on the X and Y axes respectively, and sqrt represents square root calculation.

[0134] Get the posture data in the current target posture, recorded as the target posture matrix R tar , obtain the latest posture data in the back-tracking posture data, recorded as the current posture matrix R cur . The target pose matrix R tar and the current posture matrix R cur Convert to attitude quaternion and get the target attitude quaternion Q tar and the current attitude quaternion Qcur The method of converting the attitude matrix into the attitude quaternion is an existing technology and will not be described in detail here.

[0135] According to the target attitude quaternion Q tar and the current attitude quaternion Q cur , calculate the attitude change quaternion based on the following formula 13:

[0136] dQ×Q cur Q tar *={dw, dx, dy, dz} Formula 13

[0137] Where dQ is the attitude change quaternion, Q tar * represents the target attitude quaternion Q tar The conjugate quaternion dw, dx, dy and dz represent the four elements of the attitude change quaternion dQ.

[0138] The angular distance from the latest backtracking pose data to the current target pose is calculated based on the following formula 14 as the current end angular distance:

[0139]

[0140] Where, d a_cur_tar is the final angle distance, dw, dx, dy and dz represent the four elements of the attitude change quaternion dQ respectively.

[0141] S10332: If the current final distance is less than the first distance threshold, and the current final angle distance is less than the first angle threshold, it is determined that the probe has reached the current target posture.

[0142] Specifically, if the current end distance d cur_tar Less than the first distance threshold d max , and the final angle distance d a _ cur_tar Less than the first angle threshold d a_max , it means that the probe has reached the current target posture. Among them, the first distance threshold d max It can be set as 0.001m for example, and the first angle threshold d a_max It can be set to 0.005 rad as an example.

[0143] In the embodiment of the present application, during the process of the probe moving toward the current target posture, step S107 is further executed to determine whether the robotic arm has stopped moving. If it has stopped moving, the movement process is terminated, the next target posture is obtained as the current probe posture, and the above step S103 is re-executed to control the movement of the probe toward the current target posture. Step S107 specifically includes the following steps S1071-S1072:

[0144] S1071: Calculate the position distance and angle distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and current backtracking angle distance.

[0145] Specifically, it can be a method of calculating the position distance and angular distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and the current backtracking angle distance, which is consistent with the method of calculating the current end distance and the current end angle distance in the above step S10331, and will not be repeated here.

[0146] S1072: If the current return distance is less than the second distance threshold, and the current return angle distance is less than the second angle threshold, the process of moving the probe to the current target posture is terminated, and the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the above step S103 is executed to control the probe to move to the next target posture.

[0147] Specifically, if the current distance d cur_b Less than the second distance threshold d min , and the final angle distance d a_cur_b Less than the second angle threshold d a_min , it means that the robot arm stops moving due to force protection or other reasons. At this time, the robot arm will not be able to reach the current target posture, so the process will be terminated, and the scan posture data at the end of the posture queue will be taken as the next target posture, and the above step S103 will be executed to control the probe to move to the next target posture. Among them, the second distance threshold d min It can be set as 0.0002m for example, and the second angle threshold d a_min It can be set to 0.001 rad as an example.

[0148] In step S104, the probe is controlled to acquire an optimal ultrasound image of the portal vein at the optimal probe position. Specifically, based on the optimal probe position obtained in step S103, the ultrasound robot is controlled to call an interface to control the ultrasound device to enter C-mode. However, during the execution of C-mode, the ultrasound image may still change due to factors such as human breathing, resulting in unsatisfactory C-mode effects. Therefore, the method may also include controlling the robotic arm to translate and / or rotate according to preset rules when acquiring the optimal ultrasound image.

[0149] Among them, the preset rules are manually set according to the movement patterns of the portal vein during human breathing, including the movement strategy of the robotic arm's translation or rotation, and the calculation method of the specific translation or rotation movement amount.

[0150] In a specific embodiment, the motion strategy of the preset rule is to control the robot arm to translate along the X-axis of the tool coordinate system and rotate around the Z-axis of the tool coordinate system. The tool coordinate system at the end of the robot arm is as follows: Figure 4 As shown in the figure, x t 、y t and z t They represent the X, Y, and Z axes of the tool coordinate system at the end of the robot arm.

[0151] The specific method for calculating the translational and rotational motion amounts includes: calculating the motion amount of the robot arm translation along the X-axis of the tool coordinate system at the next moment based on the following formula 15:

[0152] Δp xtcpi =d xi *p xstep Formula 15

[0153] Where Δp xtcpi is the amount of translation of the robot arm along the X-axis of the tool coordinate system at the next moment, d xi is the current movement direction of the robot arm, the value is 1 or -1, p xstep is the step length of the robot arm moving along the X-axis of the tool coordinate system. xstep It can be set to 0.0003m as an example.

[0154] The amount of rotation of the robot arm around the Z-axis of the tool coordinate system at the next moment is calculated based on the following formula 16:

[0155] Δr ztcpi =d rzi *p xstep *μ Formula 16

[0156] Where Δr ztcpi is the amount of rotation of the robot arm around the Z axis of the tool coordinate system at the next moment, d rzi is the current direction of the robot arm's rotation around the Z axis of the tool coordinate system, and has a value of 1 or -1. μ is the proportional coefficient, which can be set to 5 for example.

[0157] Among them, when the C mode is started, the movement direction of the robot arm is d xi The initial setting is 1. When the robot moves the total distance p along the X axis of the tool coordinate system xtcpi Greater than the preset maximum translation distance p along the X axis of the tool coordinate system xmax When the robot arm moves in the opposite direction, the robot arm moves in the direction of d xi The initial setting is -1, that is, when p xtcpi >p xmax When d xi =-d xiAmong them, the total distance the robot arm moves along the X axis of the tool coordinate system is The preset maximum translation distance p along the X axis of the tool coordinate system xmax It can be set as 0.004m for example. The direction of motion d of the robot arm rotating around the Z axis of the tool coordinate system is rzi Equal to the movement direction of the robotic arm d xi .

[0158] Therefore, the translation change of the robot arm in the tool coordinate system at the next moment is determined to be (Δp xtcpi ,0,Δp ztcpi ), the attitude change is (0, 0, Δr ztcpi ). Among them, Δp ztcpi The force control algorithm is a conventional technique and will not be described in detail here.

[0159] In an embodiment of the present application, the above-mentioned step S104 dynamically adapts to changes in the ultrasound image caused by factors such as breathing by synchronously controlling the translation and rotation of the probe when executing in C mode, thereby improving the stability and clarity of the ultrasound image, ensuring better imaging effect of the portal vein section, and obtaining the optimal ultrasound image.

[0160] In the above step S105, if the probe moves to the current target posture and has not yet reached the optimal probe posture, the scanning posture data at the end of the posture queue is taken as the next target posture, and the above step S103 is re-executed to control the probe to move to the next target posture until the posture queue is empty, and the above step S101 is continued to control the probe to scan the portal vein according to the new preset path until the optimal probe posture and the optimal ultrasound image are obtained.

[0161] In the embodiment of the present application, the backtracking search phase described in the above steps S103 and S105 performs detailed scanning near the optimal probe posture, dynamically adjusts the probe position and posture, and gradually optimizes and accurately locates the optimal imaging position, which not only improves the image quality, but also significantly reduces unnecessary repeated scanning, improves imaging efficiency, adapts to complex anatomical structures, increases the robustness of positioning, reduces errors, and has flexible real-time response capabilities, ensuring that the system can quickly find the imaging position that meets clinical needs and obtain higher quality C-mode imaging results.

[0162] Example 2

[0163] Based on the same inventive concept, the embodiment of the present invention also provides a device for automatically playing C mode on the portal vein of the liver for an ultrasonic robot, referring to Figure 5 As shown, the device includes:

[0164] The first scanning module 101 is configured to control the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collect scanning posture data of the probe and corresponding scanning ultrasound images in real time, add the scanning posture data to a posture queue, and calculate a score for the scanning ultrasound image; the scanning posture data in the posture queue are arranged sequentially;

[0165] A first backtracking module 102 is configured to take the scanned pose data at the end of the pose queue as the current target pose and delete the current target pose from the pose queue when the scores of the consecutive scanned ultrasound images reach a first preset score threshold;

[0166] A first judgment module 103 is configured to control the probe to move toward the current target posture, collect the probe's retrospective posture data and retrospective ultrasound images in real time, and determine whether the optimal probe posture is reached based on the score of the retrospective ultrasound image: if so, execute the first determination module; if not, continue to control the probe to move toward the current target posture;

[0167] A first determining module 104 is configured to, when it is determined that the probe has reached an optimal probe posture, control the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture;

[0168] The second backtracking module 105 is used to re-take the scanning posture data at the end of the posture queue as the next target posture if the probe has not yet reached the optimal probe posture when it moves to the current target posture, and control the probe to move toward the next target posture until the posture queue is empty, and then continue to control the probe to scan the portal vein along the preset path until the optimal probe posture and the optimal ultrasound image are obtained.

[0169] Example 3

[0170] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the method for automatically marking the liver portal vein in C-mode for an ultrasonic robot as described in the first embodiment above is implemented.

[0171] Example 4

[0172] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for automatically marking the liver portal vein in C-mode for an ultrasound robot as described in the first embodiment above.

[0173] Example 5

[0174] Based on the same inventive concept, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the method for automatically marking the liver portal vein in C mode for an ultrasonic robot as described in the above embodiment 1.

[0175] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0176] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0179] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for automatically performing C-mode on the portal vein of the liver using an ultrasound robot, characterized in that: include: Controlling the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collecting the probe's scanning posture data and corresponding scanning ultrasound images in real time, adding the scanning posture data to a posture queue, and calculating a score for the scanning ultrasound image; the scanning posture data in the posture queue are arranged sequentially; When the scores of the consecutive scanned ultrasound images reach a first preset score threshold, the scanned posture data at the end of the posture queue is taken as the current target posture, and the current target posture in the posture queue is deleted; Control the probe to move toward the current target posture, collect the probe's backtracking posture data and backtracking ultrasound images in real time, and determine whether the optimal probe posture is reached based on the score of the backtracking ultrasound image: If so, controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe position; If the probe moves to the current target posture and still has not reached the optimal probe posture, the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move toward the next target posture until the posture queue is empty, and the process of controlling the probe to scan the portal vein along the preset path is continued until the optimal probe posture and the optimal ultrasound image are obtained.

2. The method according to claim 1, characterized in that The controlling the probe to move toward the current target posture, collecting the probe's backtracking posture data and backtracking ultrasound images in real time, and determining whether the optimal probe posture is reached according to the score of the backtracking ultrasound images, includes: Control the probe to move toward the current target posture, collect backtracking posture data and backtracking ultrasound images of the probe in real time, add the backtracking posture data to a backtracking queue, calculate the score of the backtracking ultrasound image, and determine whether the scores of multiple consecutive backtracking ultrasound images all reach a second preset score threshold: If so, the latest backtracking pose data is determined as the optimal probe pose; If not, continue to control the probe to move toward the current target posture until it is determined that the probe moves to the current target posture.

3. The method according to claim 2, characterized in that The probe is moved to the current target posture by: The position distance and angular distance between the latest backtracking posture data and the current target posture are used as the current end distance and the current end angular distance; If the current-to-final distance is less than a first distance threshold, and the current-to-final angle distance is less than a first angle threshold, it is determined that the probe has reached the current target posture.

4. The method according to claim 3, characterized in that Also includes: Calculate the position distance and angular distance between the latest backtracking posture data and the head of the backtracking queue as the current backtracking distance and the current backtracking angular distance; If the current return distance is less than the second distance threshold and the current return angle distance is less than the second angle threshold, the process of moving the probe to the current target posture is terminated, and the scanning posture data at the end of the posture queue is re-taken as the next target posture, and the probe is controlled to move to the next target posture.

5. The method according to claim 1, wherein Also includes: When it is determined that the scores of the consecutive multiple scanned ultrasound images reach a first preset score threshold, recording the position of the probe at that time as the backtracking initial position; After taking the scanned pose data at the end of the pose queue as the current target pose, the method further includes: Calculating the distance between the current target position and the initial position of the backtracking as the start-end distance; If the start-end distance is greater than the third distance threshold, the process of controlling the probe to scan the portal vein along the preset path continues.

6. The method according to claim 1, characterized in that The controlling the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture includes: When acquiring the optimal ultrasound image, the robotic arm is controlled to translate and / or rotate according to preset rules.

7. A device for automatically performing C-mode on the portal vein of the liver for an ultrasound robot, characterized in that: include: a first scanning module, configured to control the probe at the end of the ultrasonic robot's mechanical arm to scan the portal vein along a preset path, collect scanning posture data of the probe and corresponding scanning ultrasound images in real time, add the scanning posture data to a posture queue, and calculate a score for the scanning ultrasound image; the scanning posture data in the posture queue are arranged sequentially; a first backtracking module, configured to take the scanned posture data at the end of the posture queue as the current target posture, and delete the current target posture from the posture queue when the scores of the consecutive scanned ultrasound images reach a first preset score threshold; a first judgment module, configured to control the probe to move toward the current target posture, collect the probe's retrospective posture data and retrospective ultrasound images in real time, and determine whether the optimal probe posture is reached based on the score of the retrospective ultrasound image; if so, execute the first determination module; if not, continue to control the probe to move toward the current target posture; A first determining module is configured to, when it is determined that the probe has reached an optimal probe posture, control the probe to acquire an optimal ultrasound image of the portal vein at the optimal probe posture; The second backtracking module is used to re-take the scanning posture data at the end of the posture queue as the next target posture if the probe has not yet reached the optimal probe posture when it moves to the current target posture, and control the probe to move toward the next target posture until the posture queue is empty, and then continue to control the probe to scan the portal vein along the preset path until the optimal probe posture and the optimal ultrasound image are obtained.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the method for automatically setting the C-mode for the liver portal vein of an ultrasonic robot according to any one of claims 1 to 6 is implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for automatically setting the C-mode for the liver portal vein of an ultrasonic robot according to any one of claims 1 to 6 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the method for automatically opening the liver portal vein in C mode for an ultrasonic robot according to any one of claims 1 to 6.

Images