Method and device for autonomously scanning thyroid focus by ultrasonic robot

Abstract

The invention discloses a method and a device for an ultrasonic robot to autonomously scan a thyroid focus. The method comprises the following steps: controlling a probe to rotate towards a target direction; ultrasonic images and force sensor data are collected in real time, and the total rotation angle is recorded and updated in real time; performing image segmentation on the ultrasonic image to obtain a focus position; judging whether the probe collides with the human body according to data of the force sensor; if yes, calculating a translation obstacle avoidance offset, and judging whether the translation obstacle avoidance offset is equal to 0 or not; if yes, calculating a rotary obstacle avoidance offset, controlling the probe to rotate according to the rotary obstacle avoidance offset until the probe does not collide with the human body any more, and re-executing the probe rotation process; if not, the probe is controlled to move according to the translation obstacle avoidance offset, and the process of collecting the ultrasonic image and carrying out collision judgment is executed again; otherwise, the probe rotation process is executed again until the total rotation angle reaches the preset angle threshold value. According to the method, thyroid focus scanning can be completed more accurately and efficiently.

Description

Technical Field

[0001] The invention relates to a method and a device for autonomously scanning thyroid lesions with an ultrasonic robot. Background Art

[0002] In the field of ultrasound examination technology, especially in the diagnosis of thyroid diseases, the use of ultrasound robots is becoming more and more common. These robots can automatically or semi-automatically perform lesion scanning tasks, significantly improving diagnostic efficiency and accuracy. However, when faced with thyroid lesions located below the clavicle, the operation of traditional ultrasound probes has encountered certain challenges. Due to anatomical limitations, when it is necessary to switch from a transverse angle to a longitudinal angle to obtain a more comprehensive view of the lesion, the probe may collide with the clavicle, which not only hinders the smooth progress of the examination, but also may affect the accuracy of the diagnostic results.

[0003] To address this problem, existing technologies have proposed some solutions, such as using an external camera to acquire the real-time neck contour shape, accurately identifying the positions of obstacles such as the clavicle and chin, and adjusting the angle and position of the probe to avoid collision and ensure acquisition of the required longitudinal section images. Summary of the invention

[0004] In order to enable the ultrasonic robot to complete the scanning of thyroid lesions more accurately and efficiently, a method and device for autonomously scanning thyroid lesions by an ultrasonic robot are provided in an embodiment of the present invention.

[0005] In a first aspect, an embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot, which may include:

[0006] Control the probe to rotate around the Z axis of the tool coordinate system in the initial position to the target direction by a preset rotation angle;

[0007] Collect ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time;

[0008] Performing image segmentation on the ultrasound image to obtain the location of the lesion;

[0009] Determine whether the probe collides with the human body according to the force sensor data:

[0010] If yes, then calculating a translational obstacle avoidance offset according to the force sensor data and the lesion position, and determining whether the translational obstacle avoidance offset is equal to 0;

[0011] If so, a rotational obstacle avoidance offset is calculated according to the force sensor data, and the probe is controlled to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, and the above-mentioned probe rotation acquisition process is executed again;

[0012] If not, the probe is controlled to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset, and the above-mentioned process of acquiring ultrasonic images and performing collision judgment is re-executed;

[0013] Otherwise, the above-mentioned probe rotation acquisition process is executed again until the total rotation angle reaches the preset angle threshold, and it is determined that the probe completes the autonomous scanning of the thyroid lesion.

[0014] In one or some optional implementations of the embodiment of the present application, controlling the probe to rotate at an initial position around the Z axis of the tool coordinate system toward a target direction by a preset rotation angle includes:

[0015] Acquire an ultrasonic image collected by the probe at an initial position, and obtain a lesion position in the ultrasonic image;

[0016] Obtaining a current probe position and a current probe posture of the probe;

[0017] A first tool coordinate system position offset is calculated according to the lesion position, and a first next probe position is calculated in combination with the current probe position and the current probe posture;

[0018] A rotation matrix is ​​calculated according to the preset rotation angle, and a first next probe posture is calculated in combination with the current probe posture;

[0019] According to the first next probe position and the first next probe posture, the probe is controlled to rotate around the Z axis of the tool coordinate system toward the target direction.

[0020] In one or some optional implementations of the embodiment of the present application, the calculating the translation obstacle avoidance offset according to the force sensor data and the lesion position includes:

[0021] Determine the direction in which the probe contacts the human body according to the force sensor data, and obtain an obstacle direction coefficient;

[0022] Calculate the translation obstacle avoidance deviation pixel value according to the obstacle direction coefficient and the lesion position;

[0023] Determine whether the translation obstacle avoidance deviation pixel value is greater than a minimum deviation pixel threshold;

[0024] If yes, the translation obstacle avoidance offset is calculated based on the translation obstacle avoidance deviation pixel value and the force sensor data;

[0025] If not, the translation obstacle avoidance offset is set to 0.

[0026] In one or some optional implementations of the embodiments of the present application, the calculating of the rotational obstacle avoidance offset according to the force sensor data, and controlling the probe to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, includes:

[0027] Determine the direction in which the probe contacts the human body according to the force sensor data, and obtain an obstacle direction coefficient;

[0028] A position offset of a second tool coordinate system is calculated according to the obstacle direction coefficient, and a second next probe position is calculated in combination with the acquired current probe position and current probe posture of the probe;

[0029] Calculating the rotation angle around the X-axis according to the obstacle reverse coefficient;

[0030] The rotation obstacle avoidance offset is calculated according to the rotation angle around the X-axis, and the second next probe posture is calculated in combination with the current probe posture;

[0031] Controlling the probe to rotate around the X-axis according to the second next probe position and the second next probe posture to collect new force sensor data in real time;

[0032] If it can be determined from the new force sensor data that the probe still collides with the human body, the above process of calculating the second next probe position and the second next probe posture and rotating is re-executed until the probe is no longer in a collision state with the human body.

[0033] In one or some optional implementations of the embodiment of the present application, before the control probe rotates at an initial position around the Z axis of the tool coordinate system in a target direction by a preset rotation angle, the method further includes:

[0034] Controlling the probe to acquire an ultrasonic image; the ultrasonic image includes a thyroid lesion;

[0035] Performing image segmentation on the ultrasound image to obtain the location of the lesion;

[0036] Calculating the pixel distance from the lesion position to the center of the ultrasound image according to the lesion position and the width value of the ultrasound image;

[0037] Calculate an initial adjustment offset according to the pixel distance;

[0038] The probe is controlled to move to the initial position according to the initial adjustment offset.

[0039] In one or some optional implementations of the embodiment of the present application, the step of performing image segmentation on the ultrasound image to obtain the lesion location includes:

[0040] Performing image segmentation on the ultrasound image to obtain a lesion contour;

[0041] The lesion contour is subjected to ellipse fitting to obtain the center position of the lesion ellipse as the lesion position.

[0042] In a second aspect, an embodiment of the present invention provides a device for autonomously scanning thyroid lesions with an ultrasonic robot, which may include:

[0043] The first rotation module is used to control the probe to rotate around the Z axis of the tool coordinate system in the target direction at an initial position by a preset rotation angle;

[0044] A first segmentation module is used to perform image segmentation on the ultrasound image to obtain the location of the lesion;

[0045] A first judgment module is used to judge whether the probe collides with the human body according to the force sensor data: if so, execute the translation obstacle avoidance module; if not, execute the second rotation module;

[0046] A translation obstacle avoidance module, which is used to calculate a translation obstacle avoidance offset according to the force sensor data and the lesion position when the probe collides with the human body, and to determine whether the translation obstacle avoidance offset is equal to 0;

[0047] A rotation obstacle avoidance module, which is used to calculate the rotation obstacle avoidance offset according to the force sensor data when the translation obstacle avoidance offset is equal to 0, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and re-execute the above-mentioned probe rotation acquisition process;

[0048] A first moving module, configured to control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset when the translation obstacle avoidance offset is not equal to 0, and to re-execute the above-mentioned process of acquiring ultrasonic images and performing collision judgment;

[0049] The second rotation module is used to re-execute the above-mentioned probe rotation acquisition process when the probe does not collide with the human body, until the total rotation angle reaches a preset angle threshold, and determine that the probe completes the autonomous scanning of the thyroid lesion.

[0050] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the method for autonomously scanning thyroid lesions by an ultrasonic robot as described above.

[0051] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for autonomously scanning thyroid lesions with an ultrasonic robot as described above.

[0052] In a fifth aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein when the processor executes the computer program, the method of autonomously scanning thyroid lesions by an ultrasonic robot as described above is implemented.

[0053] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0054] The embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot. The method controls the probe to rotate around the Z axis of the tool coordinate system at an initial position by a preset angle, collects ultrasonic images and force sensor data in real time, records the total rotation angle, segments the ultrasonic image to determine the lesion position, and determines whether the probe collides with the human body according to the force sensor data. If the collision is determined, the translation or rotation obstacle avoidance offset is calculated, and the probe position or angle is adjusted accordingly until the collision state is released. If there is no collision, the rotation collection continues until the total rotation angle reaches a preset threshold, and the autonomous scanning of the thyroid lesion is completed. The method flexibly adjusts the movement of the probe according to the actual lesion position and the changes in the human anatomical structure, ensuring that the probe can safely complete the conversion process from transverse cutting to longitudinal cutting. At the same time, by real-time monitoring of the force sensor data and dynamically adjusting the position or angle of the probe, the collision between the probe and the human body can be effectively avoided, significantly improving the safety and reliability of the inspection process, and has strong adaptability to the individual differences of different patients, ensuring the safety and comfort of the patient. Therefore, the method not only improves the efficiency and comfort of ultrasonic examination, but also ensures the accuracy of diagnosis, which is of great significance for improving the quality of thyroid disease diagnosis.

[0055] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0056] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0058] Figure 1 A schematic diagram of a flow chart of a method for autonomously scanning thyroid lesions with an ultrasonic robot provided in an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of an ultrasonic robot provided by an embodiment of the present invention;

[0060] Figure 3 The probe status of the lesion scanning provided by the embodiment of the present invention;

[0061] Figure 4 A schematic diagram of rotating a tool coordinate system Z-axis toward a target direction by a preset rotation angle provided by an embodiment of the present invention;

[0062] Figure 5 Two collision situations provided for the embodiments of the present invention;

[0063] Figure 6 Two rotation obstacle avoidance solutions provided for the embodiments of the present invention;

[0064] Figure 7 A schematic diagram of the structure of a device for autonomously scanning thyroid lesions with an ultrasonic robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0066] The inventors found that in the prior art, the real-time acquisition of the neck contour shape is achieved based on an external camera, and the angle and position of the probe are adjusted by identifying the position of obstacles such as the clavicle and chin to avoid collision and ensure the acquisition of the required longitudinal section image. This method is time-consuming, relies on external cameras, is costly, and is easily affected by human body differences or ambient light, and has poor reliability. Based on this, the inventors have made the present invention after further research and development, providing a method and device for autonomously scanning thyroid lesions with an ultrasonic robot.

[0067] Embodiment 1

[0068] In the first embodiment of the present invention, a method for autonomously scanning thyroid lesions with an ultrasonic robot is provided. Figure 1 As shown, the method may include the following steps S101-S108:

[0069] S101: Control the probe to rotate around the Z axis of the tool coordinate system in the initial position toward the target direction by a preset rotation angle.

[0070] S102: Acquire ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time.

[0071] S103: Segment the ultrasound image to obtain the location of the lesion.

[0072] S104: Determine whether the probe collides with the human body based on the force sensor data: if so, execute step S105; if not, execute step S108.

[0073] S105: Calculate the translational obstacle avoidance offset according to the force sensor data and the lesion position, and determine whether the translational obstacle avoidance offset is equal to 0: if so, execute step S106; if not, execute step S107.

[0074] S106: Calculate the rotational obstacle avoidance offset according to the force sensor data, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, and re-execute the probe rotation acquisition process described in steps S101-S104.

[0075] S107: Control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset, and re-execute the process of acquiring ultrasonic images and performing collision judgment described in steps S102-S104.

[0076] S108: Re-execute the probe rotation acquisition process described in the above steps S101-S107 until the total rotation angle reaches a preset angle threshold, and determine that the probe completes the autonomous scanning of the thyroid lesion.

[0077] The embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot. The method controls the probe to rotate around the Z axis of the tool coordinate system at an initial position by a preset angle, collects ultrasonic images and force sensor data in real time, records the total rotation angle, segments the ultrasonic image to determine the lesion position, and determines whether the probe collides with the human body according to the force sensor data. If the collision is determined, the translation or rotation obstacle avoidance offset is calculated, and the probe position or angle is adjusted accordingly until the collision state is released. If there is no collision, the rotation collection continues until the total rotation angle reaches a preset threshold, and the autonomous scanning of the thyroid lesion is completed. The method flexibly adjusts the movement of the probe according to the actual lesion position and the changes in the human anatomical structure, ensuring that the probe can safely complete the conversion process from transverse cutting to longitudinal cutting. At the same time, by real-time monitoring of the force sensor data and dynamically adjusting the position or angle of the probe, the collision between the probe and the human body can be effectively avoided, significantly improving the safety and reliability of the inspection process, and has strong adaptability to the individual differences of different patients, ensuring the safety and comfort of the patient. Therefore, the method not only improves the efficiency and comfort of ultrasonic examination, but also ensures the accuracy of diagnosis, which is of great significance for improving the quality of thyroid disease diagnosis.

[0078] In order to facilitate those skilled in the art to understand the present solution, the following first provides an example diagram of the ultrasonic robot and the probe state of the lesion scanning in the embodiment of the present application: Figure 2As shown in the figure, there are two sets of coordinate systems, the world coordinate system at the root of the robot arm and the tool coordinate system at the end of the robot arm. Figure 3 As shown, the left side of the figure is the definition of the initial scanning state, at this time the ultrasound probe is in a transverse state (horizontal state), the tool coordinate system at the end of the probe is shown in the figure, the X-axis points to the vertical direction of the probe, the Y-axis points to the horizontal direction of the probe, and the Z-axis points to the depth direction of the probe. The right side of the figure is the definition of the scanning end state, at this time the ultrasound probe is in a longitudinal state (vertical state), and the tool coordinate system at the end of the probe is shown in the figure, which is rotated 90° on the horizontal plane to complete the lesion scanning. Among them, the scanning in the figure is the scanning described in this article.

[0079] In an embodiment of the present application, when starting the autonomous scanning of thyroid lesions based on the ultrasonic robot, the probe of the ultrasonic robot will first be scanned on the human neck in a cross-sectional form. Since the probe is in a cross-sectional state, it will not collide with the clavicle or chin when moving in the Y-axis direction in the tool coordinate system. At the same time, ultrasonic images are collected in real time during the scanning process. When a thyroid lesion appears in the ultrasonic image, step S109 is executed to control the probe to pull the lesion to the center of the ultrasonic image, so that the probe is in the initial position, and then the method of autonomously scanning thyroid lesions by the ultrasonic robot described in the above steps S101-S108 is started. Among them, step S109 specifically includes the following steps S1091-S1095:

[0080] S1091: Control the probe to acquire an ultrasound image, which includes a thyroid lesion.

[0081] S1092: Perform image segmentation on the ultrasound image to obtain the location of the lesion.

[0082] Specifically, the ultrasound image may be segmented to obtain the lesion contour, and then an ellipse fitting may be performed on the lesion contour to obtain the center position of the lesion ellipse as the lesion position M=(m x ,m y ). Among them, m in the lesion location x 、m y They represent the X-axis pixel coordinate and Y-axis pixel coordinate in the ultrasound image coordinate system respectively.

[0083] S1093: Calculate the pixel distance from the lesion position to the center of the ultrasound image according to the lesion position and the width value of the ultrasound image.

[0084] Specifically, it can be, according to the lesion location M=(m x ,m y ) and the width of the ultrasound image, the pixel distance from the lesion position to the center of the ultrasound image is calculated based on the following formula 1:

[0085]

[0086] Where Δm1 represents the pixel distance from the lesion location to the center of the ultrasound image, m x represents the X-axis pixel coordinate of the lesion position in the ultrasound image coordinate system, and W represents the width value of the ultrasound image. Wherein, W can be set to 800 for example.

[0087] S1094: Calculate an initial adjustment offset based on the pixel distance.

[0088] Specifically, the initial adjustment offset may be calculated based on the following formula 2 according to the pixel distance:

[0089]

[0090] In the formula, p yoffset1 represents the initial adjustment offset, Δm1 represents the pixel distance from the lesion location to the center of the ultrasound image, and k p , k i and k d They respectively represent the proportional coefficient, integral coefficient and differential coefficient of a PID controller (Proportional-Integral-Derivative Controller), and can be set to 0.001, 0.0001 and 0.002, respectively.

[0091] S1095: Control the probe to move to the initial position according to the initial adjustment offset.

[0092] Specifically, the current probe position P of the probe may be obtained. current =[P cx , P cy , P cz ] T and the current probe posture R current Among them, P cx , P cy , P cz They respectively represent the coordinates of the tool coordinate system at the end of the probe in the XYZ directions in the world coordinate system.

[0093] According to the initial adjustment offset, the current probe position and the current probe posture, the initial next probe position is calculated based on the following formula 3:

[0094]

[0095] Where P start_next represents the initial next probe position, P current Indicates the current probe position, R current Indicates the current probe posture, p yoffset1Indicates the initial adjustment offset.

[0096] In this step, the probe does not need to rotate, so the initial next probe posture of the probe is equal to the current probe posture, that is, R start_next =R current .

[0097] The probe is controlled to move on the Y axis according to the initial next probe position and the initial next probe posture, so that the probe moves to the initial position.

[0098] In the embodiment of the present application, the above step S109 controls the ultrasonic probe to pull the thyroid lesion to the center of the image, which not only improves the accuracy of lesion positioning and ensures the best observation position of the lesion during the entire scanning process, but also reduces external interference caused by probe movement, such as avoiding distortion of the edge area of ​​the image and the influence of other structures on lesion observation, thereby significantly improving the accuracy of lesion detection and diagnostic quality. In addition, this process can also effectively prevent the probe from colliding with the clavicle or chin during movement, ensuring the safety and smoothness of the scanning process.

[0099] In the above step S101, the control probe is rotated at the initial position around the Z axis of the tool coordinate system in the target direction by a preset rotation angle, referring to Figure 4 As shown, the probe is rotated clockwise around the Z axis.

[0100] Step S101 specifically includes the following steps S1011-S1015:

[0101] S1011: Acquire an ultrasonic image collected by the probe at an initial position, and obtain a lesion position in the ultrasonic image.

[0102] Specifically, an ultrasound image collected by the probe at the initial position is obtained, the ultrasound image is segmented to obtain the lesion contour, and then an ellipse is fitted to the lesion contour to obtain the center position of the lesion ellipse as the lesion position M=(m x , m y ). Among them, m in the lesion location x 、m y They represent the X-axis pixel coordinate and Y-axis pixel coordinate in the ultrasound image coordinate system respectively.

[0103] S1012: Acquire the current probe position and current probe posture of the probe.

[0104] Specifically, the current probe position P of the probe may be obtained. current =[P cx , P cy , P cz ] T and the current probe posture R current Among them, Pcx , P cy , P cz They respectively represent the coordinates of the tool coordinate system at the end of the probe in the XYZ directions in the world coordinate system.

[0105] S1013: Calculate the first tool coordinate system position offset according to the lesion position, and calculate the first next probe position in combination with the current probe position and the current probe posture.

[0106] Specifically, the position offset of the first tool coordinate system may be calculated based on the following formula 4 according to the position of the lesion:

[0107]

[0108] In the formula, toolOffset1 represents the position offset of the first tool coordinate system, m x represents the X-axis pixel coordinate of the lesion position in the ultrasound image coordinate system, K PixelPhyDis is the physical distance of ultrasound image pixels, which refers to the actual distance represented by each pixel in the ultrasound image in the real world, and can be set to 0.00005 by way of example.

[0109] According to the first tool coordinate system position offset, the current probe position and the current probe posture, the first next probe position is calculated based on the following formula 5:

[0110] P next1 =P current +R current *toolOffset1 Formula 5

[0111] Where P next1 Indicates the first next probe position, P current Indicates the current probe position, R current Indicates the current probe posture, and toolOffset1 indicates the position offset of the first tool coordinate system.

[0112] S1014: Calculate a rotation matrix according to a preset rotation angle, and calculate a first next probe posture in combination with the current probe posture.

[0113] Specifically, the rotation matrix may be calculated based on the following formula 6 according to the preset rotation angle:

[0114]

[0115] In the formula, R z (θ) represents a rotation matrix, and θ represents a preset rotation angle, which can be set to 0.03 radians by way of example.

[0116] Multiply the rotation matrix and the current probe posture, and calculate the first next probe posture based on the following formula 7:

[0117] R next1 =Rc urrent *R z (θ) Formula 7

[0118] In the formula, R next1 Indicates the first next probe posture, R z (θ) represents the rotation matrix, R current Indicates the current probe posture.

[0119] S1015: Control the probe to rotate around the Z axis according to the first next probe position and the first next probe posture.

[0120] In the embodiment of the present application, the above-mentioned step S101 performs a rotational movement with the lesion as the center, ensuring that the lesion is always in the center of the image during the entire rotation process, which not only maintains the clarity and continuity of the lesion and improves the accuracy of lesion feature recognition, but also realizes a comprehensive scanning of the lesion from multiple angles, thereby obtaining richer lesion information, further improving the accuracy and reliability of diagnosis.

[0121] In the above step S102, ultrasonic images and force sensor data are collected in real time, and the total rotation angle is recorded and updated in real time.

[0122] Specifically, after completing the above step S101, an ultrasonic image and force sensor data F = (fx, fy, fz, tx, ty, tz) are collected, wherein the force sensor data consists of 6 components, the first 3 components fx, fy, and fz respectively represent the forces acting on the probe in the XYZ directions in the tool coordinate system, and the last three components tx, ty, and tz respectively represent the torque of the probe in the XYZ directions in the tool coordinate system, that is, the rotational force.

[0123] At the same time, the total rotation angle θ needs to be updated in real time 总 , that is, θ 总 =θ 总 +θ.

[0124] In the above step S103, the ultrasound image is segmented to obtain the location of the lesion, which may specifically include the following steps S1031-S1032:

[0125] S1031: Perform image segmentation on the ultrasound image to obtain the lesion contour.

[0126] Specifically, a preset image segmentation network may be used to perform image segmentation on the ultrasound image to obtain the lesion contour.

[0127] Those skilled in the art can select a suitable neural network for pre-training based on the detailed description of the prior art to obtain a preset image segmentation network. The training process may specifically include:

[0128] In the first step, ultrasound images of thyroid lesions are collected and the contours of the lesions in the ultrasound images are marked respectively. After preprocessing, a thyroid lesion contour dataset is obtained.

[0129] The second step is to select a suitable neural network model as the initial image segmentation network, such as U-Net model, SegNet model, etc.

[0130] The third step is to divide the thyroid lesion contour dataset into a training set and a test set.

[0131] The fourth step is to define the loss function (such as cross entropy loss function, mean square error loss function, etc.), optimization algorithm (such as Adam, SGD, etc.), etc.

[0132] The fifth step is to use the training set of the thyroid lesion contour dataset to train the initial image segmentation network to obtain the trained image segmentation model.

[0133] Repeat the training process of the above image segmentation model until the preset conditions are met, stop training, and obtain a preset image segmentation network. The preset conditions can be set as reaching a fixed number of iterations, reaching a threshold value of accuracy, and the accuracy does not change within a preset number of iterations. No specific limitation is made here.

[0134] S1032: Perform ellipse fitting on the lesion contour to obtain the center position of the lesion ellipse as the lesion position.

[0135] Specifically, the fitEllipse function in the open source computer vision library OpenCV (Open Source Computer Vision Library) can be used to perform ellipse fitting on the lesion contour to obtain the center position of the lesion ellipse as the lesion position M=(m x ,m y ). Among them, m in the lesion location x 、m y They represent the X-axis pixel coordinate and Y-axis pixel coordinate in the ultrasound image coordinate system respectively.

[0136] In the above step S104, it is determined whether the probe collides with the human body according to the force sensor data: if so, step S105 is executed; if not, step S108 is executed.

[0137] Specifically, it can be to extract the force fy in the Y-axis direction of the tool coordinate system in the force sensor data F = (fx, fy, fz, tx, ty, tz), and determine whether the absolute value of fy is greater than the preset collision threshold ∈: if so, it means that the probe collides with the clavicle or chin of the human body, and step S105 is executed to perform obstacle avoidance operation; if not, it means that the probe does not collide with the clavicle or chin of the human body, and step S108 is executed to continue the process of probe rotation collection. Among them, the preset collision threshold ∈ can be set to 2.0 for example, and the unit is N.

[0138] In the above step S105, the translation obstacle avoidance offset is calculated based on the force sensor data and the lesion position, and it is determined whether the translation obstacle avoidance offset is equal to 0: if so, step S106 is executed. If not, step S107 is executed. Specifically, the following steps S1051-S1055 are included:

[0139] S1051: Determine the direction in which the probe contacts the human body according to the force sensor data, and obtain an obstacle direction coefficient.

[0140] Specifically, the force fy in the Y-axis direction of the tool coordinate system in the force sensor data F = (fx, fy, fz, tx, ty, tz) can be extracted. When fy>=∈, it means that the probe collides with the clavicle, and the probe needs to be translated upward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to 1. When fy<=-∈, it means that the probe collides with the chin, and the probe needs to be translated downward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to -1.

[0141] In order to facilitate those skilled in the art to understand the present invention, the above two collision situations can be referred to Figure 5 As shown, the left side indicates that the probe collides with the clavicle, and the probe needs to be translated along the Y-axis of the tool coordinate system toward the top of the human body to avoid the obstacle. The right side indicates that the probe collides with the chin, and the probe needs to be translated along the Y-axis of the tool coordinate system toward the bottom of the human body to avoid the obstacle.

[0142] S1052: Calculate the pixel value of the translation obstacle avoidance deviation according to the obstacle direction coefficient and the lesion position.

[0143] Specifically, the pixel value of the translation obstacle avoidance deviation can be calculated based on the following formula 8 according to the obstacle direction coefficient and the lesion position:

[0144]

[0145] Where Δm2 represents the pixel value of the translation obstacle avoidance deviation, m x represents the X-axis pixel coordinate of the lesion position in the ultrasound image coordinate system, W represents the width value of the ultrasound image, and obstacleDir is the obstacle direction coefficient. Among them, W can be set to 800 for example.

[0146] S1053: Determine whether the translation obstacle avoidance deviation pixel value is greater than the minimum deviation pixel threshold: if so, execute step S1054; if not, execute step S1055.

[0147] Specifically, it can be determined whether the absolute value of the translation obstacle avoidance deviation pixel value |Δm2| is greater than the minimum deviation pixel threshold ε: if so, it means that the position of the probe can be translated to avoid obstacles at this time, and the lesion will not be lost, and step S1054 is executed to calculate the translation obstacle avoidance offset; if not, it means that it is no longer possible to avoid obstacles along the Y-axis direction of the tool coordinate system at this time, which may cause the lesion to be lost, and step S1055 is executed to set the translation obstacle avoidance offset to 0, and then perform the following step S106 of the rotation obstacle avoidance operation. Among them, the minimum deviation pixel threshold can be set to 10 for example.

[0148] S1054: Calculate the translational obstacle avoidance offset according to the translational obstacle avoidance deviation pixel value and the force sensor data.

[0149] Specifically, the translation obstacle avoidance offset may be calculated based on the following formula 9 according to the translation obstacle avoidance deviation pixel value and the force exerted on the probe in the Y-axis direction in the tool coordinate system in the force sensor data:

[0150] p yoffset2 =k m *Δm2+k f *fy Formula 9

[0151] In the formula, p yoffset2 is the translation obstacle avoidance offset, Δm2 represents the translation obstacle avoidance deviation pixel value, fy represents the force on the probe in the Y-axis direction in the tool coordinate system, and k m is the preset pixel deviation coefficient, k f is the preset force deviation coefficient. Wherein, W can be set to 800, k m An exemplary setting is 0.00002.

[0152] S1055: Set the translation obstacle avoidance offset to 0.

[0153] In the above step S106, the rotation obstacle avoidance offset is calculated based on the force sensor data, and the probe is controlled to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and the probe rotation acquisition process described in steps S101-S104 is re-executed. Specifically, the following steps S1061-S1066 are included:

[0154] S1061: Determine the direction in which the probe contacts the human body according to the force sensor data, and obtain an obstacle direction coefficient.

[0155] Specifically, the force fy in the Y-axis direction of the tool coordinate system in the force sensor data F = (fx, fyfz, tx, ty, tz) can be extracted. When fy>=∈, it means that the probe collides with the clavicle, and the probe needs to be translated upward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to 1. When fy<=-∈, it means that the probe collides with the chin, and the probe needs to be translated downward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to -1.

[0156] In order to facilitate those skilled in the art to understand this solution, the rotation obstacle avoidance solutions corresponding to the above two collision situations can be referred to Figure 6 As shown, the left side indicates that the probe collides with the clavicle, and the probe needs to be rotated clockwise along the X-axis of the tool coordinate system to avoid the obstacle. The left side indicates that the probe collides with the chin, and the probe needs to be rotated counterclockwise along the X-axis of the tool coordinate system to avoid the obstacle.

[0157] S1062: Calculate the position offset of the second tool coordinate system according to the obstacle direction coefficient, and calculate the second next probe position in combination with the acquired current probe position and current probe posture of the probe.

[0158] Specifically, the position offset of the second tool coordinate system may be calculated based on the following formula 10 according to the obstacle direction coefficient:

[0159]

[0160] Where toolOffset2 represents the position offset of the second tool coordinate system, obstacleDir is the obstacle direction coefficient, and W tool is the end length of the ultrasonic probe, which can be set to 0.04 m by way of example.

[0161] According to the position offset of the second tool coordinate system, the current probe position and the current probe posture, the second next probe position is calculated based on the following formula 11:

[0162] P next2 =P current +R current *toolOffset2 Formula 11

[0163] Where P next2 Indicates the second next probe position, P current Indicates the current probe position, R current Indicates the current probe posture, and toolOffset2 indicates the position offset of the second tool coordinate system.

[0164] S1063: Calculate the rotation angle around the X-axis according to the obstacle reverse coefficient.

[0165] Specifically, the rotation angle around the X-axis may be calculated according to the obstacle reverse coefficient based on the following formula 12:

[0166] α=obstacleDir*(α 0 -k t *tx) Formula 12

[0167] Where α represents the rotation angle around the X-axis, obstacleDir is the obstacle direction coefficient, and α 0 is the preset initial value of the rotation angle, k t is the torque deviation coefficient, and tx is the torque of the probe in the X-axis direction in the tool coordinate system in the force sensor data. 0 An exemplary setting may be 0.05 radians, k t An exemplary setting is 0.1.

[0168] S1064: Calculate the rotation obstacle avoidance offset according to the rotation angle around the X-axis, and calculate the second next probe posture in combination with the current probe posture.

[0169] Specifically, the rotation obstacle avoidance offset may be calculated according to the rotation angle around the X-axis based on the following formula 13:

[0170]

[0171] In the formula, R x (α) represents the rotational obstacle avoidance offset, and α represents the rotation angle around the X-axis.

[0172] Multiply the rotation obstacle avoidance offset and the current probe posture, and calculate the second next probe posture based on the following formula 14:

[0173] R next2 =R current *R x (α) Formula 14

[0174] In the formula, R next2 Indicates the second next probe posture, R x (α) represents the rotational obstacle avoidance offset, R current Indicates the current probe posture.

[0175] S1065: Control the probe to rotate around the X-axis according to the second next probe position and the second next probe posture, and collect new force sensor data in real time.

[0176] S1066: If it can be determined based on the new force sensor data that the probe and the human body have still collided, then the process of calculating the second next probe position and the second next probe posture and rotating as described in the above steps S1061-S1065 is re-executed until the probe is no longer in a collision state with the human body, and the process of probe rotation and collection as described in steps S101-S104 is re-executed.

[0177] In the embodiment of the present application, the rotation obstacle avoidance operation completed in the above step S106, when the translation obstacle avoidance along the Y-axis is not sufficient to solve the problem, ensures that the probe safely avoids obstacles without losing the lesion by rotating around the X-axis, so that the method can ensure the safe operation of lesion scanning from multiple angles, improves the stability and clarity of the lesion ultrasound image, and enhances the flexibility and adaptability of the method.

[0178] In the above step S107, the movement of the probe is controlled according to the translation obstacle avoidance offset, and the process of acquiring ultrasonic images and performing collision judgment described in steps S102-S104 is re-executed.

[0179] Specifically, the third next probe position may be calculated based on the following formula 15 according to the translation obstacle avoidance offset and the current probe position:

[0180]

[0181] Where P next3 Indicates the third next probe position, p yoffset2 Represents the translation obstacle avoidance offset, P current Indicates the current probe position.

[0182] In this step, the probe does not need to rotate, so the third next probe posture of the probe is equal to the current probe posture, that is, R next3 =R current .

[0183] The probe is controlled to move on the Y axis according to the third next probe position and the third next probe posture to complete translation obstacle avoidance, and then the process of acquiring ultrasonic images and performing collision judgment described in steps S102-S104 is re-executed.

[0184] In the embodiment of the present application, the translation obstacle avoidance operation completed in the above steps S105 and S107 ensures that the probe safely avoids obstacles without losing the lesion by translationally avoiding obstacles along the Y axis when the ultrasound probe encounters a collision with the clavicle or chin along the Y axis, which not only avoids human discomfort caused by the collision, but also maintains continuous tracking of the lesion, ensuring the stability and clarity of the lesion image. At the same time, by real-time monitoring of the force in the Y axis direction, the obstacle avoidance speed is dynamically adjusted according to the force magnitude, making the obstacle avoidance process faster and more efficient, improving the robustness and adaptability of the system, and ensuring the smooth completion of the lesion scanning task.

[0185] In the above step S108, the probe rotation acquisition process of the above steps S101-S107 is re-executed until the total rotation angle reaches the preset angle threshold, and it is determined that the probe has completed the autonomous scanning of the thyroid lesion.

[0186] Specifically, the probe rotation acquisition process of steps S101-S107 may be executed cyclically until the total rotation angle reaches a preset angle threshold π / 2, i.e., 90°, and the probe changes from a transverse cutting posture to a longitudinal cutting posture, and then it is determined that the probe has completed autonomous scanning of the thyroid lesion.

[0187] Embodiment 2

[0188] Based on the same inventive concept, an embodiment of the present invention further provides a device for autonomously scanning thyroid lesions with an ultrasonic robot, the device comprising:

[0189] The first rotation module 101 is used to control the probe to rotate around the Z axis of the tool coordinate system in the target direction at an initial position by a preset rotation angle;

[0190] The first acquisition module 102 is used to acquire ultrasound images and force sensor data in real time, and record and update the total rotation angle in real time;

[0191] A first segmentation module 103 is used to segment the ultrasound image to obtain the location of the lesion;

[0192] A first judgment module 104 is used to judge whether the probe collides with the human body according to the force sensor data: if so, execute the translation obstacle avoidance module; if not, execute the second rotation module;

[0193] The translation obstacle avoidance module 105 is used to calculate the translation obstacle avoidance offset according to the force sensor data and the lesion position when the probe collides with the human body, and determine whether the translation obstacle avoidance offset is equal to 0;

[0194] The rotation obstacle avoidance module 106 is used to calculate the rotation obstacle avoidance offset according to the force sensor data when the translation obstacle avoidance offset is equal to 0, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and re-execute the above-mentioned probe rotation acquisition process;

[0195] The first moving module 107 is used to control the probe to move along the Y axis of the tool coordinate system according to the translation obstacle avoidance offset when the translation obstacle avoidance offset is not equal to 0, and re-execute the above process of acquiring ultrasonic images and performing collision judgment;

[0196] The second rotation module 108 is used to re-execute the above-mentioned probe rotation acquisition process when there is no collision between the probe and the human body, until the total rotation angle reaches a preset angle threshold, and determine that the probe completes the autonomous scanning of the thyroid lesion.

[0197] Embodiment 3

[0198] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in the above-mentioned embodiment 1 is implemented.

[0199] Embodiment 4

[0200] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in the above-mentioned embodiment 1.

[0201] Embodiment 5

[0202] Based on the same inventive concept, an embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the method of autonomously scanning thyroid lesions with an ultrasonic robot as described in the above embodiment 1.

[0203] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0204] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0207] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for autonomously scanning thyroid lesions with an ultrasonic robot, characterized in that: include: Control the probe to rotate around the Z axis of the tool coordinate system in the initial position to the target direction by a preset rotation angle; Collect ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time; Performing image segmentation on the ultrasound image to obtain the location of the lesion; Determining whether the probe collides with a human body according to the force sensor data; If yes, then calculating a translation obstacle avoidance offset according to the force sensor data and the lesion position, and determining whether the translation obstacle avoidance offset is equal to 0; If so, a rotational obstacle avoidance offset is calculated according to the force sensor data, and the probe is controlled to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, and the above-mentioned probe rotation acquisition process is executed again; If not, the probe is controlled to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset, and the above process of acquiring ultrasonic images and performing collision judgment is re-executed; Otherwise, the above-mentioned probe rotation acquisition process is executed again until the total rotation angle reaches the preset angle threshold, and it is determined that the probe completes the autonomous scanning of the thyroid lesion.

2. The method according to claim 1, characterized in that The controlling the probe to rotate at an initial position around the Z axis of the tool coordinate system toward a target direction by a preset rotation angle comprises: Acquire an ultrasonic image collected by the probe at an initial position, and obtain a lesion position in the ultrasonic image; Obtaining a current probe position and a current probe posture of the probe; A first tool coordinate system position offset is calculated according to the lesion position, and a first next probe position is calculated in combination with the current probe position and the current probe posture; A rotation matrix is ​​calculated according to the preset rotation angle, and a first next probe posture is calculated in combination with the current probe posture; According to the first next probe position and the first next probe posture, the probe is controlled to rotate around the Z axis of the tool coordinate system toward the target direction.

3. The method according to claim 1, characterized in that The calculating the translation obstacle avoidance offset according to the force sensor data and the lesion position includes: Determine the direction in which the probe contacts the human body according to the force sensor data, and obtain an obstacle direction coefficient; Calculate the translation obstacle avoidance deviation pixel value according to the obstacle direction coefficient and the lesion position; Determine whether the translation obstacle avoidance deviation pixel value is greater than a minimum deviation pixel threshold; If yes, the translation obstacle avoidance offset is calculated based on the translation obstacle avoidance deviation pixel value and the force sensor data; If not, the translation obstacle avoidance offset is set to 0.

4. The method according to claim 1, characterized in that: The step of calculating a rotational obstacle avoidance offset according to the force sensor data, and controlling the probe to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body includes: Determine the direction in which the probe contacts the human body according to the force sensor data, and obtain an obstacle direction coefficient; A position offset of a second tool coordinate system is calculated according to the obstacle direction coefficient, and a second next probe position is calculated in combination with the acquired current probe position and current probe posture of the probe; Calculating the rotation angle around the X-axis according to the obstacle reverse coefficient; The rotation obstacle avoidance offset is calculated according to the rotation angle around the X-axis, and the second next probe posture is calculated in combination with the current probe posture; Controlling the probe to rotate around the X-axis according to the second next probe position and the second next probe posture to collect new force sensor data in real time; If it can be determined from the new force sensor data that the probe still collides with the human body, the above process of calculating the second next probe position and the second next probe posture and rotating is re-executed until the probe is no longer in a collision state with the human body.

5. The method according to claim 1, characterized in that Before the control probe is rotated at the initial position around the Z axis of the tool coordinate system in the target direction by a preset rotation angle, the method further includes: Controlling the probe to acquire an ultrasonic image; the ultrasonic image includes a thyroid lesion; Performing image segmentation on the ultrasound image to obtain the location of the lesion; Calculating the pixel distance from the lesion position to the center of the ultrasound image according to the lesion position and the width value of the ultrasound image; Calculate an initial adjustment offset according to the pixel distance; The probe is controlled to move to the initial position according to the initial adjustment offset.

6. The method according to claim 1, characterized in that The step of performing image segmentation on the ultrasound image to obtain the lesion location includes: Performing image segmentation on the ultrasound image to obtain a lesion contour; The lesion contour is subjected to ellipse fitting to obtain the center position of the lesion ellipse as the lesion position.

7. A device for autonomously scanning thyroid lesions with an ultrasonic robot, characterized in that: include: The first rotation module is used to control the probe to rotate around the Z axis of the tool coordinate system in the target direction at an initial position by a preset rotation angle; The first acquisition module is used to acquire ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time; A first segmentation module is used to perform image segmentation on the ultrasound image to obtain the location of the lesion; A first judgment module is used to judge whether the probe collides with the human body according to the force sensor data: if so, execute the translation obstacle avoidance module; if not, execute the second rotation module; A translation obstacle avoidance module, which is used to calculate a translation obstacle avoidance offset according to the force sensor data and the lesion position when the probe collides with the human body, and to determine whether the translation obstacle avoidance offset is equal to 0; A rotation obstacle avoidance module, which is used to calculate the rotation obstacle avoidance offset according to the force sensor data when the translation obstacle avoidance offset is equal to 0, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and re-execute the above-mentioned probe rotation acquisition process; A first moving module, configured to control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset when the translation obstacle avoidance offset is not equal to 0, and to re-execute the above-mentioned process of acquiring ultrasonic images and performing collision judgment; The second rotation module is used to re-execute the above-mentioned probe rotation acquisition process when the probe does not collide with the human body, until the total rotation angle reaches a preset angle threshold, and determine that the probe completes the autonomous scanning of the thyroid lesion.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by the processor, the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in any one of claims 1 to 6 is implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by the processor, the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in any one of claims 1 to 6 is implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in any one of claims 1-6.

Images