A method and device for autonomously scanning thyroid lesions using an ultrasonic robot

Abstract

The present invention discloses a method and device for autonomously scanning thyroid lesions with an ultrasonic robot. The method includes: controlling the probe to rotate in a target direction; real-time acquisition of ultrasonic images and force sensor data, and real-time recording and updating of the total rotation angle; performing image segmentation on the ultrasonic image to obtain the lesion position; judging whether the probe collides with the human body based on the force sensor data; if so, calculating the translation obstacle avoidance offset and judging whether the translation obstacle avoidance offset is equal to 0; if so, calculating the rotation obstacle avoidance offset and controlling the probe rotation according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and re-executing the probe rotation process; if not, controlling the probe movement according to the translation obstacle avoidance offset and re-executing the above-mentioned process of acquiring ultrasonic images and performing collision judgment; otherwise, re-executing the probe rotation process until the total rotation angle reaches a preset angle threshold. This method can complete thyroid lesion scanning more accurately and efficiently.

Description

Technical Field

[0001] The present invention relates to a method and device for autonomously scanning thyroid lesions with an ultrasonic robot. Background Art

[0002] The use of ultrasound robots is becoming increasingly common in the field of ultrasound examination technology, especially in the diagnosis of thyroid diseases. These robots are able to automatically or semi-automatically perform lesion scanning tasks, significantly improving diagnostic efficiency and accuracy. However, when faced with thyroid lesions located below the clavicle, the operation of traditional ultrasound probes has encountered certain challenges. Due to anatomical limitations, when switching from a transverse to a longitudinal angle to obtain a more comprehensive view of the lesion, the probe may collide with the clavicle, which not only hinders the smooth progress of the examination but also may affect the accuracy of the diagnostic results.

[0003] To address this issue, existing technologies have proposed some solutions, such as using an external camera to capture the real-time neck contour shape, accurately identifying the positions of obstacles such as the clavicle and chin, and adjusting the angle and position of the probe to avoid collisions and ensure acquisition of the required longitudinal section images. Summary of the Invention

[0004] In order to enable the ultrasonic robot to scan thyroid lesions more accurately and efficiently, an embodiment of the present invention provides a method and device for autonomously scanning thyroid lesions by an ultrasonic robot.

[0005] In a first aspect, an embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot, which may include:

[0006] Control the probe to rotate around the Z axis of the tool coordinate system in the target direction at the initial position by a preset rotation angle;

[0007] Real-time acquisition of ultrasonic images and force sensor data, and real-time recording and updating of total rotation angle;

[0008] performing image segmentation on the ultrasound image to obtain a lesion location;

[0009] Determine whether the probe collides with the human body according to the force sensor data:

[0010] If yes, calculating a translational obstacle avoidance offset based on the force sensor data and the lesion position, and determining whether the translational obstacle avoidance offset is equal to 0;

[0011] If so, a rotational obstacle avoidance offset is calculated based on the force sensor data, and the probe is controlled to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, and the above-mentioned probe rotation acquisition process is executed again;

[0012] If not, the probe is controlled to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset, and the above-mentioned process of acquiring ultrasonic images and performing collision judgment is executed again;

[0013] Otherwise, the above-mentioned probe rotation acquisition process is executed again until the total rotation angle reaches the preset angle threshold, and it is determined that the probe has completed the autonomous scanning of the thyroid lesion.

[0014] In one or some optional implementations of the embodiment of the present application, controlling the probe to rotate at an initial position about the Z axis of the tool coordinate system toward a target direction by a preset rotation angle includes:

[0015] Acquiring an ultrasound image captured by the probe at an initial position, and obtaining a lesion location in the ultrasound image;

[0016] Obtaining a current probe position and a current probe posture of the probe;

[0017] A first tool coordinate system position offset is calculated based on the lesion position, and a first next probe position is calculated based on the current probe position and the current probe posture;

[0018] A rotation matrix is ​​calculated based on the preset rotation angle, and a first next probe posture is calculated based on the current probe posture;

[0019] According to the first next probe position and the first next probe posture, the probe is controlled to rotate around the Z axis of the tool coordinate system toward the target direction.

[0020] In one or some optional implementations of the embodiment of the present application, the calculating the translational obstacle avoidance offset according to the force sensor data and the lesion position includes:

[0021] determining the direction in which the probe contacts the human body according to the force sensor data, and obtaining an obstacle direction coefficient;

[0022] Calculating a translation obstacle avoidance deviation pixel value according to the obstacle direction coefficient and the lesion position;

[0023] Determine whether the translation obstacle avoidance deviation pixel value is greater than a minimum deviation pixel threshold;

[0024] If yes, the translation obstacle avoidance offset is calculated based on the translation obstacle avoidance deviation pixel value and the force sensor data;

[0025] If not, the translation obstacle avoidance offset is set to 0.

[0026] In one or some optional implementations of the embodiment of the present application, calculating the rotational obstacle avoidance offset according to the force sensor data, and controlling the probe to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, includes:

[0027] determining the direction in which the probe contacts the human body according to the force sensor data, and obtaining an obstacle direction coefficient;

[0028] A second tool coordinate system position offset is calculated based on the obstacle direction coefficient, and a second next probe position is calculated based on the obtained current probe position and current probe posture of the probe;

[0029] Calculating the rotation angle around the X-axis according to the obstacle reverse coefficient;

[0030] The rotation obstacle avoidance offset is calculated according to the rotation angle around the X-axis, and the second next probe posture is calculated in combination with the current probe posture;

[0031] Controlling the probe to rotate around the X-axis according to the second next probe position and the second next probe posture to collect new force sensor data in real time;

[0032] If it can be determined based on the new force sensor data that the probe still collides with the human body, the above process of calculating the second next probe position and the second next probe posture and rotating is performed again until the probe is no longer in a collision state with the human body.

[0033] In one or some optional implementations of the embodiment of the present application, before the control probe rotates at an initial position around the Z axis of the tool coordinate system in a target direction by a preset rotation angle, the method further includes:

[0034] Controlling the probe to acquire an ultrasound image; the ultrasound image includes a thyroid lesion;

[0035] performing image segmentation on the ultrasound image to obtain a lesion location;

[0036] Calculating a pixel distance from the lesion position to the center of the ultrasound image according to the lesion position and the width value of the ultrasound image;

[0037] Calculating an initial adjustment offset according to the pixel distance;

[0038] The probe is controlled to move to the initial position according to the initial adjustment offset.

[0039] In one or some optional implementations of the embodiment of the present application, performing image segmentation on the ultrasound image to obtain the lesion location includes:

[0040] performing image segmentation on the ultrasound image to obtain a lesion contour;

[0041] Perform ellipse fitting on the lesion contour to obtain the center position of the lesion ellipse as the lesion position.

[0042] In a second aspect, an embodiment of the present invention provides a device for autonomously scanning thyroid lesions with an ultrasonic robot, which may include:

[0043] The first rotation module is used to control the probe to rotate around the Z axis of the tool coordinate system in the target direction at the initial position by a preset rotation angle;

[0044] A first segmentation module is used to perform image segmentation on the ultrasound image to obtain the location of the lesion;

[0045] A first judgment module is configured to judge whether the probe collides with the human body based on the force sensor data: if so, execute the translation obstacle avoidance module; if not, execute the second rotation module;

[0046] A translation obstacle avoidance module is used to calculate a translation obstacle avoidance offset based on the force sensor data and the lesion position when the probe collides with the human body, and to determine whether the translation obstacle avoidance offset is equal to 0;

[0047] A rotation obstacle avoidance module is configured to calculate a rotation obstacle avoidance offset based on the force sensor data when the translation obstacle avoidance offset is equal to 0, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and then re-execute the above-mentioned probe rotation acquisition process;

[0048] A first movement module is configured to control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset when the translation obstacle avoidance offset is not equal to 0, and to re-execute the above-mentioned process of acquiring ultrasonic images and performing collision judgment;

[0049] The second rotation module is used to re-execute the above-mentioned probe rotation acquisition process when the probe does not collide with the human body until the total rotation angle reaches a preset angle threshold, thereby determining that the probe has completed the autonomous scanning of the thyroid lesion.

[0050] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the method for autonomously scanning thyroid lesions by an ultrasonic robot as described above is implemented.

[0051] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for autonomously scanning thyroid lesions with an ultrasonic robot as described above.

[0052] In a fifth aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the method of autonomously scanning thyroid lesions by an ultrasonic robot as described above.

[0053] The beneficial effects of the above technical solutions provided by the embodiments of the present invention include at least:

[0054] The embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot. The method controls the probe to rotate around the Z axis of the tool coordinate system at an initial position by a preset angle, collects ultrasonic images and force sensor data in real time, records the total rotation angle, segments the ultrasonic image to determine the lesion location, and determines whether the probe has collided with the human body based on the force sensor data. If a collision is determined, the translation or rotation obstacle avoidance offset is calculated, and the probe position or angle is adjusted accordingly until the collision state is resolved. If no collision occurs, the rotation acquisition is continued until the total rotation angle reaches a preset threshold, completing the autonomous scanning of the thyroid lesion. The method flexibly adjusts the movement of the probe according to the actual lesion location and changes in the human anatomical structure, ensuring that the probe can safely complete the conversion process from transverse to longitudinal cutting. At the same time, by real-time monitoring of force sensor data and dynamically adjusting the position or angle of the probe, collisions between the probe and the human body can be effectively avoided, significantly improving the safety and reliability of the inspection process. It has strong adaptability to individual differences of different patients, ensuring the safety and comfort of the patient. Therefore, the method not only improves the efficiency and comfort of ultrasonic inspection, but also ensures the accuracy of diagnosis, which is of great significance for improving the quality of thyroid disease diagnosis.

[0055] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0056] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0058] Figure 1 A schematic flow chart of a method for autonomously scanning thyroid lesions using an ultrasonic robot according to an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of an ultrasonic robot provided by an embodiment of the present invention;

[0060] Figure 3 The probe status at the beginning and end of the lesion scanning provided by the embodiment of the present invention;

[0061] Figure 4 A schematic diagram of rotating a tool coordinate system Z-axis toward a target direction by a preset rotation angle provided by an embodiment of the present invention;

[0062] Figure 5 Two collision situations are provided for the embodiment of the present invention;

[0063] Figure 6 Two rotation obstacle avoidance solutions provided by the embodiments of the present invention;

[0064] Figure 7 Schematic diagram of the structure of the device for autonomously scanning thyroid lesions with an ultrasonic robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0066] The inventors discovered that existing techniques rely on external cameras to capture the neck contour in real time. By identifying the locations of obstacles such as the clavicle and chin, the angle and position of the probe are adjusted to avoid collisions and ensure the acquisition of the required longitudinal cross-sectional images. This method is time-consuming, relies on external cameras, is costly, and is susceptible to interference from human body differences and ambient light, resulting in poor reliability. Based on this, the inventors conducted further research and development, resulting in the present invention, which provides a method and apparatus for autonomously scanning thyroid lesions using an ultrasonic robot.

[0067] Example 1

[0068] The first embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot, referring to Figure 1 As shown, the method may include the following steps S101-S108:

[0069] S101: Control the probe to rotate around the Z axis of the tool coordinate system in the initial position toward the target direction by a preset rotation angle.

[0070] S102: Acquire ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time.

[0071] S103: Perform image segmentation on the ultrasound image to obtain the location of the lesion.

[0072] S104: Determine whether the probe collides with the human body based on the force sensor data: if so, execute step S105; if not, execute step S108.

[0073] S105: Calculate the translational obstacle avoidance offset based on the force sensor data and the lesion position, and determine whether the translational obstacle avoidance offset is equal to 0: if so, execute step S106; if not, execute step S107.

[0074] S106: Calculate the rotational obstacle avoidance offset based on the force sensor data, and control the probe to rotate around the X-axis of the tool coordinate system based on the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, and re-execute the probe rotation acquisition process described in steps S101-S104.

[0075] S107: Control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset, and re-execute the process of acquiring ultrasonic images and performing collision judgment described in steps S102-S104.

[0076] S108: The probe rotation acquisition process described in steps S101 to S107 is re-executed until the total rotation angle reaches a preset angle threshold, and it is determined that the probe has completed the autonomous scanning of the thyroid lesion.

[0077] The embodiment of the present invention provides a method for autonomously scanning thyroid lesions with an ultrasonic robot. The method controls the probe to rotate around the Z axis of the tool coordinate system at an initial position by a preset angle, collects ultrasonic images and force sensor data in real time, records the total rotation angle, segments the ultrasonic image to determine the lesion location, and determines whether the probe has collided with the human body based on the force sensor data. If a collision is determined, the translation or rotation obstacle avoidance offset is calculated, and the probe position or angle is adjusted accordingly until the collision state is resolved. If no collision occurs, the rotation acquisition is continued until the total rotation angle reaches a preset threshold, completing the autonomous scanning of the thyroid lesion. The method flexibly adjusts the movement of the probe according to the actual lesion location and changes in the human anatomical structure, ensuring that the probe can safely complete the conversion process from transverse to longitudinal cutting. At the same time, by real-time monitoring of force sensor data and dynamically adjusting the position or angle of the probe, collisions between the probe and the human body can be effectively avoided, significantly improving the safety and reliability of the inspection process. It has strong adaptability to individual differences of different patients, ensuring the safety and comfort of the patient. Therefore, the method not only improves the efficiency and comfort of ultrasonic inspection, but also ensures the accuracy of diagnosis, which is of great significance for improving the quality of thyroid disease diagnosis.

[0078] In order to facilitate those skilled in the art to understand this solution, the following first provides an example diagram of the ultrasonic robot and the probe status of the lesion scanning in the embodiment of the present application: Figure 2As shown in the figure, there are two sets of coordinate systems: the world coordinate system at the root of the robotic arm and the tool coordinate system at the end of the robotic arm. Figure 3 As shown in the figure, the left side of the figure defines the initial scanning state, at which the ultrasound probe is in a transverse state (horizontal state). The tool coordinate system at the end of the probe is shown in the figure, with the X-axis pointing to the vertical direction of the probe, the Y-axis pointing to the horizontal direction of the probe, and the Z-axis pointing to the depth direction of the probe. The right side of the figure defines the end of the scan, at which the ultrasound probe is in a longitudinal state (vertical state). The tool coordinate system at the end of the probe is shown in the figure, rotated 90° in the horizontal plane, completing the lesion scan. The scan in the figure is the scan described in this article.

[0079] In an embodiment of the present application, when starting the autonomous scanning of thyroid lesions based on the ultrasonic robot, the probe of the ultrasonic robot will first be scanned in a transverse form on the human neck. Since the probe is in a transverse state, it will not collide with the clavicle or chin when moving in the Y-axis direction in the tool coordinate system. At the same time, during the scanning process, ultrasonic images are collected in real time. When a thyroid lesion appears in the ultrasonic image, step S109 is executed to control the probe to pull the lesion to the center of the ultrasonic image, so that the probe is in the initial position, and then the method of autonomous scanning of thyroid lesions by the ultrasonic robot described in steps S101-S108 is started. Among them, step S109 specifically includes the following steps S1091-S1095:

[0080] S1091: Control the probe to acquire an ultrasound image, which includes a thyroid lesion.

[0081] S1092: Perform image segmentation on the ultrasound image to obtain the lesion location.

[0082] Specifically, the ultrasound image may be segmented to obtain the lesion contour, and then the lesion contour may be fitted with an ellipse to obtain the center position of the lesion ellipse as the lesion position M=(m x ,m y ). Among them, m in the lesion location x 、m y They represent the X-axis pixel coordinate and Y-axis pixel coordinate in the ultrasound image coordinate system respectively.

[0083] S1093: Calculate the pixel distance from the lesion position to the center of the ultrasound image based on the lesion position and the width value of the ultrasound image.

[0084] Specifically, it can be, according to the lesion location M=(m x ,m y ) and the width of the ultrasound image, the pixel distance from the lesion location to the center of the ultrasound image is calculated based on the following formula 1:

[0085]

[0086] Where Δm1 represents the pixel distance from the lesion location to the center of the ultrasound image, m x represents the X-axis pixel coordinate of the lesion position in the ultrasound image coordinate system, and W represents the width of the ultrasound image. W can be set to 800 for example.

[0087] S1094: Calculate an initial adjustment offset based on the pixel distance.

[0088] Specifically, the initial adjustment offset may be calculated based on the following formula 2 according to the pixel distance:

[0089]

[0090] Where p yoffset1 represents the initial adjustment offset, Δm1 represents the pixel distance from the lesion position to the center of the ultrasound image, and k p 、k i and k d They represent the proportional coefficient, integral coefficient, and differential coefficient of a PID controller (Proportional-Integral-Derivative Controller), respectively, and can be set to 0.001, 0.0001, and 0.002, respectively.

[0091] S1095: Control the probe to move to the initial position according to the initial adjustment offset.

[0092] Specifically, the current probe position P of the probe can be obtained. current =[P cx , P cy , P cz ] T and the current probe posture R current Among them, P cx , P cy , P cz They respectively represent the coordinates of the tool coordinate system at the end of the probe in the XYZ directions in the world coordinate system.

[0093] Based on the initial adjustment offset, the current probe position, and the current probe posture, the initial next probe position is calculated using the following formula 3:

[0094]

[0095] Where, P start_next Indicates the initial next probe position, P current Indicates the current probe position, R current Indicates the current probe posture, p yoffset1Indicates the initial adjustment offset.

[0096] In this step, the probe does not need to rotate, so the initial next probe posture of the probe is equal to the current probe posture, that is, R start_next =R current .

[0097] The probe is controlled to move on the Y-axis according to the initial next probe position and the initial next probe posture, so that the probe moves to the initial position.

[0098] In the embodiment of the present application, step S109 controls the ultrasound probe to bring the thyroid lesion to the center of the image. This not only improves the accuracy of lesion positioning and ensures the optimal observation position of the lesion throughout the scanning process, but also reduces external interference caused by probe movement. For example, it avoids distortion in the image edge area and the impact of other structures on lesion observation, thereby significantly improving the accuracy of lesion detection and diagnostic quality. In addition, this process can effectively prevent unnecessary collisions between the probe and the clavicle or chin during movement, ensuring the safety and smoothness of the scanning process.

[0099] In the above step S101, the control probe is rotated at the initial position around the Z axis of the tool coordinate system in the target direction by a preset rotation angle, referring to Figure 4 As shown, the figure shows that the probe rotates clockwise around the Z axis.

[0100] Step S101 specifically includes the following steps S1011-S1015:

[0101] S1011: Acquire an ultrasound image captured by the probe at an initial position, and obtain the location of the lesion in the ultrasound image.

[0102] Specifically, an ultrasound image collected by the probe at the initial position is acquired, and the ultrasound image is segmented to obtain the lesion contour. Then, an ellipse is fitted on the lesion contour to obtain the center position of the lesion ellipse as the lesion position M=(m x , m y ). Among them, m in the lesion location x 、m y They represent the X-axis pixel coordinate and Y-axis pixel coordinate in the ultrasound image coordinate system respectively.

[0103] S1012: Acquire the current probe position and current probe posture of the probe.

[0104] Specifically, the current probe position P of the probe can be obtained. current =[P cx , P cy , P cz ] T and the current probe posture R current Among them, Pcx , P cy , P cz They respectively represent the coordinates of the tool coordinate system at the end of the probe in the XYZ directions in the world coordinate system.

[0105] S1013: Calculate a first tool coordinate system position offset according to the lesion position, and calculate a first next probe position in combination with the current probe position and the current probe posture.

[0106] Specifically, the position offset of the first tool coordinate system may be calculated based on the following formula 4 according to the position of the lesion:

[0107]

[0108] In the formula, toolOffset1 represents the position offset of the first tool coordinate system, m x represents the X-axis pixel coordinate of the lesion position in the ultrasound image coordinate system, K PixelPhyDis is the physical distance of ultrasound image pixels, which refers to the actual distance represented by each pixel in the ultrasound image in the real world, and can be set to 0.00005 for example.

[0109] According to the first tool coordinate system position offset, the current probe position and the current probe posture, the first next probe position is calculated based on the following formula 5:

[0110] P next1 =P current +R current *toolOffset1 Formula 5

[0111] Where, P next1 Indicates the first next probe position, P current Indicates the current probe position, R current Indicates the current probe posture, toolOffset1 indicates the position offset of the first tool coordinate system.

[0112] S1014: Calculate a rotation matrix according to a preset rotation angle, and calculate a first next probe posture in combination with the current probe posture.

[0113] Specifically, the rotation matrix may be calculated based on the following formula 6 according to the preset rotation angle:

[0114]

[0115] Where R z (θ) represents a rotation matrix, and θ represents a preset rotation angle, which can be set to 0.03 radians by way of example.

[0116] Multiply the rotation matrix and the current probe posture, and calculate the first next probe posture based on the following formula 7:

[0117] R next1 =Rc urrent *R z (θ) Formula 7

[0118] Where R next1 Indicates the first next probe posture, R z (θ) represents the rotation matrix, R current Indicates the current probe posture.

[0119] S1015: Control the probe to rotate around the Z axis according to the first next probe position and the first next probe posture.

[0120] In the embodiment of the present application, the above-mentioned step S101 performs a rotational motion with the lesion as the center, ensuring that the lesion is always in the center of the image during the entire rotation process. It not only maintains the clarity and continuity of the lesion and improves the accuracy of lesion feature recognition, but also realizes a comprehensive scanning of the lesion from multiple angles, thereby obtaining richer lesion information and further improving the accuracy and reliability of diagnosis.

[0121] In the above step S102 , ultrasonic images and force sensor data are collected in real time, and the total rotation angle is recorded and updated in real time.

[0122] Specifically, after completing the above step S101, an ultrasonic image and force sensor data F = (fx, fy, fz, tx, ty, tz) are collected, where the force sensor data consists of 6 components, the first 3 components fx, fy, and fz respectively represent the forces acting on the probe in the XYZ directions in the tool coordinate system, and the last three components tx, ty, and tz respectively represent the torque of the probe in the XYZ directions in the tool coordinate system, that is, the rotational force.

[0123] At the same time, the total rotation angle θ needs to be updated in real time 总 , that is, θ 总 =θ 总 +θ.

[0124] In the above step S103, the ultrasound image is segmented to obtain the location of the lesion, which may specifically include the following steps S1031-S1032:

[0125] S1031: Perform image segmentation on the ultrasound image to obtain the lesion contour.

[0126] Specifically, a preset image segmentation network may be used to perform image segmentation on the ultrasound image to obtain the lesion contour.

[0127] Those skilled in the art can select a suitable neural network for pre-training based on the detailed description of the prior art to obtain a preset image segmentation network. The training process may specifically include:

[0128] In the first step, ultrasound images of thyroid lesions are collected and the contours of the lesions in the ultrasound images are marked respectively. After preprocessing, a thyroid lesion contour dataset is obtained.

[0129] The second step is to select a suitable neural network model as the initial image segmentation network, such as the U-Net model, SegNet model, etc.

[0130] In the third step, the thyroid lesion contour dataset is divided into a training set and a test set.

[0131] The fourth step is to define the loss function (such as cross entropy loss function, mean square error loss function, etc.), optimization algorithm (such as Adam, SGD, etc.), etc.

[0132] In the fifth step, the initial image segmentation network is trained using the training set of the thyroid lesion contour dataset to obtain a trained image segmentation model.

[0133] The training process of the image segmentation model is repeated until the preset conditions are met, and then training is stopped to obtain a preset image segmentation network. Exemplary preset conditions may include reaching a fixed number of iterations, accuracy reaching a threshold, and accuracy not changing within a preset number of iterations. These conditions are not specifically limited here.

[0134] S1032: Perform ellipse fitting on the lesion contour to obtain the center position of the lesion ellipse as the lesion position.

[0135] Specifically, the fitEllipse function in the open source computer vision library OpenCV (Open Source Computer Vision Library) can be used to perform ellipse fitting on the lesion contour to obtain the center position of the lesion ellipse as the lesion position M=(m x ,m y ). Among them, m in the lesion location x 、m y They represent the X-axis pixel coordinate and Y-axis pixel coordinate in the ultrasound image coordinate system respectively.

[0136] In the above step S104, it is determined based on the force sensor data whether the probe collides with the human body: if so, step S105 is executed; if not, step S108 is executed.

[0137] Specifically, the force fy in the Y-axis direction of the tool coordinate system from the force sensor data F = (fx, fy, fz, tx, ty, tz) is extracted, and a determination is made as to whether the absolute value of fy is greater than a preset collision threshold ∈. If so, the probe has collided with the clavicle or chin of the person, and step S105 is executed to perform an obstacle avoidance operation. If not, the probe has not collided with the clavicle or chin of the person, and step S108 is executed to continue the probe rotation acquisition process. The preset collision threshold ∈ can be exemplarily set to 2.0, in units of N.

[0138] In the above step S105, the translational obstacle avoidance offset is calculated based on the force sensor data and the lesion position, and it is determined whether the translational obstacle avoidance offset is equal to 0: if so, step S106 is executed. If not, step S107 is executed. Specifically, the following steps S1051-S1055 are included:

[0139] S1051: Determine the direction in which the probe contacts the human body based on the force sensor data, and obtain an obstacle direction coefficient.

[0140] Specifically, the force fy in the Y-axis direction of the tool coordinate system in the force sensor data F = (fx, fy, fz, tx, ty, tz) can be extracted. When fy>=∈, it indicates that the probe collides with the clavicle, and the probe needs to be translated upward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to 1. When fy<=-∈, it indicates that the probe collides with the chin, and the probe needs to be translated downward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to -1.

[0141] In order to facilitate those skilled in the art to understand this solution, the above two collision situations can be referred to Figure 5 As shown, the left side indicates that the probe collides with the clavicle, and the probe needs to be translated along the Y-axis of the tool coordinate system toward the upper part of the human body to avoid the obstacle. The right side indicates that the probe collides with the chin, and the probe needs to be translated along the Y-axis of the tool coordinate system toward the lower part of the human body to avoid the obstacle.

[0142] S1052: Calculate the translation obstacle avoidance deviation pixel value based on the obstacle direction coefficient and the lesion position.

[0143] Specifically, the pixel value of the translation obstacle avoidance deviation can be calculated based on the following formula 8 according to the obstacle direction coefficient and the lesion position:

[0144]

[0145] Where Δm2 represents the pixel value of the translation obstacle avoidance deviation, m x represents the X-axis pixel coordinate of the lesion location in the ultrasound image coordinate system, W represents the width of the ultrasound image, and obstacleDir is the obstacle direction coefficient. W can be set to 800 for example.

[0146] S1053: Determine whether the translation obstacle avoidance deviation pixel value is greater than the minimum deviation pixel threshold: if so, execute step S1054; if not, execute step S1055.

[0147] Specifically, it can be determined whether the absolute value of the translation obstacle avoidance deviation pixel value |Δm2| is greater than the minimum deviation pixel threshold ε. If so, it indicates that the probe position can be translated for obstacle avoidance without losing the lesion, and step S1054 is executed to calculate the translation obstacle avoidance offset. If not, it indicates that obstacle avoidance along the Y-axis of the tool coordinate system is no longer possible, and the lesion may be lost. Step S1055 is executed to set the translation obstacle avoidance offset to 0, and then the rotation obstacle avoidance operation of step S106 is executed. The minimum deviation pixel threshold can be set to 10, for example.

[0148] S1054: Calculate the translation obstacle avoidance offset based on the translation obstacle avoidance deviation pixel value and the force sensor data.

[0149] Specifically, the translation obstacle avoidance offset may be calculated based on the following formula 9 according to the translation obstacle avoidance deviation pixel value and the force exerted on the probe in the Y-axis direction in the tool coordinate system in the force sensor data:

[0150] p yoffset2 =k m *Δm2+k f *fy Formula 9

[0151] Where p yoffset2 is the translation obstacle avoidance offset, Δm2 represents the translation obstacle avoidance deviation pixel value, fy represents the force on the probe in the Y-axis direction in the tool coordinate system, k m is the preset pixel deviation coefficient, k f is the preset force deviation coefficient. Wherein, W can be set to 800, k m An exemplary setting is 0.00002.

[0152] S1055: Set the translation obstacle avoidance offset to 0.

[0153] In step S106, the rotational obstacle avoidance offset is calculated based on the force sensor data, and the probe is controlled to rotate around the X-axis of the tool coordinate system based on the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body. The probe rotation acquisition process described in steps S101-S104 is then re-executed. This specifically includes the following steps S1061-S1066:

[0154] S1061: Determine the direction in which the probe contacts the human body based on the force sensor data, and obtain an obstacle direction coefficient.

[0155] Specifically, the force fy in the Y-axis direction of the tool coordinate system in the force sensor data F = (fx, fyfz, tx, ty, tz) can be extracted. When fy>=∈, it indicates that the probe collides with the clavicle, and the probe needs to be translated upward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to 1. When fy<=-∈, it indicates that the probe collides with the chin, and the probe needs to be translated downward to the human body to avoid obstacles, and the obstacle direction coefficient obstacleDir is set to -1.

[0156] In order to facilitate those skilled in the art to understand this solution, the rotation obstacle avoidance solutions corresponding to the above two collision situations can be referred to Figure 6 As shown, the left side indicates that the probe collides with the clavicle, and the probe needs to rotate clockwise along the X-axis of the tool coordinate system to avoid the obstacle. The left side indicates that the probe collides with the chin, and the probe needs to rotate counterclockwise along the X-axis of the tool coordinate system to avoid the obstacle.

[0157] S1062: Calculate the position offset of the second tool coordinate system according to the obstacle direction coefficient, and calculate the second next probe position by combining the acquired current probe position and current probe posture of the probe.

[0158] Specifically, the position offset of the second tool coordinate system may be calculated based on the following formula 10 according to the obstacle direction coefficient:

[0159]

[0160] Where toolOffset2 represents the position offset of the second tool coordinate system, obstacleDir is the obstacle direction coefficient, and W tool is the end length of the ultrasonic probe, which can be set to 0.04m by way of example.

[0161] According to the position offset of the second tool coordinate system, the current probe position and the current probe posture, the second next probe position is calculated based on the following formula 11:

[0162] P next2 =P current +R current *toolOffset2 Formula 11

[0163] Where, P next2 Indicates the second next probe position, P current Indicates the current probe position, R current Indicates the current probe posture, and toolOffset2 indicates the position offset of the second tool coordinate system.

[0164] S1063: Calculate the rotation angle around the X-axis according to the obstacle inverse coefficient.

[0165] Specifically, the rotation angle around the X-axis may be calculated based on the obstacle inverse coefficient and the following formula 12:

[0166] α=obstacleDir*(α0-k t *tx) Formula 12

[0167] Where α represents the rotation angle around the X axis, obstacleDir is the obstacle direction coefficient, α0 is the preset initial value of the rotation angle, and k t is the torque deviation coefficient, tx is the torque of the probe in the X-axis direction in the tool coordinate system in the force sensor data. Among them, α0 can be set to 0.05 radians, k t For example, it can be set to 0.1.

[0168] S1064: Calculate the rotation obstacle avoidance offset according to the rotation angle around the X-axis, and calculate the second next probe posture in combination with the current probe posture.

[0169] Specifically, the rotation obstacle avoidance offset may be calculated according to the rotation angle around the X-axis based on the following formula 13:

[0170]

[0171] Where R x (α) represents the rotational obstacle avoidance offset, and α represents the rotation angle around the X-axis.

[0172] Multiply the rotational obstacle avoidance offset and the current probe posture, and calculate the second next probe posture based on the following formula 14:

[0173] R next2 =R current *R x (α) Formula 14

[0174] Where R next2 Indicates the second next probe posture, R x (α) represents the rotational obstacle avoidance offset, R current Indicates the current probe posture.

[0175] S1065: Control the probe to rotate around the X-axis according to the second next probe position and the second next probe posture, and collect new force sensor data in real time.

[0176] S1066: If it can be determined based on the new force sensor data that the probe has still collided with the human body, the process of calculating the second next probe position and the second next probe posture and rotating as described in the above steps S1061-S1065 is re-executed until the probe is no longer in a collision state with the human body, and the process of probe rotation and collection as described in steps S101-S104 is re-executed.

[0177] In the embodiment of the present application, the rotation obstacle avoidance operation completed in the above step S106, when the translation obstacle avoidance along the Y-axis is not enough to solve the problem, ensures that the probe safely avoids obstacles without losing the lesion by rotating around the X-axis, so that the method can ensure the safe operation of lesion scanning from multiple angles, improve the stability and clarity of the lesion ultrasound image, and enhance the flexibility and adaptability of the method.

[0178] In the above step S107, the movement of the probe is controlled according to the translation obstacle avoidance offset, and the process of acquiring ultrasound images and performing collision judgment described in steps S102 to S104 is re-executed.

[0179] Specifically, the third next probe position may be calculated based on the following formula 15 according to the translation obstacle avoidance offset and the current probe position:

[0180]

[0181] Where, P next3 Indicates the third next probe position, p yoffset2 Represents the translation obstacle avoidance offset, P current Indicates the current probe position.

[0182] In this step, the probe does not need to rotate, so the third next probe posture of the probe is equal to the current probe posture, that is, R next3 =R current .

[0183] The probe is controlled to move on the Y axis according to the third next probe position and the third next probe posture to complete translation obstacle avoidance, and then the process of acquiring ultrasound images and performing collision judgment described in steps S102-S104 is re-executed.

[0184] In the embodiment of the present application, the translational obstacle avoidance operation performed in steps S105 and S107 above ensures that the probe safely avoids the obstacle without losing the lesion by translating along the Y-axis when the ultrasound probe encounters a collision with the clavicle or chin. This not only avoids human discomfort caused by the collision, but also maintains continuous tracking of the lesion, ensuring the stability and clarity of the lesion image. At the same time, by monitoring the force applied in the Y-axis direction in real time and dynamically adjusting the obstacle avoidance speed according to the force applied, the obstacle avoidance process is made faster and more efficient, improving the robustness and adaptability of the system and ensuring the successful completion of the lesion scanning task.

[0185] In the above step S108, the probe rotation acquisition process of the above steps S101-S107 is re-executed until the total rotation angle reaches the preset angle threshold, and it is determined that the probe has completed the autonomous scanning of the thyroid lesion.

[0186] Specifically, the probe rotation acquisition process of the above steps S101-S107 may be executed cyclically until the total rotation angle reaches a preset angle threshold π / 2, i.e., 90°, and the probe changes from a transverse cutting posture to a longitudinal cutting posture, and then it is determined that the probe has completed the autonomous scanning of the thyroid lesion.

[0187] Example 2

[0188] Based on the same inventive concept, an embodiment of the present invention further provides a device for autonomously scanning thyroid lesions with an ultrasonic robot, the device comprising:

[0189] The first rotation module 101 is used to control the probe to rotate around the Z axis of the tool coordinate system in the target direction at the initial position by a preset rotation angle;

[0190] The first acquisition module 102 is used to acquire ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time;

[0191] A first segmentation module 103 is configured to segment the ultrasound image to obtain a lesion location;

[0192] A first judgment module 104 is configured to determine whether the probe collides with the human body based on the force sensor data: if so, execute the translation obstacle avoidance module; if not, execute the second rotation module;

[0193] The translation obstacle avoidance module 105 is used to calculate a translation obstacle avoidance offset based on the force sensor data and the lesion position when the probe collides with the human body, and determine whether the translation obstacle avoidance offset is equal to 0;

[0194] The rotation obstacle avoidance module 106 is configured to calculate a rotation obstacle avoidance offset based on the force sensor data when the translation obstacle avoidance offset is equal to 0, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and then re-execute the above-mentioned probe rotation acquisition process;

[0195] The first movement module 107 is configured to control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset when the translation obstacle avoidance offset is not equal to 0, and to re-execute the above-mentioned process of acquiring ultrasound images and performing collision judgment;

[0196] The second rotation module 108 is configured to re-execute the above-mentioned probe rotation acquisition process when there is no collision between the probe and the human body, until the total rotation angle reaches a preset angle threshold, thereby determining that the probe has completed the autonomous scanning of the thyroid lesion.

[0197] Example 3

[0198] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in the first embodiment above is implemented.

[0199] Example 4

[0200] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in the first embodiment above.

[0201] Example 5

[0202] Based on the same inventive concept, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, it implements the method of autonomously scanning thyroid lesions with an ultrasonic robot as described in the above embodiment 1.

[0203] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0204] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0205] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0207] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for autonomously scanning thyroid lesions using an ultrasonic robot, characterized in that: include: Control the probe to rotate around the Z axis of the tool coordinate system in the target direction at the initial position by a preset rotation angle; Real-time acquisition of ultrasonic images and force sensor data, and real-time recording and updating of total rotation angle; performing image segmentation on the ultrasound image to obtain a lesion location; Determining whether the probe collides with a human body according to the force sensor data; If yes, calculating a translational obstacle avoidance offset based on the force sensor data and the lesion position, and determining whether the translational obstacle avoidance offset is equal to 0; If so, a rotational obstacle avoidance offset is calculated based on the force sensor data, and the probe is controlled to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, and the above-mentioned probe rotation acquisition process is executed again; If not, the probe is controlled to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset, and the above-mentioned process of acquiring ultrasonic images and performing collision judgment is executed again; Otherwise, the above-mentioned probe rotation acquisition process is executed again until the total rotation angle reaches the preset angle threshold, and it is determined that the probe has completed the autonomous scanning of the thyroid lesion.

2. The method according to claim 1, characterized in that The controlling the probe to rotate at an initial position around the Z axis of the tool coordinate system toward a target direction by a preset rotation angle comprises: Acquiring an ultrasound image captured by the probe at an initial position, and obtaining a lesion location in the ultrasound image; Obtaining a current probe position and a current probe posture of the probe; A first tool coordinate system position offset is calculated based on the lesion position, and a first next probe position is calculated based on the current probe position and the current probe posture; A rotation matrix is ​​calculated based on the preset rotation angle, and a first next probe posture is calculated based on the current probe posture; According to the first next probe position and the first next probe posture, the probe is controlled to rotate around the Z axis of the tool coordinate system toward the target direction.

3. The method according to claim 1, characterized in that The calculating the translation obstacle avoidance offset according to the force sensor data and the lesion position includes: determining the direction in which the probe contacts the human body according to the force sensor data, and obtaining an obstacle direction coefficient; Calculating a translation obstacle avoidance deviation pixel value according to the obstacle direction coefficient and the lesion position; Determine whether the translation obstacle avoidance deviation pixel value is greater than a minimum deviation pixel threshold; If yes, the translation obstacle avoidance offset is calculated based on the translation obstacle avoidance deviation pixel value and the force sensor data; If not, the translation obstacle avoidance offset is set to 0.

4. The method according to claim 1, wherein The calculating of the rotational obstacle avoidance offset according to the force sensor data, and controlling the probe to rotate around the X-axis of the tool coordinate system according to the rotational obstacle avoidance offset until the probe is no longer in a collision state with the human body, includes: determining the direction in which the probe contacts the human body according to the force sensor data, and obtaining an obstacle direction coefficient; A second tool coordinate system position offset is calculated based on the obstacle direction coefficient, and a second next probe position is calculated based on the obtained current probe position and current probe posture of the probe; Calculating the rotation angle around the X-axis according to the obstacle reverse coefficient; The rotation obstacle avoidance offset is calculated according to the rotation angle around the X-axis, and the second next probe posture is calculated in combination with the current probe posture; Controlling the probe to rotate around the X-axis according to the second next probe position and the second next probe posture to collect new force sensor data in real time; If it can be determined based on the new force sensor data that the probe still collides with the human body, the above process of calculating the second next probe position and the second next probe posture and rotating is performed again until the probe is no longer in a collision state with the human body.

5. The method according to claim 1, wherein Before the control probe is rotated at the initial position around the Z axis of the tool coordinate system in the target direction by a preset rotation angle, the method further includes: Controlling the probe to acquire an ultrasound image; the ultrasound image includes a thyroid lesion; performing image segmentation on the ultrasound image to obtain a lesion location; Calculating a pixel distance from the lesion position to the center of the ultrasound image according to the lesion position and the width value of the ultrasound image; Calculating an initial adjustment offset according to the pixel distance; The probe is controlled to move to the initial position according to the initial adjustment offset.

6. The method according to claim 1, characterized in that The performing image segmentation on the ultrasound image to obtain the lesion location includes: performing image segmentation on the ultrasound image to obtain a lesion contour; Perform ellipse fitting on the lesion contour to obtain the center position of the lesion ellipse as the lesion position.

7. A device for autonomously scanning thyroid lesions with an ultrasonic robot, characterized in that: include: The first rotation module is used to control the probe to rotate around the Z axis of the tool coordinate system in the target direction at the initial position by a preset rotation angle; The first acquisition module is used to acquire ultrasonic images and force sensor data in real time, and record and update the total rotation angle in real time; A first segmentation module is used to perform image segmentation on the ultrasound image to obtain the location of the lesion; A first judgment module is configured to judge whether the probe collides with the human body based on the force sensor data: if so, execute the translation obstacle avoidance module; if not, execute the second rotation module; A translation obstacle avoidance module is used to calculate a translation obstacle avoidance offset based on the force sensor data and the lesion position when the probe collides with the human body, and to determine whether the translation obstacle avoidance offset is equal to 0; A rotation obstacle avoidance module is configured to calculate a rotation obstacle avoidance offset based on the force sensor data when the translation obstacle avoidance offset is equal to 0, and control the probe to rotate around the X-axis of the tool coordinate system according to the rotation obstacle avoidance offset until the probe is no longer in a collision state with the human body, and then re-execute the above-mentioned probe rotation acquisition process; A first movement module is configured to control the probe to move along the Y-axis of the tool coordinate system according to the translation obstacle avoidance offset when the translation obstacle avoidance offset is not equal to 0, and to re-execute the above-mentioned process of acquiring ultrasonic images and performing collision judgment; The second rotation module is used to re-execute the above-mentioned probe rotation acquisition process when the probe does not collide with the human body until the total rotation angle reaches a preset angle threshold, thereby determining that the probe has completed the autonomous scanning of the thyroid lesion.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in any one of claims 1 to 6 is implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in any one of claims 1 to 6 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the method for autonomously scanning thyroid lesions with an ultrasonic robot as described in any one of claims 1-6.

Images