Ultrasonic scanning method, system and device and computer equipment

Abstract

The invention relates to an ultrasonic scanning method, system and device and computer equipment. The method comprises the following steps: determining spatial position data of a to-be-scanned area of a target object according to a two-dimensional image of the target object, converting the spatial position data into a three-dimensional space, and determining an initial motion track of a robot; according to the initial motion track, controlling the robot to enable an ultrasonic probe to be in contact with the to-be-scanned area so as to obtain an ultrasonic image of the to-be-scanned area and pressure data between the ultrasonic probe and the to-be-scanned area; and determining target pose data of the robot according to the ultrasonic image, and adjusting the initial motion track according to the target pose data and the pressure data. By adopting the method, the ultrasonic scanning efficiency can be improved, and the ultrasonic image quality can be improved.

Description

Technical Field

[0001] The present application relates to the field of ultrasonic technology, and in particular to an ultrasonic scanning method, system, device and computer equipment. Background Art

[0002] Ultrasound scanning is a skill that is highly dependent on experience and operating skills. The image quality of the same ultrasound equipment often varies greatly in the hands of doctors with different years of experience. Due to the lack of sufficient experience and skilled operating skills, junior doctors are often easily affected by human factors, resulting in low inspection efficiency and unstable image quality. Therefore, how to improve the efficiency of ultrasound scanning and improve the quality of ultrasound images has become a technical problem that needs to be solved. Summary of the invention

[0003] Based on this, it is necessary to provide an ultrasound scanning method, system, device and computer equipment that can improve ultrasound scanning efficiency and improve ultrasound image quality in order to address the above technical problems.

[0004] In a first aspect, the present application provides an ultrasonic scanning method, comprising:

[0005] Determine spatial position data of a to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine an initial motion trajectory of the robot;

[0006] According to the initial motion trajectory, controlling the robot to bring the ultrasonic probe into contact with the area to be scanned, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned;

[0007] The target position data of the robot is determined according to the ultrasonic image, and the initial motion trajectory is adjusted according to the target position data and the pressure data.

[0008] In one embodiment, determining the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object comprises:

[0009] Performing target detection on the two-dimensional image of the target object through a target detection model to obtain a face positioning frame and a human shape positioning frame, and performing vector connection between the face positioning frame and the human shape positioning frame;

[0010] The segmentation model is used to perform body surface segmentation on the area to be scanned in the human-shaped positioning frame to obtain spatial position data of the area to be scanned.

[0011] In one embodiment, converting the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot includes:

[0012] determining an initial two-dimensional scanning path in a two-dimensional space according to the spatial position data;

[0013] Determine the spatial relationship between the camera and the human body according to the human-shaped positioning frame;

[0014] Acquire image data close to the area to be scanned, and map the initial two-dimensional scanning path to the current camera coordinate system according to the spatial relationship to obtain a target two-dimensional scanning path;

[0015] The target two-dimensional scanning path is converted into a point cloud coordinate system to obtain point cloud discrete point data, and the initial motion trajectory of the robot is determined based on the point cloud discrete point data.

[0016] In one embodiment, determining the initial motion trajectory of the robot according to the point cloud discrete point data includes:

[0017] Performing interpolation processing on the discrete point data of the point cloud to obtain densified point cloud data;

[0018] Performing trajectory planning on the densified point cloud data to obtain motion trajectory data;

[0019] The motion trajectory data is converted to the robot base coordinate system through the mapping matrix between the camera coordinate system and the robot base coordinate system to obtain the initial motion trajectory of the robot.

[0020] In one embodiment, determining the target posture data of the robot according to the ultrasonic image comprises:

[0021] The current section data of the ultrasound image is identified by a preset entity, and the current section data is scored to obtain a scoring result; the preset entity is trained according to the action space design and the reward and punishment mechanism of reinforcement learning;

[0022] The target position data of the robot are determined according to the scoring result and the spatial position data of the area to be scanned.

[0023] In one embodiment, adjusting the initial motion trajectory according to the target posture data and the pressure data comprises:

[0024] The initial motion trajectory is adjusted according to the target posture data, and the motion strategy of the robot is optimized according to the pressure data, so that the pressure data between the ultrasonic probe and the area to be scanned remains constant.

[0025] In one embodiment, the method further comprises:

[0026] Performing non-rigid registration on the actual point cloud data of the target object and the point cloud template corresponding to the target object to obtain a homography matrix;

[0027] Convert the scanning trajectory in the point cloud template to the actual point cloud coordinate system according to the homography matrix to obtain actual discrete point data;

[0028] The initial motion trajectory of the robot is determined according to the actual discrete point data.

[0029] In a second aspect, the present application also provides an ultrasonic scanning system, comprising:

[0030] A data acquisition device, used to acquire a two-dimensional image of a target object;

[0031] A robot, wherein the data acquisition device, the force sensor and the probe clamping assembly are sequentially arranged at the end of the robot, and are used to determine the spatial position data of the area to be scanned of the target object according to the two-dimensional image, and convert the spatial position data into three-dimensional space, and determine the initial motion trajectory of the robot; according to the initial motion trajectory, control the probe clamping assembly to clamp the ultrasonic probe, and control the ultrasonic probe to contact the area to be scanned;

[0032] Ultrasonic equipment, including an ultrasonic probe, wherein the ultrasonic probe is used to obtain an ultrasonic image of the area to be scanned;

[0033] A force sensor, used to obtain pressure data between the ultrasonic probe and the area to be scanned;

[0034] The robot is also used to determine target posture data of the robot according to the ultrasonic image, and adjust the initial motion trajectory of the robot according to the target posture data and the pressure data.

[0035] In a third aspect, the present application also provides an ultrasonic scanning device, comprising:

[0036] A determination module, used to determine the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot;

[0037] a control module, configured to control the robot to bring the ultrasonic probe into contact with the area to be scanned according to the initial motion trajectory, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned;

[0038] An adjustment module is used to determine the target posture data of the robot according to the ultrasonic image, and adjust the initial motion trajectory according to the target posture data and the pressure data.

[0039] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0040] Determine spatial position data of a to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine an initial motion trajectory of the robot;

[0041] According to the initial motion trajectory, controlling the robot to bring the ultrasonic probe into contact with the area to be scanned, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned;

[0042] The target position data of the robot is determined according to the ultrasonic image, and the initial motion trajectory is adjusted according to the target position data and the pressure data.

[0043] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0044] Determine spatial position data of a to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine an initial motion trajectory of the robot;

[0045] According to the initial motion trajectory, controlling the robot to bring the ultrasonic probe into contact with the area to be scanned, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned;

[0046] The target position data of the robot is determined according to the ultrasonic image, and the initial motion trajectory is adjusted according to the target position data and the pressure data.

[0047] In a sixth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0048] Determine spatial position data of a to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine an initial motion trajectory of the robot;

[0049] According to the initial motion trajectory, controlling the robot to bring the ultrasonic probe into contact with the area to be scanned, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned;

[0050] The target position data of the robot is determined according to the ultrasonic image, and the initial motion trajectory is adjusted according to the target position data and the pressure data.

[0051] The above-mentioned ultrasonic scanning method, system, device, computer equipment, storage medium and computer program product, by combining robot control, force control technology, ultrasonic image analysis and computer vision algorithm, obtains a two-dimensional image of the area to be scanned through a computer vision algorithm, determines the spatial position data of the area to be scanned according to the two-dimensional image, and converts it into three-dimensional space to determine the initial motion trajectory of the robot, which can optimize the trajectory planning accuracy and is suitable for various complex scenes. The robot is dynamically guided to scan based on the ultrasonic image, and the initial motion path and robot posture are adjusted according to real-time image feedback. Compared with the traditional rule-driven control method, it can provide a more accurate and personalized scanning path. Through intelligent image analysis and trajectory planning, the automation of the ultrasonic scanning process is realized, manual intervention is reduced, the technical threshold is lowered, the imbalance of medical resources is overcome, and the problem of doctor shortage is alleviated. Combined with the dynamic adjustment of force control technology, image quality can be ensured. This dynamic adjustment capability can adapt to different objects, thereby improving the accuracy and stability of ultrasonic scanning. In addition, through intelligent control, this method can greatly improve the inspection efficiency while ensuring image quality, reduce manual operation errors caused by repeated operations or fatigue, thereby significantly increasing the number of inspections that can be processed daily, and can also provide decision support for doctors through real-time analysis of ultrasound images. In addition, this method does not rely on volume data in traditional imaging systems, but makes full use of ultrasound images and their spatial position data for navigation and control. Ultrasound volume data is usually difficult to obtain and is limited to local areas. This application uses ultrasound image analysis to achieve efficient completion of ultrasound inspections in the absence of volume data. It can be flexibly applied in a variety of scenarios, improving operational efficiency and image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of a process of an ultrasonic scanning method in one embodiment;

[0054] Figure 2 A schematic diagram of a process for determining spatial position data of a to-be-scanned area of ​​a target object according to a two-dimensional image of the target object in one embodiment;

[0055] Figure 3 A schematic diagram of a process for converting spatial position data into three-dimensional space and determining an initial motion trajectory of a robot in one embodiment;

[0056] Figure 4 is a schematic diagram of a spatial coordinate system of an ultrasonic scanning system in one embodiment;

[0057] Figure 5 is a schematic diagram of an ultrasonic scanning system in one embodiment;

[0058] Figure 6 is a schematic diagram of a probe clamping assembly in one embodiment;

[0059] Figure 7 is a structural block diagram of an ultrasonic scanning device in one embodiment;

[0060] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] Ultrasound scanning is a skill that is highly dependent on experience and operating skills. The image quality of the same ultrasound equipment often varies greatly in the hands of doctors with different years of experience. Due to the lack of sufficient experience and skilled operating skills, junior doctors are often easily affected by human factors, resulting in low inspection efficiency and unstable image quality.

[0063] In addition, in the related art, a depth camera is used to collect the three-dimensional contour of the tissue to be scanned, and the scanning path is manually planned. Then, a normal vector-based method is used to determine the position of the end of the robotic arm corresponding to each scanning point. After the scanning is completed, three-dimensional reconstruction is performed. However, there are the following problems:

[0064] Low level of intelligence: This solution relies on manual planning of the scanning path and pose adjustment through the normal vector method, lacking intelligent guidance and real-time optimization. This solution is too dependent on human intervention and fails to fully utilize automation and artificial intelligence technologies.

[0065] Complex operation: During the scanning process, depth images need to be captured and subsequently reconstructed, and the scanning path needs to be manually planned. This process is cumbersome and time-consuming, increasing the complexity of the operation and the possibility of errors.

[0066] Not suitable for complex scenarios: This solution is mainly designed for relatively simple spinal scans and cannot effectively cope with more complex or personalized scanning needs (such as the abdomen, chest, etc.). When the scanned object has a complex or irregular shape, it is difficult to ensure the accuracy of path planning and scanning effect.

[0067] Lack of real-time feedback and adjustment capabilities: This solution cannot dynamically adjust the path and position according to actual conditions during the scanning process, resulting in low scanning accuracy and efficiency.

[0068] Poor practicality: Although this solution is effective in specific scenarios (such as spinal scanning), it has weak adaptability to complex scenarios. Therefore, in practical applications, especially in dynamically changing clinical environments, its general applicability and practicality are poor.

[0069] Therefore, to address the above problems, an ultrasonic scanning method is proposed, which achieves efficient, accurate and automated ultrasonic scanning through the combination of robot control, force control technology, ultrasonic image analysis and computer vision algorithm.

[0070] In an exemplary embodiment, Figure 1 As shown, an ultrasonic scanning method is provided, and the method is applied to an ultrasonic scanning system as an example for description, including the following steps 102 to 106. Among them:

[0071] Step 102, determining the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and converting the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot.

[0072] The target object is a user who needs to perform an ultrasonic scan. The robot refers to a robotic arm.

[0073] Optionally, the ultrasonic scanning system includes a data acquisition device, a robot, an ultrasonic device and a force sensor. For example, the data acquisition device is a camera. A two-dimensional image of the target object is collected by the data acquisition device, and the two-dimensional image is transmitted to the robot. The robot performs target detection and body surface segmentation on the two-dimensional image of the target object, and determines the spatial position data of the area to be scanned of the target object. The spatial position data is two-dimensional space data. In order to optimize the trajectory planning accuracy and be applicable to various complex scenes, the spatial position data is converted into three-dimensional space, and the initial motion trajectory of the robot is determined. Three-dimensional space refers to point cloud space, so the determined initial motion trajectory is the trajectory of three-dimensional space.

[0074] Step 104 , according to the initial motion trajectory, the robot is controlled to bring the ultrasonic probe into contact with the area to be scanned, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned.

[0075] The initial motion trajectory is converted into motion instructions that can be executed by the robot. According to the motion instructions, the robot is controlled to contact the ultrasonic probe with the area to be scanned. By contacting the ultrasonic probe with the area to be scanned, the ultrasonic device can generate an ultrasonic image of the area to be scanned in real time and transmit the ultrasonic image to the robot. At the same time, the force sensor can obtain the pressure data between the ultrasonic probe and the area to be scanned, and feed the pressure data back to the robot. Among them, the pressure data refers to the contact force between the ultrasonic probe and the area to be scanned.

[0076] Step 106, determining the target posture data of the robot according to the ultrasonic image, and adjusting the initial motion trajectory according to the target posture data and the pressure data.

[0077] The ultrasound image is processed by section recognition and scoring to determine the robot's target posture data. The target posture data refers to the next action, that is, the next optimal posture. The initial motion trajectory is adjusted in real time according to the target posture data, and the robot's motion strategy is optimized according to the pressure data between the ultrasound probe and the area to be scanned, so that the pressure data between the ultrasound probe and the area to be scanned remains constant, avoiding the degradation of image quality due to unstable force.

[0078] Optionally, the present application can also set the initial position of the robotic arm according to the usage scenario. Specifically, according to the specific usage scenario (such as the target object's body position, scanning area, the relative position of the robotic arm and the ultrasound bed, etc.) and the preset ultrasound scanning task requirements, a flexible action space suitable for the task is calculated. On this basis, the initial position and posture of the robotic arm are further determined to ensure that the robotic arm can move flexibly in the space and that most of the target object's body area is within the camera's field of view, thereby providing a reliable positioning reference for subsequent automatic scanning. This process comprehensively considers the target object's body position, scanning area, and camera field of view to ensure that the robotic arm can accurately reach the appropriate initial position and ensure the smooth progress of the ultrasound scanning task.

[0079] Furthermore, we can calculate how many postures have kinematic inverse solutions for each position in the configuration space of the manipulator, and obtain the arrival posture data of each reachable position. When a position has a posture with an inverse solution, it proves that the position is reachable and is a reachable position. When a position has more postures with inverse solutions, it proves that the flexibility of the position is higher. According to the arrival posture data of each reachable position and the number of reachable positions, the flexibility of the manipulator in each cubic grid is determined, thereby obtaining the flexible action space of the manipulator:

[0080]

[0081] Among them, OAArs represents the flexibility of the robot arm in each cubic grid, R represents the number of reachable points in the reachable sphere, that is, the number of reachable positions, and OAAi represents how many postures each reachable point can be reached in, that is, the reaching posture data of each reachable position.

[0082] Optionally, during the execution of the robot arm, collision detection and emergency stop mechanisms will be introduced to ensure the safety and stability of the scanning process. Specifically, the robot arm is equipped with a collision detection function, which will automatically stop execution when any arm is hit by an external force (the collision threshold can be set). Collision detection can also be performed according to the set flexible action space. If the initial motion trajectory of the robot exceeds this space, when the robot arm executes to this position, it will be considered that a soft collision has occurred and the execution will be stopped immediately. The emergency stop mechanism can include manual emergency stop mode and force feedback emergency stop mode. Manual emergency stop mode: The robot arm is equipped with an emergency stop handle, which can be used to quickly stop all movements of the robot arm by pressing the handle as a manual intervention method. Force feedback emergency stop mode: During the execution of the robot arm, if the pressure data felt at the end of the robot arm exceeds the set pressure threshold, the emergency stop mechanism will be automatically triggered to stop the movement immediately.

[0083] In the above-mentioned ultrasound scanning method, by combining robot control, force control technology, ultrasound image analysis and computer vision algorithm, a two-dimensional image of the area to be scanned is obtained by a computer vision algorithm, the spatial position data of the area to be scanned is determined according to the two-dimensional image, and converted to three-dimensional space to determine the initial motion trajectory of the robot, which can optimize the trajectory planning accuracy and is applicable to various complex scenes. The robot is dynamically guided to scan based on the ultrasound image, and the initial motion path and robot posture are adjusted according to real-time image feedback. Compared with the traditional rule-driven control method, a more accurate and personalized scanning path can be provided. Through intelligent image analysis and trajectory planning, the automation of the ultrasound scanning process is realized, manual intervention is reduced, the technical threshold is lowered, the imbalance of medical resources is overcome, and the problem of doctor shortage is alleviated. Combined with the dynamic adjustment of force control technology, image quality can be ensured. This dynamic adjustment capability can adapt to different objects, thereby improving the accuracy and stability of ultrasound scanning. In addition, through intelligent control, this method can greatly improve the inspection efficiency while ensuring image quality, reduce manual operation errors caused by repeated operations or fatigue, thereby significantly increasing the number of inspections that can be processed per day, and provide decision support for doctors through real-time analysis of ultrasound images. In addition, this method does not rely on volume data in traditional imaging systems, but makes full use of ultrasound images and their spatial position data for navigation and control. Ultrasound volume data is usually difficult to obtain and is limited to local areas. However, this application uses ultrasound image analysis to achieve efficient completion of ultrasound examinations without volume data. It can be flexibly applied in a variety of scenarios, improving operational efficiency and image quality.

[0084] In an exemplary embodiment, Figure 2 As shown, in step 102, determining the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object includes:

[0085] Step 202, target detection is performed on the two-dimensional image of the target object through the target detection model to obtain a face positioning frame and a human shape positioning frame, and the face positioning frame and the human shape positioning frame are vector-connected.

[0086] Step 204 , performing body surface segmentation on the area to be scanned in the human-shaped positioning frame by using the segmentation model to obtain spatial position data of the area to be scanned.

[0087] Optionally, target detection is performed on the human-shaped area and the face area in the two-dimensional image through the target detection model to obtain the face positioning frame and the human-shaped positioning frame. The center point of the face target frame is connected with the center point of the human-shaped positioning frame by vector. The vector pointing to the direction of the face is used as the positive orientation of the human body to provide necessary reference information for subsequent scanning trajectory planning. The human-shaped area is used as the region of interest for the segmentation of body surface organs, and the body surface segmentation is performed on the area to be scanned in the region of interest through the segmentation model to obtain the spatial position data of the area to be scanned of the target object. Among them, the spatial position data is the pixel coordinate information in the two-dimensional camera coordinate system. For example, the target detection model can be a Yolov5 (You Only LookOnce version 5, one-time detection) model. The segmentation model can be a PP-LiteSeg (A Superior Real-Time Semantic Segmentation, ultra-fast semantic segmentation) model.

[0088] In this embodiment, body surface segmentation based on the human-shaped area corresponding to the human-shaped positioning frame can narrow the search range of the segmentation model, eliminate the interference of complex backgrounds other than the human-shaped area, and greatly improve the accuracy and efficiency of segmentation.

[0089] In an exemplary embodiment, Figure 3 As shown, in step 102, the spatial position data is converted into three-dimensional space, and determining the initial motion trajectory of the robot includes:

[0090] Step 302, determining an initial two-dimensional scanning path in two-dimensional space according to the spatial position data.

[0091] Step 304: determine the spatial relationship between the camera and the human body according to the human-shaped positioning frame.

[0092] Step 306, obtain image data close to the area to be scanned, map the initial two-dimensional scanning path to the current camera coordinate system according to the spatial relationship, and obtain the target two-dimensional scanning path.

[0093] Step 308, converting the target two-dimensional scanning path into a point cloud coordinate system to obtain point cloud discrete point data, and determining the initial motion trajectory of the robot based on the point cloud discrete point data.

[0094] Optionally, according to the spatial position data of the area to be scanned in the camera coordinate system, an initial two-dimensional scanning path is determined in the two-dimensional space. The initial two-dimensional scanning path is usually a two-dimensional curve. The path is generated in the image plane as a scanning trajectory under the global field of view. Afterwards, the spatial relationship between the camera and the human body is inferred based on the human positioning frame, and the camera is moved to a position closer to the surface of the human body. At this time, the viewing angle of the camera changes, and the two-dimensional path under the new viewing angle is different from the path under the global field of view. After the camera approaches the area to be scanned, image data near the area to be scanned is obtained. According to the spatial relationship between the camera and the human body, the initial two-dimensional scanning path is mapped to the current camera coordinate system to obtain the target two-dimensional scanning path. The current camera coordinate system is the coordinate system where the image data near the area to be scanned is located. The target two-dimensional scanning path is a more accurate local path. The target two-dimensional scanning path is converted to the point cloud coordinate system to obtain point cloud discrete point data. Trajectory planning is performed on the point cloud discrete point data to obtain the initial motion trajectory of the robot.

[0095] Furthermore, the camera may include two lenses, a two-dimensional camera and a depth camera. By utilizing the spatial position and viewing angle relationship between the two-dimensional camera and the depth camera, the target two-dimensional scanning path is converted from the image plane to the three-dimensional space, that is, the position coordinates in the two-dimensional camera coordinate system are converted to the point cloud coordinate system of the depth camera, thereby obtaining higher quality point cloud discrete point data.

[0096] In this embodiment, a multi-stage positioning scheme of two-dimensional control and three-dimensional space conversion is used to fully utilize the full field of view of the image and the local accuracy of the point cloud. In the coarse positioning stage, a wider field of view is obtained through a two-dimensional image to identify the spatial position data of the area to be scanned. After that, the camera position is adjusted to be close to the area to be scanned, and the two-dimensional path under the global field of view is mapped to the coordinate system under the local field of view to obtain the two-dimensional path under the local field of view. Then, according to the conversion relationship from the two-dimensional camera to the depth camera, the corresponding point cloud path data under the local field of view is obtained. This data is a more accurate target point cloud data, so that an accurate initial motion trajectory of the robot can be generated. This solution can effectively improve the quality of the point cloud, optimize the trajectory planning accuracy, and is suitable for various complex scenes.

[0097] In an optional method of the above embodiment, determining the initial motion trajectory of the robot based on the discrete point data of the point cloud includes: interpolating the discrete point data of the point cloud to obtain dense point cloud data; performing trajectory planning on the dense point cloud data to obtain motion trajectory data; and converting the motion trajectory data to the robot base coordinate system through a mapping matrix between the camera coordinate system and the robot base coordinate system to obtain the initial motion trajectory of the robot.

[0098] Optionally, the obtained discrete point data of the point cloud can be interpolated, for example, cubic spline interpolation, spline interpolation or Kriging interpolation (spatial local interpolation) can be used to fill the gaps in the point cloud and generate dense point cloud data. In this way, a smoother and continuous curve can be obtained, avoiding the phenomenon of sparse or broken data.

[0099] Perform S-shaped trajectory planning on the densified point cloud data to obtain S-shaped motion trajectory data. In the S-shaped trajectory, the velocity and acceleration change rate usually present an S-shaped curve to ensure stability during the motion process and reduce vibration and impact. The acceleration and velocity of the S-shaped trajectory can be expressed by the following formula:

[0100] Speed ​​curve:

[0101] Acceleration curve:

[0102] in, is the speed at time t, is the acceleration at time t, is the maximum speed, is the total exercise time.

[0103] The generated S-shaped trajectory can be further adjusted through the optimized path planning algorithm to ensure the continuity and executableness of the trajectory. Through the mapping matrix between the camera coordinate system and the robot base coordinate system, the motion trajectory data is converted to the robot base coordinate system to obtain the initial motion trajectory of the robot, and the executable motion instructions of the robot are generated according to the initial motion trajectory.

[0104] Optionally, the mapping matrix between the camera coordinate system and the robot base coordinate system can be determined as follows:

[0105] Obtain a schematic diagram of the spatial coordinate system of the ultrasonic scanning system, such as Figure 4 As shown. The robot base coordinate system refers to the robot base coordinate system. The camera is fixed on the robot arm, and a series of coordinate systems and the conversion relationship between coordinate systems can be obtained:

[0106] A: The position and posture of the robot fixture coordinate system in the robot base coordinate system can be obtained through kinematics positive solution.

[0107] B: The pose from the camera coordinate system to the calibration plate coordinate system can be obtained through camera calibration.

[0108] X: The transformation relationship from the camera coordinate system to the robot fixture coordinate system remains unchanged during the movement of the robot.

[0109] BaseToCal: The position and orientation of the calibration plate coordinate system in the robot base coordinate system.

[0110] In practical applications, it is necessary to determine the position and posture of the target object in the robot's base coordinate system through vision to guide the robot's movement, that is, it is necessary to obtain BaseToCal. If X can be calculated, then BaseToCal can be determined.

[0111] To calibrate a certain point on the plate is the anchor point. When the robot is in posture 1, find the point The positions in the robot base coordinate system are:

[0112]

[0113]

[0114]

[0115]

[0116] Similarly, when the robot is in posture 2, there are:

[0117]

[0118] then,

[0119]

[0120]

[0121] This is a very typical After determining X, we can use the mapping matrix The points in the camera coordinate system are transformed into the robot base coordinate system. Thus, any coordinates in the camera coordinate system can be converted into the robot base coordinate system through this mapping matrix.

[0122] In this embodiment, the discrete point data of the point cloud is interpolated to obtain a smoother and continuous trajectory. The S-shaped trajectory planning is performed on the densified point cloud data. Since the velocity and acceleration change rate present an S-shaped curve, the stability of the motion process can be guaranteed and vibration and impact can be reduced. Through the mapping matrix of the camera coordinate system and the robot base coordinate system, the motion trajectory data can be converted to the robot base coordinate system.

[0123] In an exemplary embodiment, determining the target posture data of the robot based on the ultrasound image includes: identifying the current section data of the ultrasound image through a preset entity, scoring the current section data, and obtaining a scoring result; the preset entity is trained according to the reward and punishment mechanism of action space design and reinforcement learning; and determining the target posture data of the robot based on the scoring result and the spatial position data of the area to be scanned.

[0124] Among them, the preset entity is a pre-trained reinforcement learning-based intelligent agent.

[0125] Optionally, during the execution of the robot arm, the ultrasonic probe is controlled to contact the surface of the area to be scanned, the ultrasonic probe generates an ultrasonic image in real time, the ultrasonic image and the current motion data are input into the intelligent agent, the current section data of the ultrasonic image is identified by the intelligent agent, the current section data is scored, and a scoring result is obtained. According to the scoring result, the spatial position data of the area to be scanned and the current motion data, the target posture data of the robot is determined.

[0126] Furthermore, the tissue structure of the ultrasound image is detected through the intelligent agent. If certain tissue structures are correctly identified, the reward will be increased according to the confidence value; if the identification is wrong or the tissue structure is not detected, points will be deducted. The current action data includes the current probe posture, that is, the posture and position: Phi: represents the pitch angle of the ultrasound probe relative to the human body, which describes the angle between the probe and the horizontal plane, usually the angle tilted downward or upward from the horizontal plane. Theta: represents the azimuth angle of the ultrasound probe, that is, the angle of horizontal rotation of the probe relative to the center of the human body (for example, the chest or abdomen), which is the rotation angle of the probe around the longitudinal axis of the body. Radius: represents the distance from the probe to the area to be scanned, that is, the distance between the probe and the area to be scanned.

[0127] In one embodiment, adjusting the initial motion trajectory according to the target posture data and the pressure data includes: adjusting the initial motion trajectory according to the target posture data, and optimizing the motion strategy of the robot according to the pressure data so that the pressure data between the ultrasonic probe and the area to be scanned remains constant.

[0128] The target posture data is used as a correction instruction to compare and adjust the initial motion trajectory in real time. During this process, the initial motion trajectory will be dynamically corrected according to the target posture data provided by the intelligent agent to ensure that the robot arm can accurately adjust the scanning angle and position. At the same time, the movement of the robot arm will be monitored in real time, and the trajectory will be continuously optimized according to the current ultrasound image feedback to ensure that the robot arm can always stay on the ideal scanning path, thereby improving the accuracy and efficiency of ultrasound examinations, and improving the quality of ultrasound images and the degree of section standardization.

[0129] In addition, the constant force adjustment mechanism monitors the pressure data between the probe and the surface of the area to be scanned on the basis of trajectory correction, and further optimizes the movement of the robotic arm through force feedback to ensure constant pressure data and avoid image quality degradation due to unstable force.

[0130] Optionally, the agent is trained based on the action space design and the reward and punishment mechanism of reinforcement learning. During the training process, the action space design based on ultrasound images is used as rewards (positive feedback) and penalties (negative feedback) to guide the agent to learn which actions to take in different states to obtain the maximum long-term reward. Specifically, the agent adjusts its decision-making process through continuous exploration and feedback during the training process, and finally forms a set of optimal strategies.

[0131] The reward and punishment mechanism of reinforcement learning is based on the intelligent scoring design of ultrasound images, namely the action space design, as follows:

[0132] Reward design: When tissue structures that meet the standard section requirements are identified and the spatial relationships of these tissue structures meet anatomical expectations (such as the relative position and morphology of each organ or tissue), the agent will receive a reward. The degree of reward can be quantified based on the accuracy of the standard section, the correctness of the tissue structure segmentation, and the flatness of the section. For example, if the expected standard section and target tissue position appear in the ultrasound image, a positive reward will be given according to the scoring function.

[0133] Penalty design: If abnormal tissue structures that should be included in non-standard sections are detected (such as lesions that do not conform to standard anatomical structures or tissue markers that should not appear), the agent will be penalized. This penalty can be weighted according to the severity of the abnormality. If the segmentation model mistakenly detects tissue that should not appear in the standard section, it will be negatively penalized according to the evaluation results.

[0134] Positive and negative sample reward and punishment design: To further enhance the training effect of the model, the agent's reward mechanism can also be adjusted based on the accuracy of positive and negative samples. The reward for positive samples can increase with the degree of their match with the standard section; negative samples are punished accordingly according to the degree of misjudgment. This enables the agent to better identify and distinguish between correct and incorrect image content, thereby optimizing the section quality during ultrasound scanning.

[0135] Action space design: The design of the action space optimizes the quality of ultrasound images by adjusting three key parameters of the slice. Specifically, the action space contains three dimensions, which control the rotation, position and distance from the center of the volume of the slice, respectively, as detailed below:

[0136] phi: controls the angle of the slice normal vector (between 0 and π). This parameter adjusts the horizontal rotation of the slice, allowing the slice to be rotated along the horizontal axis to obtain different viewing angles.

[0137] theta: controls another angle of the slice normal vector (between π / 4 and 3π / 4). This parameter adjusts the vertical rotation of the slice, allowing the slice to change along the vertical axis, helping to scan different perspectives or depths.

[0138] Radius: The distance between the cut plane and the center of the volume (0 to the maximum size of the volume, usually the maximum value of the x or z axis of the volume). This parameter controls the distance between the cut plane and the center of the volume, helping the agent adjust the position of the cut plane to obtain the best scanning depth.

[0139] Therefore, during training, the action space is a three-dimensional continuous space that represents the three components of the slice normal vector: phi, theta, and radius. The agent adjusts the slice by choosing different actions in this space, thereby affecting the quality of the ultrasound image.

[0140] Specifically, the section normal vector is calculated by using phi, theta, and radius, and the volume data is sliced ​​by the normal vector to obtain a two-dimensional image. The content of the two-dimensional image is analyzed to obtain the section score, which is used as a reward and punishment to train the agent. The agent continuously learns and improves the strategy through the reward and punishment mechanism, and finally generates a set of optimal strategies. Strategy refers to the rules or methods for the agent to choose actions in different states. It is the code of conduct learned during the training process, which is used to guide the agent in how to choose the best posture according to the current state (such as image quality score, location of the area to be scanned, etc.) in actual applications.

[0141] In this embodiment, a reinforcement learning agent is used to perform adaptive analysis based on ultrasound images, dynamically guide the robot to scan, and adjust the scanning path and the robot arm posture according to real-time image feedback. Compared with traditional rule-driven control methods, reinforcement learning can automatically optimize strategies from a large amount of data to provide a more accurate and personalized scanning path. Combined with force control adjustment, it can not only automatically adjust the scanning angle, but also adjust the scanning strategy according to the quality of the ultrasound image. This dynamic adjustment capability can adapt to different target objects, different body shapes and different pathological conditions, thereby improving the accuracy and stability of ultrasound scanning.

[0142] In an exemplary embodiment, the above method also includes: determining the initial motion trajectory of the robot by a non-rigid registration method based on a point cloud template: performing non-rigid registration on the actual point cloud data of the target object and the point cloud template corresponding to the target object to obtain a homography matrix; converting the scanning trajectory in the point cloud template to the actual point cloud coordinate system according to the homography matrix to obtain actual discrete point data; determining the initial motion trajectory of the robot based on the actual discrete point data.

[0143] Optionally, by establishing a universal upper body point cloud template library, and combining the user's age (for example, taking adults as an example, 10 years old as a stage) and gender, the robot's initial motion trajectory is determined. The specific steps are as follows:

[0144] Construction of a universal point cloud template library: Prepare universal point cloud data of the upper body (the area below the head and above the abdomen) in advance, classify them according to different age groups and genders, and form a point cloud template library. The point cloud data in this library can be used as a standard model, covering the physiological differences of different individuals, ensuring that it is suitable for most users.

[0145] Template selection and scanning trajectory preset: According to the age and gender of the target object, select the point cloud template that matches the target object from the template library. According to the area to be scanned (such as liver, spleen, etc.), preset the corresponding scanning trajectory on the universal template. This process can be completed before the target object is in place, which not only improves efficiency, but also ensures that the preset trajectory is consistent with the anatomical structure of the target object.

[0146] Non-rigid registration and trajectory mapping: After the target object is in place, the actual point cloud data of the target object is captured by the camera. The actual point cloud data of the target object is non-rigidly registered with the selected point cloud template to obtain the homography matrix between the actual point cloud data and the point cloud template. Through the homography matrix, the scanning trajectory on the point cloud template is converted to the actual point cloud coordinate system of the target object, thereby generating the actual scanning path, that is, the actual discrete point data, which is used for subsequent scanning path planning and robot control.

[0147] In this embodiment, a non-rigid registration method based on a universal point cloud template is used to register the actual point cloud data of the target object with the point cloud template corresponding to the target object, and generate a personalized scanning trajectory suitable for the template object for a simple scanning task of a specific part. This method is simple and fast to operate, and is particularly suitable for some common or simple ultrasonic examination parts, which can significantly simplify the operation process and improve efficiency.

[0148] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0149] Based on the same inventive concept, the embodiment of the present application also provides an ultrasonic scanning system for implementing the ultrasonic scanning method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more ultrasonic scanning system embodiments provided below can refer to the limitations on the ultrasonic scanning method above, and will not be repeated here.

[0150] In an exemplary embodiment, Figure 5 As shown, an ultrasonic scanning system 500 is provided, the system comprising: a data acquisition device 501, a robot 502, an ultrasonic device 503 and a force sensor, wherein:

[0151] The data acquisition device is used to acquire a two-dimensional image of a target object.

[0152] The robot, at the end of which an image data acquisition device, a force sensor and a probe clamping assembly are sequentially arranged, is used to determine the spatial position data of the area to be scanned of the target object according to the two-dimensional image, and convert the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot; according to the initial motion trajectory, the probe clamping assembly is controlled to clamp the ultrasonic probe, and the ultrasonic probe is controlled to contact the area to be scanned.

[0153] The ultrasonic device includes an ultrasonic probe, which is used to obtain an ultrasonic image of the area to be scanned.

[0154] The force sensor is used to obtain the pressure data between the ultrasound probe and the area to be scanned.

[0155] The robot is also used to determine target posture data of the robot according to the ultrasonic image, and adjust the initial motion trajectory of the robot according to the target posture data and pressure data.

[0156] Optionally, the robot is a mechanical arm, and the end of the mechanical arm is connected in sequence with a data acquisition device, a force sensor and a probe clamping assembly. The probe clamping assembly is used to clamp the ultrasonic probe of the ultrasonic device. The robot is placed on a trolley 504 to facilitate the movement of the robot. For example, the length of the robot is 700 mm, the distance between the trolley and the ultrasonic device can be 600 mm, and the height between the ultrasonic device and the data acquisition device can be 560 mm.

[0157] The data acquisition device, force sensor and probe clamping assembly are connected to the end of the robot arm through the adapter flange. The use of the adapter flange allows multiple sensor components (such as data acquisition device, force sensor, etc.) to be flexibly configured and replaced according to needs. This design simplifies the operation and maintenance process of the equipment and improves the scalability and adaptability of the system.

[0158] The data acquisition device is connected to the end of the mechanical arm through an adapter component, and the adapter component is adapted to the threaded hole at the end of the mechanical arm to connect the camera to the end of the mechanical arm.

[0159] like Figure 6 As shown, the probe clamping assembly is a gripper designed according to ultrasonic probes of different sizes and shapes, and can replace the corresponding ultrasonic probe according to different scanning contents.

[0160] In this embodiment, by integrating key components such as data acquisition devices, force sensors, and ultrasonic probes at the end of the robot, complex external equipment configurations and redundant components are avoided, the hardware design is simpler, the system weight is reduced, and the operational flexibility and convenience are improved.

[0161] The robot is used as the actuator of the ultrasonic scanning system, and combined with hardware components such as image acquisition devices, force sensors, and ultrasonic equipment, through effective system design and the collaborative work between various components, an efficient, accurate, and scalable ultrasonic inspection platform is formed. The platform realizes the automation and intelligence of ultrasonic scanning tasks through the close cooperation of hardware and software.

[0162] The robot acts as an actuator to control the precise movement of the ultrasound probe to ensure complete coverage of the area to be scanned. The data acquisition device is used to capture two-dimensional images in real time to assist in visual analysis and positioning of the area to be scanned. The force sensor monitors the contact force between the ultrasound probe and the surface of the area to be scanned to ensure that the pressure of the ultrasound probe is moderate during operation to prevent discomfort or image quality degradation. The ultrasound machine obtains ultrasound images of the area to be scanned through the ultrasound probe, providing raw data support for image processing and analysis.

[0163] Based on the same inventive concept, the embodiment of the present application also provides an ultrasonic scanning device for implementing the ultrasonic scanning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more ultrasonic scanning device embodiments provided below can refer to the limitations on the ultrasonic scanning method above, and will not be repeated here.

[0164] In an exemplary embodiment, Figure 7 As shown, an ultrasonic scanning device is provided, including: a determination module 702, a control module 704 and an adjustment module 706, wherein:

[0165] The determination module 702 is used to determine the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot.

[0166] The control module 704 is used to control the robot to bring the ultrasonic probe into contact with the area to be scanned according to the initial motion trajectory, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned.

[0167] The adjustment module 706 is used to determine the target posture data of the robot according to the ultrasonic image, and adjust the initial motion trajectory according to the target posture data and the pressure data.

[0168] In an exemplary embodiment, the determination module 702 is used to perform target detection on the two-dimensional image of the target object through a target detection model, obtain a face positioning frame and a human shape positioning frame, and connect the face positioning frame with the human shape positioning frame by vector; perform body surface segmentation on the area to be scanned in the human shape positioning frame through a segmentation model to obtain spatial position data of the area to be scanned.

[0169] In an exemplary embodiment, the determination module 702 is used to determine an initial two-dimensional scanning path in a two-dimensional space based on spatial position data; determine the spatial relationship between the camera and the human body based on a human-shaped positioning frame; obtain image data close to the area to be scanned, and map the initial two-dimensional scanning path to the current camera coordinate system based on the spatial relationship to obtain a target two-dimensional scanning path; convert the target two-dimensional scanning path into a point cloud coordinate system to obtain point cloud discrete point data, and determine the initial motion trajectory of the robot based on the point cloud discrete point data.

[0170] In an exemplary embodiment, the determination module 702 is used to perform interpolation processing on discrete point data of the point cloud to obtain dense point cloud data; perform trajectory planning on the dense point cloud data to obtain motion trajectory data; and convert the motion trajectory data to the robot base coordinate system through a mapping matrix between the camera coordinate system and the robot base coordinate system to obtain the initial motion trajectory of the robot.

[0171] In an exemplary embodiment, the adjustment module 706 is used to identify the current section data of the ultrasound image through a preset entity, score the current section data, and obtain a scoring result; the preset entity is trained according to the reward and punishment mechanism of action space design and reinforcement learning; and the target posture data of the robot is determined according to the scoring result and the spatial position data of the area to be scanned.

[0172] In an exemplary embodiment, the adjustment module 706 is used to adjust the initial motion trajectory according to the target posture data, and optimize the robot's motion strategy according to the pressure data to keep the pressure data between the ultrasound probe and the area to be scanned constant.

[0173] In an exemplary embodiment, the determination module 702 is used to perform non-rigid registration on the actual point cloud data of the target object and the point cloud template corresponding to the target object to obtain a homography matrix; convert the scanning trajectory in the point cloud template to the actual point cloud coordinate system according to the homography matrix to obtain actual discrete point data; and determine the initial motion trajectory of the robot according to the actual discrete point data.

[0174] Each module in the above-mentioned ultrasonic scanning device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0175] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store two-dimensional images of the target object, ultrasonic images of the area to be scanned, pressure data and other data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an ultrasonic scanning method is implemented.

[0176] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0177] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0178] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0179] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0181] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0182] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An ultrasonic scanning method, characterized in that: The method comprises: Determine spatial position data of a to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine an initial motion trajectory of the robot; According to the initial motion trajectory, controlling the robot to bring the ultrasonic probe into contact with the area to be scanned, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned; The target position data of the robot is determined according to the ultrasonic image, and the initial motion trajectory is adjusted according to the target position data and the pressure data.

2. The method according to claim 1, characterized in that Determining the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object comprises: Performing target detection on the two-dimensional image of the target object through a target detection model to obtain a face positioning frame and a human shape positioning frame, and performing vector connection between the face positioning frame and the human shape positioning frame; The segmentation model is used to perform body surface segmentation on the area to be scanned in the human-shaped positioning frame to obtain spatial position data of the area to be scanned.

3. The method according to claim 2, characterized in that The converting the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot includes: determining an initial two-dimensional scanning path in two-dimensional space according to the spatial position data; Determine the spatial relationship between the camera and the human body according to the human-shaped positioning frame; Acquire image data close to the area to be scanned, and map the initial two-dimensional scanning path to the current camera coordinate system according to the spatial relationship to obtain a target two-dimensional scanning path; The target two-dimensional scanning path is converted into a point cloud coordinate system to obtain point cloud discrete point data, and the initial motion trajectory of the robot is determined based on the point cloud discrete point data.

4. The method according to claim 3, characterized in that Determining the initial motion trajectory of the robot according to the point cloud discrete point data includes: Performing interpolation processing on the discrete point data of the point cloud to obtain densified point cloud data; Performing trajectory planning on the densified point cloud data to obtain motion trajectory data; The motion trajectory data is converted to the robot base coordinate system through the mapping matrix between the camera coordinate system and the robot base coordinate system to obtain the initial motion trajectory of the robot.

5. The method according to claim 1, characterized in that Determining the target posture data of the robot according to the ultrasonic image comprises: The current section data of the ultrasound image is identified by a preset entity, and the current section data is scored to obtain a scoring result; the preset entity is trained according to the action space design and the reward and punishment mechanism of reinforcement learning; The target position data of the robot are determined according to the scoring result and the spatial position data of the area to be scanned.

6. The method according to claim 1, characterized in that The adjusting the initial motion trajectory according to the target posture data and the pressure data comprises: The initial motion trajectory is adjusted according to the target posture data, and the motion strategy of the robot is optimized according to the pressure data, so that the pressure data between the ultrasonic probe and the area to be scanned remains constant.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Performing non-rigid registration on the actual point cloud data of the target object and the point cloud template corresponding to the target object to obtain a homography matrix; Convert the scanning trajectory in the point cloud template to the actual point cloud coordinate system according to the homography matrix to obtain actual discrete point data; The initial motion trajectory of the robot is determined according to the actual discrete point data.

8. An ultrasonic scanning system, characterized in that: The system comprises: A data acquisition device, used to acquire a two-dimensional image of a target object; A robot, wherein the data acquisition device, the force sensor and the probe clamping assembly are sequentially arranged at the end of the robot, and are used to determine the spatial position data of the area to be scanned of the target object according to the two-dimensional image, and convert the spatial position data into three-dimensional space, and determine the initial motion trajectory of the robot; according to the initial motion trajectory, control the probe clamping assembly to clamp the ultrasonic probe, and control the ultrasonic probe to contact the area to be scanned; Ultrasonic equipment, including an ultrasonic probe, wherein the ultrasonic probe is used to obtain an ultrasonic image of the area to be scanned; A force sensor, used to obtain pressure data between the ultrasonic probe and the area to be scanned; The robot is also used to determine target posture data of the robot according to the ultrasonic image, and adjust the initial motion trajectory of the robot according to the target posture data and the pressure data.

9. An ultrasonic scanning device, characterized in that: The device comprises: A determination module, used to determine the spatial position data of the to-be-scanned area of ​​the target object according to the two-dimensional image of the target object, and convert the spatial position data into three-dimensional space to determine the initial motion trajectory of the robot; a control module, configured to control the robot to bring the ultrasonic probe into contact with the area to be scanned according to the initial motion trajectory, so as to obtain an ultrasonic image of the area to be scanned and pressure data between the ultrasonic probe and the area to be scanned; An adjustment module is used to determine the target posture data of the robot according to the ultrasonic image, and adjust the initial motion trajectory according to the target posture data and the pressure data.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images