Mechanical arm automatic ultrasonic scanning method based on binocular vision and related product

Abstract

The invention discloses a mechanical arm automatic ultrasonic scanning method based on binocular vision and a related product. The method comprises the steps that first three-dimensional information of a first object in a first coordinate system is obtained, the first three-dimensional information is determined on the basis of a binocular image comprising the first object, the first object comprises a first point, and the first coordinate system is a coordinate system constructed on the basis of a base of the mechanical arm. A first normal vector of the first point is determined based on the first three-dimensional information. Determining a first pose under the first coordinate system based on the first normal vector; the first pose is a pose when the ultrasonic probe emits ultrasonic waves and generates an ultrasonic image based on a reflected signal of the ultrasonic waves, and an ultrasonic scanning surface of the ultrasonic waves passes through the first point. Wherein under the condition that the pose of the ultrasonic probe is the first pose, the included angle between the ultrasonic scanning surface and the first normal vector is smaller than or equal to a first threshold value. The invention further discloses a corresponding product. Through the method provided by the invention, the efficiency of determining the pose of the ultrasonic probe can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a binocular vision-based automatic ultrasonic scanning method for a robotic arm and related products. Background Art

[0002] Ultrasound scanning technology has gained widespread and important applications in numerous fields, including medicine and industrial non-destructive testing, thanks to its significant advantages, including being non-invasive, highly real-time, relatively low-cost, and harmless to humans and objects. When scanning an object with an ultrasound probe, the probe's position is a key factor affecting the quality of the resulting ultrasound image.

[0003] Currently, in most ultrasound scanning operations, the position of the ultrasound probe is primarily determined and controlled manually. Specifically, the operator holds the ultrasound probe and moves and presses it on the surface of the scanned object. Simultaneously, the operator observes the image displayed by the ultrasound probe in real time, combining their own operating experience and visual judgment to adjust the position, angle, and fit of the ultrasound probe to the surface of the scanned object.

[0004] However, this manual method of determining the ultrasound probe's posture has low efficiency. Summary of the Invention

[0005] The present application provides a binocular vision-based automatic ultrasonic scanning method and related products of a robotic arm to improve the efficiency of determining the position and posture of an ultrasonic probe.

[0006] In a first aspect, a method for automatic ultrasonic scanning of a robotic arm based on binocular vision is provided, wherein the method comprises:

[0007] Acquire first three-dimensional information of a first object in a first coordinate system, where the first three-dimensional information is determined based on a binocular image including the first object, the first object includes a first point, and the first coordinate system is a coordinate system constructed based on a base of the robotic arm;

[0008] determining a first normal vector of the first point based on the first three-dimensional information;

[0009] Determine a first pose in the first coordinate system based on the first normal vector; the first pose is a pose of the ultrasound probe when it emits an ultrasonic wave and generates an ultrasonic image based on a reflected signal of the ultrasonic wave, including an ultrasonic scanning plane of the ultrasonic wave passing through the first point;

[0010] When the ultrasound probe is in the first posture, the angle between the ultrasound scanning plane and the first normal vector is less than or equal to a first threshold.

[0011] In combination with any embodiment of the present application, before determining the first pose based on the first normal vector, the binocular vision-based robotic arm automatic ultrasonic scanning method further includes:

[0012] Acquire a first direction, where the first direction is a direction from the first point to a second point in the first object;

[0013] The determining the first pose based on the first normal vector includes:

[0014] Based on the first normal vector and the first direction, the first posture is determined. When the posture of the ultrasound probe is the first posture, the angle between the second normal vector of the ultrasound scanning surface and the first direction is less than or equal to a second threshold.

[0015] In combination with any embodiment of the present application, the first object includes a first tissue, the angle between the first direction and the second direction is less than or equal to a third threshold, and the length of the first tissue in the second direction is the maximum length of the first tissue in all directions.

[0016] In combination with any embodiment of the present application, obtaining the first direction includes:

[0017] determining second three-dimensional information of the first point and third three-dimensional information of the second point based on the first three-dimensional information;

[0018] The first direction is determined based on the second three-dimensional information and the third three-dimensional information.

[0019] In combination with any embodiment of the present application, the first point and the second point are both points on the skin of the first object.

[0020] In combination with any embodiment of the present application, the ultrasound probe is connected to the first structure of the robotic arm;

[0021] The binocular vision-based robotic arm automatic ultrasonic scanning method further includes:

[0022] determining a second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe; wherein the first relative position relationship is the relative position relationship between the first structure and the ultrasound probe, and when the posture of the first structure is the second posture, the posture of the ultrasound probe is the first posture;

[0023] Based on the second posture, the robotic arm is controlled to drive the first structure to move, so that the posture of the first structure is the second posture.

[0024] In combination with any embodiment of the present application, before determining the second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe, the binocular vision-based robotic arm automatic ultrasound scanning method further includes:

[0025] Acquiring a second relative position relationship, where the second relative position relationship is a relative position relationship between the base and the binocular imaging device, and the binocular image is acquired by the binocular imaging device;

[0026] Acquire fourth three-dimensional information of the first object in a second coordinate system, where the second coordinate system is a coordinate system constructed based on the binocular imaging device;

[0027] The first three-dimensional information is determined based on the second relative positional relationship and the fourth three-dimensional information.

[0028] In a second aspect, a binocular vision-based automatic ultrasonic scanning device for a robotic arm is provided. The binocular vision-based automatic ultrasonic scanning device for a robotic arm comprises:

[0029] an acquisition unit, configured to acquire first three-dimensional information of a first object in a first coordinate system, where the first three-dimensional information is determined based on a binocular image including the first object, the first object including a first point, and the first coordinate system is a coordinate system constructed based on a base of the robotic arm;

[0030] a determining unit, configured to determine a first normal vector of the first point based on the first three-dimensional information;

[0031] The determining unit is configured to determine a first pose in the first coordinate system based on the first normal vector; the first pose is a pose of the ultrasound probe when it emits an ultrasonic wave and generates an ultrasonic image based on a reflected signal of the ultrasonic wave, including an ultrasonic scanning plane of the ultrasonic wave passing through the first point;

[0032] When the ultrasound probe is in the first posture, the angle between the ultrasound scanning plane and the first normal vector is less than or equal to a first threshold.

[0033] In combination with any embodiment of the present application, the acquiring unit is further configured to acquire a first direction, where the first direction is a direction from the first point to a second point in the first object;

[0034] The determination unit is also used to determine the first posture based on the first normal vector and the first direction. When the posture of the ultrasound probe is the first posture, the angle between the second normal vector of the ultrasound scanning surface and the first direction is less than or equal to a second threshold.

[0035] In combination with any embodiment of the present application, the first object includes a first tissue, the angle between the first direction and the second direction is less than or equal to a third threshold, and the length of the first tissue in the second direction is the maximum length of the first tissue in all directions.

[0036] In combination with any embodiment of the present application, the acquiring unit is further configured to:

[0037] determining second three-dimensional information of the first point and third three-dimensional information of the second point based on the first three-dimensional information;

[0038] The first direction is determined based on the second three-dimensional information and the third three-dimensional information.

[0039] In combination with any embodiment of the present application, the first point and the second point are both points on the skin of the first object.

[0040] In combination with any embodiment of the present application, the ultrasound probe is connected to the first structure of the robotic arm;

[0041] The determining unit is further configured to:

[0042] determining a second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe; wherein the first relative position relationship is the relative position relationship between the first structure and the ultrasound probe, and when the posture of the first structure is the second posture, the posture of the ultrasound probe is the first posture;

[0043] The binocular vision-based robotic arm automatic ultrasonic scanning device also includes: a control unit for controlling the robotic arm to drive the first structure to move based on the second posture, so that the posture of the first structure is the second posture.

[0044] In combination with any embodiment of the present application, the acquiring unit is further configured to:

[0045] Acquiring a second relative position relationship, where the second relative position relationship is a relative position relationship between the base and the binocular imaging device, and the binocular image is acquired by the binocular imaging device;

[0046] Acquire fourth three-dimensional information of the first object in a second coordinate system, where the second coordinate system is a coordinate system constructed based on the binocular imaging device;

[0047] The determining unit is further configured to determine the first three-dimensional information based on the second relative position relationship and the fourth three-dimensional information.

[0048] In a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

[0049] In a fourth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

[0050] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method as described in the first aspect above and any possible implementation method thereof.

[0051] In a sixth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is enabled to execute the method of the above-mentioned first aspect and any possible implementation thereof.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0053] In an embodiment of the present application, the three-dimensional information of a first object in a first coordinate system is first three-dimensional information, the first object includes a first point, the first three-dimensional information is determined based on a binocular image including the first object, and the first coordinate system is a coordinate system constructed based on the base of the robotic arm. After acquiring the first three-dimensional information, the scanning device determines a first normal vector for the first point based on the first three-dimensional information. Then, based on the first normal vector, a first pose in the first coordinate system is determined. The first pose is the pose of the ultrasonic probe when it emits ultrasonic waves and generates an ultrasonic image based on the reflected signals of the ultrasonic waves, including an ultrasonic scanning plane passing through the first point. Furthermore, when the ultrasonic probe is in the first pose, the angle between the ultrasonic scanning plane and the first normal vector is less than or equal to a first threshold. Thus, when the ultrasonic probe is in the first pose, emitting ultrasonic waves at the first point and generating an ultrasonic image based on the reflected signals of the ultrasonic waves within the ultrasonic scanning plane can improve the quality of the ultrasonic image. In other words, determining the first pose of the ultrasonic probe based on the method provided in the embodiment of the present application can improve the accuracy of the first pose. Moreover, the first position of the ultrasound probe can be determined without human intervention, which is faster and more efficient than manually determining the position of the ultrasound probe during scanning. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0055] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0056] Figure 1 A schematic flow chart of a method for automatic ultrasonic scanning by a robotic arm based on binocular vision provided in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of a scenario for performing ultrasonic scanning based on a robotic arm and a binocular imaging device provided in an embodiment of the present application;

[0058] Figure 3 A schematic diagram of first three-dimensional information and a first normal vector provided in an embodiment of the present application;

[0059] Figure 4 A schematic structural diagram of a binocular vision-based robotic arm automatic ultrasonic scanning device provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0063] Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one (item)" refers to one or more, "a plurality of" refers to two or more, and "at least two (items)" refers to two or three and more.

[0064] As described in the background, ultrasound scanning technology has gained widespread and significant application in numerous fields, including medical and industrial nondestructive testing, thanks to its significant advantages, including its non-invasive nature, high real-time performance, relatively low cost, and lack of radiation damage to humans and objects. When scanning an object with an ultrasound probe, the probe's position is a key factor affecting the quality of the resulting ultrasound image.

[0065] Currently, in most ultrasound scanning operations, the position of the ultrasound probe is primarily determined and controlled manually. Specifically, the operator holds the ultrasound probe and moves and presses it on the surface of the scanned object. Simultaneously, the operator observes the image displayed by the ultrasound probe in real time, combining their own operating experience and visual judgment to adjust the position, angle, and fit of the ultrasound probe to the surface of the scanned object.

[0066] However, this manual method of determining the ultrasound probe's position has many limitations. For example, individual differences in operator experience, skill level, and operating habits can lead to varying degrees of accuracy in controlling the ultrasound probe's position, making it difficult to ensure the consistency and reliability of the scan results. Another example is that during manual operation, operators need to simultaneously balance the movement of the ultrasound probe and the observation and judgment of images. Long hours of work can easily lead to fatigue, which not only reduces scanning efficiency but can also cause deviations in the ultrasound probe's position due to lack of concentration, leading to the omission of critical detection information.

[0067] In summary, manual methods for determining the position of ultrasound probes have shortcomings in terms of accuracy and efficiency, and cannot well adapt to the requirements of modern ultrasound scanning technology for high precision, automation, and standardization. Based on this, the embodiments of the present application provide a method for automated ultrasound scanning using a robotic arm based on binocular vision to improve the accuracy and efficiency of determining the position of the ultrasound probe.

[0068] The embodiment of this application is implemented by a binocular vision-based robotic arm automated ultrasonic scanning device (hereinafter referred to as the scanning device). The scanning device can be any electronic device capable of executing the technical solutions disclosed in the method embodiments of this application. Optionally, the scanning device can be one of the following: a mobile phone, a computer, a tablet computer, or a wearable smart device.

[0069] It should be understood that the method embodiment of the present application can also be implemented by a processor executing computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 1 , Figure 1 A schematic flow chart of a method for automatic ultrasonic scanning by a robotic arm based on binocular vision is provided in an embodiment of the present application.

[0070] 101. Obtain first three-dimensional information of a first object in a first coordinate system, wherein the first three-dimensional information is determined based on a binocular image including the first object, the first object includes a first point, and the first coordinate system is a coordinate system constructed based on a base of a robotic arm.

[0071] In the embodiment of the present application, the first point is any point on the first object. In one possible implementation, the first object is a human body, and the first point is any point on the human skin.

[0072] The first three-dimensional information is three-dimensional information of the first object in a first coordinate system. Based on the first three-dimensional information, the coordinates of the first point in the first coordinate system can be determined, where the first coordinate system is a coordinate system constructed based on the base of the robotic arm. Optionally, the first three-dimensional information is a coordinate system of the base of the robotic arm. Optionally, the first three-dimensional information is a three-dimensional point cloud.

[0073] The first three-dimensional information is determined based on a binocular image, wherein the binocular image includes two images, and both images include the first object. Optionally, the two images in the binocular image are acquired by a binocular imaging device, wherein the binocular imaging device includes two different imaging devices. The binocular image can be obtained by simultaneously photographing the first object using the two different imaging devices. For example, the binocular imaging device includes two different cameras, and the first object is a human body. By simultaneously photographing the human body using the two different cameras, a binocular image including the human body can be obtained.

[0074] The first three-dimensional information includes three-dimensional information of any point in the first object. For example, the first object includes a first point, and the first three-dimensional information includes the three-dimensional information of the first point. It should be understood that the three-dimensional information of any point in the binocular image can be determined based on the binocular image. Optionally, the disparity map of the binocular image is determined based on the displacement difference between matching point pairs in the binocular image. The matching point pair includes two points, each from different binocular images, and both points in the matching point pair correspond to the same object point. For example, the binocular image includes image i1 and image i2, where image i1 includes point p1 and image i2 includes point p2. If both points p1 and p2 correspond to point p3 on human skin, then points p1 and p2 are a matching point pair. The displacement difference between points p1 and p2 is the distance between the position of point p1 in image i1 and the position of point p2 in image i2. Therefore, the three-dimensional information of any point in the first object can be determined based on the binocular image.

[0075] 102. Determine a first normal vector of the first point based on the first three-dimensional information.

[0076] In the embodiments of the present application, the first normal vector is the normal vector of a first point in a first object in a first coordinate system. In one possible implementation, the first object is a human body, and the first point is a point in the human body. Thus, the first normal vector is the normal vector of the first point in the human body. Alternatively, the first point is a point in the skin of the first object, and the first normal vector is the normal vector of the point in the skin of the first object.

[0077] Based on the first three-dimensional information, an equation of the surface to which the first point belongs in the first coordinate system can be determined, and further, a normal vector of the first point can be determined based on the equation. The first three-dimensional information is three-dimensional information in the first coordinate system. For example, the first point is a point on the skin of a first subject, and the first three-dimensional information includes three-dimensional information of the skin of the first subject. Based on the three-dimensional information of the skin, an equation of the skin in the first coordinate system can be determined, and a first normal vector of the first point can be determined.

[0078] 103. Based on the first normal vector, determine the first pose in the first coordinate system.

[0079] In the embodiment of the present application, the first pose is the pose of the ultrasound probe when emitting ultrasound waves and generating an ultrasound image based on the reflected signals of the ultrasound waves, including the ultrasound scanning plane of the ultrasound waves emitted by the ultrasound probe passing through the first point. The ultrasound waves emitted by the ultrasound probe during scanning are within the ultrasound scanning plane. Optionally, the ultrasound scanning plane is a plane, for example, a sector-shaped plane.

[0080] Because when an ultrasound probe is used to scan and obtain an ultrasound image, the probe transmits ultrasound waves toward the object being scanned and generates an ultrasound image based on the reflected signals from the object. Therefore, when the ultrasound scanning plane passes through a first point, the object being scanned by the ultrasound probe includes the first point. In other words, the first pose is the pose of the ultrasound probe when scanning the first point.

[0081] In a possible implementation, the first point is a point on the skin of the first object, and the first pose is a pose when the ultrasound probe is used to scan the first point on the skin of the first object.

[0082] When scanning an object, the smaller the angle between the ultrasonic scanning plane and the object's normal vector, the stronger the energy of the ultrasonic wave reflected from the object within the ultrasonic scanning plane. Consequently, the ultrasonic image generated by the ultrasonic probe based on the reflected signal is richer in information. Therefore, a smaller angle between the ultrasonic scanning plane and the first normal vector enriches the information in the ultrasonic image generated by the ultrasonic probe scanning the first point, thereby improving the quality of the generated ultrasonic image.

[0083] In the embodiment of the present application, a first threshold is used as a basis to determine whether the angle between the ultrasound scanning plane and the first normal vector is large or small. Specifically, if the angle between the ultrasound scanning plane and the first normal vector is less than or equal to the first threshold, it indicates that the angle between the ultrasound scanning plane and the first normal vector is small. Conversely, if the angle between the ultrasound scanning plane and the first normal vector is greater than the first threshold, it indicates that the angle between the ultrasound scanning plane and the first normal vector is large.

[0084] Therefore, in the embodiment of the present application, when the ultrasound probe is in the first posture, the angle between the ultrasound scanning plane and the first normal vector is less than or equal to the first threshold. In this way, when the ultrasound probe is in the first posture, scanning the first point can improve the quality of the generated ultrasound image.

[0085] It should be understood that the relative position relationship between the ultrasonic scanning surface of the ultrasonic probe and the ultrasonic probe is fixed. Therefore, when the angle between the ultrasonic scanning surface and the first normal vector is less than or equal to the first threshold, the first posture can be determined based on the relative position relationship and the first normal vector.

[0086] In an embodiment of the present application, the three-dimensional information of a first object in a first coordinate system is first three-dimensional information, the first object includes a first point, the first three-dimensional information is determined based on a binocular image including the first object, and the first coordinate system is a coordinate system constructed based on the base of the robotic arm. After acquiring the first three-dimensional information, the scanning device determines a first normal vector for the first point based on the first three-dimensional information. Then, based on the first normal vector, a first pose in the first coordinate system is determined. The first pose is the pose of the ultrasonic probe when it emits ultrasonic waves and generates an ultrasonic image based on the reflected signals of the ultrasonic waves, including an ultrasonic scanning plane passing through the first point. Furthermore, when the ultrasonic probe is in the first pose, the angle between the ultrasonic scanning plane and the first normal vector is less than or equal to a first threshold. Thus, when the ultrasonic probe is in the first pose, emitting ultrasonic waves at the first point and generating an ultrasonic image based on the reflected signals of the ultrasonic waves within the ultrasonic scanning plane can improve the quality of the ultrasonic image. In other words, determining the first pose of the ultrasonic probe based on the method provided in the embodiment of the present application can improve the accuracy of the first pose. Moreover, the first position of the ultrasound probe can be determined without human intervention, which is faster and more efficient than manually determining the position of the ultrasound probe during scanning.

[0087] It should be understood that since the first pose is the pose in the first coordinate system, and the first pose is a coordinate system constructed based on the base of the robotic arm, the robotic arm can recognize the first pose. Therefore, the ultrasound probe can be fixed to the robotic arm, and the movement of the robotic arm can be controlled based on the first pose, so that the pose of the ultrasound probe fixed to the robotic arm is the first pose. In this way, automatic ultrasound scanning can be achieved using the robotic arm.

[0088] As an optional implementation, the scanning device further performs the following steps before performing step 102:

[0089] 201. Obtain a first direction, where the first direction is a direction from a first point to a second point in a first object.

[0090] In the embodiment of the present application, the first point and the second point are both points in the first object, and the first point is different from the second point. The first direction is the direction from the first point to the second point. In one possible implementation, the scanning device determines the second three-dimensional information of the first point and the third three-dimensional information of the second point based on the first three-dimensional information. The first direction is determined based on the second three-dimensional information and the third three-dimensional information. For example, based on the second three-dimensional information and the third three-dimensional information, a vector is determined from the first point to the second point. In this case, the direction of the vector is the first direction.

[0091] After acquiring the first direction, the scanning device performs the following steps during step 102:

[0092] 202. Determine a first posture based on the first normal vector and the first direction, wherein, when the posture of the ultrasound probe is the first posture, an angle between a second normal vector of the ultrasound scanning surface and the first direction is less than or equal to a second threshold.

[0093] When the angle between the ultrasound scanning plane and the first normal vector is less than or equal to the first threshold, the position of the ultrasound scanning plane in the first coordinate system is still uncertain. For example, if the angle between the first candidate plane and the first normal vector is less than or equal to the second threshold, the first candidate plane is rotated around the first normal vector while maintaining the angle between the first candidate plane and the first normal vector unchanged to obtain at least one second candidate plane. In this case, the angle between any second candidate plane and the first normal vector is less than or equal to the first threshold. In other words, the first candidate plane and any of the at least one second candidate planes satisfy the constraint that the angle between the first candidate plane and the first normal vector is less than or equal to the second threshold.

[0094] Therefore, in step 202, the scanning device determines the first posture based on the first normal vector and the first direction, so that the angle between the ultrasonic scanning surface and the first normal vector is less than or equal to the first threshold, and the angle between the second normal vector of the ultrasonic scanning surface and the first direction is less than or equal to the second threshold, so that the posture of the ultrasonic scanning surface in the first coordinate system is determined.

[0095] In other words, the number of postures of the ultrasound scanning surface determined based on step 102 is greater than or equal to 1, and accordingly, the number of first postures is greater than or equal to 1. The number of postures of the ultrasound scanning surface determined based on step 202 is 1, and accordingly, the number of first postures is greater than or equal to 1.

[0096] In one possible implementation scenario, the first direction is the direction of movement of an ultrasonic probe when scanning a first object. Specifically, when scanning the first object, the ultrasonic probe first scans a first point, then moves from the first point to a second point, and scans the second point. Because moving from the first point to the second point in the first direction minimizes the distance from the first point to the second point, the scanning device determines the first pose based on the first direction, which facilitates the movement of the ultrasonic probe from the first point to the second point.

[0097] As an optional implementation, the first object includes a first tissue, the angle between the first direction and the second direction is less than or equal to a third threshold, and the length of the first tissue in the second direction is the maximum length of the first tissue in all directions.

[0098] In the embodiment of the present application, the first tissue may be any tissue in the target subject, for example, the first tissue is a blood vessel, another example, the first tissue is an organ, another example, the first tissue is cartilage. Optionally, the first tissue is a kidney.

[0099] Because the ultrasound probe moves along a first direction and can scan during this movement, the scanning direction of the ultrasound probe during its movement is the first direction. If the angle between the first direction and the second direction is small, then the angle between the ultrasound probe's scanning direction and the second direction is small. Accordingly, the ultrasound probe can scan more information in the second direction, thereby enabling the resulting ultrasound image to include more information in the second direction.

[0100] In an embodiment of the present application, the scanning device determines whether the angle between the first direction and the second direction is large or small based on the third threshold value. Specifically, if the angle between the first direction and the second direction is less than or equal to the third threshold value, it means that the angle between the first direction and the second direction is small. Conversely, if the angle between the first direction and the second direction is greater than the third threshold value, it means that the angle between the first direction and the second direction is large. Therefore, when the angle between the first direction and the second direction is less than or equal to the third threshold value, the angle between the scanning direction of the ultrasonic probe and the second direction is small, and accordingly, the ultrasonic probe can scan more information in the second direction.

[0101] Because the length of the first tissue in the second direction is the maximum length of the first tissue in all directions, scanning the first tissue along the second direction provides the most information about the first tissue. Therefore, when the angle between the first and second directions is less than or equal to the third threshold, the amount of information about the first tissue that can be obtained by scanning with the ultrasound probe can be increased.

[0102] In one possible implementation, the second direction is parallel to the long axis of the first tissue. The long axis is parallel to the coronal plane. For example, if the first tissue is a kidney of a first subject, then the long axis of the kidney is parallel to the coronal plane of the first subject. In another possible implementation, the second direction is parallel to the coronal axis of the first tissue.

[0103] In one possible scenario, the first object is a human body, the first tissue is a kidney, the second direction is the long axis of the kidney, and both the first point and the second point are points on the human skin. In this scenario, an ultrasound probe is moved from the first point to the second point, scanning the human body during the movement. This scan can produce an ultrasound image that includes the first tissue. Furthermore, because the angle between the ultrasound probe's movement direction and the second direction is small, information about the first tissue in the resulting ultrasound image can be enhanced.

[0104] As an optional embodiment, the ultrasonic probe is connected to a first structure of the robotic arm. Optionally, the first structure is a flange of the robotic arm, i.e., the ultrasonic probe is connected to the flange of the robotic arm. Optionally, the connection between the ultrasonic probe and the first structure of the robotic arm is a fixed connection, i.e., the relative positional relationship between the ultrasonic probe and the first structure remains unchanged.

[0105] In this embodiment, the scanning device further performs the following steps:

[0106] 301. Determine a second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe.

[0107] In the embodiment of the present application, the first relative position relationship is the relative position relationship between the first structure and the ultrasound probe. Therefore, the second position of the first structure in the first coordinate system can be determined based on the first relative position relationship and the first position of the ultrasound probe in the first coordinate system.

[0108] 302. Based on the second posture, control the robotic arm to drive the first structure to move, so that the posture of the first structure is the second posture.

[0109] Since the ultrasound probe's posture is the first posture when the first structure's posture is the second posture, the scanning device sets the ultrasound probe's posture to the first posture by executing step 302 to set the first structure's posture to the second posture.

[0110] In this embodiment, the robotic arm can only control the structure of the robotic arm and cannot control structures other than the robotic arm. In other words, the robotic arm can control the posture of the robotic arm structure as a control target. However, the robotic arm cannot control the posture of a structure other than the robotic arm. Furthermore, because the first structure is the structure of the robotic arm and the ultrasonic probe is not the structure of the robotic arm, the robotic arm can control the posture of the first structure as a control target, but cannot control the posture of the ultrasonic probe as a control target. Therefore, the scanning device determines a second posture corresponding to the first posture by executing step 301. Then, in step 302, the robotic arm is controlled to drive the movement of the first structure, so that the posture of the first structure is the second posture, thereby adjusting the posture of the ultrasonic probe connected to the first structure to the first posture. This achieves the adjustment of the posture of the ultrasonic probe to the first posture via the robotic arm.

[0111] It should be understood that the structure of the manipulator controlling the manipulator arm can be understood as including a central processing unit (CPU) that can control the position and orientation of the manipulator arm structure. Specifically, the CPU can adjust the position and orientation of the manipulator arm structure in the manipulator arm's base coordinate system by controlling the movement of the manipulator arm. For example, the manipulator arm's CPU can adjust the position and orientation of the manipulator arm's reference structure in the base coordinate system by controlling the movement of each joint of the manipulator arm.

[0112] As an optional embodiment, before executing the step of “determining the second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe”, the scanning device further performs the following steps:

[0113] 401. Obtain a second relative position relationship.

[0114] In the embodiment of the present application, the second relative position relationship is the relative position relationship between the base and the binocular imaging device, and the binocular image is acquired by the binocular imaging device.

[0115] 402. Obtain fourth three-dimensional information of the first object in the second coordinate system.

[0116] In the embodiment of the present application, the second coordinate system is a coordinate system constructed based on the binocular imaging device. Optionally, the second coordinate system is defined as the optical center of any one of the binocular imaging devices. The horizontal axis of the second coordinate system passes through the optical centers of the two imaging devices of the binocular imaging device. The vertical axis of the second coordinate system is parallel to the optical axis of any one of the binocular imaging devices.

[0117] In one possible implementation, fourth three-dimensional information can be determined based on binocular vision technology and binocular images. Optionally, after a binocular image including the first object is captured using a binocular imaging device, fourth three-dimensional information of the first object in a second coordinate system can be determined based on the binocular image. Optionally, the fourth three-dimensional information is a three-dimensional point cloud in the second coordinate system.

[0118] 403. Determine the first three-dimensional information based on the second relative position relationship and the fourth three-dimensional information.

[0119] Since the second coordinate system is a coordinate system constructed based on the binocular imaging device, the first coordinate system is a coordinate system constructed based on the base of the robotic arm, and the relative position relationship between the base and the binocular imaging device is the second relative position relationship, the conversion relationship between the second coordinate system and the first coordinate system can be determined based on the second relative position relationship, and then the fourth three-dimensional information in the second coordinate system can be converted into the first three-dimensional information in the first coordinate system based on the conversion relationship.

[0120] Based on the above-mentioned binocular vision-based automatic ultrasonic scanning method of a robotic arm, the embodiment of the present application also provides a possible implementation scenario. Figure 2 , Figure 2 A schematic diagram of a scenario for performing ultrasonic scanning based on a robotic arm and a binocular imaging device is provided in an embodiment of the present application. Figure 2 As shown, the base of the robotic arm is connected to the terminal, and the first structure of the robotic arm is connected to the ultrasound probe.

[0121] like Figure 2As shown, the first object is a human body, and there are four markers on the back of the first object. Based on the four markers, a first area can be determined from the skin of the first object. For example, the first area can be obtained by connecting the four markers in sequence. The first area includes a first point and a second point. The binocular imaging device provided on the terminal can capture a binocular image including the first area, and then determine the three-dimensional information of the first area in the second coordinate system based on the binocular image (that is, the fourth three-dimensional information mentioned above). Based on the second relative position relationship and the fourth three-dimensional information, the point cloud of the first area in the first coordinate system (that is, the first three-dimensional information mentioned above) can be determined.

[0122] See also Figure 3 , Figure 3 This is a schematic diagram of a first three-dimensional information and a first normal vector provided in an embodiment of the present application. Figure 3 As shown, the first three-dimensional information is a point cloud, which includes a plurality of points. The first normal vector is the normal vector of the first point in the point cloud.

[0123] After obtaining the first three-dimensional information, the scanning device can determine the first pose of the ultrasound probe based on the first three-dimensional information and the first direction. Based on the first relative positional relationship and the first pose, the second pose of the first structure can be determined. The scanning device then controls the robotic arm, using the second pose of the first structure as a control target, to move the first structure to the second pose, thereby returning the ultrasound probe to the first pose. The ultrasound probe can then scan the area of ​​the first object including the first point to generate an ultrasound image. This enables automated scanning with the ultrasound probe.

[0124] Optionally, the terminal is a movable terminal, for example, the terminal has rolling wheels. The terminal includes a display screen and an input component. The display screen can display information, for example, an ultrasound image scanned by an ultrasound probe, and the input component can input commands for controlling the robotic arm, for example, a first direction.

[0125] Optionally, the scanning device is Figure 2 in the terminal.

[0126] In practical applications, when scanning a first object using an ultrasound probe, the content of the ultrasound image obtained varies depending on the ultrasound probe's position. Therefore, to increase the amount of information in the ultrasound image, it is necessary to determine a target position of the ultrasound probe. When the ultrasound probe is in the target position, the ultrasound image obtained by scanning with the ultrasound probe satisfies at least one of the following: the amount of information in the first tissue is greater than or equal to a fourth threshold, and the resolution is greater than or equal to a fifth threshold.

[0127] Optionally, the specific process for determining the target posture is as follows: Using the method described above, the ultrasound probe is controlled to scan the test object in different postures to generate multiple test ultrasound images. Then, based on the information content of the first tissue in the test ultrasound images and the resolution of the test ultrasound images, a reference ultrasound image is selected from the test images. The reference ultrasound image satisfies at least one of the following: the information content of the first tissue is greater than or equal to a fourth threshold, and the resolution is greater than or equal to a fifth threshold. Finally, the probe posture when the reference ultrasound image was acquired is determined as the target posture.

[0128] Since the aforementioned binocular vision-based robotic arm automated ultrasound scanning method utilizes a robotic arm and binocular imaging device to enable automated scanning of the ultrasound probe, after determining the target posture, the ultrasound probe can be automatically scanned based on the aforementioned method and target posture. Specifically, when the ultrasound probe is positioned at the target posture, a target ultrasound image of the first tissue is obtained. Based on the target ultrasound image, a lesion in the first tissue can be identified, as well as a first path passing through the lesion.

[0129] To ensure that the first path through the lesion does not intersect with blood vessels in the first tissue when determining the first path, a contrast agent may be injected into the first object before scanning the first object with an ultrasound probe. Thus, if the target ultrasound image is obtained by scanning with the ultrasound probe within the development time of the contrast agent, the blood vessels in the first tissue are displayed in the target ultrasound image, that is, the blood vessels are highlighted in the target ultrasound image. Conversely, if the target ultrasound image is not obtained by scanning with the ultrasound probe within the development time of the contrast agent, the blood vessels in the first tissue are not displayed in the target ultrasound image, that is, the blood vessels are not highlighted in the target ultrasound image.

[0130] Due to the limitation of the development time of contrast agents, the target ultrasound image will inevitably contain some images that exceed the development time. However, since all puncture ultrasound images are acquired by the ultrasound probe in the target posture, and the body position of the first object usually remains unchanged during the acquisition process (that is, the relative position relationship between the first object and the ultrasound probe remains unchanged), the position of the blood vessels in each target ultrasound image is fixed. Based on this feature, the scanning device can first filter out the images acquired within the development time of the contrast agent from all target ultrasound images as developed images, and the images in the target ultrasound image other than the developed images are called non-developed images, that is, the non-developed images are scanned outside the display time of the contrast agent.

[0131] The imaging location of a blood vessel in the developed image is determined. The developed location is then synchronized with the non-developed image. Specifically, the area in the non-developed image corresponding to the developed location is highlighted. This process allows the blood vessels in the non-developed image to also be highlighted. Based on the highlighted non-developed image and the developed image, the lesion in the first tissue and the first path passing through the lesion are then determined, thereby improving the accuracy of both the lesion and the first path.

[0132] As an optional implementation, since the first path is a path in the target ultrasound image, the first path can be converted to the world coordinate system based on the conversion relationship between the pixel coordinate system of the target ultrasound image and the world coordinate system to obtain the second path. This facilitates determining the lesion of the first object based on the second path.

[0133] Optionally, the conversion relationship between the pixel coordinate system of the target ultrasound image and the world coordinate system is obtained by testing the following process: using an ultrasound probe to scan a test workpiece to obtain a test ultrasound image of the test workpiece, wherein the test workpiece has a first test point, a second test point, and a third test point. The scanned test ultrasound image thus obtained includes a first ultrasound point corresponding to the first test point, a second ultrasound point corresponding to the second test point, and a third ultrasound point corresponding to the third test point. The line connecting the first ultrasound point and the second ultrasound point is parallel to the horizontal axis of the pixel coordinate system of the test ultrasound image. The line connecting the first ultrasound point and the third ultrasound point is parallel to the vertical axis of the pixel coordinate system of the test ultrasound image.

[0134] Determine a first distance between a first ultrasonic point and a second ultrasonic point, determine a second distance between the first ultrasonic point and a third ultrasonic point, determine a third distance between the first test point and the second test point, and determine a fourth distance between the first test point and the third test point. Determine a first ratio of the first distance to the third distance, and determine a second ratio of the second distance to the fourth distance. Determine a conversion relationship between a pixel coordinate system and a world coordinate system of the target ultrasound image based on the first ratio and the second ratio, wherein the conversion relationship includes a first ratio and a second ratio, the first ratio representing the conversion relationship between the abscissa of the pixel coordinate system and the abscissa of the world coordinate system, and the second ratio representing the conversion relationship between the ordinate of the pixel coordinate system and the ordinate of the world coordinate system.

[0135] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0136] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0137] The above describes in detail the method of the embodiment of the present application, and the following provides an apparatus of the embodiment of the present application.

[0138] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a binocular vision-based robotic arm automatic ultrasonic scanning device provided in an embodiment of the present application. The binocular vision-based robotic arm automatic ultrasonic scanning device 1 includes: an acquisition unit 11, a determination unit 12, and optionally, the binocular vision-based robotic arm automatic ultrasonic scanning device 1 also includes a control unit 13.

[0139] an acquiring unit 11, configured to acquire first three-dimensional information of a first object in a first coordinate system, where the first three-dimensional information is determined based on a binocular image including the first object, the first object including a first point, and the first coordinate system is a coordinate system constructed based on a base of the robotic arm;

[0140] a determining unit 12, configured to determine a first normal vector of the first point based on the first three-dimensional information;

[0141] The determining unit 12 is configured to determine a first pose in the first coordinate system based on the first normal vector; the first pose is a pose of the ultrasound probe when it emits an ultrasonic wave and generates an ultrasonic image based on a reflected signal of the ultrasonic wave, including an ultrasonic scanning plane of the ultrasonic wave passing through the first point;

[0142] When the ultrasound probe is in the first posture, the angle between the ultrasound scanning plane and the first normal vector is less than or equal to a first threshold.

[0143] In combination with any embodiment of the present application, the acquiring unit 11 is further configured to acquire a first direction, where the first direction is a direction from the first point to a second point in the first object;

[0144] The determination unit 12 is also used to determine the first posture based on the first normal vector and the first direction. When the posture of the ultrasound probe is the first posture, the angle between the second normal vector of the ultrasound scanning surface and the first direction is less than or equal to a second threshold.

[0145] In combination with any embodiment of the present application, the first object includes a first tissue, the angle between the first direction and the second direction is less than or equal to a third threshold, and the length of the first tissue in the second direction is the maximum length of the first tissue in all directions.

[0146] In combination with any embodiment of the present application, the acquiring unit 11 is further configured to:

[0147] determining second three-dimensional information of the first point and third three-dimensional information of the second point based on the first three-dimensional information;

[0148] The first direction is determined based on the second three-dimensional information and the third three-dimensional information.

[0149] In combination with any embodiment of the present application, the first point and the second point are both points on the skin of the first object.

[0150] In combination with any embodiment of the present application, the ultrasound probe is connected to the first structure of the robotic arm;

[0151] The determining unit 12 is further configured to:

[0152] determining a second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe; wherein the first relative position relationship is the relative position relationship between the first structure and the ultrasound probe, and when the posture of the first structure is the second posture, the posture of the ultrasound probe is the first posture;

[0153] The binocular vision-based robotic arm automatic ultrasonic scanning device further includes: a control unit 13, which is used to control the robotic arm to drive the first structure to move based on the second posture, so that the posture of the first structure is the second posture.

[0154] In combination with any embodiment of the present application, the acquiring unit 11 is further configured to:

[0155] Acquiring a second relative position relationship, where the second relative position relationship is a relative position relationship between the base and the binocular imaging device, and the binocular image is acquired by the binocular imaging device;

[0156] Acquire fourth three-dimensional information of the first object in a second coordinate system, where the second coordinate system is a coordinate system constructed based on the binocular imaging device;

[0157] The determining unit 12 is further configured to determine the first three-dimensional information based on the second relative position relationship and the fourth three-dimensional information.

[0158] In an embodiment of the present application, the three-dimensional information of a first object in a first coordinate system is first three-dimensional information, the first object includes a first point, the first three-dimensional information is determined based on a binocular image including the first object, and the first coordinate system is a coordinate system constructed based on the base of the robotic arm. After acquiring the first three-dimensional information, the scanning device determines a first normal vector for the first point based on the first three-dimensional information. Then, based on the first normal vector, a first pose in the first coordinate system is determined. The first pose is the pose of the ultrasonic probe when it emits ultrasonic waves and generates an ultrasonic image based on the reflected signals of the ultrasonic waves, including an ultrasonic scanning plane passing through the first point. Furthermore, when the ultrasonic probe is in the first pose, the angle between the ultrasonic scanning plane and the first normal vector is less than or equal to a first threshold. Thus, when the ultrasonic probe is in the first pose, emitting ultrasonic waves at the first point and generating an ultrasonic image based on the reflected signals of the ultrasonic waves within the ultrasonic scanning plane can improve the quality of the ultrasonic image. In other words, determining the first pose of the ultrasonic probe based on the method provided in the embodiment of the present application can improve the accuracy of the first pose. Moreover, the first position of the ultrasound probe can be determined without human intervention, which is faster and more efficient than manually determining the position of the ultrasound probe during scanning.

[0159] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0160] Figure 5A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, connection through various interfaces, transmission lines, buses, etc.

[0161] The processor 21 may be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Alternatively, the processor 21 may be a processor group consisting of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Alternatively, the processor may be another type of processor, etc., which is not limited in the present embodiment.

[0162] The memory 22 can be used to store computer program instructions and various computer program codes, including program codes for executing the solution of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0163] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0164] It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. For example, the memory 22 can be used to store the first three-dimensional information of the first object in the first coordinate system obtained through the input device 23, or the memory 22 can also be used to store the first posture determined by the processor 21, etc. The embodiment of the present application does not limit the specific data stored in the memory.

[0165] It is understandable that Figure 5Only a simplified design of an electronic device is shown. In actual applications, the electronic device may further include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the scope of protection of the present application.

[0166] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here. Those skilled in the art will also clearly understand that the descriptions of the various embodiments of this application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, reference can be made to the descriptions of other embodiments.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0170] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0171] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0172] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by a computer program instructing related hardware to perform the processes. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A binocular vision-based robotic arm automatic ultrasonic scanning method, characterized in that: The automatic ultrasonic scanning method of a robotic arm based on binocular vision includes: Acquire first three-dimensional information of a first object in a first coordinate system, where the first three-dimensional information is determined based on a binocular image including the first object, the first object includes a first point, and the first coordinate system is a coordinate system constructed based on a base of the robotic arm; determining a first normal vector of the first point based on the first three-dimensional information; Determine a first pose in the first coordinate system based on the first normal vector; the first pose is a pose of the ultrasound probe when it emits an ultrasonic wave and generates an ultrasonic image based on a reflected signal of the ultrasonic wave, including an ultrasonic scanning plane of the ultrasonic wave passing through the first point; When the ultrasound probe is in the first posture, the angle between the ultrasound scanning plane and the first normal vector is less than or equal to a first threshold.

2. The binocular vision-based automatic ultrasonic scanning method of a robotic arm according to claim 1, characterized in that: Before determining the first pose based on the first normal vector, the binocular vision-based automatic ultrasonic scanning method for a robotic arm further includes: Acquire a first direction, where the first direction is a direction from the first point to a second point in the first object; The determining the first pose based on the first normal vector includes: Based on the first normal vector and the first direction, the first posture is determined. When the posture of the ultrasound probe is the first posture, the angle between the second normal vector of the ultrasound scanning surface and the first direction is less than or equal to a second threshold.

3. The binocular vision-based automatic ultrasonic scanning method of a robotic arm according to claim 2, characterized in that: The first object includes a first tissue, an angle between the first direction and the second direction is less than or equal to a third threshold, and a length of the first tissue in the second direction is a maximum length of the first tissue in all directions.

4. The binocular vision-based automatic ultrasonic scanning method of a robotic arm according to claim 2 or 3, characterized in that: The obtaining of the first direction includes: determining second three-dimensional information of the first point and third three-dimensional information of the second point based on the first three-dimensional information; The first direction is determined based on the second three-dimensional information and the third three-dimensional information.

5. The binocular vision-based automatic ultrasonic scanning method of a robotic arm according to any one of claims 1 to 3, characterized in that: The ultrasonic probe is connected to the first structure of the robotic arm; The binocular vision-based robotic arm automatic ultrasonic scanning method further includes: determining a second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe; wherein the first relative position relationship is the relative position relationship between the first structure and the ultrasound probe, and when the posture of the first structure is the second posture, the posture of the ultrasound probe is the first posture; Based on the second posture, the robotic arm is controlled to drive the first structure to move, so that the posture of the first structure is the second posture.

6. The binocular vision-based automatic ultrasonic scanning method of a robotic arm according to claim 5, characterized in that: Before determining the second posture of the first structure in the first coordinate system based on the first posture and the first relative position relationship of the ultrasound probe, the binocular vision-based robotic arm automatic ultrasound scanning method further includes: Acquiring a second relative position relationship, where the second relative position relationship is a relative position relationship between the base and the binocular imaging device, and the binocular image is acquired by the binocular imaging device; Acquire fourth three-dimensional information of the first object in a second coordinate system, where the second coordinate system is a coordinate system constructed based on the binocular imaging device; The first three-dimensional information is determined based on the second relative positional relationship and the fourth three-dimensional information.

7. A robotic arm automatic ultrasonic scanning device based on binocular vision, characterized in that: The binocular vision-based robotic arm automatic ultrasonic scanning device comprises: an acquisition unit, configured to acquire first three-dimensional information of a first object in a first coordinate system, where the first three-dimensional information is determined based on a binocular image including the first object, the first object including a first point, and the first coordinate system is a coordinate system constructed based on a base of the robotic arm; a determining unit, configured to determine a first normal vector of the first point based on the first three-dimensional information; The determining unit is configured to determine a first pose in the first coordinate system based on the first normal vector; the first pose is a pose of the ultrasound probe when it emits an ultrasonic wave and generates an ultrasonic image based on a reflected signal of the ultrasonic wave, including an ultrasonic scanning plane of the ultrasonic wave passing through the first point; When the ultrasound probe is in the first posture, the angle between the ultrasound scanning plane and the first normal vector is less than or equal to a first threshold.

8. An electronic device, characterized in that: include: A processor and a memory, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program; when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images