Robotic target positioning method, ultrasonic robotic positioning method, and related devices

Abstract

The application provides a robot target positioning method, an ultrasonic robot positioning method and related devices, and the method comprises the following steps: acquiring position information of a target point in an image, wherein the target point is mapped to a support surface of a detection table under a monocular camera visual angle; determining a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image; and controlling a mechanical arm to move along the target path, and determining an actual position of the target object through data obtained by a force sensor. The application determines the target path corresponding to the target point and the center point of the monocular camera in combination with the monocular camera, and controls the mechanical arm to move along the target path, and determines the actual position of the target object through the data of the force sensor in the movement process. The whole process has a small amount of calculation, and the positioning stability of the target object is improved by the monocular camera cooperating with the robot, and the cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a robot target positioning method, an ultrasonic robot positioning method and related devices. BACKGROUND

[0002] It has been a long-term goal for computer vision and robotics to endow robots with the ability to perceive. The prerequisite for a robot to perform a specific task on a target object is to obtain the spatial position of the target object relative to the robot.

[0003] Currently, the spatial position of the target object relative to the robot can only be obtained using a binocular camera or a depth camera. The method using a binocular camera is to estimate the spatial position of the target object based on the distance between the two monocular cameras (referred to as the baseline). The greater the baseline distance, the farther the distance that can be measured. However, the disadvantage of this method is that the configuration and calibration are relatively complex, the depth range and accuracy are limited by the baseline and resolution of the binocular camera, and the visual calculation is very computationally intensive, requiring the use of GPU and FPGA devices for acceleration to output distance information for the entire image in real time. Therefore, under the existing conditions, the computational load is one of the main problems of the binocular camera.

[0004] The method using a depth camera is to measure the distance of the target object relative to the depth camera based on the infrared structured light or Time-of-Flight (ToF) principle by actively emitting light to the target object and receiving the returned light, to position the target object. However, the commonly used depth camera has many problems such as narrow measurement range, large noise, small field of view, susceptible to sunlight interference, and inability to measure transparent materials, resulting in poor positioning stability of the target object.

[0005] In summary, the current methods have the problems of large computational load and poor positioning stability, and the binocular and depth cameras are relatively expensive. SUMMARY

[0006] Therefore, the present application provides a robot target positioning method, an ultrasonic robot positioning method and related devices to solve the problems of large computational load, poor positioning stability and high cost in the prior art. The technical solutions are as follows:

[0007] A robot target positioning method based on a monocular camera, the monocular camera is used to collect images of a target object on a detection table, the monocular camera is relatively fixed to the detection table, the robot includes a base fixed relative to the detection table, a mechanical arm movably connected relative to the base, and a detection probe installed at the free end of the mechanical arm, the detection probe includes a force sensor, the robot target positioning method based on the monocular camera includes:

[0008] acquire position information of a target point in an image, the target point being a point in a support surface of a detection table to which a target object is mapped in a view of a monocular camera;

[0009] determine a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image;

[0010] control the robot arm to move along the target path, and determine an actual position of the target object according to data obtained by the force sensor.

[0011] Optionally, the determining the shortest path between the target point and the center point of the monocular camera according to the position information of the target point in the image comprises:

[0012] convert the position information of the target point in the image into a robot coordinate system to obtain position information of the target point in the robot coordinate system;

[0013] calculate position information of the center point of the monocular camera in the robot coordinate system;

[0014] determine the shortest path between the target point and the center point of the monocular camera according to the position information of the target point in the robot coordinate system and the position information of the center point of the monocular camera in the robot coordinate system.

[0015] Optionally, the converting the position information of the target point in the image into the robot coordinate system to obtain the position information of the target point in the robot coordinate system comprises:

[0016] determine position information of the target point in a camera coordinate system according to the position information of the target point in the image, a distance ratio and a distance between the center point of the monocular camera and the detection table, wherein the distance ratio is a ratio of an actual distance between two points in the support surface of the detection table to a pixel distance between the two points in the image, and the camera coordinate system takes the center point of the monocular camera as a coordinate origin;

[0017] determine the position information of the target point in the robot coordinate system according to the position information of the target point in the camera coordinate system and a homogeneous transformation matrix, wherein the homogeneous transformation matrix is a transformation matrix from the camera coordinate system to the robot coordinate system.

[0018] Optionally, the calculating the position information of the center point of the monocular camera in the robot coordinate system comprises:

[0019] determine the position information of the center point of the monocular camera in the robot coordinate system according to the position information of the center point of the monocular camera in the camera coordinate system and the homogeneous transformation matrix.

[0020] Optionally, the controlling the robot arm to move along the target path and determining the actual position of the target object according to the data obtained by the force sensor comprises:

[0021] In the process that the mechanical arm moves along the target path, it is determined in real time whether the data obtained by the force sensor is greater than a preset threshold, and when the data obtained by the force sensor is greater than the preset threshold, the position of the mechanical arm is determined as the actual position of the target object.

[0022] Optionally, the position information of the target point in the image is acquired, and the position information includes:

[0023] The corresponding relationship between the target point and the target object is received, and the position information is determined in combination with the corresponding relationship.

[0024] A robot target positioning device based on a monocular camera, the monocular camera is used to collect an image of a target object on a detection table, the monocular camera is relatively fixed with the detection table, the robot includes a base fixed relative to the detection table, a mechanical arm movably connected relative to the base, and a detection probe installed at a free end of the mechanical arm, the detection probe includes a force sensor, the robot target positioning device based on the monocular camera includes a position information acquisition module, a target path determination module and an actual position determination module.

[0025] The position information acquisition module is used to acquire position information of a target point in an image, the target point being mapped into a support surface of the detection table under a visual angle of the monocular camera;

[0026] The target path determination module is used to determine a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image;

[0027] The actual position determination module is used to control the mechanical arm to move along the target path, and determine an actual position of the target object through data obtained by the force sensor.

[0028] Optionally, the target path determination module includes a position information conversion module, a position information calculation module and a shortest path determination module.

[0029] The position information conversion module is used to convert the position information of the target point in the image into a robot coordinate system to obtain position information of the target point in the robot coordinate system;

[0030] The position information calculation module is used to calculate position information of the center point of the monocular camera in the robot coordinate system;

[0031] The shortest path determination module is used to determine the shortest path between the target point and the center point of the monocular camera according to the position information of the target point in the robot coordinate system and the position information of the center point of the monocular camera in the robot coordinate system.

[0032] Optionally, the actual position determining module is specifically configured to determine whether the data obtained by the force sensor is greater than a preset threshold in real time during movement of the mechanical arm along the target path, and determine the position of the mechanical arm as the actual position of the target object when the data obtained by the force sensor is greater than the preset threshold.

[0033] An ultrasonic detection device comprises:

[0034] A detection table comprising a support surface for supporting a target object;

[0035] A monocular camera fixed relative to the detection table, the monocular camera being configured to capture images of the target object;

[0036] An ultrasonic robot comprising a base fixed relative to the detection table, a mechanical arm movably connected relative to the base, and an ultrasonic probe mounted at a free end of the mechanical arm, the ultrasonic probe comprising a force sensor; and

[0037] A controller configured to perform each step of the monocular camera-based robot target positioning method according to any one of the preceding embodiments.

[0038] Optionally, the method further comprises:

[0039] A remote communication module connected to the controller, the remote communication module being configured to transmit at least the correspondence between the target point and the target object to the controller.

[0040] Optionally, the method further comprises:

[0041] A support trolley for supporting the ultrasonic robot, the detection table being provided with a positioning portion for positioning the support trolley, the support trolley being fixed in cooperation with the positioning portion, and the base being mounted on the support trolley.

[0042] An ultrasonic robot positioning method, the ultrasonic robot comprising a base fixed relative to a detection table, a mechanical arm movably connected relative to the base, and an ultrasonic probe mounted at a free end of the mechanical arm, the ultrasonic probe comprising a force sensor, the ultrasonic robot positioning method comprising:

[0043] Capturing, by a monocular camera fixed relative to the detection table, an image of a target object on a support surface of the detection table;

[0044] Obtaining position information of a target point, which is mapped to the support surface of the detection table, in the image under a view angle of the monocular camera;

[0045] Determining, according to the position information of the target point in the image, a shortest path between the target point and a center point of the monocular camera as a target path;

[0046] Controlling the mechanical arm to move along the target path, and determining an actual position of the target object by data obtained by the force sensor.

[0047] A positioning device comprises a memory and a processor;

[0048] a memory for storing a program;

[0049] a processor for executing the program to implement the steps of the monocular camera based robot target positioning method of any one of the above or the steps of the ultrasonic robot positioning method of the above.

[0050] A readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the monocular camera based robot target positioning method of any one of the above or the steps of the ultrasonic robot positioning method of the above.

[0051] According to the technical solutions described above, the monocular camera based robot target positioning method provided by the application first acquires the position information of the target point in the image of the target object mapped to the support surface of the detection table under the perspective of the monocular camera, then determines the shortest path between the target point and the center point of the monocular camera as the target path according to the position information of the target point in the image, and finally controls the movement of the mechanical arm along the target path to determine the actual position of the target object through the data obtained by the force sensor. Considering that the target object must be on the line connecting the target point and the center point of the monocular camera, the application determines the target path corresponding to the target point and the center point of the monocular camera in combination with the monocular camera, and then makes the mechanical arm move along the target path, so that the actual position of the target object can be determined through the data obtained by the force sensor. The entire process has a smaller amount of calculation, and the way of combining the monocular camera with the robot can effectively improve the positioning stability of the target object and reduce the cost. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0053] Figure 1 The flowchart of the monocular camera based robot target positioning method provided by the embodiments of the application;

[0054] Figure 2 The position diagram of the center point of the monocular camera, the target object and the target point provided by the embodiments of the application;

[0055] Figure 3 The diagram of the image coordinate system provided by the embodiments of the application;

[0056] Figure 4A schematic diagram showing the relationship between the image coordinate system and the camera coordinate system provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the structure of the robot provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the structure of a robot target positioning device based on a monocular camera provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the structure of an ultrasonic detection device provided in an embodiment of the present application;

[0060] Figure 8 A schematic diagram of the flow of the ultrasonic robot positioning method provided in an embodiment of the present application;

[0061] Figure 9 This is a hardware structure block diagram of the positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] The present application provides a robot target positioning method based on a monocular camera. The robot target positioning method based on a monocular camera is suitable for scenarios where only the position of the target object needs to be positioned, posture estimation is not required, and the robot end is equipped with a force sensor. The monocular camera-based robot target positioning method provided by the present application is introduced in detail through the following embodiments.

[0064] See also Figure 1 , shows a flow chart of a robot target positioning method based on a monocular camera provided in an embodiment of the present application. The robot target positioning method based on a monocular camera may include:

[0065] Step S101: Obtain position information of a target point in an image mapped to a support surface of a detection platform from the perspective of a monocular camera.

[0066] Here, the image in this step refers to the image of the target object on the inspection platform captured by the monocular camera. The monocular camera and the inspection platform are relatively fixed. For example, a camera bracket can be fixed to the inspection platform, and then the monocular camera can be placed on the camera bracket to ensure that the relative position of the monocular camera and the inspection platform remains fixed during the inspection process.

[0067] It should be noted that the above fixing mode of the monocular camera and the detection table is only an example, and other fixing modes can also be used in addition, which are not limited in the present application.

[0068] In some scenarios, the target object on the detection table needs to be detected, and then the actual position of the target object needs to be determined. For example, in the scenario of automatic robot ultrasonic scanning (in which an ultrasonic robot needs to be equipped with a force sensor for force control when drawing a picture), the detection part in the human body (such as the lung) needs to be located first, and then the detection part is scanned according to the located position.

[0069] In the above scenario, when the target object needs to be located, the position information of the target point in the image, which is mapped by the target object in the monocular camera view to the support surface of the detection table, is obtained first.

[0070] Specifically, when the monocular camera and the detection table are relatively fixed, the target object (for example, the target object is a human body) on the detection table can be photographed by the monocular camera.

[0071] It can be understood that in actual detection, the target object generally has a certain height and does not exactly lie in the support surface of the detection table. For example, when a person lies flat on the detection table, the lung has a certain height from the support surface of the detection table. Referring to the position diagram of the center point of the monocular camera, the target object and the target point shown in Figure 2 , it is assumed that the center point of the monocular camera is located at point O, and the target object is located at point A. In the image formed by the monocular camera, the target object is located at point B in the support surface of the detection table. The point B is the "target point in the support surface of the detection table, which is obtained by mapping the target object in the monocular camera view" in this step.

[0072] In this step, the position information of the target point in the image can be obtained. It should be noted that if the position information of the target point in the image is represented by coordinate values, since the image is a two-dimensional image, the position information of the target point in the image refers to the two-dimensional coordinates of the target point in the image coordinate system. Here, the image coordinate system is a coordinate system formed with the center point of the monocular camera as the coordinate origin. For details, refer to the image coordinate system shown in Figure 3 , the point O is the coordinate origin, the W axis is the direction of the width of the image, and the H axis is the direction of the height of the image. In Figure 3 , the position information of the target point in the image can be represented as (w, h).

[0073] In an optional embodiment, the process of this step of "obtaining the position information of the target point in the image mapped to the support surface of the detection platform from the perspective of the monocular camera" may include: receiving the correspondence between the target point and the target object sent by the remote communication module, and determining the position information of the target point in the image mapped to the support surface of the detection platform from the perspective of the monocular camera based on the correspondence.

[0074] Optionally, the remote communication module may specifically be a remote control device.

[0075] Step S102: Determine the shortest path between the target point and the center point of the monocular camera based on the position information of the target point in the image, as the target path.

[0076] As described in the previous steps, the actual position of the target object ( Figure 2 Point A in the figure must be located on the line connecting the target point and the center point of the monocular camera. Therefore, the target path needs to be determined through this step. The target path is Figure 2 OB segment in.

[0077] Optionally, the process of the step of "determining the shortest path between the target point and the center point of the monocular camera based on the position information of the target point in the image" may include:

[0078] Step S1021 : Convert the position information of the target point in the image into the robot coordinate system to obtain the position information of the target point in the robot coordinate system.

[0079] Specifically, the implementation process of this step includes:

[0080] a1. Determine the position information of the target point in the camera coordinate system based on the position information of the target point in the image, the distance ratio, and the distance between the monocular camera and the detection platform.

[0081] Here, the origin of the camera coordinate system and the image coordinate system coincides with the center point of the monocular camera, and the X and Y axes coincide with the W and H axes respectively. The XYZ coordinate system conforms to the right-hand rule. For details, see Figure 4 .

[0082] The above-mentioned distance ratio refers to the ratio of the actual distance between two points on the support surface of the detection platform to the pixel distance between the two points in the image. In this embodiment, the distance ratio is recorded as K, that is, K = actual distance / pixel distance. It can be understood that when the relative position of the monocular camera and the detection platform is fixed and the angle between the imaging plane of the monocular camera and the support surface of the detection platform is fixed, the distance ratio K is a fixed value.

[0083] In an optional embodiment, the process of the step of "determining the position information of the target point in the camera coordinate system according to the position information of the target point in the image, the distance ratio, and the distance between the center point of the monocular camera and the detection table" can include: calculating the coordinate value of the target point in the X-axis direction of the camera coordinate system by the formula x=K*w; calculating the coordinate value of the target point in the Y-axis direction of the camera coordinate system by the formula y=K*h; and calculating the coordinate value of the target point in the Z-axis direction of the camera coordinate system by the formula z=h or z=h*cosθ.

[0084] wherein if the imaging plane of the monocular camera is parallel to the support surface of the detection table, the coordinate value of the target point in the Z-axis direction of the camera coordinate system is calculated by the formula z=h; and if the angle between the imaging plane of the monocular camera and the support surface of the detection table is θ, the coordinate value of the target point in the Z-axis direction of the camera coordinate system is calculated by the formula z=h*cosθ.

[0085] In the above formula, w refers to the coordinate value of the target point in the W-axis direction of the image coordinate system, x refers to the coordinate value of the target point in the X-axis direction of the camera coordinate system, h refers to the coordinate value of the target point in the H-axis direction of the image coordinate system, y refers to the coordinate value of the target point in the Y-axis direction of the camera coordinate system, z refers to the coordinate value of the target point in the Z-axis direction of the camera coordinate system, and h refers to the distance between the center point of the monocular camera and the detection table.

[0086] a2, determining the position information of the target point in the robot coordinate system according to the position information of the target point in the camera coordinate system and the homogeneous transformation matrix.

[0087] wherein the homogeneous transformation matrix is the transformation matrix from the camera coordinate system to the robot coordinate system.

[0088] Specifically, the position information of the target point in the camera coordinate system is denoted as c P B the homogeneous transformation matrix is denoted as the position information of the target point in the robot coordinate system is denoted as r P B and the step can determine the position information of the target point in the robot coordinate system by the formula (notably, the vectors are calculated by using the normalized homogeneous coordinates).

[0089] Step S1022, calculating the position information of the center point of the monocular camera in the robot coordinate system.

[0090] Similar to the process of a2, the step can determine the position information of the center point of the monocular camera in the robot coordinate system according to the position information of the center point of the monocular camera in the camera coordinate system and the homogeneous transformation matrix.

[0091] Specifically, the position information of the center point of the monocular camera in the camera coordinate system is denoted as c P O The position information of the center point of the monocular camera in the robot coordinate system is denoted as r P O The position information of the center point of the monocular camera in the robot coordinate system can be determined by the formula

[0092] It is worth noting that, since the center point of the monocular camera is located at the coordinate origin of the camera coordinate system, it has c P O = (0, 0, 0).

[0093] Step S1023, according to the position information of the target point in the robot coordinate system and the position information of the center point of the monocular camera in the robot coordinate system, determine the shortest path between the target point and the center point of the monocular camera.

[0094] Step S103, control the robot arm to move along the target path, and determine the actual position of the target object through the data obtained by the force sensor.

[0095] In this embodiment, as shown in Figure 5 , the robot includes a base fixed relative to the detection table, a robot arm movably connected relative to the base, and a detection probe mounted on the free end of the robot arm. Here, the detection probe includes a force sensor.

[0096] In this embodiment, the robot path can be planned by the upper computer, that is, the upper computer can control the robot arm (i.e. the robot) to move along the target path, and in the process of moving the robot arm, it is determined whether the robot arm has contacted the target object through the data obtained by the detection force sensor, if so, the position of the robot arm is taken as the actual position of the target object.

[0097] Here, the upper computer can determine the position of the robot arm because the robot has the function of real-time returning its real-time position to the upper computer, and each joint of the robot is equipped with an encoder, which can record the position of each joint in real time, and the real-time position in the robot coordinate system can be obtained by solving the joint position.

[0098] This step uses the method of robot path planning and force detection, which can make up for the lack of information in the depth direction of the monocular camera, so as to achieve the purpose of three-dimensional positioning.

[0099] ​In an optional embodiment, the process of the step of "controlling the robot arm to move along the target path, and determining the actual position of the target object based on the data obtained by the force sensor" can include: determining whether the data obtained by the force sensor is greater than a preset threshold in real time during the movement of the robot arm along the target path, and determining the position of the robot arm as the actual position of the target object when the data obtained by the force sensor is greater than the preset threshold.

[0100] It can be understood that the greater the data obtained by the force sensor, the closer the robot arm is to the target object, and vice versa. Thus, a threshold can be preset for the step, and then whether the data obtained by the force sensor is greater than the preset threshold is determined in real time during the movement of the robot arm along the target path, and the position of the robot arm is determined as the actual position of the target object when the data obtained by the force sensor is greater than the preset threshold.

[0101] It should be noted that the above-mentioned preset threshold is not limited in the embodiment, and can be determined according to actual conditions, for example, the above-mentioned preset threshold can be determined according to experience.

[0102] The robot target positioning method based on a monocular camera provided in the application first obtains the position information of a target point in an image, which is mapped from a target object in a view of a monocular camera to a support surface of a detection table, then determines a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image, and finally controls a robot arm to move along the target path and determines an actual position of the target object based on data obtained by a force sensor. Considering that the target object must be on a line connecting the target point and the center point of the monocular camera, the application determines a target path corresponding to the target point and the center point of the monocular camera in combination with the monocular camera, and then controls the robot arm to move along the target path, so that the actual position of the target object can be determined based on the data obtained by the force sensor. The entire process has a small amount of calculation, and the monocular camera in cooperation with the robot can effectively improve the positioning stability of the target object and reduce the cost.

[0103] The application also provides a robot target positioning device based on a monocular camera. Here, the monocular camera is configured to collect an image of a target object on a detection table, the monocular camera is fixed relative to the detection table, the robot includes a base fixed relative to the detection table, a robot arm movably connected relative to the base, and a detection probe mounted on a free end of the robot arm, and the detection probe includes a force sensor.

[0104] The robot target positioning device based on a monocular camera provided in the embodiments of the application is described below, and the robot target positioning device based on a monocular camera described below can be mutually referred to the robot target positioning method based on a monocular camera described above.

[0105] Please refer to Figure 6 , a structure schematic diagram of a robot target positioning device based on a monocular camera provided by an embodiment of the present application is shown, as Figure 6 shown, the robot target positioning device based on the monocular camera can include a position information acquisition module 601, a target path determination module 602, and an actual position determination module 603.

[0106] The position information acquisition module 601 is configured to acquire position information of a target point in an image, where the target point is mapped to a support surface of a detection table under a view angle of a monocular camera.

[0107] The target path determination module 602 is configured to determine a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image.

[0108] The actual position determination module 603 is configured to control a robot arm to move along the target path, and determine an actual position of a target object according to data obtained by a force sensor.

[0109] The robot target positioning device based on the monocular camera provided by the present application first acquires, by the position information acquisition module, position information of a target point in an image, where the target point is mapped to a support surface of a detection table under a view angle of a monocular camera, then determines, by the target path determination module, a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image, and finally controls, by the actual position determination module, a robot arm to move along the target path, and determines an actual position of a target object according to data obtained by a force sensor. Considering that the target object must be on a line connecting the target point and the center point of the monocular camera, the present application determines a target path corresponding to the target point and the center point of the monocular camera in combination with the monocular camera, and then controls the robot arm to move along the target path, so that the actual position of the target object can be determined according to the data obtained by the force sensor. The entire process has a small amount of calculation, and the monocular camera in cooperation with the robot can effectively improve the positioning stability of the target object and reduce the cost.

[0110] In a possible implementation, the target path determination module 602 can include a position information conversion module, a position information calculation module, and a shortest path determination module.

[0111] The position information conversion module is configured to convert the position information of the target point in the image to a robot coordinate system to obtain position information of the target point in the robot coordinate system.

[0112] The position information calculation module is configured to calculate position information of the center point of the monocular camera in the robot coordinate system.

[0113] The shortest path determination module is configured to determine a shortest path between the target point and the center point of the monocular camera according to position information of the target point in the robot coordinate system and position information of the center point of the monocular camera in the robot coordinate system.

[0114] In a possible implementation, the position information conversion module can include a first position information determination module and a second position information determination module.

[0115] The first position information determination module is configured to determine position information of the target point in the camera coordinate system according to position information of the target point in the image, a distance ratio, and a distance between the center point of the monocular camera and the detection table, where the distance ratio refers to a ratio of an actual distance between two points in a support surface of the detection table to a pixel distance of the two points in the image, and the camera coordinate system takes the center point of the monocular camera as a coordinate origin.

[0116] The second position information determination module is configured to determine position information of the target point in the robot coordinate system according to position information of the target point in the camera coordinate system and a homogeneous conversion matrix, where the homogeneous conversion matrix is a conversion matrix from the camera coordinate system to the robot coordinate system.

[0117] In a possible implementation, the position information calculation module can be specifically configured to determine position information of the center point of the monocular camera in the robot coordinate system according to position information of the center point of the monocular camera in the camera coordinate system and the homogeneous conversion matrix.

[0118] In a possible implementation, the actual position determination module can be specifically configured to determine, in a process in which the mechanical arm moves along the target path, whether data obtained by the force sensor is greater than a preset threshold value, and determine a position of the mechanical arm as the actual position of the target object when the data obtained by the force sensor is greater than the preset threshold value.

[0119] On the basis of the foregoing embodiments, the embodiment of the present application further provides an ultrasonic detection device, which will be introduced below.

[0120] Referring to Figure 7 Fig. 1, the ultrasonic detection device provided by the embodiment of the present application can at least include a detection table 701, a monocular camera 702 fixed relative to the detection table, an ultrasonic robot 703, and a controller 704 (the controller 704 is not shown). Figure 7

[0121] Specifically, the detection table 701 includes a support surface for supporting a target object, for example, in Figure 7 In the embodiment, the target object is a human body.

[0122] ​The monocular camera 702 shown above collects images of the target object. Optionally, in the embodiment, the monocular camera can be fixed on a camera support, for example, Figure 7 The monocular camera 702 shown above is fixed on the camera support, and the camera support is fixed on the detection table 701, so that the monocular camera 702 and the detection table 701 are relatively fixed.

[0123] The ultrasonic robot 703 shown above includes a base fixed relative to the detection table, a mechanical arm movably connected relative to the base, and an ultrasonic probe installed at the free end of the mechanical arm, and the ultrasonic probe includes a force sensor. Here, the structure of the ultrasonic robot 703 is the same as that of the robot shown in Figure 5 The structure of the robot shown above is the same as that of the robot shown in Figure 5 The detection probe in the above embodiment is specifically an ultrasonic probe.

[0124] The controller 704 shown above is used to execute each step of the monocular camera-based robot target positioning method described above. For the introduction of each step of the monocular camera-based robot target positioning method, please refer to the foregoing embodiment, which will not be described here again.

[0125] Optionally, referring to Figure 7 The ultrasonic detection device provided by the embodiment of the present application can further include a support trolley 705 supporting the ultrasonic robot.

[0126] In the embodiment, the detection table 701 is provided with a positioning portion 7011 positioning the support trolley 705, and the support trolley 705 is fixed in cooperation with the positioning portion 7011, as shown in Figure 7 The base in the ultrasonic robot 703 is installed on the support trolley 705.

[0127] Optionally, the ultrasonic detection device provided by the embodiment of the present application can further include a remote communication module connected to the controller, and the remote communication module at least transmits the correspondence between the target point and the target object to the controller, so that the controller determines the position information of the target point in the image in the target object mapped to the detection table support surface under the monocular camera view in combination with the correspondence.

[0128] It should be noted that the ultrasonic detection device provided by the embodiment of the present application includes various components, which are only examples, and can include other components, which are not limited by the present application.

[0129] Corresponding to the ultrasonic detection device in the foregoing embodiment, the embodiment of the present application further provides an ultrasonic robot positioning method, and the ultrasonic robot includes a base fixed relative to the detection table, a mechanical arm movably connected relative to the base, and an ultrasonic probe installed at the free end of the mechanical arm, and the ultrasonic probe includes a force sensor, as shown in Figure 8 The ultrasonic robot positioning method provided by the embodiment of the present application includes:

[0130] Step S801, acquiring an image of the target object on the support surface of the detection platform by using a monocular camera fixed relative to the detection platform.

[0131] Step S802, obtaining position information of a target point in the image, where the target point is mapped to the support surface of the detection platform under the perspective of the monocular camera.

[0132] Step S803, determining a shortest path between the target point and a center point of the monocular camera as a target path according to the position information of the target point in the image.

[0133] Step S804, controlling the robot arm to move along the target path, and determining an actual position of the target object by using data obtained by the force sensor.

[0134] It should be noted that the ultrasonic robot positioning method provided in the embodiments of the present application corresponds to the robot target positioning method based on the monocular camera introduced in the foregoing embodiments, and details can be referred to the introduction in the foregoing embodiments, which will not be described herein.

[0135] The embodiments of the present application also provide a positioning device. Optionally, Figure 9 A hardware structure block diagram of the positioning device is shown, which is described with reference to Figure 9 The hardware structure of the positioning device can include at least one processor 901, at least one communication interface 902, at least one memory 903, and at least one communication bus 904.

[0136] In the embodiments of the present application, the number of the processor 901, the communication interface 902, the memory 903, and the communication bus 904 is at least one, and the processor 901, the communication interface 902, and the memory 903 complete communication with each other through the communication bus 904.

[0137] The processor 901 can be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application, etc.

[0138] The memory 903 can include a high-speed RAM memory, and can also include a non-volatile memory, etc., such as at least one disk memory.

[0139] The memory 903 stores a program, and the processor 901 can call the program stored in the memory 903, and the program is used to implement each step of the robot target positioning method based on the monocular camera or each step of the ultrasonic robot positioning method.

[0140] Optionally, the refinement function and the extension function of the program can refer to the description above.

[0141] The embodiment of the present application further provides a readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize each step of the robot target positioning method based on a monocular camera or the ultrasonic robot positioning method.

[0142] Optionally, the refinement function and the extension function of the program can refer to the description above.

[0143] Finally, it needs to be explained that, in this document, the relationship terms such as and the second and the like are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0144] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between various embodiments can be referred to each other.

[0145] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robot target positioning method based on a monocular camera, wherein the monocular camera is used to capture an image of a target object on a detection platform, the monocular camera is fixed relative to the detection platform, and the robot includes a base fixed relative to the detection platform, a robotic arm movably connected to the base, and a detection probe mounted on the free end of the robotic arm; characterized in that: The detection probe includes a force sensor, and the robot target positioning method based on a monocular camera includes: Obtaining position information in the image of a target point of the target object mapped to the support surface of the detection platform under the perspective of the monocular camera; Determine, according to the position information of the target point in the image, the shortest path between the target point and the center point of the monocular camera as the target path; controlling the robotic arm to move along the target path, and determining the actual position of the target object through data obtained by the force sensor; The determining, based on the position information of the target point in the image, the shortest path between the target point and the center point of the monocular camera comprises: Converting the position information of the target point in the image into a robot coordinate system to obtain the position information of the target point in the robot coordinate system; Calculating the position information of the center point of the monocular camera in the robot coordinate system; The shortest path between the target point and the center point of the monocular camera is determined according to the position information of the target point in the robot coordinate system and the position information of the center point of the monocular camera in the robot coordinate system.

2. The robot target positioning method based on a monocular camera according to claim 1, characterized in that: The converting the position information of the target point in the image into the robot coordinate system to obtain the position information of the target point in the robot coordinate system includes: Determining the position information of the target point in the camera coordinate system based on the position information of the target point in the image, the distance ratio, and the distance between the center point of the monocular camera and the detection platform, wherein the distance ratio refers to the ratio of the actual distance between two points on the support surface of the detection platform to the pixel distance between the two points in the image, and the camera coordinate system uses the center point of the monocular camera as the coordinate origin; The position information of the target point in the robot coordinate system is determined according to the position information of the target point in the camera coordinate system and a homogeneous transformation matrix, wherein the homogeneous transformation matrix is ​​a transformation matrix from the camera coordinate system to the robot coordinate system.

3. The robot target positioning method based on a monocular camera according to claim 2, characterized in that: Calculating the position information of the center point of the monocular camera in the robot coordinate system includes: The position information of the center point of the monocular camera in the robot coordinate system is determined according to the position information of the center point of the monocular camera in the camera coordinate system and the homogeneous transformation matrix.

4. The robot target positioning method based on a monocular camera according to claim 1, characterized in that: The controlling the robotic arm to move along the target path and determining the actual position of the target object through data obtained by the force sensor includes: During the movement of the robotic arm along the target path, it is determined in real time whether the data obtained by the force sensor is greater than a preset threshold. When the data obtained by the force sensor is greater than the preset threshold, the position of the robotic arm is determined as the actual position of the target object.

5. The robot target positioning method based on a monocular camera according to claim 1, characterized in that: The obtaining of position information of a target point in the image mapped to the target object within the support surface of the detection platform under the perspective of the monocular camera includes: The corresponding relationship between the target point and the target object sent by the remote communication module is received, and the position information is determined based on the corresponding relationship.

6. A robot target positioning device based on a monocular camera, wherein the monocular camera is used to capture an image of a target object on a detection platform, the monocular camera is fixed relative to the detection platform, and the robot includes a base fixed relative to the detection platform, a robotic arm movably connected to the base, and a detection probe mounted on the free end of the robotic arm; characterized in that: The detection probe includes a force sensor, and the robot target positioning device based on a monocular camera includes: a position information acquisition module, a target path determination module and an actual position determination module; The position information acquisition module is used to obtain the position information of the target point of the target object mapped to the support surface of the detection platform in the image under the perspective of the monocular camera; The target path determination module is used to determine the shortest path between the target point and the center point of the monocular camera according to the position information of the target point in the image, as the target path; The actual position determination module is used to control the robot arm to move along the target path and determine the actual position of the target object through data obtained by the force sensor; The target path determination module includes: a position information conversion module, a position information calculation module and a shortest path determination module; The position information conversion module is used to convert the position information of the target point in the image into the robot coordinate system to obtain the position information of the target point in the robot coordinate system; The position information calculation module is used to calculate the position information of the center point of the monocular camera in the robot coordinate system; The shortest path determination module is used to determine the shortest path between the target point and the center point of the monocular camera based on the position information of the target point in the robot coordinate system and the position information of the center point of the monocular camera in the robot coordinate system.

7. The robot target positioning device based on a monocular camera according to claim 6, characterized in that: The actual position determination module is specifically used to determine in real time whether the data obtained by the force sensor is greater than a preset threshold during the movement of the robotic arm along the target path. When the data obtained by the force sensor is greater than the preset threshold, the position of the robotic arm is determined as the actual position of the target object.

8. An ultrasonic detection device, characterized in that: include: an inspection platform, the inspection platform including a support surface for supporting a target object; a monocular camera fixed relative to the detection platform, wherein the monocular camera captures an image of the target object; an ultrasonic robot comprising a base fixed relative to the testing platform, a robotic arm movably connected relative to the base, and an ultrasonic probe mounted at a free end of the robotic arm, the ultrasonic probe comprising a force sensor; and A controller, wherein the controller is used to execute each step of the robot target positioning method based on a monocular camera as described in any one of claims 1 to 5.

9. The ultrasonic detection device according to claim 8, wherein: Also includes: A remote communication module connected to the controller, wherein the remote communication module at least transmits the corresponding relationship between the target point and the target object to the controller.

10. The ultrasonic detection device according to claim 8 or 9, characterized in that: Also includes: A supporting trolley supports the ultrasonic robot, the detection platform is provided with a positioning portion for positioning the supporting trolley, the supporting trolley is fixed in cooperation with the positioning portion, and the base is installed on the supporting trolley.

11. A method for positioning an ultrasonic robot, the ultrasonic robot comprising a base fixed relative to a test platform, a robotic arm movably connected relative to the base, and an ultrasonic probe mounted at a free end of the robotic arm; characterized in that: The ultrasonic probe includes a force sensor, and the ultrasonic robot positioning method includes: Using a monocular camera fixed relative to the detection platform to capture an image of a target object on the support surface of the detection platform; Obtaining position information in the image of a target point of the target object mapped to the support surface of the detection platform under the perspective of the monocular camera; Determine, according to the position information of the target point in the image, the shortest path between the target point and the center point of the monocular camera as the target path; controlling the robotic arm to move along the target path, and determining the actual position of the target object through data obtained by the force sensor; The determining, based on the position information of the target point in the image, the shortest path between the target point and the center point of the monocular camera comprises: Converting the position information of the target point in the image into a robot coordinate system to obtain the position information of the target point in the robot coordinate system; Calculating the position information of the center point of the monocular camera in the robot coordinate system; The shortest path between the target point and the center point of the monocular camera is determined according to the position information of the target point in the robot coordinate system and the position information of the center point of the monocular camera in the robot coordinate system.

12. A positioning device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the robot target positioning method based on a monocular camera as described in any one of claims 1 to 5, or the various steps of the ultrasonic robot positioning method as described in claim 11.

13. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the robot target positioning method based on a monocular camera according to any one of claims 1 to 5 or each step of the ultrasonic robot positioning method according to claim 11 is implemented.

Images