Positioning method, apparatus and electronic device

Abstract

Embodiments of the present application provide a positioning method and device and electronic equipment, and relate to the technical field of image processing. An image acquisition device acquires a to-be-detected image of a to-be-detected object; an object detection is performed on the to-be-detected image to obtain an image center point of a detected target object; a preset conversion relationship is used to map a target three-dimensional point set of the to-be-detected object to the to-be-detected image; an initial mapping point corresponding to the image center point is determined from each mapping point obtained by mapping the target three-dimensional point set, and a target mapping point corresponding to the image center point is determined based on the initial mapping point; and based on the target mapping point, a target center point of the target object is located in the to-be-detected object. Compared with related technologies, the embodiments of the present application can improve positioning efficiency.

Description

A positioning method, apparatus and electronic device Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a positioning method, apparatus and electronic device. Background Technology

[0002] When performing object detection on an object, an image acquisition device is typically used to capture an image of the object to be detected. Then, object detection is performed on this image to determine the location of defects, trademarks, and other target objects within the image. Next, points corresponding one-to-one with each point of the detected target object need to be found in the world coordinate system, thus locating the target object on the object being detected.

[0003] In related technologies, a reference image of a reference object can be pre-acquired using an image acquisition device. Then, a mathematical model is used to calculate the mapping relationship between the pixel coordinates of the reference image and the spatial coordinates of the reference object in the world coordinate system. This mapping relationship can then be stored as a look-up table. After acquiring the image to be detected using the image acquisition device, it can be first determined whether the image to be detected has any anomalies compared to the reference image. If anomalies are found, the parameter information of the image acquisition device needs to be obtained, and the coordinate look-up table is reconstructed using this parameter information. Then, the reconstructed coordinate look-up table is used to find the spatial coordinates corresponding to the detected pixel coordinates in the image to be detected.

[0004] For example, Figure 1 is a coordinate lookup table calculated using a reference image acquired by an image acquisition device. In this coordinate lookup table, the spatial coordinates (X, Y) are stored at the pixel coordinates (X, Y). xy Y xy Z xy ).

[0005] However, in related technologies, when using coordinate lookup tables to locate the target object detected on the image to be detected, the process of anomaly detection of the image to be detected and reconstructing the coordinate lookup table using the parameter information of the image acquisition device takes a long time, resulting in low positioning efficiency. Summary of the Invention

[0006] The purpose of this application is to provide a positioning method, device, and electronic device to improve positioning efficiency. The specific technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a positioning method, the method comprising:

[0008] Acquire the image of the object to be detected captured by the image acquisition device;

[0009] Object detection is performed on the image to be detected to obtain the image center point of the detected target object;

[0010] Using a preset transformation relationship, the target 3D point set of the object to be detected is mapped to the image to be detected; wherein, the preset transformation relationship includes: a first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected;

[0011] Among the mapping points obtained by mapping the target 3D point set, an initial mapping point corresponding to the image center point is determined, and based on the initial mapping point, a target mapping point corresponding to the image center point is determined;

[0012] Based on the target mapping point, the target center point of the target object is located in the object to be detected.

[0013] Optionally, in one specific implementation, before mapping the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship, the method further includes:

[0014] Obtain the reference three-dimensional point set of the object to be detected;

[0015] Based on the reference 3D point set, the target 3D point set of the object to be detected is determined.

[0016] Optionally, in one specific implementation, obtaining the reference 3D point set of the object to be detected includes:

[0017] Uniform sampling is performed on the three-dimensional model of the object to be detected to obtain the reference three-dimensional point set of the object to be detected;

[0018] or,

[0019] The three-dimensional point cloud of the surface of the object to be detected is acquired by a preset sensor to obtain the reference three-dimensional point set of the object to be detected.

[0020] Optionally, in one specific implementation, determining the target 3D point set of the object to be detected based on the reference 3D point set includes:

[0021] The reference three-dimensional point set is determined as the target three-dimensional point set of the object to be detected;

[0022] or,

[0023] The reference 3D point set is downsampled at preset intervals to obtain the target 3D point set of the object to be detected.

[0024] Optionally, in one specific implementation, determining the target mapping point corresponding to the image center point based on the initial mapping point includes:

[0025] The initial mapping point is determined as the target mapping point corresponding to the center point of the image;

[0026] The step of locating the target center point of the target object within the object to be detected based on the target mapping point includes:

[0027] In the target three-dimensional point set, the target three-dimensional point corresponding to the target mapping point is determined, and the target point indicated by the target three-dimensional point is located in the object to be detected to obtain the target center point of the target object.

[0028] Optionally, in one specific implementation, the target three-dimensional point set is: a three-dimensional point set obtained by downsampling the reference three-dimensional point set at a preset interval;

[0029] The method further includes:

[0030] According to the specified data structure, the baseline three-dimensional point set is divided into regions to obtain multiple sub-point sets;

[0031] The step of determining the target mapping point corresponding to the image center point based on the initial mapping point includes:

[0032] Determine the initial 3D point corresponding to the initial mapping point in the target 3D point set, and determine the target subset to which the initial 3D point belongs;

[0033] Using the preset transformation relationship, the target sub-point set is mapped onto the image to be detected, and the target mapping point corresponding to the image center point is determined among the mapping points obtained from the target sub-point set.

[0034] The step of locating the target center point of the target object within the object to be detected based on the target mapping point includes:

[0035] In the reference three-dimensional point set, the target three-dimensional point corresponding to the target mapping point is determined, and the target point indicated by the target three-dimensional point is located in the object to be detected to obtain the target center point of the target object.

[0036] Optionally, in one specific implementation, the initial three-dimensional point corresponding to the initial mapping point is determined from the target three-dimensional point set;

[0037] The three-dimensional point that corresponds to the initial mapping point and is closest to the image acquisition device in the target three-dimensional point set is determined as the initial three-dimensional point.

[0038] Optionally, in one specific implementation, determining the initial mapping point corresponding to the image center point among the mapping points obtained from the mapping of the target 3D point set includes:

[0039] Among the mapping points obtained by mapping the target 3D point set, the mapping point whose distance to the image center point meets the preset condition is determined as the initial mapping point corresponding to the image center point; wherein, the preset condition includes any one of: equal to a specified distance, not greater than a preset distance, and minimum distance.

[0040] Optionally, in one specific implementation, the acquisition of the image of the object to be detected acquired by the image acquisition device includes:

[0041] Iterate through each preset acquisition angle, and when each preset acquisition angle is reached, acquire the image of the object to be detected captured by the image acquisition device at that preset acquisition angle;

[0042] The method further includes:

[0043] When the center point of each target is obtained, determine whether all preset acquisition angles have been traversed.

[0044] If the traversal is not completed, the next preset acquisition angle is traversed, and the step of acquiring the image of the object to be detected acquired by the image acquisition device at each preset acquisition angle is returned.

[0045] Optionally, in one specific implementation, the image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on a fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0046] Secondly, embodiments of this application provide a positioning device, the device comprising:

[0047] The image acquisition module is used to acquire the image of the object to be detected captured by the image acquisition device.

[0048] The detection module is used to perform object detection on the image to be detected and obtain the image center point of the detected target object;

[0049] The mapping module is used to map the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship; wherein, the preset transformation relationship includes: a first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected;

[0050] The target mapping point determination module is used to determine the initial mapping point corresponding to the image center point among the mapping points obtained by mapping the target three-dimensional point set, and to determine the target mapping point corresponding to the image center point based on the initial mapping point;

[0051] The positioning module is used to locate the target center point of the target object in the object to be detected based on the target mapping point.

[0052] Optionally, in one specific implementation, the apparatus further includes:

[0053] The reference 3D point set determination module is used to obtain the reference 3D point set of the object to be detected before mapping the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship.

[0054] The target 3D point set determination module is used to determine the target 3D point set of the object to be detected based on the reference 3D point set.

[0055] Optionally, in one specific implementation, the reference three-dimensional point set determination module is specifically used for:

[0056] The reference three-dimensional point set of the object to be detected is obtained by uniformly sampling the three-dimensional model of the object to be detected; or, the reference three-dimensional point cloud of the surface of the object to be detected is obtained by acquiring the data collected by a preset sensor.

[0057] Optionally, in one specific implementation, the target three-dimensional point set determination module is specifically used for:

[0058] The reference 3D point set is determined as the target 3D point set of the object to be detected; or, the reference 3D point set is downsampled at a preset interval to obtain the target 3D point set of the object to be detected.

[0059] Optionally, in one specific implementation, the target mapping point determination module is specifically used for:

[0060] The initial mapping point is determined as the target mapping point corresponding to the center point of the image;

[0061] The positioning module is specifically used for:

[0062] In the target three-dimensional point set, the target three-dimensional point corresponding to the target mapping point is determined, and the target point indicated by the target three-dimensional point is located in the object to be detected to obtain the target center point of the target object.

[0063] Optionally, in one specific implementation, the target three-dimensional point set is: a three-dimensional point set obtained by downsampling the reference three-dimensional point set at a preset interval;

[0064] The device further includes:

[0065] The region division module is used to divide the baseline three-dimensional point set into regions according to a specified data structure, thereby obtaining multiple sub-point sets;

[0066] The target mapping point determination module includes:

[0067] The initial 3D point determination submodule is used to determine the initial 3D point corresponding to the initial mapping point in the target 3D point set, and to determine the target sub-point set to which the initial 3D point belongs;

[0068] The target mapping point determination submodule is used to map the target sub-point set to the image to be detected using the preset transformation relationship, and to determine the target mapping point corresponding to the image center point among the mapping points obtained from the target sub-point set.

[0069] The positioning module is specifically used for:

[0070] In the reference three-dimensional point set, the target three-dimensional point corresponding to the target mapping point is determined, and the target point indicated by the target three-dimensional point is located in the object to be detected to obtain the target center point of the target object.

[0071] Optionally, in one specific implementation, the initial three-dimensional point determination submodule is specifically used for:

[0072] The three-dimensional point that corresponds to the initial mapping point and is closest to the image acquisition device in the target three-dimensional point set is determined as the initial three-dimensional point.

[0073] Optionally, in one specific implementation, the target mapping point determination module includes:

[0074] Among the mapping points obtained by mapping the target 3D point set, the mapping point whose distance to the image center point meets the preset condition is determined as the initial mapping point corresponding to the image center point; wherein, the preset condition includes any one of: equal to a specified distance, not greater than a preset distance, and minimum distance.

[0075] Optionally, in one specific implementation, the image acquisition module is specifically used for:

[0076] Iterate through each preset acquisition angle, and when each preset acquisition angle is reached, acquire the image of the object to be detected captured by the image acquisition device at that preset acquisition angle;

[0077] The device further includes:

[0078] The judgment module is used to determine whether all preset acquisition angles have been traversed when the center point of each target is obtained; if not, the traversal module is triggered.

[0079] The trigger traversal module is used to traverse the next preset acquisition angle and return the step of acquiring the image of the object to be detected acquired by the image acquisition device at each preset acquisition angle.

[0080] Optionally, in one specific implementation, the image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on a fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0081] Thirdly, embodiments of this application provide an electronic device, including:

[0082] Memory, used to store computer programs;

[0083] When a processor executes a program stored in memory, it implements the steps of any of the above method embodiments.

[0084] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above method embodiments.

[0085] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the positioning methods described above.

[0086] Beneficial effects of the embodiments in this application:

[0087] As can be seen from the above, by applying the solution provided in this application, the target three-dimensional point set of the object to be detected and the image to be detected can be obtained, and the target three-dimensional point set can be mapped onto the image to be detected to determine the target mapping point corresponding to the image center point of the target object in the object to be detected. Then, using the target mapping point, the three-dimensional point corresponding to the target mapping point can be determined from the target three-dimensional point set. Therefore, after determining the three-dimensional point corresponding to the image center point of the target object, the position of this three-dimensional point in the object to be detected can be used as the position of the target center point of the target object, thus achieving the purpose of locating the target center point of the target object. Compared with related technologies, the location of the target center point of the target object is achieved using a three-dimensional point set. Since there is no need to use a coordinate lookup table to locate the target object, there is no need to perform anomaly judgment on the image to be detected or to reconstruct the coordinate lookup table using the parameter information of the image acquisition device. Therefore, the location time can be shortened and the location efficiency improved.

[0088] Furthermore, by simply acquiring the target 3D point set of the object to be detected and the images of each surface to be detected, the target object can be located on each surface of the object to be detected using the aforementioned target 3D point set, thereby further improving the positioning efficiency.

[0089] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0090] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0091] Figure 1 is a schematic diagram of a coordinate lookup table provided in an embodiment of this application;

[0092] Figure 2(a) is a schematic diagram of the third transformation relationship provided in the embodiment of this application;

[0093] Figure 2(b) is a schematic diagram of the installation location of the image acquisition device provided in the embodiment of this application;

[0094] Figure 2(c) is a schematic diagram of the positional relationship between the image acquisition device and the robotic arm provided in the embodiment of this application;

[0095] Figure 3 is a flowchart illustrating a positioning method provided in an embodiment of this application;

[0096] Figure 4 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0097] Figure 5 is a schematic diagram of the imaging model provided in the embodiment of this application;

[0098] Figure 6 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0099] Figure 7 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0100] Figures 8(a) and 8(b) are schematic flowcharts of specific examples provided in the embodiments of this application;

[0101] Figure 9 is a structural schematic diagram of a positioning device provided in an embodiment of this application;

[0102] Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0103] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0104] In related technologies, when using coordinate lookup tables to locate the target object detected on the image to be detected, the process of anomaly detection of the image to be detected and reconstructing the coordinate lookup table using the parameter information of the image acquisition device takes a long time, resulting in low positioning efficiency.

[0105] To address the aforementioned technical problems, this application provides a positioning method.

[0106] This method is applicable to various scenarios that require locating target objects on an object, such as locating defects on a workpiece or locating product labels on product packaging.

[0107] Furthermore, the subject executing this method can be an image acquisition device with data processing capabilities, or various electronic devices with data processing capabilities that communicate with the image acquisition device, such as mobile phones, laptops, desktop computers, etc. In addition, the electronic device can be an independent electronic device or a cluster of devices composed of multiple electronic devices, hereinafter referred to as electronic devices.

[0108] Therefore, the embodiments of this application do not specifically limit the application scenarios and execution entities of the method.

[0109] A positioning method provided in this application embodiment may include the following steps:

[0110] Acquire the image of the object to be detected captured by the image acquisition device;

[0111] Object detection is performed on the image to be detected to obtain the image center point of the detected target object;

[0112] Using a preset transformation relationship, the target 3D point set of the object to be detected is mapped to the image to be detected; wherein, the preset transformation relationship includes: a first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected;

[0113] Among the mapping points obtained by mapping the target 3D point set, an initial mapping point corresponding to the image center point is determined, and based on the initial mapping point, a target mapping point corresponding to the image center point is determined;

[0114] Based on the target mapping point, the target center point of the target object is located in the object to be detected.

[0115] As can be seen from the above, by applying the solution provided in this application, the target three-dimensional point set of the object to be detected and the image to be detected can be obtained, and the target three-dimensional point set can be mapped onto the image to be detected to determine the target mapping point corresponding to the image center point of the target object in the object to be detected. Then, using the target mapping point, the three-dimensional point corresponding to the target mapping point can be determined from the target three-dimensional point set. Therefore, after determining the three-dimensional point corresponding to the image center point of the target object, the position of this three-dimensional point in the object to be detected can be used as the position of the target center point of the target object, thus achieving the purpose of locating the target center point of the target object. Compared with related technologies, the location of the target center point of the target object is achieved using a three-dimensional point set. Since there is no need to use a coordinate lookup table to locate the target object, there is no need to perform anomaly judgment on the image to be detected or to reconstruct the coordinate lookup table using the parameter information of the image acquisition device. Therefore, the location time can be shortened and the location efficiency improved.

[0116] Furthermore, by simply acquiring the target 3D point set of the object to be detected and the images of each surface to be detected, the target object can be located on each surface of the object to be detected using the aforementioned target 3D point set, thereby further improving the positioning efficiency.

[0117] To better understand the positioning method provided in the embodiments of this application, the relevant concepts involved in the embodiments of this application will first be explained.

[0118] Since the exact location of the target object on the object to be detected is unknown, when using an image acquisition device to locate the target object on the object to be detected, the object to be detected can first be placed in a fixed position. Then, by changing the acquisition angle of the image acquisition device, the image acquisition device can be controlled to acquire images of the entire area of ​​the object to be detected.

[0119] Based on this, multiple preset acquisition angles can be set. Then, each preset acquisition angle is traversed, and when each preset acquisition angle is reached, the image acquisition device is used to acquire the image of the object to be detected at that preset acquisition angle.

[0120] When locating a target object on an object to be detected using an image acquired by an image acquisition device, a first transformation relationship between the physical coordinate system of the object to be detected and the camera coordinate system of the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system of the image to be detected can be determined. The physical coordinate system corresponding to the object to be detected is the world coordinate system corresponding to the object.

[0121] Both the first and second transformation relationships mentioned above can be achieved through camera calibration.

[0122] The purpose of camera calibration is to determine the coordinates of an object in the world coordinate system based on its coordinates in the image coordinate system, thereby enabling the positioning of the object based on the image of the object.

[0123] Generally, camera calibration can be divided into camera intrinsic parameter calibration and camera extrinsic parameter calibration. Camera intrinsic parameters are used to represent the transformation relationship between the camera coordinate system and the pixel coordinate system. Based on the camera intrinsic parameters, the coordinates of the object in the image coordinate system can be determined from the pixel coordinates of the object in the camera coordinate system. Camera extrinsic parameters are used to represent the transformation relationship between the world coordinate system and the camera coordinate system. Based on the camera extrinsic parameters, the coordinates of the object in the camera coordinate system can be determined from the coordinates of the object in the world coordinate system.

[0124] Figure 2(a) is a schematic diagram of a world coordinate system, camera coordinate system, pixel coordinate system, and image coordinate system provided in an embodiment of this application. In Figure 2(a), O W -X W Y W Z W For the world coordinate system, O C -X C Y C Z C Let P be the camera coordinate system, uv be the pixel coordinate system, and o-xy be the image coordinate system. Wherein, point P(X...) W Y W ZW Let P be a point in the world coordinate system. The image point p of point P in the image has coordinates (u, v) in the pixel coordinate system and (x, y) in the image coordinate system.

[0125] The world coordinate system is a three-dimensional Cartesian coordinate system that describes the spatial position of a camera and an object (for example, point P). It reflects the position of objects in the real world. The origin of the world coordinate system can be determined based on the specific circumstances.

[0126] Optionally, the world coordinate system can be constructed based on a calibration plate. For example, the origin of the world coordinate system can be the upper left corner feature point of the calibration plate, and the X-axis of the world coordinate system can be... W -Y W The plane can coincide with the calibration plate plane, and the Z-axis of the world coordinate system... W The axis can pass through the plane of the calibration plate and point vertically upwards.

[0127] The camera coordinate system is a three-dimensional Cartesian coordinate system with its origin located at the optical center of the camera lens. The x and y axes are parallel to the two sides of the image plane, respectively, and the z-axis is the optical axis of the lens, perpendicular to the image plane. The image plane is the plane on which the camera forms an image.

[0128] The image coordinate system is a two-dimensional Cartesian coordinate system. Because the pixel coordinate system is not conducive to coordinate transformations, the image coordinate system was established. The units of its coordinate axes can be easily transformed from the units of the coordinate axes in the world coordinate system and the camera coordinate system. The origin of the image coordinate system is the intersection of the camera's optical axis and the image plane (or the principal point), which is the center point of the image. The x-axis and y-axis are parallel to the u-axis and v-axis of the pixel coordinate system, respectively. Therefore, the pixel coordinate system and the image coordinate system can be considered as a translation relationship, meaning they can be obtained through translation. The difference lies in the units of the coordinate axes of the pixel coordinate system and the image coordinate system.

[0129] Optionally, in one specific implementation, the image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on the fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0130] In this specific implementation, in order to switch the preset acquisition angle, the image acquisition device can be installed at one end of the movable device, and the other end of the movable device can be installed on the fixed base. In this way, the acquisition angle of the image acquisition device can be changed by adjusting the movable device.

[0131] The positional relationship between the fixed position where the object to be detected is placed and the fixed base can be known, that is, the fixed position where the object to be detected is placed is a preset position; correspondingly, the positional relationship between the fixed position where the object to be detected is placed and the fixed base can also be unknown, that is, the fixed position where the object to be detected is placed is random.

[0132] For example, as shown in Figure 2(b), the image acquisition device can be a camera 210, the movable device can be a robotic arm 220, and the fixed base can be a robotic arm base 230. The robotic arm base 230 is fixedly placed on the ground, one end of the robotic arm 220 is mounted on the robotic arm base, and the other end is equipped with a camera 210. Since the robotic arm 220 is movable, for example, it can perform actions such as lifting, lowering, and rotating; thus, by moving the robotic arm 220, the acquisition angle of the camera 210 mounted on the robotic arm 220 can be adjusted. In this way, the camera 210 can acquire images of the object to be detected placed on the ground from different acquisition angles, obtaining multiple images of the object to be detected.

[0133] When using the image to be detected acquired by the image acquisition device to locate the target object on the object to be detected, the first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and the second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected can be determined.

[0134] The first transformation relationship mentioned above may include a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0135] Therefore, in order to locate the target object using the image to be detected, the first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, the second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device, and the second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected can be determined.

[0136] The method for determining the second transformation relationship is as described above and will not be repeated here.

[0137] Optionally, a reference image including the object to be detected and the fixed base can be acquired by a specified image acquisition device fixed at a specified position. Then, based on the reference image, the first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base can be determined.

[0138] The specified location is the position where reference images of the object to be detected and the fixed base can be acquired. It can be set according to actual needs. In this specific implementation, the specified location is not specifically limited. Furthermore, the specified image acquisition device can be a 3D (three-dimensional) camera or other cameras. This is all reasonable and is not specifically limited in this application embodiment.

[0139] Before acquiring reference images of the object to be detected and the fixed base using the specified image acquisition device, a first identifiable marker can be set on the fixed base, and the first coordinate of the first marker in the base coordinate system corresponding to the fixed base can be determined. A second marker is set on the object to be detected, and the second coordinate of the second marker in the object coordinate system can be determined. Then, reference images of the object to be detected and the fixed base are acquired, and the third coordinate of the first marker and the fourth coordinate of the second marker are determined in the image coordinate system of the reference image. Thus, using the first coordinate, the third coordinate, and the transformation relationship between the camera coordinate system and the image coordinate system, the transformation relationship between the camera coordinate system corresponding to the specified image acquisition device and the base coordinate system corresponding to the fixed base can be calibrated; and using the second coordinate, the fourth coordinate, and the transformation relationship between the camera coordinate system and the image coordinate system, the transformation relationship between the camera coordinate system corresponding to the specified image acquisition device and the physical coordinate system corresponding to the object to be detected can be calibrated.

[0140] Furthermore, based on the transformation relationship between the camera coordinate system corresponding to the specified image acquisition device and the base coordinate system corresponding to the fixed base, and the transformation relationship between the camera coordinate system corresponding to the specified image acquisition device and the physical coordinate system corresponding to the object to be detected, the first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base can be calibrated.

[0141] Optionally, a second sub-transformation relationship can be first established between the camera coordinate system corresponding to the image acquisition device and the base coordinate system corresponding to the fixed base; then, a first transformation relationship can be established between the camera coordinate system of the image acquisition device and the physical coordinate system corresponding to the object to be detected; and then, using the second sub-transformation relationship and the first transformation relationship, a first sub-transformation relationship can be established between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base.

[0142] Optionally, the hand-eye calibration method can be used to determine the second sub-transformation relationship between the aforementioned base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0143] Typically, the positional relationship between a robotic arm and an image acquisition device includes "eye on the hand" and "eye outside the hand." Here, "eye" refers to the image acquisition device, and "hand" refers to the robotic arm. In this application, the positional relationship between the robotic arm and the image acquisition device is "eye on the hand."

[0144] As shown in Figure 2(c), the so-called "eye in hand" means that the image acquisition device is fixed on the end of the robotic arm and moves with the end of the robotic arm, while the relative position of the camera and the end of the robotic arm remains unchanged.

[0145] Furthermore, based on the relative positions of the image acquisition device, the preset calibration plate, and the end effector of the robotic arm on the fixed base, the second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device can be determined.

[0146] The following is a detailed description of a positioning method provided in an embodiment of this application, with reference to the accompanying drawings.

[0147] Figure 3 is a flowchart of a positioning method provided in an embodiment of this application. As shown in Figure 3, the method may include the following steps S301-S305.

[0148] S301: Acquire the image of the object to be detected acquired by the image acquisition device;

[0149] In order to detect the target object on the object to be detected, an image acquisition device can be used to acquire an image of the object to be detected. In this way, the image acquisition device can acquire an image of the object to be detected.

[0150] The target object on the object to be detected can be a defect, such as a workpiece, which can identify defects on the workpiece; or it can be a designated identifier, such as a product name, production date, or a designated symbol. All of these are reasonable and are not specifically limited in this embodiment.

[0151] For example, images of each product to be inspected can be collected, and the production batch number of the product can be detected on each image.

[0152] Optionally, before step S301 above, a first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected can be obtained as preset transformation relationships.

[0153] The methods for determining the first and second transformation relationships have already been described in detail above, and will not be repeated here.

[0154] Optionally, in one specific implementation, the image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on the fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0155] In this specific implementation, in order to facilitate changing the acquisition angle of the image acquisition device, the image acquisition device can be installed at one end of the movable device, while the other end of the movable device is installed on the fixed base.

[0156] Thus, in order to locate the target object on the object to be detected, it is necessary to obtain the first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, as well as the second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected.

[0157] The first transformation relationship mentioned above may include: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0158] To locate the target object within the object to be detected, it is necessary to determine the first sub-transformation relation, the second sub-transformation relation, and the aforementioned second transformation relation. The specific implementation methods for determining the first sub-transformation relation, the second sub-transformation relation, and the aforementioned second transformation relation have been explained above, and therefore will not be repeated here.

[0159] S302: Perform object detection on the image to be detected and obtain the image center point of the detected target object;

[0160] After acquiring the image to be detected, object detection can be performed on the image to obtain the image center point of the detected target object.

[0161] Object detection can be performed using deep learning methods or other methods, and these are all reasonable and are not specifically limited in this embodiment.

[0162] Optionally, when using an object detection method to select a target object, the geometric center point of the bounding box is the image center point of the target object.

[0163] S303: Using a preset transformation relationship, the target 3D point set of the object to be detected is mapped onto the image to be detected;

[0164] The preset transformation relationships include: a first transformation relationship between the physical coordinate system of the object to be detected and the camera coordinate system of the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system of the image to be detected.

[0165] To locate the image center point on the object to be detected, we can first determine the target 3D point set of the object to be detected, and then determine the 3D point corresponding to the image center point within that target 3D point set. Furthermore, the target point indicated by the 3D point corresponding to the image center point in the object to be detected can be used as the target center point of the target object.

[0166] In this application, the target three-dimensional point set of the object to be detected can be determined by using the three-dimensional model of the object to be detected; or the target three-dimensional point set of the object to be detected can be obtained by using a preset sensor to collect the three-dimensional point cloud of the surface of the object to be detected; or other methods can be used to obtain the target three-dimensional point set of the object to be detected. All of these are reasonable and are not specifically limited in the embodiments of this application.

[0167] Furthermore, the aforementioned preset sensor can be a laser profile measuring instrument, a lidar scanner, or other sensors, all of which are reasonable and are not specifically limited in the embodiments of this application.

[0168] It should be noted that, for the sake of clarity, examples will be provided later to illustrate the method for determining the target 3D point set and the above step S303.

[0169] After determining the target 3D point set of the object to be detected, the aforementioned preset transformation relationship can be used to map the target 3D point set of the object to be detected onto the image to be detected.

[0170] As mentioned above, the preset transformation relationship includes the first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and the second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected.

[0171] Optionally, the first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device can be used to transform the target 3D point set from the physical coordinate system to the camera coordinate system corresponding to the image acquisition device, thereby obtaining the first mapping point of the 3D points in the target 3D point set in the camera coordinate system. Then, the second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected can be used to transform the first mapping point from the camera coordinate system to the image coordinate system, thereby obtaining the second mapping point of the total 3D points of the target 3D point set in the image coordinate system, that is, obtaining the mapping points of the target 3D point set in the image to be detected.

[0172] Optionally, when the image acquisition device is installed at one end of the movable device and the other end of the movable device is installed on the fixed base, i.e., the first transformation relationship includes: the first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and the second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device, the first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base can be used first to transform the target three-dimensional point set from the object coordinate system to the base coordinate system corresponding to the fixed base, so as to obtain the third mapping point of the three-dimensional point set of the target in the base coordinate system; Then, using the second sub-transformation relationship between the aforementioned base coordinate system and the camera coordinate system corresponding to the image acquisition device, the aforementioned third mapping point can be transformed from the aforementioned base coordinate system to the camera coordinate system corresponding to the image acquisition device, thus obtaining the fourth mapping point of the three-dimensional points in the target three-dimensional reference point set in the camera coordinate system; next, using the second transformation relationship between the aforementioned camera coordinate system and the image coordinate system corresponding to the image to be detected, the aforementioned fourth mapping point can be transformed from the camera coordinate system to the image coordinate system corresponding to the image to be detected, thus obtaining the fifth mapping point of the three-dimensional points in the target three-dimensional reference point set in the camera coordinate system, that is, obtaining the mapping points of the target three-dimensional point set in the image to be detected.

[0173] S304: Among the mapping points obtained by mapping the target 3D point set, determine the initial mapping point corresponding to the image center point, and based on the initial mapping point, determine the target mapping point corresponding to the image center point;

[0174] After mapping the target 3D point set to the image to be detected, the initial mapping point corresponding to the image center point can be determined from each mapping point obtained by mapping the target 3D point set. Then, the target mapping point of the image center point can be determined based on the initial mapping point.

[0175] Optionally, in one specific implementation, step S304 may include the following step 11:

[0176] Step 11: Among the mapping points obtained by mapping the target 3D point set, the mapping points whose distance to the image center point meets the preset conditions are determined as the initial mapping points corresponding to the image center point;

[0177] The preset conditions include any one of the following: equal to the specified distance, not greater than the preset distance, and minimum distance.

[0178] In this specific implementation, after mapping the target 3D point set to the image to be detected, the mapping points among the mapping points obtained from the target 3D point set whose distance from the image center point meets the preset conditions can be determined as the initial mapping points corresponding to the image center point.

[0179] The aforementioned preset conditions may include any one of the following: equal to a specified distance, not greater than a preset distance, or minimum distance. For example, when the object to be detected is a transparent object, the aforementioned preset condition may be equal to a specified distance or not greater than a preset distance, while when the object to be detected is a non-transparent object, the aforementioned preset condition may be minimum distance.

[0180] Of course, depending on the specific circumstances of the actual application, the above-mentioned preset conditions can be other conditions, which are all reasonable.

[0181] Furthermore, both the specified distance and the preset distance can be set according to actual needs. For example, the specified distance can be 0.1 cm, 0.01 cm, etc., while the preset distance can be 0.005 cm, 0.02 cm, etc. These are all reasonable and are not specifically limited in this application embodiment.

[0182] Optionally, when the preset condition is equal to the specified distance, the mapping point that is equal to the specified distance from the center point of the image can be determined as the initial mapping point corresponding to the center point of the image.

[0183] Optionally, when the preset condition is no greater than the preset distance, the mapping point whose distance to the image center point is no greater than the preset distance among the mapping points obtained by mapping the target three-dimensional point set can be determined as the initial mapping point corresponding to the image center point;

[0184] Optionally, when the preset condition is minimum distance, the mapping point with the smallest distance to the image center point among the mapping points obtained from the target 3D point set can be determined as the initial mapping point corresponding to the image center point.

[0185] Optionally, the above initial mapping points can be used to determine the target mapping point corresponding to the center point of the image.

[0186] S305: Based on the target mapping point, locate the target center point of the target object in the object to be detected.

[0187] After determining the target mapping point of the image center point, the position of the target center point of the target object can be determined in the object to be detected, that is, the target center point.

[0188] Optionally, in one specific implementation, as shown in Figure 4, step S305 above may include the following step S3051:

[0189] S3051: In the target 3D point set, determine the target 3D point corresponding to the target mapping point, and locate the target point indicated by the target 3D point in the object to be detected, so as to obtain the target center point of the target object.

[0190] In this specific implementation, the target three-dimensional point corresponding to the target mapping point can be determined from the target three-dimensional point set. Then, the target point indicated by the target three-dimensional point can be located in the object to be detected, thereby obtaining the target center point of the target object.

[0191] When determining the target 3D point corresponding to the above target mapping point, the 3D point corresponding to the above target 3D point and closest to the image acquisition device can be used as the target 3D point.

[0192] Furthermore, since multiple 3D points can be mapped to the same location in the image to be detected during 3D point mapping, that is, the mapping point of multiple different 3D points can be the same mapping point, thus, a mapping point obtained by mapping the target 3D point set can correspond to multiple 3D points in the target 3D point set.

[0193] For example, Figure 5 is a schematic diagram of imaging. According to Figure 5, when acquiring the image of the object to be detected, point P' in the physical imaging plane can be the imaging point of point P2 on the object to be detected. Then, since the mapping paths of point P1 and point P2 are the same when mapping the three-dimensional point set of the object to be detected, both point P1 and point P2 will be mapped to point P'. That is, point P' can be used as the mapping point of point P1 and point P2. The mapping point P' corresponds to point P1 and point P2 in the target three-dimensional point set of the object to be detected.

[0194] Based on this, in one optional implementation, the target 3D point that corresponds to the target mapping point and is closest to the image acquisition device in the target 3D point set can be determined as the target 3D point corresponding to the target mapping point.

[0195] In this specific implementation, the target 3D point that corresponds to the target mapping point and is closest to the image acquisition device in the target 3D point set can be determined as the target 3D point corresponding to the target mapping point.

[0196] For example, as shown in Figure 5, since the Z coordinate of point P2 in the camera coordinate system is less than the Z coordinate of point P1 in the camera coordinate system, that is, the distance between point P2 and the camera is less than the distance between point P1 and the camera, point P2 can be used as the three-dimensional point corresponding to the mapping point P'.

[0197] Since the above target 3D point set is obtained based on the object to be detected, each 3D point in the target 3D point set can indicate a point on the object to be detected; and since the above target 3D point is the 3D point corresponding to the image center point of the target object, the target point indicated by the target 3D point on the object to be detected is the point corresponding to the image center point of the target object on the object to be detected, that is, the target center point of the target object.

[0198] As can be seen from the above, by applying the solution provided in this application, the target three-dimensional point set of the object to be detected and the image to be detected can be obtained, and the target three-dimensional point set can be mapped onto the image to be detected to determine the target mapping point corresponding to the image center point of the target object in the object to be detected. Then, using the target mapping point, the three-dimensional point corresponding to the target mapping point can be determined from the target three-dimensional point set. Therefore, after determining the three-dimensional point corresponding to the image center point of the target object, the position of this three-dimensional point in the object to be detected can be used as the position of the target center point of the target object, thus achieving the purpose of locating the target center point of the target object. Compared with related technologies, the location of the target center point of the target object is achieved using a three-dimensional point set. Since there is no need to use a coordinate lookup table to locate the target object, there is no need to perform anomaly judgment on the image to be detected or to reconstruct the coordinate lookup table using the parameter information of the image acquisition device. Therefore, the location time can be shortened and the location efficiency improved.

[0199] Furthermore, by simply acquiring the target 3D point set of the object to be detected and the images of each surface to be detected, the target object can be located on each surface of the object to be detected using the aforementioned target 3D point set, thereby further improving the positioning efficiency.

[0200] The following provides an example illustrating the method for determining the target three-dimensional point set and the above step S303.

[0201] Optionally, in one specific implementation, before step S303 above, which maps the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship, the localization method provided in this application embodiment may further include the following steps 21-22:

[0202] Step 21: Obtain the reference 3D point set of the object to be detected;

[0203] Step 22: Based on the baseline 3D point set, determine the target 3D point set of the object to be detected.

[0204] In this specific implementation, the reference three-dimensional point set of the object to be detected can be obtained first, and then the target three-dimensional point set of the object to be detected can be determined using the reference three-dimensional point set of the object to be detected.

[0205] The aforementioned reference 3D point set and target 3D point set can be represented in the form of voxels, 3D point clouds, or other forms, all of which are reasonable and are not specifically limited in the embodiments of this application.

[0206] Optionally, in one specific implementation, step 21 above may include the following step 211:

[0207] Step 211: Uniformly sample the 3D model of the object to be detected to obtain the reference 3D point set of the object to be detected.

[0208] In this specific implementation, a three-dimensional model of the object to be detected can be obtained first, and the three-dimensional model can be uniformly sampled to obtain the reference three-dimensional points of the object to be detected. In this way, the target three-dimensional points of the object to be detected can be determined using the reference three-dimensional points.

[0209] The aforementioned three-dimensional model can be a CAD (Computer Aided Design) model or other models, which is reasonable and is not specifically limited in this embodiment.

[0210] For example, a reference 3D point set of the object to be detected can be obtained by uniformly sampling the CAD model of the object to be detected.

[0211] Optionally, in one specific implementation, step 21 above may also include the following step 212:

[0212] Step 212: Obtain the three-dimensional point cloud of the surface of the object to be detected collected by the preset sensor, and obtain the reference three-dimensional point set of the object to be detected.

[0213] In this specific implementation, a preset sensor is first used to collect three-dimensional point clouds of each surface of the object to be detected. Then, the three-dimensional point clouds of the object's surface collected by the preset sensor are obtained, thus obtaining the reference three-dimensional points of the object. Next, the target three-dimensional points of the object can be determined using the reference three-dimensional points.

[0214] The aforementioned preset sensor can be a laser profile measuring instrument, a lidar scanner, or other sensors, all of which are reasonable and are not specifically limited in this application embodiment.

[0215] For example, a laser profilometer can be used to collect 3D point clouds of the surface of the object to be inspected, thereby obtaining a reference 3D point set of the object to be inspected.

[0216] Furthermore, steps 211 and 212 described above are merely examples of methods for obtaining the reference 3D point set of the object to be detected. Of course, other methods can also be used to obtain the reference 3D point set of the object to be detected, which will not be listed here.

[0217] Optionally, in one specific implementation, step 22 above may include the following step 221:

[0218] Step 221: Determine the reference 3D point set as the target 3D point set of the object to be detected;

[0219] In this specific implementation, after obtaining the above-mentioned reference three-dimensional point set, the above-mentioned reference three-dimensional point set can be directly determined as the target three-dimensional point set of the object to be detected.

[0220] Because the number of 3D points in the reference 3D point set obtained by uniformly sampling the 3D model of the object to be detected is quite large, it takes a long time to locate the target center point of the target object using the above reference 3D point set, resulting in low positioning efficiency.

[0221] Based on this, in one optional implementation, step 22 above may include the following step 222:

[0222] Step 222: Downsample the reference 3D point set at preset intervals to obtain the target 3D point set of the object to be detected.

[0223] In this specific implementation, in order to improve positioning efficiency, the above-mentioned reference three-dimensional point set can be downsampled at preset intervals, so as to obtain the target three-dimensional point set of the object to be detected after downsampling.

[0224] Downsampling involves taking samples from the initial sample sequence at intervals of several samples. Each sample obtained is then used as a new sample sequence. This new sample sequence is the sample sequence obtained by downsampling the initial sample sequence.

[0225] The aforementioned preset interval can be set according to actual needs, such as 20, 100, etc., which are all reasonable and are not specifically limited in this application embodiment.

[0226] Furthermore, since the number and density of three-dimensional points in the reference three-dimensional point set are greater than those in the three-dimensional point set obtained by downsampling the reference three-dimensional point set, the reference three-dimensional point set can be called a dense three-dimensional point set according to the number and density of three-dimensional points included in the point set, while the three-dimensional point set obtained by downsampling the reference three-dimensional point set can be called a sparse three-dimensional point set.

[0227] As can be seen, the above target 3D point set can be a reference 3D point set determined based on the 3D model of the object to be detected, or it can be a 3D point set obtained by downsampling the reference 3D point set determined by the 3D model of the object to be detected.

[0228] When the target 3D point set is a reference 3D point set, a preset transformation relationship can be used to map the reference 3D point set onto the image to be detected. Among the mapping points obtained from the reference 3D point set, the initial mapping point corresponding to the image center point can be determined. Then, the initial mapping point can be used as the target mapping point corresponding to the image center point. Next, the target 3D point corresponding to the target mapping point can be determined in the reference 3D point set, and the target point indicated by the target 3D point can be located in the object to be detected, thereby obtaining the target center point of the target object.

[0229] Because the number and density of the three-dimensional points in the reference three-dimensional point set are large, the determined target three-dimensional points are closer to the target center point of the target object. Therefore, the accuracy of locating the target center point of the target object using the target three-dimensional point set is high.

[0230] When the target 3D point set is obtained by downsampling the reference 3D point set, a preset transformation relationship can be used to map the 3D point set onto the image to be detected. Among the mapping points obtained from the 3D point set, the initial mapping point corresponding to the image center point can be determined. Then, the initial mapping point can be used as the target mapping point corresponding to the image center point. Next, the target 3D point corresponding to the target mapping point can be determined in the 3D point set, and the target point indicated by the target 3D point can be located in the object to be detected, thereby obtaining the target center point of the target object.

[0231] Because the number and density of the three-dimensional points in this three-dimensional point set are relatively small, the amount of computation required to locate the target center point of the target object using this three-dimensional point set is small, faster, and more efficient.

[0232] When the target 3D point set is a 3D point set obtained by downsampling the reference 3D point set at a preset interval, the number of points in the target 3D point set is significantly less than that in the reference 3D point set obtained based on the 3D model of the object to be detected. Therefore, when using the target 3D point set to locate the target object in the object to be detected, its positioning accuracy will be less than that using the reference 3D point set to locate the target object in the object to be detected. Therefore, in order to improve the positioning accuracy, the target mapping point corresponding to the image center point can be determined in the reference 3D point set based on the initial mapping point of the image center point mentioned above.

[0233] Optionally, in one specific implementation, the positioning method provided in this application embodiment may further include the following step 31:

[0234] Step 31: Divide the baseline 3D point set into regions according to the specified data structure to obtain multiple sub-point sets;

[0235] Accordingly, in this specific implementation, step S304 above, which determines the target mapping point corresponding to the image center point based on the initial mapping point, may include the following steps 32-33:

[0236] Step 32: Determine the initial 3D point corresponding to the initial mapping point in the target 3D point set, and determine the target sub-point set to which the initial 3D point belongs;

[0237] Step 33: Using a preset transformation relationship, map the target subset of points onto the image to be detected, and among the mapped points obtained from the target subset of points, determine the target mapping point corresponding to the image center point.

[0238] Accordingly, in this specific implementation, step S305 above may include the following step 34:

[0239] Step 34: In the reference 3D point set, determine the target 3D point corresponding to the target mapping point, and locate the target point indicated by the target 3D point in the object to be detected, so as to obtain the target center point of the target object.

[0240] In this specific implementation, since each point in the three-dimensional point set obtained by downsampling has a corresponding relationship with each point in the reference three-dimensional point set, the correspondence between the reference three-dimensional point set and the three-dimensional point set obtained by downsampling can be obtained after downsampling the reference three-dimensional point set.

[0241] In this process, when sampling is performed at preset intervals, each sampled point corresponds to other points within that preset interval. For example, if the preset interval is 20, one 3D point is sampled from every 20 3D points, and thus, the sampled point corresponds to the remaining 19 3D points within that interval.

[0242] Based on this, the above correspondence can be used as a specified data structure. This specified data structure can be a KD-Tree (k-dimensional tree), an octree, or a data structure that stores indexes in blocks according to spatial coordinates. All of these are reasonable and are not specifically limited in this embodiment.

[0243] Among them, the so-called KD-Tree is a data structure in a k-dimensional data space, which is mainly used for searching key data in a multi-dimensional space, such as performing nearest neighbor search and range search.

[0244] Optionally, step 31 above: dividing the baseline 3D point set into regions according to the specified data structure to obtain multiple sub-point sets can be: constructing a KD-Tree using the above baseline 3D point set.

[0245] In this way, the aforementioned reference 3D point set can be divided into regions according to the specified data structure, thereby obtaining multiple sub-point sets of the reference 3D point set.

[0246] For example, the preset interval can be 20, and every 20 three-dimensional points can be used as a subset of points according to the preset interval.

[0247] Therefore, after determining the initial mapping point, the initial three-dimensional point corresponding to the initial mapping point can be determined in the target three-dimensional point. Then, the target sub-point set to which the initial three-dimensional point belongs in each sub-point set of the reference three-dimensional point set can be determined.

[0248] In this process, multiple 3D points can be mapped to the same location in the image to be detected during 3D point mapping. That is, the mapping points of multiple 3D points can be the same. Therefore, a mapping point obtained by mapping the target 3D point set can correspond to multiple 3D points in the target 3D point set.

[0249] Based on this, optionally, the initial three-dimensional point can be determined as the three-dimensional point that corresponds to the initial mapping point in the target three-dimensional point set and is closest to the image acquisition device.

[0250] Thus, after determining the initial three-dimensional point corresponding to the above initial mapping point in the target reference point set, it can be determined that the above initial three-dimensional point belongs to the target sub-set in the reference three-dimensional point set.

[0251] The initial mapping point can be considered to be the mapping point closest to the image center point. Therefore, the initial 3D point corresponding to the initial mapping point can be considered to be the 3D point in the target 3D point set closest to the target object's center point. However, since the initial 3D point is obtained by downsampling its target subset within the reference 3D point set, each 3D point in the target subset can be considered to be a 3D point close to the target object's center point. Furthermore, there may be 3D points in the target subset that are even closer to the target object's center point than the initial 3D point.

[0252] Based on this, the above-mentioned preset transformation relationship can be used to map all three-dimensional points in the target sub-point set to the image to be detected. Thus, among the mapping points obtained by mapping the target sub-point set, the target mapping point corresponding to the image center point is determined. That is, among the mapping points obtained by mapping the target sub-point set, the mapping point whose distance from the image center point meets the preset condition is used as the target mapping point corresponding to the image center point.

[0253] The obtained target mapping points are obtained by mapping each 3D point in the target sub-point set of the reference 3D point set onto the image to be detected. Therefore, the target 3D point corresponding to the target mapping point can be determined in the reference 3D point set. Then, in the object to be detected, the target point indicated by the target 3D point is determined, thereby locating the target center point of the target object.

[0254] As mentioned above, as shown in Figure 6, the positioning method provided in this application embodiment may include the following steps S601-S607:

[0255] S601: Acquire the image of the object to be detected acquired by the image acquisition device;

[0256] S602: Perform object detection on the image to be detected and obtain the image center point of the detected target object;

[0257] S603: Using a preset transformation relationship, the target 3D point set of the object to be detected is mapped to the image to be detected;

[0258] The preset transformation relationships include: a first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected; the target three-dimensional point set is: a three-dimensional point set obtained by downsampling the reference three-dimensional point set according to a preset interval;

[0259] S604: Divide the baseline 3D point set into regions according to the specified data structure to obtain multiple sub-point sets;

[0260] S605: Among the mapping points obtained by mapping the target three-dimensional point set, determine the initial mapping point corresponding to the image center point, determine the initial three-dimensional point corresponding to the initial mapping point in the target three-dimensional point set, and determine the target sub-point set to which the initial three-dimensional point belongs;

[0261] S606: Using a preset transformation relationship, the target sub-point set is mapped onto the image to be detected, and the target mapping point corresponding to the image center point is determined among the mapping points obtained from the target sub-point set;

[0262] S607: In the reference three-dimensional point set, determine the target three-dimensional point corresponding to the target mapping point, and locate the target point indicated by the target three-dimensional point in the object to be detected, so as to obtain the target center point of the target object.

[0263] The target three-dimensional point set mentioned above is a three-dimensional point set obtained by downsampling the reference three-dimensional point set according to a preset interval; the specific implementation of the above steps S601-S603 is the same as the specific implementation of the above steps S301-S303, and the specific implementation of the above steps S604-S607 is the same as the specific implementation of the above steps 31-34, so it will not be described again here.

[0264] When using an image acquisition device to locate a target object on an object to be detected, multiple preset acquisition angles can be set. This allows the image acquisition device to be controlled to acquire images of the object to be detected at each preset acquisition angle.

[0265] Optionally, in one specific implementation, as shown in Figure 7, step S301, acquiring the image of the object to be detected acquired by the image acquisition device, may include the following step S3011:

[0266] S3011: Traverse each preset acquisition angle, and when traversing to each preset acquisition angle, acquire the image of the object to be detected acquired by the image acquisition device at that preset acquisition angle;

[0267] The positioning method provided in this application embodiment may further include the following steps S306-S307:

[0268] S306: When the center point of each target is obtained, determine whether all preset acquisition angles have been traversed; if not, proceed to step S307.

[0269] S307: Traverse the next preset acquisition angle and return to step S3011 above.

[0270] In this specific implementation, each preset acquisition angle can be traversed, and when each preset acquisition angle is reached, the image acquisition device is controlled to acquire an image of the object to be detected at that preset acquisition angle. Thus, the image of the object to be detected acquired by the image acquisition device at that preset acquisition angle can be obtained.

[0271] Furthermore, upon obtaining each center point, it can be determined whether all the aforementioned preset acquisition angles have been traversed. Then, if not all preset acquisition angles have been traversed, the next preset acquisition angle can be traversed, and upon reaching the next preset acquisition angle, the image of the object to be detected captured by the image acquisition device at that preset acquisition angle is obtained.

[0272] When all the preset acquisition angles have been traversed, the positioning results of the target object location of the object to be detected can be output.

[0273] Optionally, when the image acquisition device is installed at one end of the movable device and the other end of the movable device is installed on the fixed base, the above-mentioned traversal of each preset acquisition angle can be achieved by moving the movable device.

[0274] For example, when the image acquisition device is installed at one end of the robotic arm and the robotic arm is installed on a fixed base, the robotic arm can be moved to traverse various preset acquisition angles.

[0275] Optionally, after traversing all preset acquisition angles, the aforementioned image acquisition device can be used to continue acquiring the image of the next object to be detected.

[0276] For example, each object to be detected can be placed on a conveyor belt and sequentially transported to an image acquisition area. The object remains in the image acquisition area for a specified duration, allowing the image acquisition device to capture images of the object at various preset acquisition angles. After the image acquisition device has traversed all preset acquisition angles, the object to be detected can be transported to the next area, and the next object to be detected can be transported to the image acquisition area.

[0277] To facilitate understanding of the positioning method provided in the embodiments of this application, the following description is provided in conjunction with Figures 8(a) and 8(b).

[0278] Defects in the object to be inspected can be located using steps S801-S811 as shown in Figure 8(a), wherein steps S801-S811 are as follows:

[0279] S801: Calibrates the camera's intrinsic parameters and the camera's and robotic arm's extrinsic parameters;

[0280] S802: Calibrate the conversion relationship between the robotic arm and the object to be detected;

[0281] S803: Establish an initial 3D point set based on the CAD model of the object to be detected;

[0282] S804: Obtain a sparse KD-Tree 3D point set based on the initial 3D point set;

[0283] S805: The camera captures an image of the object to be detected;

[0284] S806: Defect location is completed on the image, and the defect center point is obtained on the two-dimensional image;

[0285] S807: Defect center point mapping processing to obtain the corresponding three-dimensional defect points;

[0286] S808: Determine if the detection position traversal is complete; if yes, proceed to step S810; otherwise, proceed to step S809.

[0287] S809: Move to the next detection pose;

[0288] S810: Determine if the traversal of the object to be detected is complete; if yes, end; otherwise, proceed to step S811.

[0289] S811: Replace the object to be tested.

[0290] Before using a camera to acquire an image of the object to be inspected, the camera can first be mounted on one end of a robotic arm, and the other end of the robotic arm can be fixed to the robotic arm base. Next, the camera's intrinsic parameters, the camera's and robotic arm's extrinsic parameters can be calibrated, and the conversion relationship between the robotic arm and the object to be inspected can be determined.

[0291] Before locating defects in the object to be inspected, an initial three-dimensional point set of the object can be established based on its CAD model. Then, the initial three-dimensional point set is downsampled to obtain a sparse KD-Tree three-dimensional point set of the object.

[0292] Then, by moving the robotic arm, the camera can be controlled to capture images of the object to be detected at each preset detection pose, and the defect can be located in the image. Thus, the center point of the defect of the object to be detected can be determined on the two-dimensional image.

[0293] Next, the sparse KD-Tree three-dimensional point set can be used to map the defect center point to obtain the corresponding three-dimensional defect point. In this way, the defect center point can be located on the object to be inspected using the three-dimensional defect point.

[0294] After obtaining the defect center point corresponding to the image to be detected, it can be determined whether the detection position has been traversed. If not, it can move to the next detection pose. If so, it can be determined whether the object to be detected has been traversed. Then, when all the objects to be detected have been traversed, the localization can end. If each object to be detected has not been traversed, the detection object can be changed and the defect localization can be performed on the next object to be detected.

[0295] As shown in Figure 8(b), the three-dimensional defect point corresponding to the defect center point can be determined using the following steps S8071-S8075:

[0296] S8071: Mapping a sparse KD-Tree 3D point set to a 2D image;

[0297] S8072: Select the optimal three-dimensional point of the defect as the coarse positioning point;

[0298] S8073: Find the k nearest neighbors of a coarsely located point on an initial set of three-dimensional points;

[0299] S8074: Map k three-dimensional nearest neighbor points to an image;

[0300] S8075: Select the optimal point from k three-dimensional nearest neighbors as the target three-dimensional point for the defect.

[0301] After locating the defect on the image and determining the defect center point on the two-dimensional image, the sparse KD-Tree three-dimensional point set can be mapped onto the two-dimensional image. Then, the optimal three-dimensional point of the defect center point is selected as the coarse positioning point, that is, the mapping point with the smallest distance from the defect center point is taken as the optimal mapping point of the defect center point, and the optimal three-dimensional point corresponding to the optimal mapping point is taken as the coarse positioning point of the defect center point.

[0302] Subsequently, through nearest neighbor search on the initial 3D point set, k 3D nearest neighbor points of the coarse positioning point are found, and the above transformation relationship is used to map the above k 3D nearest neighbor points onto the image. Thus, among the mapped points of the above k 3D nearest neighbor points, the target mapped point with the smallest distance to the defect center point is determined, and the 3D nearest neighbor point corresponding to the target mapped point and closest to the camera is determined as the target 3D point of the defect center point.

[0303] Based on the same inventive concept, and corresponding to the positioning method shown in FIG3 of the above-described embodiments of this application, this application also provides a positioning device.

[0304] Figure 9 is a schematic diagram of a positioning device provided in an embodiment of this application. As shown in Figure 9, the device may include the following modules:

[0305] The image acquisition module 910 is used to acquire the image of the object to be detected acquired by the image acquisition device.

[0306] Detection module 920 is used to perform object detection on the image to be detected and obtain the image center point of the detected target object;

[0307] The mapping module 930 is used to map the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship; wherein, the preset transformation relationship includes: a first transformation relationship between the physical coordinate system corresponding to the object to be detected and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the image to be detected;

[0308] The target mapping point determination module 940 is used to determine the initial mapping point corresponding to the image center point among the mapping points obtained by mapping the target three-dimensional point set, and to determine the target mapping point corresponding to the image center point based on the initial mapping point;

[0309] The positioning module 950 is used to locate the target center point of the target object in the object to be detected based on the target mapping point.

[0310] As can be seen from the above, by applying the solution provided in this application, the target three-dimensional point set of the object to be detected and the image to be detected can be obtained, and the target three-dimensional point set can be mapped onto the image to be detected to determine the target mapping point corresponding to the image center point of the target object in the object to be detected. Then, using the target mapping point, the three-dimensional point corresponding to the target mapping point can be determined from the target three-dimensional point set. Therefore, after determining the three-dimensional point corresponding to the image center point of the target object, the position of this three-dimensional point in the object to be detected can be used as the position of the target center point of the target object, thus achieving the purpose of locating the target center point of the target object. Compared with related technologies, the location of the target center point of the target object is achieved using a three-dimensional point set. Since there is no need to use a coordinate lookup table to locate the target object, there is no need to perform anomaly judgment on the image to be detected or to reconstruct the coordinate lookup table using the parameter information of the image acquisition device. Therefore, the location time can be shortened and the location efficiency improved.

[0311] Furthermore, by simply acquiring the target 3D point set of the object to be detected and the images of each surface to be detected, the target object can be located on each surface of the object to be detected using the aforementioned target 3D point set, thereby further improving the positioning efficiency.

[0312] Optionally, in one specific implementation, the apparatus further includes:

[0313] The reference 3D point set determination module is used to obtain the reference 3D point set of the object to be detected before mapping the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship.

[0314] The target 3D point set determination module is used to determine the target 3D point set of the object to be detected based on the reference 3D point set.

[0315] Optionally, in one specific implementation, the reference three-dimensional point set determination module is specifically used for:

[0316] The reference three-dimensional point set of the object to be detected is obtained by uniformly sampling the three-dimensional model of the object to be detected; or, the reference three-dimensional point cloud of the surface of the object to be detected is obtained by acquiring the data collected by a preset sensor.

[0317] Optionally, in one specific implementation, the target three-dimensional point set determination module is specifically used for:

[0318] The reference three-dimensional point set is determined as the target three-dimensional point set of the object to be detected;

[0319] or,

[0320] The reference 3D point set is downsampled at preset intervals to obtain the target 3D point set of the object to be detected.

[0321] Optionally, in one specific implementation, the target mapping point determination module 940 is specifically used for:

[0322] The initial mapping point is determined as the target mapping point corresponding to the center point of the image;

[0323] The positioning module 950 is specifically used for:

[0324] In the target three-dimensional point set, the target three-dimensional point corresponding to the target mapping point is determined, and the target point indicated by the target three-dimensional point is located in the object to be detected to obtain the target center point of the target object.

[0325] Optionally, in one specific implementation, the target three-dimensional point set is: a three-dimensional point set obtained by downsampling the reference three-dimensional point set at a preset interval;

[0326] The device further includes:

[0327] The region division module is used to divide the baseline three-dimensional point set into regions according to a specified data structure, thereby obtaining multiple sub-point sets;

[0328] The target mapping point determination module 940 includes:

[0329] The initial 3D point determination submodule is used to determine the initial 3D point corresponding to the initial mapping point in the target 3D point set, and to determine the target sub-point set to which the initial 3D point belongs;

[0330] The target mapping point determination submodule is used to map the target sub-point set to the image to be detected using the preset transformation relationship, and to determine the target mapping point corresponding to the image center point among the mapping points obtained from the target sub-point set.

[0331] The positioning module 950 is specifically used for:

[0332] In the reference three-dimensional point set, the target three-dimensional point corresponding to the target mapping point is determined, and the target point indicated by the target three-dimensional point is located in the object to be detected to obtain the target center point of the target object.

[0333] Optionally, in one specific implementation, the initial three-dimensional point determination submodule is specifically used for:

[0334] The three-dimensional point that corresponds to the initial mapping point and is closest to the image acquisition device in the target three-dimensional point set is determined as the initial three-dimensional point.

[0335] Optionally, in one specific implementation, the target mapping point determination module 940 includes:

[0336] Among the mapping points obtained by mapping the target 3D point set, the mapping point whose distance to the image center point meets the preset condition is determined as the initial mapping point corresponding to the image center point; wherein, the preset condition includes any one of: equal to a specified distance, not greater than a preset distance, and minimum distance.

[0337] Optionally, in one specific implementation, the image acquisition module 910 is specifically used for:

[0338] Iterate through each preset acquisition angle, and when each preset acquisition angle is reached, acquire the image of the object to be detected captured by the image acquisition device at that preset acquisition angle;

[0339] The device further includes:

[0340] The judgment module is used to determine whether all preset acquisition angles have been traversed when the center point of each target is obtained; if not, the traversal module is triggered.

[0341] The trigger traversal module is used to traverse the next preset acquisition angle and return the step of acquiring the image of the object to be detected acquired by the image acquisition device at each preset acquisition angle.

[0342] Optionally, in one specific implementation, the image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on a fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

[0343] This application also provides an electronic device, as shown in FIG10, including:

[0344] Memory 1001 is used to store computer programs;

[0345] When the processor 1002 executes the program stored in the memory 1001, it implements the steps of any of the positioning methods provided in the embodiments of this application.

[0346] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1002, the communication interface, and the memory 1001 communicating with each other via the communication bus.

[0347] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0348] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0349] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0350] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0351] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described positioning methods.

[0352] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the positioning methods described above.

[0353] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0354] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0355] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0356] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A positioning method, characterized in that, The method includes: acquiring a target image of an object to be detected acquired by an image acquisition device; performing object detection on the target image to obtain the image center point of the detected target object; and mapping the target 3D point set of the target object to the target image using a preset transformation relationship; wherein the preset transformation relationship includes: a first transformation relationship between the physical coordinate system corresponding to the target object and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the target image; the target 3D point set is: a 3D point set obtained by downsampling a reference 3D point set of the target object according to a preset interval; the reference 3D point set includes points according to a specified... The data structure is divided into regions to obtain multiple subsets of points. Among the mapping points obtained by mapping the target 3D point set, the initial mapping point corresponding to the image center point is determined, and the initial 3D point corresponding to the initial mapping point in the target 3D point set is determined. The target subset to which the initial 3D point belongs is determined. Using the preset transformation relationship, the target subset is mapped to the image to be detected. Among the mapping points obtained by mapping the target subset, the target mapping point corresponding to the image center point is determined. In the reference 3D point set, the target 3D point corresponding to the target mapping point is determined, and the target point indicated by the target 3D point is located in the object to be detected to obtain the target center point of the target object.

2. The method according to claim 1, characterized in that, Before mapping the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship, the method further includes: obtaining a reference 3D point set of the object to be detected.

3. The method according to claim 2, characterized in that, The step of obtaining the reference three-dimensional point set of the object to be detected includes: uniformly sampling the three-dimensional model of the object to be detected to obtain the reference three-dimensional point set of the object to be detected; or, obtaining the three-dimensional point cloud of the surface of the object to be detected collected by a preset sensor to obtain the reference three-dimensional point set of the object to be detected.

4. The method according to claim 1, characterized in that, The step of determining the initial three-dimensional point corresponding to the initial mapping point in the target three-dimensional point set includes: determining the three-dimensional point in the target three-dimensional point set that corresponds to the initial mapping point and is closest to the image acquisition device as the initial three-dimensional point.

5. The method according to any one of claims 1-4, characterized in that, Determining the initial mapping point corresponding to the image center point from among the mapping points obtained by mapping the target 3D point set includes: determining the mapping point whose distance from the image center point satisfies a preset condition from among the mapping points obtained by mapping the target 3D point set as the initial mapping point corresponding to the image center point; wherein the preset condition includes any one of: equal to a specified distance, not greater than a preset distance, and minimum distance.

6. The method according to claim 1, characterized in that, The method of acquiring the image of the object to be detected acquired by the image acquisition device includes: traversing each preset acquisition angle, and acquiring the image of the object to be detected acquired by the image acquisition device at each preset acquisition angle; the method further includes: when obtaining each target center point, determining whether all preset acquisition angles have been traversed; if not, traversing the next preset acquisition angle, and returning to the step of acquiring the image of the object to be detected acquired by the image acquisition device at each preset acquisition angle.

7. The method according to claim 1, characterized in that, The image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on the fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

8. A positioning device, characterized in that, The device includes: an image acquisition module for acquiring a target image of an object to be detected acquired by an image acquisition device; a detection module for performing object detection on the target image to obtain the image center point of the detected target object; and a mapping module for mapping the target three-dimensional point set of the target object to the target image using a preset transformation relationship; wherein the preset transformation relationship includes: a first transformation relationship between the physical coordinate system corresponding to the target object and the camera coordinate system corresponding to the image acquisition device, and a second transformation relationship between the camera coordinate system and the image coordinate system corresponding to the target image; the target three-dimensional point set is: a three-dimensional point set obtained by downsampling the reference three-dimensional point set of the target object at preset intervals; the reference three-dimensional point set includes points according to a preset interval. A data structure is used to divide the region, resulting in multiple subsets of points. A target mapping point determination module is used to determine the initial mapping point corresponding to the image center point among the mapping points obtained from the target 3D point set, and to determine the initial 3D point corresponding to the initial mapping point in the target 3D point set, and to determine the target subset to which the initial 3D point belongs. Using the preset transformation relationship, the target subset is mapped onto the image to be detected, and the target mapping point corresponding to the image center point is determined among the mapping points obtained from the target subset. A positioning module is used to determine the target 3D point corresponding to the target mapping point in the reference 3D point set, and to locate the target point indicated by the target 3D point in the object to be detected, thereby obtaining the target center point of the target object.

9. The apparatus according to claim 8, characterized in that, The device further includes a reference 3D point set determination module, used to obtain the reference 3D point set of the object to be detected before mapping the target 3D point set of the object to be detected to the image to be detected using a preset transformation relationship.

10. The apparatus according to claim 9, characterized in that, The reference 3D point set determination module is specifically used to: uniformly sample the 3D model of the object to be detected to obtain the reference 3D point set of the object to be detected; or, acquire the 3D point cloud of the surface of the object to be detected collected by a preset sensor to obtain the reference 3D point set of the object to be detected.

11. The apparatus according to claim 8, characterized in that, The target mapping point determination module is specifically used to: determine the three-dimensional point that corresponds to the initial mapping point and is closest to the image acquisition device as the initial three-dimensional point from the target three-dimensional point set.

12. The apparatus according to any one of claims 8-11, characterized in that, The target mapping point determination module includes: determining the initial mapping point corresponding to the image center point as the mapping point among the mapping points obtained by mapping the target three-dimensional point set, where the distance between the mapping point and the image center point satisfies a preset condition; wherein, the preset condition includes any one of: equal to a specified distance, not greater than a preset distance, and minimum distance.

13. The apparatus according to claim 8, characterized in that, The image acquisition module is specifically used for: traversing each preset acquisition angle, and acquiring the image of the object to be detected captured by the image acquisition device at each preset acquisition angle; the device further includes: a judgment module, used to determine whether all preset acquisition angles have been traversed when each target center point is obtained; if not, the traversal module is triggered; the triggered traversal module is used to traverse the next preset acquisition angle and return to the step of acquiring the image of the object to be detected captured by the image acquisition device at each preset acquisition angle.

14. The apparatus according to claim 8, characterized in that, The image acquisition device is installed at one end of the movable device, and the other end of the movable device is installed on the fixed base; the first transformation relationship includes: a first sub-transformation relationship between the physical coordinate system corresponding to the object to be detected and the base coordinate system corresponding to the fixed base, and a second sub-transformation relationship between the base coordinate system and the camera coordinate system corresponding to the image acquisition device.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images