Positioning method, device, electronic device and readable storage medium

Abstract

The present application provides a positioning method, apparatus, electronic device, and readable storage medium, which calculates the first coordinates of a target point of a target device using a visual algorithm; calculates the second coordinates of the target point of the target device using a point laser algorithm; and obtains the target coordinates of the target point of the target device based on the first coordinates and the second coordinates. The embodiments of the present application calculate the target coordinates of the target point of the target device using two coordinate calculation methods, a visual algorithm and a laser point algorithm. The actual coordinates of the target point of the target device can be calculated using only an image and some calculation rules, making the target coordinate calculation of the target point of the target device more flexible.

Description

Technical Field

[0001] The present application relates to the field of positioning technology, and in particular to a positioning method, device, electronic device, and readable storage medium. Background Art

[0002] In the field of robotic automation, there is usually a certain deviation between the actual coordinates of the item to be picked up and the coordinates of the item to be picked up in the robot's coordinate system. Therefore, before picking up the item, it is necessary to calculate the actual position of the item to be picked up. For example, in terms of nucleic acid testing, nucleic acid testing is currently still mainly based on manual testing by medical personnel, with low testing efficiency and a high risk of infection during the testing process. Therefore, it is very important to realize automated nucleic acid testing. However, in actual work, there will always be a certain deviation between the actual placement of the item to be picked up and the robot's built-in coordinate system. Before the robot picks up the test item, the coordinates of the item to be picked up need to be mapped to the coordinate system where the robot is located to obtain the actual coordinates of the item to be picked up. However, the actual coordinates of the item to be picked up are generally obtained through mechanical hard positioning, which is inflexible. Summary of the Invention

[0003] In view of this, the purpose of the embodiments of the present application is to provide a positioning method, device, electronic device and readable storage medium, which can flexibly locate an item to be picked up.

[0004] In the first aspect, an embodiment of the present application provides a positioning method, which calculates the first coordinates of a target point of a target device through a visual algorithm, where the first coordinates are the coordinates of the target point on a first plane in a calibration coordinate system; calculates the second coordinates of the target point of the target device through a point laser algorithm, where the second coordinates are the coordinates of the target point on a first plane in the calibration coordinate system; and determines the target coordinates of the target point of the target device based on the first coordinates and the second coordinates.

[0005] In the above implementation process, the coordinates of the target point on the first plane in the calibration coordinate system are calculated by the visual algorithm, and the coordinates of the target point on the second plane in the calibration coordinate system are calculated by the point laser algorithm. The coordinate values of each target point can be accurately calculated to obtain the precise target coordinates of the target point. In addition, when the visual algorithm is performed, the real-time image data of the target device is obtained by the image acquisition device for processing. When the point laser is used for processing, the partial coordinates of the reference point are directly obtained by the laser device for processing. Since both the laser device and the image acquisition device can be flexibly installed and applied and are suitable for various scenarios, the visual algorithm and the point laser algorithm are both relatively flexible positioning methods. The flexibility of the positioning method is increased by the visual algorithm and the point laser algorithm.

[0006] In combination with the first aspect, an embodiment of the present application provides a first possible implementation method of the first aspect, wherein: the calculation of the first coordinates of the target point of the target device through a visual algorithm includes: matching the captured image of the target device with an image template, wherein the image template is an image of the target device calibrated with a coordinate system; if the match is successful, processing the marker point through a feature matching algorithm to output the actual coordinates of the marker point, wherein the marker point is a set marker point on the target device, and the actual coordinates of the marker point are the coordinates of the marker point on the image of the target device; and calculating the first coordinates of the target point of the target device based on the actual coordinates of the marker point.

[0007] In the above implementation process, the image of the target device is compared with the image template to determine the matching status of the image of the target device and the image template. If the image of the target device and the image template match successfully, the image template can be determined to be the image template of the target device. The actual coordinates of the marker point are then calculated using a feature matching algorithm to obtain the actual coordinates of the marker point. Since the marker point and the target point have a fixed positional relationship, after determining the actual coordinate value of the marker point, the actual coordinates of the target point can be calculated based on the positional relationship between the actual coordinates of the marker point and the target point. The coordinate value of the target point is calculated using the fixed coordinate value of the marker point. By matching the image information to the coordinate information, the calculation method is simplified, and the calculation efficiency and accuracy are improved.

[0008] In combination with the first possible implementation of the first aspect, the embodiment of the present application provides a second possible implementation of the first aspect, wherein: the calculation of the first coordinates of the target device based on the actual coordinates of the marking point includes: calculating the coordinate difference based on the actual coordinates of the marking point and the marking point template coordinates, the marking point template coordinates are the coordinates of the marking point on the image template; determining the first coordinates of the target point position of the target device through the coordinate difference.

[0009] In the above implementation, the coordinate difference is calculated by calculating the actual coordinates of the marker point and the coordinates of the marker template. Since the positions of the marker point and the target point are relatively fixed, the coordinate difference of the marker point can also be directly used as the coordinate difference of the target point to directly calculate the actual coordinates of the target point. The actual coordinates of the target point are calculated based on the coordinates of the marker point and the coordinate difference. Since all calculations and processing are based on coordinates, they are not restricted by usage scenarios, equipment, etc., which increases the flexibility of the positioning method.

[0010] In combination with the second possible implementation of the first aspect, an embodiment of the present application provides a third possible implementation of the first aspect, wherein, before calculating the second coordinate of the target point of the target device through the point laser algorithm, the method also includes: obtaining multiple reference points of the target device; calculating the first coordinate axis values and the second coordinate axis values of the multiple reference points through a visual algorithm.

[0011] In the above implementation process, based on the first coordinate of the target point calculated by the visual algorithm, the first coordinate axis value and the second coordinate axis value of the reference point are calculated. Since a series of data such as the coordinate difference has been obtained when calculating the target point, when calculating the first coordinate axis value and the second coordinate axis value of the reference point, it is only necessary to compensate the coordinate difference to the template coordinate of the reference point, which simplifies the calculation of the first coordinate axis value and the second coordinate axis value of the reference point and improves the calculation accuracy and efficiency.

[0012] In combination with the second possible implementation of the first aspect, the embodiment of the present application provides a fourth possible implementation of the first aspect, wherein the obtaining of the second coordinate of the target point of the target device through the point laser algorithm includes: obtaining the third coordinate axis values of multiple reference points and the third coordinate axis value of the target point through the point laser; obtaining the second coordinate of the target point of the target device according to the reference point coordinates of multiple reference points and the third coordinate axis value of the target point, the reference point coordinates of the reference point including the first coordinate axis value of the reference point, the second coordinate axis value of the reference point and the third coordinate axis value of the reference point.

[0013] In the above implementation process, since the visual algorithm can only calculate the value on a plane and it is difficult to calculate the value in the height direction, the point laser is used to obtain the values of the height direction of multiple reference points and target points, which can well make up for the shortcomings of the visual algorithm and obtain the coordinate values of the reference points and target points in various directions. Based on the complete coordinate value of the reference point, the second rotation angle and the third rotation angle of the target point can be further calculated. The complete coordinate value of the target point is calculated by combining the above two methods, which improves the accuracy of the actual coordinates of the target point. Since both the image acquisition equipment and the laser equipment can be easily installed and the replacement cost is low when the scene is frequently changed, the flexibility of the use of this positioning method is improved while also reducing the cost.

[0014] In combination with the fourth possible implementation of the first aspect, an embodiment of the present application provides a fifth possible implementation of the first aspect, wherein obtaining multiple reference point positions of the target device includes: obtaining a first reference point, a second reference point and a third reference point, the first reference point and the second reference point are at the same position on the first coordinate axis, and the first reference point and the third reference point are at the same position on the second coordinate axis.

[0015] In the above implementation, by setting the first coordinate axis of the first reference coordinate to be the same as the first coordinate axis of the second reference coordinate, and the second coordinate axis of the first reference coordinate to be the same as the second coordinate axis of the third reference coordinate, the selected first, second, and third reference coordinates are all interrelated, representative, and easy to calculate. Therefore, calculations can be performed more simply and conveniently, simplifying the calculations and improving efficiency and accuracy.

[0016] In combination with the fifth possible implementation of the first aspect, the embodiment of the present application provides a sixth possible implementation of the first aspect, wherein the obtaining the second coordinates of the target point of the target device based on the multiple reference point coordinates and the third coordinate axis value of the target point includes: calculating a first angle based on the first reference point coordinates and the second reference point coordinates, the first angle being the angle between the line connecting the first reference point and the second reference point and the second coordinate axis; calculating a second angle based on the first reference point coordinates and the third reference point coordinates, the second angle being the angle between the line connecting the first reference point and the third reference point and the first coordinate axis; obtaining the second coordinates of the target point of the target device based on the first angle, the second angle and the third coordinate axis value of the target point.

[0017] In the above implementation process, since the coordinates of the first reference point and the second reference point are in the same position on the first coordinate axis, according to the mathematical relationship, it can be known that the angle between the line connecting the first reference point and the second reference point and the second coordinate axis is the second rotation angle of the target point. Since the coordinates of the first reference point and the third reference point are in the same position on the second coordinate axis, according to the mathematical relationship, it can be known that the angle between the line connecting the first reference point and the third reference point and the first coordinate axis is the third rotation angle of the target point. By calculating the angle between the reference point line and the coordinate axis, the second rotation angle and the third rotation angle of the target point can be further obtained. Since the entire process only involves the calculation and conversion of numerical values, it can be applied in various scenarios, which improves the flexibility of the method.

[0018] In the second aspect, an embodiment of the present application also provides a positioning device, including: a first calculation module: used to calculate the first coordinates of the target point of the target device through a visual algorithm; a second calculation module: used to calculate the second coordinates of the target point of the target device through a point laser algorithm; a processing module: used to obtain the target coordinates of the target point of the target device based on the first coordinates and the second coordinates.

[0019] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method in the above-mentioned first aspect, or any possible implementation of the first aspect.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the positioning method in the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0021] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the following embodiments are given in conjunction with the accompanying drawings for detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A block diagram of an execution device provided in an embodiment of the present application;

[0024] Figure 2 A flowchart of the positioning method provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a coordinate system provided in an embodiment of the present application;

[0026] Figure 4 Flowchart of step 201 of the positioning method provided in an embodiment of the present application;

[0027] Figure 5 Flowchart of step 202 of the positioning method provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the functional modules of the positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0031] With the rapid development of automation, machines are increasingly replacing simple manual labor in areas such as cargo handling, material handling, material movement, and express delivery sorting. Robots can effectively replace manual labor by performing tasks such as sorting, placing, moving, and transporting goods, materials, or express deliveries. However, when using robots for handling and placing items, the actual position of the item must be consistent with the robot's internal coordinate system. This ensures that the robot can accurately retrieve the target item during handling and other operations.

[0032] Currently, the most common positioning method is mechanical hard positioning, which is mainly based on various positioning design objects such as positioning plates, positioning pins, and material dowel pins for individual materials. Since the positioning parts of the above-mentioned mechanical hard positioning are designed based on the properties of the material to be positioned, after the material is replaced, the original positioning parts are not suitable for the new material, so new positioning parts need to be redesigned. The above-mentioned mechanical hard positioning method has a strong restrictive effect on the material, and when there are a lot of materials, the number of positioning parts that need to be replaced is also large. This is not only prone to waste, but also inflexible to use. The positioning parts need to be changed in a timely manner according to the changes in the material.

[0033] In view of this, the inventors of the present application have proposed a new positioning method, which uses a visual algorithm to obtain an image of the object to be positioned, matches and calculates a portion of the coordinates of the object to be positioned, then uses a point laser algorithm to obtain the coordinates of the reference point, and calculates another portion of the coordinates of the object to be positioned based on the coordinates of the reference point. By combining the image acquisition device and the laser device, image data and partial point data are obtained, and then the actual coordinates of the target device are calculated based on the obtained image data and partial point data. Since both the image acquisition device and the laser device can be flexibly installed and the replacement cost is low when the scene is frequently changed, the positioning method of the present application can be flexibly used in various scenarios.

[0034] To facilitate understanding of this embodiment, the operating environment for executing a positioning method disclosed in the embodiment of this application is first introduced in detail.

[0035] The positioning system of the present application may include an execution device, a target device and a processing device. The target device is used to place items and may be a loading platform, a loading trolley, a robot, an automatic navigation device, etc. The execution device is used to perform automatic operations and may be a robot, a robotic arm, a stacker, a filling machine, etc. The processing device is used to determine the actual position of the target point and may be a computer, a programmable controller, a tablet computer, a smart phone, a personal digital assistant (PDA), etc. The processing device may be connected to the execution device for communication or interaction via a network. The processing device may also be connected to the target device for communication or interaction via a network. The target device may be connected to the execution device for communication or interaction via a network.

[0036] It can be understood that the execution device and the processing device can be two independent devices, or two parts of the same physical device corresponding to different functions. For example, the processing device can be the system in the device that performs automatic operations that is responsible for external communication, interaction, and support for programming development, while the execution device can be the part of the device that performs automatic operations that is responsible for automatic operations. The target device and the processing device can be two independent devices, or two parts of the same physical device corresponding to different functions. For example, the processing device can be the system in the device that performs automatic operations that is responsible for external communication, interaction, and support for programming development, while the target device can be the part used to place objects. The target device, the execution device, and the processing device can be three independent devices, or three parts of the same physical device corresponding to different functions. For example, the processing device can be the system in the device that performs automatic operations that is responsible for external communication, interaction, and support for programming development, while the target device can be the part used to place objects, and the execution device can be the part of the device that performs automatic operations that is responsible for automatic operations.

[0037] If the target device, the execution device, and the processing device are three independent devices, a loading platform is provided on the target device for placing multiple items. The processing device is provided with a program to execute the positioning method described in this application and transmit the actual coordinates of the target point on the target device to the execution device. After receiving the actual coordinates of the target point, the execution device performs a picking task on the target device based on the actual coordinates of the target point.

[0038] If the execution device and the processing device are two parts of the same physical device with different functions, a loading platform is provided on the target device for placing multiple items. The execution device is provided with a program to execute the positioning method of this application, obtain the actual coordinates of the target point on the target device, and perform the picking task on the target device based on the actual coordinates of the target point.

[0039] If the target device and the processing device are two parts of the same physical device with different functions, the target device is provided with a loading platform for placing multiple items, and the target device is provided with a relevant program to execute the positioning method of this application and transmit the actual coordinates of the target point on the target device to the execution device. After receiving the actual coordinates of the target point, the execution device performs the picking task on the target device based on the actual coordinates of the target point.

[0040] The positioning system also includes an image acquisition device and a laser device. The image acquisition device is used to acquire image information, such as an image template and an image of the target device, and can be installed on either the execution device or the target device. The laser device is used to acquire partial coordinate values of a point and can be installed on either the execution device or the target device. The image acquisition device can communicate with the processing device via a network for data communication or interaction. The laser device can also communicate with the processing device via a network for data communication or interaction.

[0041] It can be understood that the positioning method of the present application can be applied to nucleic acid detection robots, loading robots, handling robots, hotel robots, service robots, etc.

[0042] For example, if this positioning method is applicable to a nucleic acid detection robot, the target device can be a stage with an area specifically designed for placing test samples, a tip box, a liquid holder, etc. During nucleic acid testing, before the nucleic acid detection robot retrieves the test sample, the processing device uses an image acquisition device to capture an image of the target device and matches it with an internally stored template of the target device image. If a successful match is found, the captured image is deemed to be an image of the target device. A feature matching algorithm is then used to process the marker points to obtain their actual coordinates. Since the marker points maintain a fixed positional relationship with other points on the target device, the coordinates of a portion of the target sample can be calculated based on these coordinates. Furthermore, a laser device uses a point laser to capture the coordinates of a reference point and the coordinates of another portion of the target sample. The coordinates of the other portion of the target sample are calculated based on the mathematical relationship and the coordinates of the reference point. The actual coordinates of the target sample are determined based on these two coordinates and transmitted to the nucleic acid detection robot. Based on these actual coordinates, the nucleic acid detection robot controls the gripper to move to the actual coordinates of the target sample for retrieval.

[0043] To facilitate understanding of this embodiment, a processing device for executing a positioning method disclosed in an embodiment of the present application is described in detail below.

[0044] like Figure 1 FIG. 1 is a block diagram of a processing device 100. The processing device 100 may include a memory 111 and a processor 112. A person skilled in the art will appreciate that Figure 1 The structure shown is only for illustration and does not limit the structure of the processing device 100. For example, the processing device 100 may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0045] The memory 111 and processor 112 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines. The processor 112 is configured to execute the executable modules stored in the memory.

[0046] The memory 111 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 111 is used to store programs, and the processor 112 executes the programs after receiving an execution instruction. The method executed by the processing device 100 defined by the process disclosed in any embodiment of the present application can be applied to the processor 112 or implemented by the processor 112.

[0047] The memory 111 may be used to store image templates, marker template coordinates, and data such as images of target devices, actual marker coordinates, coordinate differences, and reference point coordinates.

[0048] The processor 112 may be an integrated circuit chip with signal processing capabilities. The processor 112 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0049] The processor 112 is configured to obtain information such as the image template, the marker template coordinates, and the reference point coordinates from the memory 111 to execute the positioning method of the present application, and to send the obtained actual coordinates of the target point to the memory 111 for storage.

[0050] The processing device 100 in this embodiment can be used to execute each step in each method provided in the embodiments of the present application. The implementation process of the positioning method is described in detail below through several embodiments.

[0051] See also Figure 2 , is a flow chart of the positioning method provided by the embodiment of the present application. Figure 1 The specific process shown is explained in detail.

[0052] Step 201: Calculate the first coordinates of a target point of a target device using a visual algorithm.

[0053] A visual algorithm is an algorithm that uses an image acquisition device to capture an image of a target device, extract feature points from the image, obtain the coordinates of the feature points, and then calculate the coordinates of the target point based on the coordinates of the feature points. For example, this visual algorithm may include the SIFT algorithm, the SURF algorithm, the ORB algorithm, etc.

[0054] The target device here is a device that can be used to place items. The target device can be a loading platform, a loading cart, a robot, an automatic navigation device, etc. The target point is one or more points in the loading area of the target device. The first coordinate is the coordinate of the target point on the first plane in the calibration coordinate system. The first plane in the calibration coordinate system here can be the plane where the X-axis and Y-axis in the calibration coordinate system are located, such as Figure 3 As shown, if Figure 3 The coordinate system in is the calibration coordinate system, then the first plane can be the horizontal plane where the X axis and the Y axis are located. The first plane in the calibration coordinate system here can also be the plane parallel to the Z axis in the calibration coordinate system. If Figure 3 The coordinate system in is a calibration coordinate system, and the first plane may be a vertical plane where the Z axis is located. It can be understood that the first plane in the calibration coordinate system may be selected according to actual needs.

[0055] Step 202: Calculate the second coordinate of the target point of the target device using a point laser algorithm.

[0056] The point laser algorithm here is an algorithm that obtains partial coordinate information of the reference point and partial coordinate information of the target point through point laser, and calculates the coordinates of the target point on the second plane by calculating the partial coordinate information of the reference point. The second coordinate is the coordinate of the target point on the second plane in the calibration coordinate system. The reference point is one or more points in the loading area of the target device. The second plane in the calibration coordinate system here can be the plane where the X-axis and Y-axis in the calibration coordinate system are located, such as Figure 3 As shown, if Figure 3 The coordinate system in is the calibration coordinate system, then the second plane can be the horizontal plane where the X axis and the Y axis are located. The second plane in the calibration coordinate system here can also be the plane parallel to the Z axis in the calibration coordinate system. If Figure 3The coordinate system in is a calibration coordinate system, and the second plane may be a vertical plane where the Z axis is located. It can be understood that the second plane in the calibration coordinate system may be selected according to actual needs.

[0057] Step 203 : determining the target coordinates of the target point of the target device according to the first coordinates and the second coordinates.

[0058] After obtaining the first and second coordinates from the above calculations, the first and second coordinates can be directly combined to obtain the target coordinates of the target point of the target device. For example, if the first coordinates obtained from the above calculations are (X1, Y1, Rz1) and the second coordinates are (Z1, Rx1, Ry1), then the target coordinates are the combination of the first and second coordinates, i.e., (X1, Y1, Z1, Rx1, Ry1, Rz1).

[0059] In the above implementation process, the coordinates of the target point on the first plane in the calibration coordinate system are calculated by the visual algorithm, and then the coordinates of the target point on the first plane in the calibration coordinate system are calculated by the point laser algorithm. The coordinate values of each target point can be accurately calculated to obtain the precise target coordinates of the target point. In addition, when the visual algorithm is performed, the real-time image data of the target device is obtained by the image acquisition device for processing, and when the point laser is used for processing, the partial coordinates of the reference point are directly obtained by the laser device for processing. Since both the laser device and the image acquisition device can be flexibly installed and applied and are suitable for various scenarios, the visual algorithm and the point laser algorithm are both relatively flexible positioning methods, and the flexibility of the positioning method is increased by the visual algorithm and the point laser algorithm.

[0060] In one possible implementation, Figure 4 As shown, step 201 includes: steps 2011-2013.

[0061] Step 2011: Match the captured image of the target device with the image template.

[0062] When matching the image of the target device with the image template, it is necessary to match the marking points on the image of the target device with the marking points on the image template. Therefore, before matching the image of the target device with the image template, it is necessary to first set a marking point on the target device, and take a photo with the feature of the marking point to create an image template. After the execution device reaches the default position, it is necessary to take a photo of the marking point to obtain the actual image of the marking point. By matching the actual image of the marking point with the image template, a successful match indicates that the marking point exists in the collected image of the target device. The marking point here is a point on the target device used for marking. The actual coordinates of the marking point are the coordinates of the marking point on the image of the target device. It can be understood that the marking point can be a point set at a set position on the target device specifically for marking, or the marking point can be a fixed point selected in the loading area on the target device. The marking point has a fixed positional relationship with the point in the loading area on the target device.

[0063] The image template here is an image of the target device calibrated with the coordinate system. The image of the target device is an image of the target device captured by the executing device at a set position before the picking task is performed. This image of the target device is a real-time image of the target device. This image of the target device can be acquired by a capture device on the executing device, or by a capture device located at a set position.

[0064] Based on the above embodiment, the matching method may be: matching the image of the target device with the image template by sliding the image template on the image of the target device.

[0065] Step 2012: If the matching is successful, the marker point is processed by a feature matching algorithm to output the actual coordinates of the marker point.

[0066] The feature matching algorithm here can be a SURF feature matching algorithm. The SURF feature matching algorithm is an algorithm for extracting a feature point. To achieve invariance to directional rotation and scale transformation, the output of the SURF feature matching algorithm may include the location information and orientation information of the feature point. The location information includes the location of the feature point on the image, including the coordinate values of the first coordinate axis and the coordinate values of the second coordinate axis. The orientation information includes the angle of rotation of the feature point about the third coordinate axis.

[0067] In addition, the image of the target device may fail to match the image template. If the match fails, an alarm will be issued to indicate the match failure.

[0068] Step 2013: Calculate the first coordinates of the target point of the target device according to the actual coordinates of the marking point.

[0069] The first coordinates here may include a first coordinate axis coordinate value, a second coordinate axis coordinate value, and a rotation angle (ie, a first rotation angle) of the target point about a third coordinate axis.

[0070] It is understandable that since the positions of the marker point and the target point on the target device are relatively fixed, the first coordinates of the target point can be calculated based on the actual coordinates of the marker point and the fixed value between the target point and the marker point. For example, if the actual coordinates of the marker point are calculated to be (X1, Y1, Rz1), and the actual coordinates of the marker point and the fixed value between the target point and the marker point are (X, Y, Rz), then the fixed value is compensated to the actual coordinates of the marker point to obtain the first coordinates of the target point (X2, X2, Rz2).

[0071] It is understandable that the first coordinate of the target point can also be calculated by calculating the coordinate difference of the marker point. For example, if the actual coordinates of the marker point are calculated to be (X3, X3, Rz3), and the template of the marker point is (X4, Y4, Rz4), then the coordinate difference between the actual coordinates of the marker point and the template coordinates is (ΔX, ΔY, ΔRz). This coordinate difference is compensated to the template coordinates of the target point to obtain the first coordinate of the target point (X5, Y5, Rz5).

[0072] In the above implementation process, the image of the target device is compared with the image template to determine the matching status of the image of the target device and the image template. If the image of the target device and the image template match successfully, the image template can be determined to be the image template of the target device. The actual coordinates of the marker point are then calculated using a feature matching algorithm to obtain the actual coordinates of the marker point. Since the marker point and the target point have a fixed positional relationship, after determining the actual coordinate value of the marker point, the actual coordinates of the target point can be calculated based on the positional relationship between the actual coordinates of the marker point and the target point. The coordinate value of the target point is calculated using the fixed coordinate value of the marker point. By matching the image information to the coordinate information, the calculation method is simplified, and the calculation efficiency and accuracy are improved.

[0073] In a possible implementation, step 2013 includes: calculating a coordinate difference value based on the actual coordinates of the marking point and the coordinates of the marking point template, and determining a first coordinate of the target point of the target device through the coordinate difference value.

[0074] The marker template coordinates here are the coordinates of the marker point on the image template.

[0075] It can be understood that the coordinate difference can be obtained directly by subtracting the actual coordinates of the marking point from the marking point template coordinates, the coordinate difference can also be obtained by performing difference processing on the actual coordinates of the marking point and the marking point template coordinates, and the coordinate difference can also be obtained by combining the above two methods.

[0076] In the above implementation, the coordinate difference is calculated by calculating the actual coordinates of the marker point and the coordinates of the marker template. Since the positions of the marker point and the target point are relatively fixed, the coordinate difference of the marker point can also be directly used as the coordinate difference of the target point to directly calculate the actual coordinates of the target point. The actual coordinates of the target point are calculated based on the coordinates of the marker point and the coordinate difference. Since all calculations and processing are based on coordinates, they are not restricted by usage scenarios, equipment, etc., which increases the flexibility of the positioning method.

[0077] In a possible implementation, obtaining the first coordinate of the target point of the target device through the coordinate difference includes: compensating the coordinate difference to the template coordinate of the target point of the target device to obtain the first coordinate of the target point of the target device.

[0078] The template coordinates of the target point here are the coordinates of the target point in the image template.

[0079] In the above implementation process, by compensating the coordinate difference to the template coordinates of the target point, the deviation between the template coordinates and the actual coordinates of the target point is eliminated, and a more accurate actual coordinate of the target point is obtained, thereby improving the accuracy of the first coordinate.

[0080] In a possible implementation, before step 202, the positioning method further includes: acquiring multiple reference points of the target device; and calculating first coordinate axis values and second coordinate axis values of the multiple reference points by a visual algorithm.

[0081] The reference point here is a point in the target device. The reference point can be obtained by the target device, the execution device, the processing device, or an external input to the target device, the processing device, or the execution device.

[0082] First, the first coordinate axis values and the second coordinate axis values of multiple reference points are calculated through a visual algorithm. Specifically, the coordinate difference is calculated based on the actual coordinates of the marked point and the template coordinates of the marked point, and the coordinate difference is compensated to the template coordinates of multiple reference points to obtain the first coordinate axis values and the second coordinate axis values of multiple reference points.

[0083] In the above implementation process, based on the first coordinate of the target point calculated by the visual algorithm, the first coordinate axis value and the second coordinate axis value of the reference point are calculated. Since a series of data such as the coordinate difference has been obtained when calculating the target point, when calculating the first coordinate axis value and the second coordinate axis value of the reference point, it is only necessary to compensate the coordinate difference to the template coordinate of the reference point, which simplifies the calculation of the first coordinate axis value and the second coordinate axis value of the reference point and improves the calculation accuracy and efficiency.

[0084] In one possible implementation, Figure 5 As shown, step 202 includes: steps 2021-2022.

[0085] Step 2021: Acquire the third coordinate axis values of multiple reference points and the third coordinate axis value of the target point by using a point laser.

[0086] On the basis of the above embodiment, the point laser can obtain the third coordinate axis value of the reference point, and the point laser can also obtain the third coordinate axis value of the target point.

[0087] Step 2022: Acquire the second coordinate of the target point of the target device according to the reference point coordinates of the multiple reference points and the third coordinate axis value of the target point.

[0088] The reference point coordinates of the reference point here include a first coordinate axis value of the reference point, a second coordinate axis value of the reference point, and a third coordinate axis value of the reference point.

[0089] In the above embodiment, the first, second, and third axis values of multiple reference points are obtained. Based on these first, second, and third axis values, the second and third rotation angles of the target point can be calculated. Furthermore, because the point laser acquires the third axis value of the target point, the second coordinate of the target point can be obtained based on the second rotation angle, third rotation angle, and third axis value.

[0090] Since the vision algorithm can only calculate the first and second axis values of the reference point, in order to obtain the complete coordinates of the reference point, it is necessary to use a point laser to obtain the third axis values of the target point and multiple reference points. The above point laser and vision algorithm can jointly obtain the first, second, and third axis values of multiple reference points and the third axis value of the target point.

[0091] In the above implementation process, after the actual coordinates of the marked point are calculated using the aforementioned visual algorithm, the first coordinates of the reference point can also be calculated using the visual algorithm, similar to the algorithm for the first coordinates of the target point. Because the first coordinate does not contain the value of the third coordinate axis, the third coordinate axis values of the target point and the reference point are obtained using a point laser to obtain the values of each coordinate axis of the reference point. The point laser algorithm compensates for the shortcomings of the visual algorithm and obtains values that cannot be obtained by the visual algorithm. This facilitates the further calculation of the actual coordinates of the target point, simplifies the calculation difficulty, and improves the efficiency and accuracy of the calculation of the actual coordinates of the target point.

[0092] In a possible implementation, step 2021 includes: acquiring a first reference point, a second reference point, and a third reference point.

[0093] It is understood that the first reference point, the second reference point, and the third reference point can be inputted externally to an execution device, a processing device, or a target device. The first reference point, the second reference point, and the third reference point can be obtained by a target device according to a set rule. The first reference point, the second reference point, and the third reference point can also be obtained by an execution device according to a set rule. The first reference point, the second reference point, and the third reference point can also be obtained by a processing device according to a set rule.

[0094] The first reference point and the second reference point have the same position on the first coordinate axis, and the first reference point and the third reference point have the same position on the second coordinate axis. For example, if the first reference point and the second reference point have the same X axis, then the first reference point and the third reference point have the same Y axis. If the first reference point and the second reference point have the same Y axis, then the first reference point and the third reference point have the same X axis. Figure 3 As shown, if Figure 3 The coordinate system shown in is the coordinate system of the target device. Point B shown in the figure is the first reference point, point A is the second reference point, and point C is the third reference point.

[0095] In the above implementation, by setting the first coordinate axis of the first reference coordinate to be the same as the first coordinate axis of the second reference coordinate, and the second coordinate axis of the first reference coordinate to be the same as the second coordinate axis of the third reference coordinate, the selected first, second, and third reference coordinates are all interrelated, representative, and easy to calculate. Therefore, calculations can be performed more simply and conveniently, simplifying the calculations and improving efficiency and accuracy.

[0096] In one possible implementation, step 2023 includes: calculating a first angle based on the coordinates of the first reference point and the coordinates of the second reference point, calculating a second angle based on the coordinates of the first reference point and the coordinates of the third reference point, and obtaining the second coordinates of the target point of the target device based on the first angle, the second angle, and the coordinates of multiple reference points.

[0097] Among them, the first angle is the angle between the line connecting the first reference point and the second reference point and the second coordinate axis. The second angle is the angle between the line connecting the first reference point and the third reference point and the first coordinate axis. For example, if the X-axis of the first reference point and the second reference point is the same, and the Y-axis of the first reference point and the third reference point is the same. Then the first angle is the angle between the line connecting the first reference point and the second reference point and the Y-axis, and the second angle is the angle between the line connecting the first reference point and the third reference point and the X-axis. If the Y-axis of the first reference point and the second reference point is the same, and the X-axis of the first reference point and the third reference point is the same. Then the first angle is the angle between the line connecting the first reference point and the second reference point and the X-axis, and the second angle is the angle between the line connecting the first reference point and the third reference point and the Y-axis.

[0098] Among them, the first angle corresponds to the second rotation angle of the target point, and the second angle corresponds to the third rotation angle of the target point. For example, if the first angle is the angle between the line connecting the first reference point and the second reference point and the Y-axis, and the second angle is the angle between the line connecting the first reference point and the third reference point and the X-axis. Then the first angle corresponds to Rx of the target point, and the second angle corresponds to Ry of the target point. If the first angle is the angle between the line connecting the first reference point and the second reference point and the X-axis, and the second angle is the angle between the line connecting the first reference point and the third reference point and the Y-axis. Then the first angle corresponds to Ry of the target point, and the second angle corresponds to Rx of the target point.

[0099] In the above implementation process, since the coordinates of the first reference point and the second reference point are in the same position on the first coordinate axis, according to the mathematical relationship, it can be known that the angle between the line connecting the first reference point and the second reference point and the second coordinate axis is the second rotation angle of the target point. Since the coordinates of the first reference point and the third reference point are in the same position on the second coordinate axis, according to the mathematical relationship, it can be known that the angle between the line connecting the first reference point and the third reference point and the first coordinate axis is the third rotation angle of the target point. By calculating the angle between the reference point line and the coordinate axis, the second rotation angle and the third rotation angle of the target point can be further obtained. Since the entire process only involves the calculation and conversion of numerical values, it can be applied in various scenarios, which improves the flexibility of the method.

[0100] Based on the same application concept, a positioning device corresponding to the positioning method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to that of the aforementioned positioning method embodiment, the implementation of the device in this embodiment can refer to the description in the embodiment of the above method, and the repeated parts will not be repeated.

[0101] See also Figure 6 , is a functional module diagram of the positioning device provided in the embodiment of the present application. Each module in the positioning device in this embodiment is used to perform each step in the above method embodiment. The positioning device includes a first calculation module 301, a second calculation module 302, and a processing module 303; wherein,

[0102] The first calculation module 301 is used to calculate the first coordinates of the target point of the target device through a visual algorithm, where the first coordinates are the coordinates of the target point on a first plane in the calibration coordinate system.

[0103] The second calculation module 302 is used to calculate the second coordinates of the target point of the target device by using a point laser algorithm, where the second coordinates are the coordinates of the target point on a second plane in the calibration coordinate system.

[0104] The processing module 303 is configured to obtain target coordinates of a target point of the target device according to the first coordinates and the second coordinates.

[0105] In one possible implementation, the first calculation module 301 is further used to: match the captured image of the target device with an image template, where the image template is the image of the target device calibrated with the coordinate system; if the match is successful, process the marker point through a feature matching algorithm to output the actual coordinates of the marker point, where the marker point is a set marker point position on the target device, and the actual coordinates of the marker point are the coordinates of the marker point on the image of the target device; and calculate the first coordinates of the target point position of the target device based on the actual coordinates of the marker point.

[0106] In one possible implementation, the first calculation module 301 is specifically used to: calculate a coordinate difference based on the actual coordinates of the marker point and the marker point template coordinates, where the marker point template coordinates are the coordinates of the marker point on the image template; and obtain the first coordinate of the target point of the target device through the coordinate difference.

[0107] In a possible implementation, the first calculation module 301 is specifically configured to: obtain multiple reference points of the target device; and calculate first coordinate axis values and second coordinate axis values of the multiple reference points using a visual algorithm.

[0108] In one possible implementation, the first calculation module 301 is specifically configured to: compensate the coordinate difference to the template coordinates of the target point of the target device to obtain the first coordinate of the target point of the target device. In one possible implementation, the second calculation module 302 is further configured to: obtain the third coordinate axis values of multiple reference points and the third coordinate axis value of the target point using a point laser; and obtain the second coordinate of the target point of the target device based on the reference point coordinates of the multiple reference points and the third coordinate axis value of the target point, where the reference point coordinates of the reference point include the first coordinate axis value of the reference point, the second coordinate axis value of the reference point, and the third coordinate axis value of the reference point.

[0109] In one possible implementation, the second calculation module 302 is specifically used to obtain a first reference point, a second reference point, and a third reference point, wherein the first reference point and the second reference point have the same position on the first coordinate axis, and the first reference point and the third reference point have the same position on the second coordinate axis.

[0110] In one possible implementation, the second calculation module 302 is specifically used to: calculate a first angle based on the coordinates of the first reference point and the coordinates of the second reference point, where the first angle is the angle between the line connecting the first reference point and the second reference point and the second coordinate axis; calculate a second angle based on the coordinates of the first reference point and the coordinates of the third reference point, where the second angle is the angle between the line connecting the first reference point and the third reference point and the first coordinate axis; and obtain the second coordinate of the target point of the target device based on the first angle, the second angle and the third coordinate axis value of the target point.

[0111] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the positioning method described in the above method embodiment are executed.

[0112] The computer program product of the positioning method provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the positioning method described in the above method embodiments. For details, please refer to the above method embodiments and will not be repeated here.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0114] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0115] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0116] The foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0117] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A positioning method, characterized in that: include: Calculate first coordinates of a target point of the target device by a visual algorithm, where the first coordinates are coordinates of the target point on a first plane in a calibration coordinate system; Calculating a second coordinate of a target point of the target device using a point laser algorithm, where the second coordinate is the coordinate of the target point on a second plane in the calibration coordinate system; wherein the point laser algorithm is an algorithm that obtains partial coordinate information of a reference point and partial coordinate information of a target point using a point laser, and calculates the partial coordinate information of the reference point to obtain the coordinate of the target point on the second plane; Determine the target coordinates of the target point of the target device according to the first coordinates and the second coordinates; Before calculating the second coordinates of the target point of the target device by the point laser algorithm, the method further includes: Acquire multiple reference points of the target device; Calculate the first coordinate axis values of the plurality of reference points and the second coordinate axis values of the reference points by a visual algorithm; The method of obtaining the second coordinate of the target point of the target device by using a point laser algorithm includes: Acquire the third coordinate axis values of the plurality of reference points and the third coordinate axis values of the target point by using a point laser; The second coordinate of the target point of the target device is obtained according to the reference point coordinates of multiple reference points and the third coordinate axis value of the target point, wherein the reference point coordinates of the reference point include the first coordinate axis value of the reference point, the second coordinate axis value of the reference point and the third coordinate axis value of the reference point.

2. The method according to claim 1, characterized in that Calculating the first coordinate of the target point of the target device by using a visual algorithm includes: Matching the captured image of the target device with an image template, wherein the image template is an image of the target device with a calibrated coordinate system; If the match is successful, the marker point is processed by the feature matching algorithm to output the actual coordinates of the marker point, where the marker point is the set marker point position on the target device and the actual coordinates of the marker point are the coordinates of the marker point on the image of the target device; Calculate the first coordinates of the target point of the target device according to the actual coordinates of the marking point.

3. The method according to claim 2, characterized in that The calculating the first coordinates of the target device according to the actual coordinates of the marking point includes: Calculating a coordinate difference between the actual coordinates of the marker point and the marker point template coordinates, where the marker point template coordinates are the coordinates of the marker point on the image template; The first coordinate of the target point of the target device is determined by the coordinate difference.

4. The method according to claim 1, wherein The acquiring of multiple reference points of the target device includes: A first reference point, a second reference point, and a third reference point are obtained, wherein the first reference point and the second reference point are at the same position on the first coordinate axis, and the first reference point and the third reference point are at the same position on the second coordinate axis.

5. The method according to claim 4, characterized in that The step of obtaining the second coordinate of the target point of the target device according to the coordinates of the plurality of reference points and the third coordinate axis value of the target point includes: A first angle is calculated based on the coordinates of the first reference point and the second reference point, where the first angle is the angle between a line connecting the first reference point and the second reference point and the second coordinate axis; A second angle is calculated based on the coordinates of the first reference point and the third reference point, where the second angle is the angle between the line connecting the first reference point and the third reference point and the first coordinate axis; The second coordinate of the target point of the target device is obtained according to the first angle, the second angle and the third coordinate axis value of the target point.

6. A positioning device, characterized in that: include: A first calculation module: configured to calculate a first coordinate of a target point of a target device by using a visual algorithm, where the first coordinate is a coordinate of the target point on a first plane in a calibration coordinate system; A second calculation module is configured to calculate the second coordinates of a target point of the target device by using a point laser algorithm, where the second coordinates are the coordinates of the target point on a second plane in the calibration coordinate system; wherein the point laser algorithm is an algorithm that obtains partial coordinate information of a reference point and partial coordinate information of a target point by using a point laser, and calculates the partial coordinate information of the reference point to obtain the coordinates of the target point on the second plane; A processing module: configured to obtain the target coordinates of the target point of the target device according to the first coordinates and the second coordinates; The first calculation module is further configured to obtain a plurality of reference points of the target device; and calculate the first coordinate axis values and the second coordinate axis values of the plurality of reference points by a visual algorithm; The second calculation module is also used to obtain the third coordinate axis values of multiple reference points and the third coordinate axis value of the target point through point laser; obtain the second coordinate of the target point of the target device according to the reference point coordinates of multiple reference points and the third coordinate axis value of the target point, and the reference point coordinates of the reference point include the first coordinate axis value of the reference point, the second coordinate axis value of the reference point and the third coordinate axis value of the reference point.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of any one of the methods according to claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method according to any one of claims 1 to 5.

Images