Positioning method and device based on two-dimensional code, equipment, medium and product

Abstract

This application relates to a QR code-based positioning method, apparatus, device, medium, and product. The method includes: controlling an ultraviolet light source and a near-infrared light source to illuminate a QR code label according to a preset time sequence, acquiring an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image; placing the QR code label on a robot hand; determining a target image from the ultraviolet light image, ultraviolet light depth image, near-infrared light image, and near-infrared light depth image based on the relationship between the average brightness of the ultraviolet light image and a preset brightness range; acquiring feature point data of the QR code label in the image coordinate system based on the target image; and acquiring the pose of the robot hand in three-dimensional space based on the feature point data. The technical solution of this application can improve the positioning accuracy of the robot hand's pose under complex lighting conditions.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a positioning method, apparatus, device, medium, and product based on QR codes. Background Technology

[0002] In the field of modern intelligent manufacturing, real-time, high-precision tracking of robot hand movements is one of the key technologies for improving human-computer interaction. Existing methods for robot hand localization are mainly divided into deep learning-based estimation methods and optical tag-based capture methods. However, both of these methods have low accuracy under complex lighting conditions. Summary of the Invention

[0003] The technical problem solved by this application is to provide a positioning method, device, equipment, medium and product based on QR codes, which can improve the positioning accuracy of robot hand pose under complex lighting conditions.

[0004] According to a first aspect of the embodiments of this application, a QR code-based positioning method is provided, comprising: controlling an ultraviolet light source and a near-infrared light source to illuminate a QR code label according to a preset time sequence, acquiring an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image, wherein the QR code label is disposed on a robot hand; determining a target image from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image based on the relationship between the average brightness value of the ultraviolet light image and a preset brightness range, and acquiring feature point data of the QR code label in the image coordinate system based on the target image; and acquiring the pose of the robot hand in three-dimensional space based on the feature point data.

[0005] It is understandable that the above-mentioned QR code-based positioning method can determine different ambient lighting conditions by using a dual-band alternating illumination strategy of ultraviolet and near-infrared light sources and detecting ambient lighting conditions. This can be achieved by comparing the average brightness of the ultraviolet light image with a preset brightness range. Consequently, the corresponding target image can be selected for positioning based on different ambient lighting conditions, enabling stable recognition of QR code tags under various lighting conditions. This significantly improves the environmental robustness of the system and thus enhances the accuracy of positioning based on QR code tags.

[0006] In one embodiment, the preset brightness range includes a range where the brightness value is greater than or equal to a second threshold and less than or equal to a first threshold. Based on the relationship between the average brightness value of the ultraviolet light image and the preset brightness range, a target image is determined from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image. This includes: acquiring the average brightness value of the ultraviolet light image; if the average brightness value of the ultraviolet light image is greater than the first threshold, determining that the target image includes the near-infrared light image and the near-infrared light depth image; and if the average brightness value of the ultraviolet light image is less than the second threshold, determining that the target image includes the ultraviolet light image and the ultraviolet light depth image.

[0007] It is understood that the above implementation method enables the selection of the corresponding target image for positioning when the ambient lighting conditions are determined to be strong light or weak light, avoiding the loss of information of the QR code label caused by excessively strong light or insufficient light conditions, and improving the accuracy of positioning based on the QR code label.

[0008] In one embodiment, a target image is determined from an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image based on the relationship between the average brightness of the ultraviolet light image and a preset brightness range. The method further includes: if the average brightness of the ultraviolet light image is within the preset brightness range, determining whether there are overexposed areas in the ultraviolet light image; corresponding to the presence of overexposed areas in the ultraviolet light image, replacing the pixels in the overexposed areas with the pixels in the near-infrared light image corresponding to the overexposed areas to obtain a composite image, thus determining that the target image includes the composite image and the ultraviolet light depth image; and corresponding to the absence of overexposed areas in the ultraviolet light image, determining that the target image includes both the ultraviolet light image and the ultraviolet light depth image.

[0009] It is understood that the above implementation method enables the selection of the corresponding target image for positioning when the ambient lighting conditions are determined to be between strong light and weak light and there are local shadows, effectively suppressing inaccurate positioning when the ambient lighting conditions are between strong light and weak light and there is overexposure.

[0010] In one embodiment, based on the target image, feature point data of the QR code label in the image coordinate system is obtained, including: based on a deep learning network or an image processing algorithm, the feature point data of the QR code label in the image coordinate system is obtained, wherein the feature point data of the QR code label includes: the pixel coordinates and depth information of four corner points in the ultraviolet light image, the ultraviolet light depth image, or the near-infrared light image and the near-infrared light depth image in the image coordinate system.

[0011] In one embodiment, obtaining the position of the robot hand in three-dimensional space based on the feature point data includes: obtaining the physical coordinates of the corner points in the QR code coordinate system; obtaining corner point offset data under different radii of curvature; obtaining the corresponding corner point offset from the corner point offset data according to the radius of curvature of the hand attachment area, and correcting the physical coordinates of the corner points according to the corresponding corner point offset to obtain the corrected physical coordinates of the corner points in the QR code coordinate system; and obtaining the pose of the wrist and / or the pose of the elbow in three-dimensional space based on the corrected physical coordinates of the corner points and the feature point data.

[0012] It is understandable that by correcting the physical coordinates of the corner points of the QR code label according to the radius of curvature of the area where the hand is attached, that is, correcting the physical size of the QR code label, the projection distortion error caused by the attached curved surface is reduced, and the positioning accuracy is improved, such as improving the positioning accuracy to the sub-millimeter level.

[0013] In one implementation, the QR code label is a fluorescent QR code label.

[0014] According to a second aspect of the embodiments of this application, a QR code-based positioning device is provided, including an image acquisition module, a feature point data acquisition module, and a hand pose acquisition module. The image acquisition module is configured to control an ultraviolet light source and a near-infrared light source to illuminate the QR code label according to a preset time sequence, and acquire an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image, wherein the QR code label is disposed on the robot hand. The feature point data acquisition module is configured to determine a target image from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image based on the relationship between the average brightness value of the ultraviolet light image and a preset brightness range, and acquire feature point data of the QR code label in the image coordinate system based on the target image. The hand pose acquisition module is configured to acquire the position of the robot hand in three-dimensional space based on the feature point data.

[0015] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor. The memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the above-described QR code-based positioning method.

[0016] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, can implement the above-described QR code-based positioning method.

[0017] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described QR code-based positioning method. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a QR code-based positioning method according to an exemplary embodiment.

[0019] Figure 2 This is a flowchart illustrating a method for determining a target image based on the relationship between the average brightness of an ultraviolet light image and a preset brightness range, according to an exemplary embodiment.

[0020] Figure 3 This is a flowchart illustrating a method for obtaining the pose of a hand in three-dimensional space based on feature point data, according to an exemplary embodiment.

[0021] Figure 4 This is a block diagram illustrating a QR code-based positioning device according to an exemplary embodiment.

[0022] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0023] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. Specific embodiments of this application will be described below in conjunction with the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. Without departing from the spirit and scope of this application, those skilled in the art can modify and substitute the embodiments of this application, and the resulting embodiments are also within the protection scope of this application.

[0024] As mentioned earlier, methods for locating a robot's hand mainly fall into two categories: deep learning-based estimation methods and optical marker-based capture methods. However, both methods suffer from low accuracy under complex lighting conditions. For example, in conditions of drastic lighting changes, the robot's hand is prone to joint drift or loss when it self-occludes, resulting in low accuracy for locating the robot's hand using deep learning-based estimation methods. Similarly, glare is easily generated under strong direct sunlight, or low contrast occurs in low-light conditions, and markers are easily lost when shadows obscure the surface, leading to low accuracy for locating the robot's hand pose using optical marker-based capture methods.

[0025] To address the aforementioned technical issues, this application proposes a QR code-based positioning method, device, equipment, medium, and product. Through a dual-band alternating illumination strategy using ultraviolet and near-infrared light sources and a detection strategy based on ambient lighting conditions, the QR code label can be stably identified under various complex lighting conditions, such as strong light, weak light, and local shadows, thereby improving the positioning accuracy of the robot's hand.

[0026] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] One embodiment of this application provides a QR code-based positioning method. This QR code-based positioning method can be applied to electronic devices. Please refer to... Figure 1 The QR code-based positioning method may include the following steps: Step S101: Control the ultraviolet light source and the near-infrared light source to illuminate the QR code label according to the preset time sequence, and obtain the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image and the near-infrared light depth image. The QR code label is set on the robot hand.

[0028] In some embodiments, the QR code label can be placed on the end effector of the robot's hand. In this application embodiment, the placement location is not specifically limited. The QR code label may include anti-glare features of a microscopic optical structure layer.

[0029] In some embodiments, the QR code label can be a fluorescent QR code label. Compared to other ordinary QR code labels, fluorescent QR code labels enhance the light signal of the QR code label. For example, the fluorescent QR code label can be a dual-mode fluorescent QR code. The intensity of the fluorescent signal emitted by the dual-mode fluorescent QR code under the illumination of ultraviolet and near-infrared light sources is higher than the intensity of the reflected light signal of ordinary QR codes, improving the accuracy of positioning based on the fluorescent QR code label. In some embodiments, the ultraviolet and near-infrared light sources can be controlled to illuminate the QR code label according to a preset time sequence, such as alternating illumination of the QR code label with ultraviolet light for 2 seconds and near-infrared light for 2 seconds. A camera, such as a 3D vision camera, can also be used to simultaneously acquire ultraviolet light images, ultraviolet light depth images, near-infrared light images, and near-infrared light depth images.

[0030] In the depth image, the pixel value of each pixel represents the distance from that pixel to the camera.

[0031] It's understandable that ultraviolet light sources produce better contrast in low-light conditions, but may cause pixel saturation, highlight clipping, and overexposure in strong light. For example, the ultraviolet light source could be ultraviolet light itself.

[0032] It is understandable that near-infrared light sources have a wide dynamic range, are less prone to overexposure under strong light conditions, and can maintain stable imaging. However, under low light conditions, the imaging contrast is insufficient, the signal-to-noise ratio is low, and weak signals and blurred features are easily observed. For example, near-infrared light can be used as a near-infrared light source.

[0033] In some embodiments, other light waves can be used to illuminate the QR code label for positioning; however, no specific limitations are made in this application embodiment.

[0034] Step S102: Based on the relationship between the average brightness of the ultraviolet light image and the preset brightness range, determine the target image from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image, and obtain the feature point data of the QR code label in the image coordinate system based on the target image.

[0035] In some embodiments, the preset brightness range is a range where the brightness value is greater than or equal to the second threshold and less than or equal to the first threshold. The first and second thresholds can be set as needed, and are not specifically limited in this embodiment.

[0036] In some embodiments, a target image can be determined from an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image based on the relationship between the average brightness of the ultraviolet light image and a preset brightness range, and the target image can be decoded to obtain feature point data of the QR code label in the image coordinate system.

[0037] In some embodiments, feature point data of a QR code label in an image coordinate system can be obtained based on a deep learning network or image processing algorithm, according to the target image. The feature point data of the QR code label includes the pixel coordinates and depth information of the four corner points in the target image. The four corner points are feature points, and their pixel coordinates and depth information constitute feature point data. It is understood that other points can also be identified as feature points, and pixel coordinates can also be sub-pixel coordinates; however, this embodiment does not impose specific limitations.

[0038] Specifically, the method for determining the target image from ultraviolet light images, ultraviolet light depth images, near-infrared light images, and near-infrared light depth images based on the relationship between the average brightness of ultraviolet light images and preset brightness ranges can be found in [reference needed]. Figure 2 This method can be applied to electronic devices and may include the following steps: Step S201: Obtain the average brightness of the ultraviolet light image.

[0039] In some embodiments, the average brightness of an ultraviolet light image can be calculated. In this application embodiment, the method for calculating the average brightness of an ultraviolet light image is not specifically limited.

[0040] Step S202: If the average brightness of the ultraviolet light image is greater than the first threshold, determine that the target image includes the near-infrared light image and the near-infrared light depth image.

[0041] In some embodiments, ambient lighting conditions can be determined based on the average brightness of the ultraviolet light image. For example, if the average brightness of the ultraviolet light image is greater than a first threshold, the ambient lighting condition is determined to be a strong light condition. The first threshold can be set as needed, and is not specifically limited in this embodiment.

[0042] In some embodiments, when the average brightness of the ultraviolet light image is greater than a first threshold, the ambient lighting condition is determined to be a strong light condition. To avoid the impact of using ultraviolet light images and ultraviolet light depth images on positioning accuracy, near-infrared light images and near-infrared light depth images can be decoded.

[0043] Step S203: If the average brightness of the ultraviolet light image is less than the second threshold, determine that the target image includes the ultraviolet light image and the ultraviolet light depth image.

[0044] In some embodiments, if the average brightness of the ultraviolet light image is less than a second threshold, the ambient lighting condition is determined to be a low-light condition. The second threshold can be set as needed, and is not specifically limited in this embodiment.

[0045] In some embodiments, if the average brightness of the ultraviolet light image is less than a second threshold, the ambient lighting condition is determined to be a weak light condition, and the ultraviolet light image and the ultraviolet light depth image can be decoded.

[0046] Step S204: If the average brightness of the ultraviolet image is greater than or equal to the second threshold and less than or equal to the first threshold, determine whether there are overexposed areas in the ultraviolet image. If yes, proceed to step S205: Replace the pixels of the overexposed areas with the pixels of the areas corresponding to the overexposed areas in the near-infrared image to obtain a composite image, and determine that the target image includes the composite image and the ultraviolet depth image. If no, proceed to step S206: Determine that the target image includes the ultraviolet image and the ultraviolet depth image.

[0047] In some embodiments, if an overexposed area is determined to exist in the ultraviolet image, such as if the area of ​​a region in the ultraviolet image with a pixel value greater than a third threshold is greater than or equal to an area threshold, then an overexposed area can be determined to exist in the ultraviolet image. Otherwise, it can be determined that no overexposed area exists in the ultraviolet image.

[0048] Step S205: Replace the pixels in the overexposed area with the pixels in the near-infrared image corresponding to the overexposed area to obtain a composite image, and determine that the target image includes the composite image and the ultraviolet depth image.

[0049] In some embodiments, if it is determined that there are overexposed areas in the ultraviolet light image, the pixels in the overexposed areas of the ultraviolet light image can be replaced with the pixels in the corresponding areas of the near-infrared light image to obtain a composite image.

[0050] Step S206: Determine that the target image includes an ultraviolet light image and an ultraviolet light depth image.

[0051] In some embodiments, if it is determined that there are no overexposed areas in the ultraviolet image, the ultraviolet image and the ultraviolet depth image can be selected for decoding.

[0052] Thus, by employing a dual-band alternating illumination strategy using ultraviolet and near-infrared light sources and a detection strategy based on ambient lighting conditions, QR code tags can be stably identified under various complex lighting conditions, including strong light, weak light, and local shadows, significantly improving the system's environmental robustness.

[0053] Furthermore, the use of ultraviolet and near-infrared light sources can separate the signal wavelength generated by the QR code tag from the ambient visible light. Combined with the anti-glare design of the microscopic optical structure layer of the QR code tag, it can effectively suppress overexposure caused by strong direct light and signal loss caused by shadow occlusion under insufficient light conditions, thereby improving the accuracy of positioning based on the fluorescent QR code tag.

[0054] The method described above, which determines the target image from ultraviolet light images, ultraviolet light depth images, near-infrared light images, and near-infrared light depth images based on the relationship between the average brightness of ultraviolet light images and preset brightness ranges, can adapt to ambient lighting conditions, select appropriate images for decoding and positioning, and improve the accuracy of positioning results.

[0055] Step S103: Obtain the pose of the robot hand in three-dimensional space based on the feature point data.

[0056] For specific methods on obtaining the hand pose in 3D space based on feature point data, please refer to [link / reference]. Figure 3 This method can be applied to electronic devices and may include the following steps: Step S301: Obtain the physical coordinates of the lower corner point of the QR code coordinate system.

[0057] In some embodiments, the physical coordinates of the four corner points in the QR code coordinate system can be obtained.

[0058] In the QR code coordinate system, the physical coordinates of the four corner points are P_qr,i. i is the index of the corner point, and the value of i can be any one of 1, 2, 3, and 4.

[0059] Step S302: Obtain the offset data of the lower corner point under different radii of curvature.

[0060] The corner offset of the QR code label is the deviation between the theoretical corner position and the actual projected position of the QR code label due to surface deformation.

[0061] In some embodiments, a lookup table for the corner offset of QR code labels under different radii of curvature can be established based on data obtained from finite element simulation or actual measurement data. The lookup table for the corner offset of QR code labels under different radii of curvature may include multiple radii of curvature and the corner offset corresponding to each radius of curvature.

[0062] It is understood that the method of obtaining the corner offset of the QR code label under different radii of curvature can be selected as needed, and no specific limitation is made in this embodiment.

[0063] Step S303: Based on the radius of curvature of the hand attachment area, obtain the corresponding corner offset from the corner offset data, and correct the physical coordinates of the corner based on the corresponding corner offset to obtain the corrected physical coordinates of the corner in the QR code coordinate system.

[0064] In some embodiments, the physical coordinates of the four corner points in the QR code coordinate system can be corrected by using the corresponding corner offset to obtain the corrected corner point physical coordinates.

[0065] Where P'_qr,i are the corrected physical coordinates of the corner point. i is the index of the corner point, and the value of i can be any one of 1, 2, 3, or 4.

[0066] In this way, by correcting the physical coordinates of the corner points of the QR code label according to the radius of curvature of the area where the hand is attached, that is, correcting the physical size of the QR code label, the projection distortion error caused by the attached curved surface is reduced, and the positioning accuracy is improved, such as improving the positioning accuracy to the sub-millimeter level.

[0067] Step S304: Based on the corrected corner point physical coordinates and feature point data, obtain the pose of the wrist and / or elbow in three-dimensional space.

[0068] In some embodiments, step S304, the method for obtaining the wrist pose and elbow pose in three-dimensional space based on the corrected corner point physical coordinates and feature point data, may include the following: For example, the pixel coordinates of the four corner points of a QR code label in the feature point data in the image coordinate system are (u_i, v_i), where i takes the value of 1, 2, 3, or 4. Here, i is the index of the corner point, u_i is the x-coordinate of the corner point in the image coordinate system, and v_i is the y-coordinate of the corner point in the image coordinate system.

[0069] In some embodiments, the depth information of the four corner points in the feature point data, i.e., the depth value d_i, can be converted into three-dimensional coordinates P_cam, i in the camera coordinate system based on the feature point data, where P_cam, i = ((u_i - cx) / fx *d_i, (v_i - cy) / fy * d_i, d_i).

[0070] Where i can take any value from 1, 2, 3, or 4, and i is the index of the corner point. P_cam, i represents the 3D coordinates of the corner point's depth information in the camera coordinate system. u_i is the x-coordinate of the corner point in the image coordinate system, and v_i is the y-coordinate of the corner point in the image coordinate system. (cx, cy) are the coordinates of the camera principal point, where cx is the x-coordinate of the corner point in the pixel coordinate system, and cy is the y-coordinate of the corner point in the pixel coordinate system. d_i is the depth information of the corner point, i.e., the depth value. fx is the camera's lateral focal length, and fy is the camera's longitudinal focal length.

[0071] In some embodiments, the rotation matrix R and translation vector T of the QR code coordinate system relative to the camera coordinate system can be calculated when the least squares objective function ∑||P_cam,i - (R·P'_qr,i + T)||² is minimized. Here, P_cam,i represents the 3D coordinates of the depth information of a corner point in the camera coordinate system. P'_qr,i represents the corrected physical coordinates of the corner point. i is the index of the corner point, taking any value from 1, 2, 3, or 4. R is the rotation matrix from the QR code coordinate system to the camera coordinate system. T is the translation vector from the QR code coordinate system to the camera coordinate system.

[0072] In some embodiments, a fixed transformation matrix T_qr_wrist between the wrist joint and the QR code attachment point can be obtained based on the rotation matrix R and the translation vector T.

[0073] Where, T_qr_wrist= .

[0074] In some embodiments, the wrist pose in three-dimensional space can be obtained based on a fixed transformation matrix.

[0075] Wherein, the wrist pose in three-dimensional space = T_qr_wrist * (T_qr_wrist) -1 .

[0076] In some embodiments, the elbow pose in three-dimensional space can be obtained based on the wrist pose and forearm length in three-dimensional space.

[0077] In three-dimensional space, the elbow pose includes both the elbow's position and its orientation. The elbow's orientation can be the same as the wrist's orientation.

[0078] The elbow position Pelbow = TL * R * direction.

[0079] Where R is the rotation matrix from the QR code coordinate system to the camera coordinate system. T is the translation vector from the QR code coordinate system to the camera coordinate system. L is the length of the forearm. direction is the fixed direction vector of the forearm in the local coordinate system of the wrist.

[0080] The aforementioned QR code-based localization method achieves real-time, high-precision estimation of the pose of hand extremities, such as the wrist and / or elbow, by fusing depth information with feature point data from the QR code label image and combining it with the PnP algorithm to solve the pose.

[0081] Another exemplary embodiment of this application also provides a QR code-based positioning device. For example... Figure 4 As shown, in this embodiment, the device includes: an image acquisition module 410, a feature point data acquisition module 420, and a hand pose acquisition module 430.

[0082] The image acquisition module 410 is configured to control an ultraviolet light source and a near-infrared light source to illuminate the QR code label according to a preset time sequence, acquiring an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image. The QR code label is placed on the robot's hand. For specific applications of the image acquisition module 410, please refer to... Figure 1 Step S101 in the process will not be described again here.

[0083] The feature point data acquisition module 420 is configured to determine the target image from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image based on the relationship between the average brightness of the ultraviolet light image and a preset brightness range, and to acquire the feature point data of the QR code label in the image coordinate system based on the target image. For a detailed application of the feature point data acquisition module 420, please refer to [reference needed]. Figure 1 Step S102 in the process will not be described again here.

[0084] The hand pose acquisition module 430 is configured to acquire the position and orientation of the robot's hand in three-dimensional space based on feature point data. For a detailed application of the hand pose acquisition module 430, please refer to [reference needed]. Figure 1 Step S103 in the process will not be described again here.

[0085] The method and apparatus embodiments of this application can complement each other.

[0086] Embodiments of this application also provide an electronic device including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the method in any of the above embodiments.

[0087] Embodiments of this application also propose a computer-readable storage medium that, when an executable computer program in the storage medium is executed by a processor, enables the implementation of the methods in any of the above embodiments.

[0088] Embodiments of this application also propose a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the above embodiments.

[0089] Regarding the apparatus in the above embodiments, the specific manner in which the processor performs the operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0090] Embodiments of this application also provide an electronic device 600, such as... Figure 5 As shown, the electronic device 600 includes a memory 601 and a processor 602. The memory 601 is used to store computer programs executable by the processor 602; the processor 602 is used to execute the computer programs in the memory 601 to implement the methods provided in any of the above embodiments.

[0091] The electronic device 600 also includes a communication interface 603. The processor 602, memory 601, and communication interface 603 are connected via a communication bus and communicate with each other.

[0092] Processor 602 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.

[0093] Communication interface 603 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0094] The memory 601 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0095] In this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0096] The above description of the embodiments is intended to enable those skilled in the art to understand and apply this application. It will be apparent to those skilled in the art that various modifications can be easily made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, this application is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. A positioning method based on QR codes, characterized in that, The method includes: The ultraviolet light source and the near-infrared light source are controlled to illuminate the QR code label according to a preset time sequence to obtain an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image. The QR code label is set on the robot's hand. Based on the relationship between the average brightness of the ultraviolet light image and the preset brightness range, the target image is determined from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image, and based on the target image, the feature point data of the QR code label in the image coordinate system is obtained. Based on the feature point data, the pose of the robot's hand in three-dimensional space is obtained.

2. The positioning method based on QR codes according to claim 1, characterized in that, The preset brightness range is a range where the brightness value is greater than or equal to a second threshold and less than or equal to a first threshold. Determining the target image from the ultraviolet image, the ultraviolet depth image, the near-infrared image, and the near-infrared depth image based on the relationship between the average brightness value of the ultraviolet image and the preset brightness range includes: Obtain the average brightness of the ultraviolet light image; If the average brightness of the ultraviolet light image is greater than the first threshold, the target image is determined to include the near-infrared light image and the near-infrared light depth image. If the average brightness of the ultraviolet light image is less than the second threshold, the target image is determined to include the ultraviolet light image and the ultraviolet light depth image.

3. The positioning method based on QR codes according to claim 2, characterized in that, The step of determining the target image from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image based on the relationship between the average brightness value of the ultraviolet light image and the preset brightness range further includes: If the average brightness of the ultraviolet light image is within the preset brightness range, determine whether there are overexposed areas in the ultraviolet light image; If there is an overexposed area in the ultraviolet light image, the pixels in the overexposed area in the near-infrared light image are replaced to obtain a composite image, and the target image is determined to include the composite image and the depth image of the ultraviolet light. Since there are no overexposed areas in the ultraviolet light image, the target image is determined to include the ultraviolet light image and the ultraviolet light depth image.

4. The QR code-based positioning method according to claim 1, characterized in that, The step of obtaining feature point data of the QR code label in the image coordinate system based on the target image includes: obtaining feature point data of the QR code label in the image coordinate system based on the target image using a deep learning network or an image processing algorithm, wherein the feature point data of the QR code label includes: the pixel coordinates and depth information of the four corner points in the target image.

5. The QR code-based positioning method according to claim 1, characterized in that, The step of obtaining the position state of the robot hand in three-dimensional space based on the feature point data includes: Obtain the physical coordinates of the lower corner point of the QR code coordinate system; Obtain the offset data of the lower corner point under different radii of curvature; Based on the radius of curvature of the hand attachment area, the corresponding corner offset is obtained from the corner offset data, and the physical coordinates of the corner are corrected according to the corresponding corner offset to obtain the corrected physical coordinates of the corner in the QR code coordinate system. Based on the corrected corner point physical coordinates and the feature point data, the pose of the wrist and / or elbow in three-dimensional space is obtained.

6. The QR code-based positioning method according to claim 1, characterized in that, The QR code label is a fluorescent QR code label.

7. A positioning device based on a QR code, characterized in that, The QR code-based positioning device includes an image acquisition module, a feature point data acquisition module, and a hand pose acquisition module. The image acquisition module is configured to control the ultraviolet light source and the near-infrared light source to illuminate the QR code label according to a preset time sequence, and acquire an ultraviolet light image, an ultraviolet light depth image, a near-infrared light image, and a near-infrared light depth image, wherein the QR code label is set on the robot hand; The feature point data acquisition module is configured to determine the target image from the ultraviolet light image, the ultraviolet light depth image, the near-infrared light image, and the near-infrared light depth image based on the relationship between the average brightness value of the ultraviolet light image and the preset brightness range, and to acquire the feature point data of the QR code label in the image coordinate system based on the target image. The hand pose acquisition module is configured to acquire the pose of the robot hand in three-dimensional space based on the feature point data.

8. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program executable by the processor; and the processor executes the computer program in the memory to implement the QR code-based positioning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the executable computer program in the storage medium is executed by a processor, it can implement the QR code-based positioning method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the QR code-based positioning method as described in any one of claims 1 to 6.

Images