Object pose recognition method and device based on two-dimensional code, equipment and storage medium

Abstract

The invention provides an object pose recognition method and device based on a two-dimensional code, equipment and a storage medium, and the method comprises the steps: obtaining a two-dimensional code image disposed on a marker through a depth camera, and enabling the two-dimensional code to comprise the index information of a target object; searching a standard point cloud model of the marker according to the index information of the target object; acquiring an actual point cloud model of a marker through the depth camera; the standard point cloud model of the marker and the actual point cloud model of the marker are registered, and the relative pose of the marker and the depth camera is obtained, excessive information does not need to be obtained from a two-dimensional code, accurate positioning of the two-dimensional code is not needed either, and the method and the device have the advantages that the method and the device are easy to implement. The accurate pose of the object can be obtained through the two-dimensional code image and the marker point cloud model, and the method can be used for determining pose positioning of the marker under a camera local coordinate system or positioning of a grabbed target object.

Description

Technical Field

[0001] The present invention relates to the field of digital intelligent manufacturing technology, and in particular to a method, device, equipment and storage medium for object posture recognition based on a two-dimensional code. Background Art

[0002] At present, most of the object posture calculations rely on binocular cameras or inertial sensors, which are not only expensive, but also large in size, which is not conducive to being installed on mobile devices. The object posture acquisition system based on QR code has broad application prospects in industrial production, logistics and transportation, film and television special effects, exhibitions, education and other fields. At the same time, it is of great significance to realize the posture calculation of objects on portable devices and devices with insufficient computing power. The traditional positioning method based on QR code reads the position, attitude angle and geometric size information of the QR code in the world coordinate system, calculates the position and attitude angle of the monocular camera in the QR code coordinate system by using the mapping relationship, and determines the relative posture of the QR code and the camera through corner point auxiliary positioning according to the size information provided by the QR code, and then determines the posture of the camera in the map according to the position of the QR code in the global coordinate system. By comparing the deviation between the posture of the QR code in the image and the set posture target, the camera is adjusted to reach the set posture. However, since the position of the QR code is closely related to the posture calculation, this method has high requirements for its position accuracy, and too much information needs to be obtained from the QR code, resulting in a complex QR code image that can only be recognized at close range at high resolution. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for object posture recognition based on QR code, so as to solve the defects of traditional positioning methods based on QR code, such as high requirements for position accuracy, too much information needs to be obtained from the QR code, the QR code image is complex, and it can only be recognized at close range under high resolution.

[0004] The present invention provides a method for object posture recognition based on a two-dimensional code, comprising: Acquire a two-dimensional code image set on the marker through a depth camera, wherein the two-dimensional code includes index information of the target object; Searching for a standard point cloud model of the landmark according to the index information of the target object; Acquire an actual point cloud model of the landmark through the depth camera; The standard point cloud model of the marker and the actual point cloud model of the marker are registered to obtain the relative pose of the marker and the depth camera.

[0005] According to the object posture recognition method based on the two-dimensional code provided by the present invention, the two-dimensional code image set on the marker is obtained by a depth camera, including: Acquire the original image of the QR code set on the marker through the depth camera; Performing denoising on the original image using Gaussian filtering; Use histogram equalization to enhance the denoised image; Use the QR code detection function to identify the codes corresponding to all QR codes in the enhanced image; According to the coding rules, the QR code images that meet the landmark information naming specifications are screened out from the codes corresponding to all QR codes.

[0006] According to the object posture recognition method based on two-dimensional code provided by the present invention, the encoding rule includes: The first to second characters of the code represent the type of marker, the third to sixth characters generate a hexadecimal hash value based on the timestamp as a unique code, and the seventh character is used to check whether the information is valid, wherein the marker type corresponds to the point cloud model, and the unique code corresponds to the marker number.

[0007] According to the object posture recognition method based on a two-dimensional code provided by the present invention, searching for a standard point cloud model of the marker according to the index information of the target object includes: Determine whether the unique code is a marker required for the task based on the index information of the target object. If so, identify the type of marker based on the first and second characters of the code and the corresponding point cloud model.

[0008] According to the object posture recognition method based on QR code provided by the present invention, the actual point cloud model of the marker is obtained by the depth camera, including: Acquire an initial point cloud of a landmark through the depth camera; Downsampling the initial point cloud to remove outlier points in the downsampled point cloud; According to the coordinate transformation within the depth camera, the pixel points of the image acquired by the depth camera correspond to the point cloud with outliers removed; Obtaining the point cloud area corresponding to the two-dimensional code image from the correspondence between the pixel points of the image and the point cloud; Based on the point cloud area, the point cloud is segmented to extract the point cloud belonging to the landmark; The surface is fitted according to the point cloud of the marker, and the points in multiple point cloud neighborhoods corresponding to the point cloud area are moved to the surface using the moving least squares method to obtain the actual point cloud model of the marker.

[0009] According to the object posture recognition method based on QR code provided by the present invention, the registering the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative posture of the marker and the depth camera includes: Setting a search neighborhood of the standard point cloud model of the marker relative to the actual point cloud model of the marker, and calculating the normal of the local area where each point in the standard point cloud model of the marker is located; Calculate the differential Gaussian function of the normals of adjacent point clouds to detect points with significant local changes at different scales; Calculate the gradient direction histogram in the neighborhood of the key point to generate a scale-invariant feature description; Calculating the Euclidean distance between the feature description quantity and the features in the standard point cloud model of the marker, and finding matching point pairs between the standard point cloud model of the marker and the actual point cloud model of the marker according to a nearest neighbor search algorithm; Randomly select some point pairs from the matching point pairs to calculate the transformation matrix, apply the transformation matrix to all point pairs, calculate and record the distance of the point pairs after the transformation, retain the transformation matrix with the smallest distance as the initial transformation matrix, and obtain the relative pose of the marker and the depth camera according to the initial transformation matrix.

[0010] According to the object posture recognition method based on a two-dimensional code provided by the present invention, the step of obtaining the relative posture of the marker and the depth camera according to the initial transformation matrix includes: Use the kd tree to accelerate the nearest neighbor search. For each point in the initial transformation matrix, match the nearest point in the standard point cloud model of the marker, calculate an optimal rigid transformation matrix, apply the calculated transformation matrix to the actual point cloud model of the marker, update the position, iterate the above steps until the end condition is met, and obtain the rigid transformation matrix. Use the rigid transformation matrix as the relative pose of the marker and the depth camera.

[0011] The termination condition includes that the number of iterations reaches a preset maximum number of iterations or the average square error of each point between the initial transformation matrix and the standard point cloud model of the marker is less than a preset threshold.

[0012] The present invention also provides an object posture recognition device based on a two-dimensional code, comprising: A first acquisition module, configured to acquire a two-dimensional code image set on a marker through a depth camera, wherein the two-dimensional code includes index information of a target object; A search module, used for searching a standard point cloud model of the marker according to the index information of the target object; A second acquisition module, used for acquiring an actual point cloud model of the marker through the depth camera; The registration module is used to register the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative position and posture of the marker and the depth camera.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the object posture recognition method based on a two-dimensional code as described in any one of the above items is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the object posture recognition method based on a two-dimensional code is implemented as described in any one of the above items.

[0015] The object posture recognition method, device, equipment and storage medium based on QR code provided by the present invention obtain a QR code image set on a marker through a depth camera, wherein the QR code includes index information of the target object; a standard point cloud model of the marker is searched according to the index information of the target object; an actual point cloud model of the marker is obtained through the depth camera; the standard point cloud model of the marker and the actual point cloud model of the marker are aligned to obtain the relative posture of the marker and the depth camera. It does not require excessive information obtained from the QR code, nor does it require precise positioning of the QR code. The accurate posture of the object can be obtained through the QR code image and the marker point cloud model, which can be used to determine the posture positioning of the marker in the local coordinate system of the camera or to capture the positioning of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flow charts of the object posture recognition method based on the two-dimensional code provided in the embodiment of the present invention; Figure 2 This is a second flow chart of a method for object posture recognition based on a QR code provided in an embodiment of the present invention; Figure 3 Schematic diagram of the functional structure of an object posture recognition device based on a QR code provided by an embodiment of the present invention; Figure 4 It is a functional structure diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1 A flowchart of a method for object posture recognition based on a two-dimensional code provided in an embodiment of the present invention, such as Figure 1 As shown, the object posture recognition method based on QR code provided by the embodiment of the present invention includes: Step 101: Acquire a two-dimensional code image set on a marker through a depth camera, wherein the two-dimensional code includes index information of the target object; In the embodiment of the present invention, the marker is, for example, a robot charging station, or an object on a shelf to be grasped by the robot. The present invention does not limit the marker, and those skilled in the art can select it according to actual needs.

[0020] Step 102: searching for a standard point cloud model of the landmark according to the index information of the target object; Step 103: Acquire an actual point cloud model of the landmark through the depth camera; Step 104: align the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative pose of the marker and the depth camera.

[0021] The traditional positioning method based on QR code reads the position, attitude angle and geometric size information of the QR code in the world coordinate system, and uses the mapping relationship to calculate the position and attitude angle of the monocular camera in the QR code coordinate system. According to the size information provided by the QR code, the relative position and posture of the QR code and the camera are determined through corner point auxiliary positioning, and then the position and posture of the camera in the map are determined according to the position of the QR code in the global coordinate system. By comparing the deviation between the posture of the QR code in the image and the set posture target, the camera is adjusted to reach the set posture. However, since the position of the QR code is closely related to the posture calculation, this method has high requirements for its position accuracy. In addition, too much information needs to be obtained from the QR code, resulting in a complex QR code image that can only be recognized at close range at high resolution.

[0022] The object posture recognition method based on QR code provided by an embodiment of the present invention obtains a QR code image set on a marker through a depth camera, wherein the QR code includes index information of the target object; searches for a standard point cloud model of the marker according to the index information of the target object; obtains an actual point cloud model of the marker through the depth camera; aligns the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative posture of the marker and the depth camera. It does not require excessive information obtained from the QR code, nor does it require precise positioning of the QR code. The accurate posture of the object can be obtained through the QR code image and the marker point cloud model, which can be used to determine the posture positioning of the marker in the local coordinate system of the camera or to capture the positioning of the target object.

[0023] Based on any of the above embodiments, obtaining a two-dimensional code image set on a marker through a depth camera includes: Step 201: Acquire an original image of a QR code set on a marker through a depth camera; Step 202: Use Gaussian filtering to perform denoising on the original image; In the embodiment of the present invention, the quality and accuracy of the two-dimensional code image can be improved through filtering and denoising, which facilitates subsequent feature recognition.

[0024] Step 203, using histogram equalization to enhance the denoised image; In the embodiment of the present invention, using histogram equalization to enhance the denoised image can enhance the local contrast without affecting the overall contrast, such as subsequent image recognition.

[0025] Step 204: Use the QR code detection function to identify the codes corresponding to all QR codes in the enhanced image; In an embodiment of the present invention, the QR code detection function includes, for example, detect() and detectAndDecode(). The detect() function is used to detect whether the image contains a QR code, and returns the coordinates of the four vertices of the minimum area quadrilateral containing the QR code, that is, the four vertices of the QR code. This function accepts two parameters: the image to be detected (which can be a grayscale image or a color image) and an output array for storing the detection results. Through this function, the position and direction of the QR code can be determined to prepare for subsequent decoding operations. The detectAndDecode() function combines the functions of detect() and decode() to complete the detection and decoding of the QR code at one time. This function also accepts the image to be detected as input and returns the decoded text information. This function simplifies the process of QR code detection and decoding, allowing developers to obtain information in the QR code more quickly.

[0026] Step 205: Filter out the two-dimensional code images that meet the landmark information naming specification from the codes corresponding to all two-dimensional codes according to the coding rules.

[0027] In the embodiment of the present invention, the encoding rule includes: The first to second characters of the code represent the type of marker, the third to sixth characters generate a hexadecimal hash value based on the timestamp as a unique code, and the seventh character is used to check whether the information is valid, wherein the marker type corresponds to the point cloud model, and the unique code corresponds to the marker number.

[0028] In an embodiment of the present invention, searching for a standard point cloud model of the landmark according to the index information of the target object includes: Determine whether the unique code is a marker required for the task based on the index information of the target object. If so, identify the type of marker based on the first and second characters of the code and the corresponding point cloud model.

[0029] Through the above steps, it is possible to verify whether the acquired QR code image is valid, thereby further improving the accuracy of posture recognition.

[0030] In an embodiment of the present invention, if the depth camera acquires multiple two-dimensional code images, the two-dimensional code image corresponding to the target object can be automatically screened out from the multiple two-dimensional code images, thereby improving the intelligence of recognition.

[0031] Based on any of the above embodiments, the step of acquiring the actual point cloud model of the marker through the depth camera includes: Step 301: Acquire an initial point cloud of a landmark through the depth camera; Step 302: downsample the initial point cloud to remove outliers in the downsampled point cloud; In the embodiment of the present invention, downsampling processing can improve the computing speed, and removing outliers can improve the accuracy of the point cloud.

[0032] Step 303: according to the coordinate transformation in the depth camera, the pixel points of the image acquired by the depth camera correspond to the point cloud with outliers removed; Step 304: Obtain the point cloud area corresponding to the two-dimensional code image based on the correspondence between the pixel points of the image and the point cloud; Step 305: segment the point cloud based on the point cloud area to extract the point cloud belonging to the landmark; Step 306: Fit a surface according to the point cloud of the marker, and use the moving least squares method to move points in multiple point cloud neighborhoods corresponding to the point cloud area to the surface to obtain an actual point cloud model of the marker.

[0033] Based on any of the above embodiments, the registering the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative pose of the marker and the depth camera includes: Step 401: setting a search neighborhood of the standard point cloud model of the marker relative to the actual point cloud model of the marker, and calculating the normal of the local area where each point in the standard point cloud model of the marker is located; Step 402: Calculate the differential Gaussian function of the normals of adjacent point clouds to detect points with significant local changes at different scales; Step 403, calculating the gradient direction histogram in the neighborhood of the key point to generate a scale-invariant feature description; Step 404: Calculate the Euclidean distance between the feature description and the features in the standard point cloud model of the marker, and find matching point pairs between the standard point cloud model of the marker and the actual point cloud model of the marker according to the nearest neighbor search algorithm; Step 405: randomly select some point pairs from the matching point pairs to calculate the transformation matrix, apply the transformation matrix to all point pairs, calculate and record the distance of the point pairs after the transformation, retain the transformation matrix with the smallest distance as the initial transformation matrix, and obtain the relative posture of the marker and the depth camera according to the initial transformation matrix.

[0034] In an embodiment of the present invention, acquiring the relative position and posture of the marker and the depth camera according to the initial transformation matrix includes: Use the kd tree to accelerate the nearest neighbor search. For each point in the initial transformation matrix, match the nearest point in the standard point cloud model of the marker, calculate an optimal rigid transformation matrix, apply the calculated transformation matrix to the actual point cloud model of the marker, update the position, iterate the above steps until the end condition is met, and obtain the rigid transformation matrix. Use the rigid transformation matrix as the relative pose of the marker and the depth camera.

[0035] The termination condition includes that the number of iterations reaches a preset maximum number of iterations or the average square error of each point between the initial transformation matrix and the standard point cloud model of the marker is less than a preset threshold.

[0036] In the embodiment of the present invention, a kd tree (k-dimensional tree) is a tree structure for storing points in a k-dimensional space, so as to efficiently perform range query and nearest neighbor query.

[0037] The method of constructing a kd tree includes: starting from a point in space, selecting a dimension for division, creating a root node, and then recursively dividing the remaining points, each node is the middle division point of the space where its child nodes are located.

[0038] Starting from the root node, recursively search the kd tree and calculate the closest distance on the search path. According to the current closest distance and the distance between the current node and the search point, it can be decided whether to recurse to the child node or stop the search, thereby accelerating the nearest neighbor search.

[0039] like Figure 2 As shown, the object posture recognition method based on QR code provided by the embodiment of the present invention specifically includes the steps of: QR code detection, landmark information retrieval, regional point cloud extraction, landmark point cloud screening and point cloud and model alignment.

[0040] The steps of QR code detection include: (1) using Gaussian filtering to denoise the image; (2) using histogram equalization to enhance the image and improve the contrast and clarity of the QR code; (3) using the QR code detection function of the OpenCV library to identify all QR codes in the image; (4) filtering out QR codes that meet the naming specifications of landmark information according to the encoding rules; (5) selecting the required landmark information and recording the position of the corresponding QR code in the image.

[0041] The steps of landmark information retrieval include: (1) designing a coding rule: the first two characters represent the type of landmark, the third to sixth characters generate a hash value hexadecimal code based on the timestamp to maintain uniqueness, and the seventh check digit verifies whether the information is valid, where the landmark type corresponds to the point cloud model, and the unique code corresponds to the landmark number; (2) confirming whether the landmark corresponding to the identification number in the image is required for the task; (3) searching for the landmark point cloud model based on the QR code information recognized in the image.

[0042] The steps of extracting the point cloud of the landmark include: (1) downsampling the point cloud to improve the computing speed; (2) removing outliers in the point cloud; (3) making the pixel points correspond to the point cloud generated by the depth map according to the coordinate transformation in the camera; (4) obtaining the point cloud area corresponding to the QR code in the image from the corresponding relationship, and the plane where the point cloud is located is a certain surface on the landmark; (5) using the point cloud area where the QR code is located as the reference, using the Euclidean clustering method to segment the point cloud and extract the point cloud belonging to the landmark; (6) using the moving least squares method to fit a surface in the neighborhood of each point cloud and move it to the fitting surface, so as to smooth the surface of the landmark point cloud while retaining the geometric features.

[0043] The steps of landmark point cloud screening and point cloud and model registration include: (1) setting the search neighborhood and calculating the normal of the local area where each point is located; (2) calculating the differential Gaussian function of the normals of adjacent point clouds and detecting points with significant local changes at different scales; (3) calculating the gradient direction histogram of the key point in the neighborhood to generate a scale-invariant feature description; (4) calculating the Euclidean distance between the features, and using the nearest neighbor search algorithm to find the matching point pairs between the point cloud and the model; (5) randomly selecting some point pairs from the matching point pairs to calculate the transformation matrix, applying the transformation matrix to all point pairs, calculating and recording the distance between the transformed point pairs, iterating and repeating this step multiple times, and retaining the transformation matrix with the best consistency as the initial transformation matrix; (6) using the kd tree to accelerate the nearest neighbor search, for each point in the source point cloud, matching the nearest point in the model point cloud, calculating an optimal rigid transformation matrix, applying the calculated transformation matrix to the model point cloud, updating the position, and repeating this step until the predetermined maximum number of iterations is reached or the mean square error between the source point cloud and the model point cloud is less than a threshold. The final rigid transformation matrix is ​​the position and posture of the depth camera relative to the landmark.

[0044] The QR code-based object posture recognition method provided in an embodiment of the present invention identifies the type of marker according to the QR code, extracts the marker point cloud by the depth camera and compares and matches it with the corresponding point cloud model, thereby obtaining the relative posture of the depth camera and the marker. The method provided in an embodiment of the present invention can be used to determine the posture of the camera in the local coordinate system where the marker is located or to capture the positioning of the target object.

[0045] The object posture recognition device based on the two-dimensional code provided by the present invention is described below. The object posture recognition device based on the two-dimensional code described below and the object posture recognition method based on the two-dimensional code described above can be referred to each other.

[0046] Figure 3 A schematic diagram of the structure of an object posture recognition device based on a two-dimensional code according to an embodiment of the present invention is shown in FIG. Figure 3 As shown, the object posture recognition device based on QR code provided by the embodiment of the present invention includes: A first acquisition module 301 is used to acquire a two-dimensional code image set on a marker through a depth camera, wherein the two-dimensional code includes index information of a target object; A search module 302 is used to search for a standard point cloud model of the landmark according to the index information of the target object; A second acquisition module 303 is used to acquire an actual point cloud model of the landmark through the depth camera; The registration module 304 is used to register the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative position and posture of the marker and the depth camera.

[0047] The QR code-based object posture recognition device provided in an embodiment of the present invention obtains a QR code image set on a marker through a depth camera, wherein the QR code includes index information of the target object; searches for a standard point cloud model of the marker according to the index information of the target object; obtains an actual point cloud model of the marker through the depth camera; aligns the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative posture of the marker and the depth camera. It does not require excessive information obtained from the QR code, nor does it require precise positioning of the QR code. The accurate posture of the object can be obtained through the QR code image and the marker point cloud model, which can be used to determine the posture positioning of the marker in the local coordinate system of the camera or to capture the positioning of the target object.

[0048] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The memory 430 includes a computer program, an operating system and acquired data, and the processor 410 may call the logic instructions in the memory 430 to execute a method for object posture recognition based on a two-dimensional code, the method comprising: acquiring a two-dimensional code image set on a marker through a depth camera, wherein the two-dimensional code includes index information of the target object; searching for a standard point cloud model of the marker according to the index information of the target object; acquiring an actual point cloud model of the marker through the depth camera; registering the standard point cloud model of the marker with the actual point cloud model of the marker, and obtaining the relative posture of the marker and the depth camera.

[0049] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0050] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the object posture recognition method based on the QR code provided by the above-mentioned methods, the method comprising: obtaining a QR code image set on a marker through a depth camera, the QR code including index information of the target object; searching for a standard point cloud model of the marker according to the index information of the target object; obtaining an actual point cloud model of the marker through the depth camera; and aligning the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative posture of the marker and the depth camera.

[0051] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0052] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for object posture recognition based on a two-dimensional code, characterized in that: include: Acquire a two-dimensional code image set on the marker through a depth camera, wherein the two-dimensional code includes index information of the target object; Searching for a standard point cloud model of the marker according to the index information of the target object; Acquire an actual point cloud model of the landmark through the depth camera; The standard point cloud model of the marker and the actual point cloud model of the marker are registered to obtain the relative pose of the marker and the depth camera.

2. The object posture recognition method based on a two-dimensional code according to claim 1, characterized in that: The depth camera is used to obtain the QR code image set on the marker, including: Acquire the original image of the QR code set on the marker through the depth camera; Performing denoising on the original image using Gaussian filtering; Use histogram equalization to enhance the denoised image; Use the QR code detection function to identify the codes corresponding to all QR codes in the enhanced image; According to the coding rules, the QR code images that meet the landmark information naming specifications are screened out from the codes corresponding to all QR codes.

3. The object posture recognition method based on two-dimensional code according to claim 2 is characterized in that: The encoding rules include: The first to second characters of the code represent the type of marker, the third to sixth characters generate a hexadecimal hash value based on the timestamp as a unique code, and the seventh character is used to check whether the information is valid, wherein the marker type corresponds to the point cloud model, and the unique code corresponds to the marker number.

4. The object posture recognition method based on two-dimensional code according to claim 3 is characterized in that: The step of searching for a standard point cloud model of the marker according to the index information of the target object comprises: Determine whether the unique code is a marker required for the task based on the index information of the target object. If so, identify the type of marker based on the first and second characters of the code and the corresponding point cloud model.

5. The object posture recognition method based on two-dimensional code according to claim 1 is characterized in that: The actual point cloud model of the landmark is obtained by the depth camera, comprising: Acquire an initial point cloud of a landmark through the depth camera; Downsampling the initial point cloud to remove outlier points in the downsampled point cloud; According to the coordinate transformation within the depth camera, the pixel points of the image acquired by the depth camera correspond to the point cloud with outliers removed; Obtaining the point cloud area corresponding to the two-dimensional code image from the correspondence between the pixel points of the image and the point cloud; Based on the point cloud area, the point cloud is segmented to extract the point cloud belonging to the landmark; The surface is fitted according to the point cloud of the marker, and the points in multiple point cloud neighborhoods corresponding to the point cloud area are moved to the surface using the moving least squares method to obtain the actual point cloud model of the marker.

6. The object posture recognition method based on two-dimensional code according to claim 1 is characterized in that: The registering the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative position and posture of the marker and the depth camera includes: Setting a search neighborhood of the standard point cloud model of the marker relative to the actual point cloud model of the marker, and calculating the normal of the local area where each point in the standard point cloud model of the marker is located; Calculate the differential Gaussian function of the normals of adjacent point clouds to detect points with significant local changes at different scales; Calculate the gradient direction histogram in the neighborhood of the key point to generate a scale-invariant feature description; Calculating the Euclidean distance between the feature description quantity and the features in the standard point cloud model of the marker, and finding matching point pairs between the standard point cloud model of the marker and the actual point cloud model of the marker according to a nearest neighbor search algorithm; Randomly select some point pairs from the matching point pairs to calculate the transformation matrix, apply the transformation matrix to all point pairs, calculate and record the distance of the point pairs after the transformation, retain the transformation matrix with the smallest distance as the initial transformation matrix, and obtain the relative pose of the marker and the depth camera according to the initial transformation matrix.

7. The object posture recognition method based on two-dimensional code according to claim 6 is characterized in that: The acquiring the relative position and posture of the marker and the depth camera according to the initial transformation matrix comprises: Use kd tree to accelerate nearest neighbor search. For each point in the initial transformation matrix, match the nearest point in the standard point cloud model of the marker, calculate an optimal rigid transformation matrix, apply the calculated transformation matrix to the actual point cloud model of the marker, update the position, iterate the above steps until the end condition is met, and obtain the rigid transformation matrix. Use the rigid transformation matrix as the relative pose of the marker and the depth camera. The termination condition includes that the number of iterations reaches a preset maximum number of iterations or the average square error of each point between the initial transformation matrix and the standard point cloud model of the marker is less than a preset threshold.

8. An object posture recognition device based on a two-dimensional code, characterized in that: include: A first acquisition module, configured to acquire a two-dimensional code image set on a marker through a depth camera, wherein the two-dimensional code includes index information of a target object; A search module, used for searching a standard point cloud model of the marker according to the index information of the target object; A second acquisition module, used for acquiring an actual point cloud model of the marker through the depth camera; The registration module is used to register the standard point cloud model of the marker with the actual point cloud model of the marker to obtain the relative position and posture of the marker and the depth camera.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the object posture recognition method based on the two-dimensional code is implemented as described in any one of claims 1 to 7.

10. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the object posture recognition method based on a two-dimensional code as described in any one of claims 1 to 7 is implemented.

Images