Spatial positioning method, spatial positioning device and computer equipment

Abstract

The invention relates to a spatial positioning method, a spatial positioning device and computer equipment. The spatial positioning method comprises the following steps: calibrating camera parameters, and obtaining a mapping table between image coordinates and physical coordinates; positioning a target matrix code block according to the data matrix code block in the view of the camera; acquiring the position of the camera in the target coordinate system according to a mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and a preset conversion rule; the preset conversion rule is a conversion rule of the target coordinate system and the target matrix code block. According to the method, before positioning, the camera parameters are calibrated in advance, so that long positioning time consumption caused by calibration in a positioning stage can be avoided, and the time consumption of space positioning is reduced. Due to the fact that the actual size of the target matrix code block and the image coordinates of the target matrix code block are combined, positioning of different heights can be compatible, and plane positioning of the 3D space is achieved.

Description

Technical Field

[0001] This application relates to the field of spatial positioning technology, and in particular to a spatial positioning method, a spatial positioning device, and a computer device. Background Technology

[0002] In the field of spatial positioning, a single checkerboard calibration board is commonly used as a spatial reference. A planar calibration board printed with black and white squares is placed at the test station, and multiple images are taken to solve for the camera's intrinsic and extrinsic parameters and distortion coefficients, and to establish a pixel-physical coordinate mapping.

[0003] However, this scheme implicitly assumes that all feature points are coplanar and have a constant height. Once there are multiple types and thicknesses of workpieces, or when the same workstation needs to inspect surfaces of different heights simultaneously, a single checkerboard pattern reveals its incompatibility with heights—the pixel equivalent and actual physical scale of any height layer outside the calibration plane will change due to perspective scaling, causing a systematic error in the positioning results as the height shifts.

[0004] Therefore, single-plane chessboard positioning technology cannot be compatible with positioning at different heights. Summary of the Invention

[0005] Therefore, it is necessary to provide a spatial positioning method, spatial positioning device, and computer equipment that can be compatible with different altitudes to address the above-mentioned technical problems.

[0006] Firstly, this application provides a spatial positioning method, the method comprising:

[0007] The camera parameters are calibrated to obtain a mapping table between image coordinates and physical coordinates;

[0008] Locate the target matrix code block based on the data matrix code blocks in the camera's field of view;

[0009] The position of the camera in the target coordinate system is obtained based on the mapping table between the image coordinates and physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and the preset transformation rule; the preset transformation rule is the transformation rule between the target coordinate system and the target matrix code block.

[0010] In one embodiment, obtaining the position of the camera in the target coordinate system based on the mapping table between the image coordinates and physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and a preset transformation rule includes:

[0011] The physical coordinates of the target matrix code block are obtained based on the image coordinates of the target matrix code block and the mapping table between the image coordinates and physical coordinates;

[0012] According to the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and the preset conversion rule, the position of the camera in the target coordinate system is obtained.

[0013] In one of the embodiments, the position of the camera in the target coordinate system is obtained according to the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and the preset conversion rule, including:

[0014] According to the actual size of the target matrix code block and the physical coordinates of the target matrix code block, the positional relationship of the target matrix code block relative to the camera is obtained.

[0015] According to the positional relationship of the target matrix code block relative to the camera and the preset conversion rule, the positional relationship of the target matrix code block relative to the camera is converted into the position of the camera in the target coordinate system.

[0016] In one of the embodiments, it further includes:

[0017] Caliper positioning and sub-pixel corner positioning are performed on the target matrix code block to obtain the positions of four corner points of the target matrix code block.

[0018] According to the positions of the four corner points of the target matrix code block, the image coordinates of the target matrix code block are obtained.

[0019] In one of the embodiments, the target matrix code block is positioned according to the data matrix code blocks in the field of view of the camera, including:

[0020] According to the gray scale gradient changes in the field of view of the camera, a plurality of data matrix code blocks in the field of view of the camera are positioned.

[0021] From the plurality of data matrix code blocks, a code block to be positioned is determined.

[0022] The code block to be positioned is decoded to determine the type of the code block to be positioned.

[0023] According to the type of the code block to be positioned, the target matrix code block is determined.

[0024] In one of the embodiments, the code block to be positioned is determined from the plurality of data matrix code blocks, including:

[0025] The edge straight lines of the plurality of data matrix code blocks are scored.

[0026] The data matrix code block with the highest score is taken as the code block to be positioned.

[0027] In one embodiment, determining the target matrix code block based on the type of the code block to be located includes:

[0028] When the type of the code block to be located is a single code, the code block to be located is used as the target matrix code block;

[0029] When the type of the code block to be located is a code array or a code band, the sub-code in the code array or the code band that is closest to the center of the camera's field of view is taken as the target matrix code block.

[0030] In one embodiment, calibrating the camera parameters and obtaining a mapping table between image coordinates and physical coordinates includes:

[0031] Identify the response code of the chessboard image and extract the physical parameters of the chessboard image;

[0032] Based on the position of the response code in the chessboard image, corner detection is performed on the chessboard image, and sub-pixel optimization is performed on the chessboard image to obtain the image coordinates of the chessboard image.

[0033] Based on the physical parameters of the chessboard image and the image coordinates, the physical coordinates corresponding to each pixel in the chessboard image are obtained, and a mapping table between the image coordinates and the physical coordinates is generated.

[0034] In one embodiment, the step of obtaining the physical coordinates corresponding to each pixel in the chessboard image based on the physical parameters of the chessboard image and the image coordinates, and generating a mapping table between the image coordinates and the physical coordinates, includes:

[0035] The correspondence between pixels and physical coordinates is obtained based on the physical parameters and the image coordinates;

[0036] The camera parameters are nonlinearly optimized based on the correlation coefficient in the correspondence between the pixel points and physical coordinates to generate a mapping table between the image coordinates and physical coordinates.

[0037] Secondly, this application provides a spatial positioning device, comprising:

[0038] The calibration module is used to calibrate camera parameters and obtain a mapping table between image coordinates and physical coordinates;

[0039] The code block recognition module is used to locate the target matrix code block based on the data matrix code blocks in the camera's field of view;

[0040] The positioning module is configured to acquire the position of the camera in a target coordinate system according to a mapping table between image coordinates and physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and a preset conversion rule. The preset conversion rule is a conversion rule between the target coordinate system and the target matrix code block.

[0041] In a third aspect, the present application provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0042] The spatial positioning method, the spatial positioning device and the computer device can avoid long positioning time caused by calibration in the positioning stage, thereby reducing the time consumption of spatial positioning. The target matrix code block is positioned according to the data matrix code block in the field of view of the camera, and then the position of the camera in the target coordinate system is acquired according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and the preset conversion rule. The preset conversion rule is the conversion rule between the target coordinate system and the target matrix code block. Since the actual size of the target matrix code block and the image coordinates of the target matrix code block are combined, different heights of positioning can be compatible, thereby realizing the planar positioning of the 3D space. The visual positioning of the full height can be realized by one calibration. Regardless of the height of the workpiece surface, the coordinate mapping can be completed by the spatial positioning method in the embodiment of the present application, without the need to replace the calibration plate or reacquire the image, thereby significantly improving the spatial positioning accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0044] Figure 1 The flowchart of the spatial positioning method in one embodiment of the present application;

[0045] Figure 2 The flowchart of acquiring the position of the camera in the target coordinate system according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and the preset conversion rule in one embodiment of the present application;

[0046] Figure 3A flowchart of a process of obtaining the position of the camera in the target coordinate system according to the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and a preset conversion rule in an embodiment of the present application;

[0047] Figure 4 A flowchart of another process in a spatial positioning method in an embodiment of the present application;

[0048] Figure 5 A flowchart of a process of positioning a target matrix code block according to a data matrix code block in the field of view of the camera in an embodiment of the present application;

[0049] Figure 6 A flowchart of a process of calibrating the camera parameters and obtaining a mapping table between the image coordinates and the physical coordinates in an embodiment of the present application;

[0050] Figure 7 A structural block diagram of a spatial positioning apparatus in an embodiment of the present application;

[0051] Figure 8 An internal structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the objects, technical solutions, and advantages of the present application clearer, further detailed descriptions will be given to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0053] The spatial positioning method provided in the embodiments of the present application can be applied to an automated guided vehicle (AGV) or a rail guided vehicle (RGV). Further, as a visual navigation technology, the spatial positioning method can be applied to scenarios such as warehouse logistics and intelligent manufacturing. The spatial positioning method fuses a data matrix (DM) positioning technology, and can provide millimeter-level positioning and path planning for a vehicle by decoding a two-dimensional code coordinate, thereby breaking away from the constraints of a magnetic nail track and a laser reflector. Compared with magnetic navigation (first generation) and laser navigation (second generation), the spatial positioning method can improve the navigation deployment speed, reduce the cost, and be flexible in route, and thus exhibits higher flexibility and environmental adaptability in dynamic environments such as warehouse logistics and intelligent manufacturing.

[0054] The embodiments of the present application provide a spatial positioning method. The embodiments take the method as an example applied to a terminal, and it can be understood that the method can also be applied to a server, and can also be applied to a system including the terminal and the server, and is implemented through the interaction of the terminal and the server. In the embodiments, refer to Figure 1 , and Figure 1A flowchart of a spatial positioning method in the embodiment is shown. The method comprises the following steps S101 to S103.

[0055] In step S101, camera parameters are calibrated to obtain a mapping table between image coordinates and physical coordinates.

[0056] The camera parameters include camera intrinsic parameters, camera extrinsic parameters, distortion parameters, and the like, but are not limited thereto. The camera intrinsic parameters can be a camera intrinsic parameter matrix, which converts a physical length on an image plane into a pixel coordinate. The camera intrinsic parameter matrix K can be expressed as fx, fy, cx, cy, where fx and fy are focal lengths in pixels, and cx and cy are pixel coordinates of a principal point (intersection of an optical axis and an image plane).

[0057] The camera extrinsic parameters can be a camera extrinsic parameter matrix, which converts a point in a world coordinate system to a camera coordinate system. The camera extrinsic parameter matrix can be represented by [R | t], where R is a rotation matrix of the world coordinate system to the camera coordinate system, and t is a translation vector of the world coordinate system to the camera coordinate system.

[0058] The distortion parameters can be represented by a distortion parameter vector D, which can include a plurality of distortion coefficients, such as k1, k2, p1, p2, and the like, but is not limited thereto. Among them, k1 and k2 can represent radial distortion parameters, and p1 and p2 can represent tangential distortion parameters.

[0059] The mapping table between the image coordinates and the physical coordinates can be represented by the core formula p=D(K·[R | t]·P), where p=[u, v]T is a pixel. P=[X, Y, Z, 1]T is a three-dimensional world point (homogeneous coordinates), and exemplarily, X and Y in it are the actual physical values of the intersection points of the checkerboard in the checkerboard picture, which can be calculated by the information in the QR code (Quick Response Code, matrix two-dimensional bar code). D(.) can be understood as a distortion correction function (including the distortion coefficients k1, k2, p1, p2 described above). K is the camera intrinsic parameter matrix described above. [R | t] is the camera extrinsic parameter matrix described above.

[0060] In step S102, a target matrix code block is positioned according to a data matrix code block in the field of view of the camera.

[0061] The data matrix (DM) code block is a super-small, high-error-correcting, and laser-etchable two-dimensional bar code. The data matrix code block has black and white modules, and can be a square or a matrix. The data matrix code block has four sub-pixel corner points (L outer contour intersection points) and can be used as a high-precision marker.

[0062] The type of the data matrix code block includes single code, code array and code band. The single code can be understood as an independent DM code, which can be the minimum size of 10*10 or the maximum size of 144*144, and can be decoded independently. The code array can be understood as a code array including multiple DM single codes, and the multiple DM single codes can be separated by a blank interval. The code band can be understood as a code strip formed by copying multiple DM single codes.

[0063] The target matrix code block can be positioned from the multiple data matrix code blocks in the field of view of the camera.

[0064] Exemplarily, the target matrix code block can be positioned according to the type of the data matrix code block.

[0065] Exemplarily, the camera in the embodiment of the application can be a monocular camera.

[0066] In step S103, the position of the camera in the target coordinate system is obtained according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and a preset conversion rule. The preset conversion rule is a conversion rule of the target coordinate system and the target matrix code block.

[0067] The actual size of the target matrix code block can be the actual size of the target matrix code block.

[0068] The target coordinate system can refer to a self-defined coordinate system. The conversion rule of the target coordinate system and the target matrix code block is a preset known rule.

[0069] Exemplarily, the size of the target matrix code block under the calibration plane can be obtained according to the mapping table between the image coordinates and the physical coordinates and the image coordinates of the target matrix code block. According to the size of the target matrix code block under the calibration plane and the actual size, and in combination with the preset conversion rule, the position of the target matrix code block in the target coordinate system can be determined, so as to realize positioning.

[0070] In the above spatial positioning method, the camera parameters are calibrated in advance before positioning, which can avoid long positioning time caused by calibration during positioning, thereby reducing the time consumption of spatial positioning. According to the data matrix code block in the field of view of the camera, the target matrix code block is positioned, and then according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and the preset conversion rule, the position of the camera in the target coordinate system is obtained; the preset conversion rule is the conversion rule of the target coordinate system and the target matrix code block. Since the actual size of the target matrix code block and the image coordinates of the target matrix code block are combined, different height positioning can be compatible, thereby realizing the planar positioning of the 3D space. Once calibration is completed, the visual positioning of the full height can be realized. Regardless of the height of the workpiece surface, the coordinate mapping can be completed by the spatial positioning method in the embodiment of the application, without the need to replace the calibration plate or reacquire the image, which significantly improves the spatial positioning accuracy.

[0071] In one exemplary embodiment, as shown in Figure 2 According to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and the preset conversion rule, the position of the camera in the target coordinate system is obtained, including steps S201 to S202.

[0072] In step S201, the physical coordinates of the target matrix code block are obtained according to the image coordinates of the target matrix code block and the mapping table between the image coordinates and the physical coordinates.

[0073] Exemplarily, after obtaining the image coordinates of the target matrix code block, the physical coordinates of the target matrix code block can be found based on the mapping table between the image coordinates and the physical coordinates. The physical coordinates of the target matrix code block can represent the coordinates of the four corner points of the target matrix code block on the calibration plane, and the size of the target matrix code block on the calibration plane can be calculated according to the physical coordinates of the target matrix code block.

[0074] In step S202, the position of the camera in the target coordinate system is obtained according to the actual size of the target matrix code block, the physical coordinates of the target matrix code block and the preset conversion rule.

[0075] In this embodiment, the physical coordinates of the target matrix code block can be quickly found according to the image coordinates of the target matrix code block and the mapping table between the image coordinates and the physical coordinates, thereby reducing the time consumption of positioning. Then, according to the actual size of the target matrix code block and the physical coordinates of the target matrix code block, the position of the target matrix code block is converted from two dimensions to three dimensions, which is compatible with different height positioning. In combination with the preset conversion rule of the target coordinate system and the target matrix code block, the position of the camera in the target coordinate system is determined, thereby improving the accuracy of spatial positioning.

[0076] In one example embodiment, as shown in Figure 3 According to the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and the preset conversion rule, the position of the camera in the target coordinate system is obtained, including the following steps S301 to S302.

[0077] In step S301, the positional relationship of the target matrix code block relative to the camera is obtained according to the actual size of the target matrix code block and the physical coordinates of the target matrix code block.

[0078] In step S302, the positional relationship of the target matrix code block relative to the camera is converted into the position of the camera in the target coordinate system according to the positional relationship of the target matrix code block relative to the camera and the preset conversion rule.

[0079] In the space positioning method in the embodiment, since there is a deviation between the actual size of the target matrix code block and the size of the target matrix code block in the calibration plane obtained according to the physical coordinates of the target matrix code block, the positional relationship of the target matrix code block relative to the camera is obtained according to the actual size of the target matrix code block and the physical coordinates of the target matrix code block, which can compensate for the above deviation, so that the position of the target matrix code block in the target coordinate system positioned finally according to the positional relationship of the target matrix code block relative to the camera and the preset conversion rule can be more accurate, and the accuracy of space positioning is improved.

[0080] In one example embodiment, refer to the accompanying Figure 4 The space positioning method provided in the embodiment further includes the following steps S401 to S402.

[0081] In step S401, caliper positioning and sub-pixel corner positioning are performed on the target matrix code block to obtain the positions of the four corner points of the target matrix code block.

[0082] Caliper positioning is a one-dimensional edge measurement method. It projects the gray scale of each pixel in the search region along the vertical direction to obtain a one-dimensional gray scale curve by setting a search line (rotatable) on the image; and then detects the maximum gradient point on the curve to accurately locate the sub-pixel level one-dimensional coordinate of the object edge on the search line.

[0083] Sub-pixel corner positioning is a two-dimensional feature refinement technique. The integer pixel coordinates of the corner point are obtained through a preset algorithm, and then the sub-pixel level two-dimensional floating point coordinates of the corner point are solved by using gray scale gradient information or surface fitting in an n×n (n is a positive integer, for example, 3×3 or 5×5) neighborhood, and then the pixel is accurately positioned. For example, the positioning accuracy can reach 0.02-0.1 pixels.

[0084] Exemplarily, the step S401 can first locate the horizontal / vertical edges of the corner point by using the caliper, and since the intersection point (corner point) is calculated by the edge extension line, it is easy to be affected by the edge noise, and the accuracy of the intersection point is low. Therefore, according to the sub-pixel corner point positioning, the corner point coordinates are refined in the intersection area of the edge, and high-precision positioning of the corner point can be realized.

[0085] In some embodiments, before the caliper positioning of the target matrix code block, the target matrix code block can also be preprocessed to improve the edge definition of the target matrix code block and reduce the noise interference of the edge detection. Exemplarily, the target matrix code block can be filtered and denoised while the edge features are preserved. Or the edge of the target matrix code block can be enhanced to improve the edge gray level mutation degree and improve the stability of the caliper positioning for edge detection.

[0086] Step S402, obtaining the image coordinates of the target matrix code block according to the positions of the four corner points of the target matrix code block.

[0087] In the spatial positioning method in the embodiment, the caliper positioning and the sub-pixel corner point positioning are performed on the target matrix code block, and then the image coordinates of the target matrix code block are determined according to the positions of the four corner points of the target matrix code block. Since the accuracy of the corner point detection of the target matrix code block is improved, the image coordinates of the target matrix code block can be further determined.

[0088] In one exemplary embodiment, refer to the accompanying drawings Figure 5 According to the data matrix code block in the field of view of the camera, the target matrix code block is positioned, which includes the following steps S501 to S504.

[0089] Step S501, according to the gray level gradient change in the field of view of the camera, a plurality of data matrix code blocks in the field of view of the camera are positioned.

[0090] Exemplarily, the plurality of data matrix code blocks can be positioned according to the black and white gradient change in the field of view of the camera.

[0091] Step S502, determining the to-be-positioned code block from the plurality of data matrix code blocks.

[0092] Step S503, decoding the to-be-positioned code block to determine the type of the to-be-positioned code block.

[0093] Exemplarily, the to-be-positioned code block can be decoded, and the type of the to-be-positioned code block is determined according to the matching between the decoding content and the pre-set type of the code block. Exemplarily, the type of the to-be-positioned code block can include a single code, a code array and a code band.

[0094] Step S504, determining the target matrix code block according to the type of the to-be-positioned code block.

[0095] In the embodiment, the change of the gray scale gradient can reflect the stable structural features of the edge and texture of the data matrix code block which are not affected by the overall shift of the light intensity. The whole pixel matrix code block in the camera view can be located by comparing the gradient feature similarity, and the fast data matrix code block searching is realized. According to the type of the to-be-located code block, the target matrix code block is determined, which is helpful to accurately locate the target matrix according to the form of the to-be-located code block, and further improve the accuracy of spatial positioning.

[0096] In an exemplary embodiment, the to-be-located code block is determined from the plurality of data matrix code blocks, including: scoring the edge straight lines of the plurality of data matrix code blocks; and taking the data matrix code block with the highest score as the to-be-located code block.

[0097] Exemplarily, the change of the gradient around the plurality of data matrix code blocks, the ratio of the length and width of the data matrix code block, and the included angle of the two straight line edges of the data matrix code block can be scored respectively, and finally the total score is obtained by weighted calculation of each score, and the data matrix code block with the highest total score is taken as the to-be-located code block.

[0098] In the spatial positioning method in the embodiment, the plurality of data matrix code blocks are scored, and the data matrix code block with the highest score is determined as the to-be-located code block, which improves the reliability of the to-be-located code block, and further determines the target matrix code block from the to-be-located code block, which can improve the accuracy of spatial positioning according to the target matrix code block.

[0099] In an exemplary embodiment, the target matrix code block is determined according to the type of the to-be-located code block, including: in the case that the type of the to-be-located code block is a single code, the to-be-located code block is taken as the target matrix code block; and in the case that the type of the to-be-located code block is a code array or a code band, the sub-code closest to the center of the camera view in the code array or the code band is taken as the target matrix code block.

[0100] In some embodiments, if the type of the to-be-located code block is a single code, the to-be-located code block is directly taken as the target matrix code block, and in step S301, the positional relationship of the single code relative to the camera can be directly obtained according to the actual size of the single code and the physical coordinates of the single code.

[0101] In some embodiments, if the type of the to-be-located code block is a code array or a code band, the sub-code closest to the center of the camera view in the code array or the code band is taken as the target matrix code block, and in step S301, the positional relationship of the sub-code relative to the camera can be obtained according to the actual size of the sub-code and the physical coordinates of the sub-code, and further the positional relationship of the code array or the code band relative to the camera can be obtained according to the coding rule of the code band or the code array and in combination with the positional relationship of the sub-code relative to the camera.

[0102] The space positioning method in the embodiment can be compatible with different types of data matrix code blocks and more space positioning scenes, and can realize more accurate space positioning.

[0103] In one exemplary embodiment, refer to the accompanying drawings Figure 6 The camera parameters are calibrated to obtain a mapping table between image coordinates and physical coordinates, including the following steps S601 to S603.

[0104] In step S601, a response code of a checkerboard picture is identified, and physical parameters of the checkerboard picture are extracted.

[0105] The checkerboard picture refers to a planar array image formed by high-contrast two-color (for example, black and white) squares arranged alternately in a fixed number of rows and columns, and can be used as a precise two-dimensional reference target for camera calibration, distortion correction, and coordinate mapping.

[0106] The response code can be a QR code (Quick Response Code, matrix two-dimensional barcode), which can store the physical parameter information of the checkerboard picture.

[0107] The physical parameters of the checkerboard picture include the actual physical values of the intersection points of the checkerboard.

[0108] In step S602, corner point detection is performed on the checkerboard picture according to the position of the response code in the checkerboard picture, and sub-pixel optimization is performed on the checkerboard picture to obtain image coordinates of the checkerboard picture.

[0109] For example, the center of the checkerboard picture is determined according to the geometric features of the response code, and then the approximate position of the checkerboard picture is estimated to realize coarse positioning of the checkerboard picture. Then the outer contour lines of the checkerboard picture are coarsely positioned through corner point detection. Each edge is sampled by a caliper, fitted by a straight line, etc. to obtain initial corner point coordinates of the checkerboard picture, and then sub-pixel optimization is performed to accurately position the corner point coordinates to obtain accurate image coordinates of the checkerboard picture.

[0110] In step S603, the physical coordinates corresponding to each pixel point in the checkerboard picture are obtained according to the physical parameters and image coordinates of the checkerboard picture, and a mapping table between image coordinates and physical coordinates is generated.

[0111] In the spatial positioning method in the embodiment, the physical parameters of the checkerboard picture can be obtained by recognizing the response code in the checkerboard picture with the checkerboard picture customized with the response code, without the need of the checkerboard picture being completely in the field of view, thus simplifying the difficulty of calibrating the camera parameters. According to the position of the response code in the checkerboard picture, corner point detection is performed on the checkerboard picture, and sub-pixel optimization is performed on the checkerboard picture, so as to obtain accurate image coordinates of the checkerboard picture. Finally, the physical coordinates corresponding to each pixel point in the checkerboard picture are obtained according to the physical parameters and the image coordinates of the checkerboard picture, and a mapping table between the image coordinates and the physical coordinates is generated.

[0112] In an exemplary embodiment, the physical coordinates corresponding to each pixel point in the checkerboard picture are obtained according to the physical parameters and the image coordinates of the checkerboard picture, and a mapping table between the image coordinates and the physical coordinates is generated, including: obtaining the corresponding relationship between the pixel points and the physical coordinates according to the physical parameters and the image coordinates; and performing nonlinear optimization on the camera parameters according to the correlation coefficients in the corresponding relationship between the pixel points and the physical coordinates, so as to generate the mapping table between the image coordinates and the physical coordinates.

[0113] Exemplarily, the corresponding relationship between the pixel points and the physical coordinates can be represented as: ; wherein s, K, r1, r2 are the internal and external parameters of the camera, which can also be understood as a simplified form of the D, K, R, t in the foregoing, and X and Y are the actual physical values of the intersection points of the checkerboard, which can be calculated by the information in the QR code. According to the least square method, the H value in the above mapping relationship is solved according to the point pair, and the H value is decomposed as the initial value of the following nonlinear optimization:

[0114] D, K, [R|t] are iteratively optimized, the physical space value corresponding to each pixel value is calculated in the calibration process, and then the mapping table between the image coordinates and the physical coordinates is generated.

[0115] In the embodiment, the distortion error of the camera has been calculated in advance in the process of calibrating the camera parameters, so that distortion compensation is not needed in the positioning stage. The mapping table between the image coordinates and the physical coordinates generated in the calibration process can be directly applied in the positioning stage, which can reduce the time consumption of positioning and improve the positioning accuracy.

[0116] In some embodiments, a spatial positioning method is provided, including: identifying a response code of a checkerboard picture, extracting physical parameters of the checkerboard picture; obtaining a corresponding relationship between pixel points and physical coordinates according to the physical parameters and image coordinates; performing nonlinear optimization on camera parameters according to the correlation coefficients in the corresponding relationship between the pixel points and the physical coordinates, to generate a mapping table between the image coordinates and the physical coordinates; inputting a picture with a data matrix code block; positioning a plurality of data matrix code blocks in the field of view of the camera according to the gray gradient change in the field of view of the camera; positioning a plurality of data matrix code blocks in the field of view of the camera according to the gray gradient change in the field of view of the camera; scoring the edge straight lines of the plurality of data matrix code blocks; taking the data matrix code block with the highest score as a to-be-positioned code block; decoding the to-be-positioned code block to determine the type of the to-be-positioned code block; in the case that the type of the to-be-positioned code block is a single code, taking the to-be-positioned code block as a target matrix code block; performing caliper positioning and sub-pixel corner positioning on the single-code target matrix code block to obtain the positions of four corner points of the single-code target matrix code block; obtaining the image coordinates of the single-code target matrix code block according to the positions of the four corner points of the single-code target matrix code block; obtaining the physical coordinates of the single-code target matrix code block according to the image coordinates of the single-code target matrix code block and the mapping table between the image coordinates and the physical coordinates; obtaining the positional relationship of the single-code target matrix code block relative to the camera according to the actual size of the single-code target matrix code block and the physical coordinates of the single-code target matrix code block; and converting the positional relationship of the target matrix code block relative to the camera into the position of the camera in a target coordinate system according to the positional relationship of the target matrix code block relative to the camera and a preset conversion rule.

[0117] In the case that the type of the to-be-positioned code block is a code array or a code band, taking a sub-code closest to the center of the field of view of the camera in the code array or the code band as a target matrix code block; performing caliper positioning and sub-pixel corner positioning on the target matrix code block to obtain the positions of four corner points of the target matrix code block; obtaining the image coordinates of the target matrix code block according to the positions of the four corner points of the target matrix code block; obtaining the physical coordinates of the target matrix code block according to the image coordinates of the target matrix code block and the mapping table between the image coordinates and the physical coordinates; obtaining the positional relationship of the sub-code relative to the camera according to the actual size of the sub-code and the physical coordinates of the sub-code, and further obtaining the positional relationship of the code array or the code band relative to the camera in combination with the positional relationship of the sub-code relative to the camera according to the encoding rule of the code band or the code array; and converting the positional relationship of the target matrix code block of the code array or the code band relative to the camera into the position of the camera in a target coordinate system according to the positional relationship of the target matrix code block of the code array or the code band relative to the camera and a preset conversion rule.

[0118] The spatial positioning method in the embodiments of the present application has calculated the distortion error in the camera parameter in advance in the calibration stage, and does not need to be calculated in the positioning stage, so that the time consumption of positioning can be reduced. The camera parameter is calibrated by using the customized checkerboard picture with response code, so that the checkerboard picture does not need to be completely in the field of view, and the calibration difficulty is reduced. In the positioning stage, the target matrix code block is positioned according to the data matrix code block in the field of view of the camera, and then the position of the camera in the target coordinate system is obtained according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block and the preset conversion rule. The preset conversion rule is the conversion rule of the target coordinate system and the target matrix code block. Since the actual size of the target matrix code block and the image coordinates of the target matrix code block are combined, different height positioning can be compatible, so that the plane positioning of the 3D space is realized. The visual positioning of the full height can be realized by one calibration. No matter which height layer the workpiece surface is located, the coordinate mapping can be completed by the spatial positioning method in the embodiments of the present application, without the need of replacing the calibration board or re-collecting the image, so that the spatial positioning precision is significantly improved.

[0119] It should be understood that, although the steps in the flowcharts involved in the embodiments as described above are displayed in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowcharts involved in the embodiments as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiments of the present application also provide a spatial positioning device for implementing the spatial positioning method as described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more spatial positioning device embodiments provided below can refer to the limitations of the spatial positioning method described above, which will not be described here again.

[0121] In one exemplary embodiment, as shown in Figure 7 a spatial positioning device 700 is provided, which includes a calibration module 710, a code block identification module 720 and a positioning module 730.

[0122] The calibration module 710 is configured to calibrate the camera parameter and obtain the mapping table between the image coordinates and the physical coordinates.

[0123] The code block identifying module 720 is configured to locate the target matrix code block according to the data matrix code blocks in the field of view of the camera.

[0124] The positioning module 730 is configured to obtain the position of the camera in the target coordinate system according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and a preset conversion rule; the preset conversion rule is a conversion rule between the target coordinate system and the target matrix code block.

[0125] In an exemplary embodiment, the positioning module is further configured to obtain the physical coordinates of the target matrix code block according to the image coordinates of the target matrix code block and the mapping table between the image coordinates and the physical coordinates; and obtain the position of the camera in the target coordinate system according to the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and the preset conversion rule.

[0126] In an exemplary embodiment, the positioning module is further configured to obtain the positional relationship of the target matrix code block relative to the camera according to the actual size of the target matrix code block and the physical coordinates of the target matrix code block; and convert the positional relationship of the target matrix code block relative to the camera into the position of the camera in the target coordinate system according to the positional relationship of the target matrix code block relative to the camera and the preset conversion rule.

[0127] In an exemplary embodiment, the positioning module is further configured to locate the target matrix code block according to the mapping table between the image coordinates and the physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and a preset conversion rule; the preset conversion rule is a conversion rule between the target coordinate system and the target matrix code block.

[0128] In an exemplary embodiment, the code block identifying module is further configured to locate a plurality of data matrix code blocks in the field of view of the camera according to the changes in the gray scale gradient in the field of view of the camera; determine the code block to be located from the plurality of data matrix code blocks; decode the code block to be located to determine the type of the code block to be located; and determine the target matrix code block according to the type of the code block to be located.

[0129] In an exemplary embodiment, the code block identifying module is further configured to score the edge straight lines of the plurality of data matrix code blocks; and take the data matrix code block with the highest score as the code block to be located.

[0130] In an exemplary embodiment, the code block identifying module is further configured to, in a case where the type of the code block to be located is a single code, take the code block to be located as the target matrix code block; and in a case where the type of the code block to be located is a code array or a code band, take the sub-code closest to the center of the field of view of the camera in the code array or the code band as the target matrix code block.

[0131] In an exemplary embodiment, the calibration module is further configured to identify the response code of the checkerboard picture, and extract the physical parameters of the checkerboard picture.

[0132] According to the position of the response code in the checkerboard picture, corner point detection is performed on the checkerboard picture, and sub-pixel optimization is performed on the checkerboard picture to obtain image coordinates of the checkerboard picture; and according to the physical parameters and the image coordinates of the checkerboard picture, physical coordinates corresponding to each pixel point in the checkerboard picture are obtained to generate a mapping table between the image coordinates and the physical coordinates.

[0133] In an exemplary embodiment, the calibration module is further configured to obtain a correspondence between the pixel points and the physical coordinates according to the physical parameters and the image coordinates; and perform nonlinear optimization on the camera parameters according to the correlation coefficients in the correspondence between the pixel points and the physical coordinates to generate the mapping table between the image coordinates and the physical coordinates.

[0134] The modules in the above spatial positioning apparatus can be implemented wholly or partially by software, hardware, and combinations thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the modules.

[0135] In an exemplary embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store various data involved in the spatial positioning method. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a spatial positioning method.

[0136] Those skilled in the art can understand that Figure 8 the structure shown in the above

[0137] In an example embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0138] In an example embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps in the above method embodiments when executed by a processor.

[0139] In an example embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the steps in the above method embodiments when executed by a processor.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0141] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0142] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A spatial positioning method, characterized in that, The method includes: The camera parameters are calibrated to obtain a mapping table between image coordinates and physical coordinates; Locate the target matrix code block based on the data matrix code blocks in the camera's field of view; The position of the camera in the target coordinate system is obtained based on the mapping table between the image coordinates and physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and the preset transformation rule; the preset transformation rule is the transformation rule between the target coordinate system and the target matrix code block.

2. The spatial positioning method according to claim 1, characterized in that, The step of obtaining the camera's position in the target coordinate system based on the mapping table between image coordinates and physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and preset transformation rules includes: The physical coordinates of the target matrix code block are obtained based on the image coordinates of the target matrix code block and the mapping table between the image coordinates and physical coordinates; The position of the camera in the target coordinate system is obtained based on the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and the preset transformation rule.

3. The spatial positioning method according to claim 2, characterized in that, The step of obtaining the position of the camera in the target coordinate system based on the actual size of the target matrix code block, the physical coordinates of the target matrix code block, and the preset transformation rule includes: Based on the actual size of the target matrix code block and the physical coordinates of the target matrix code block, the positional relationship of the target matrix code block relative to the camera is obtained; Based on the positional relationship of the target matrix code block relative to the camera and the preset transformation rule, the positional relationship of the target matrix code block relative to the camera is transformed into the position of the camera in the target coordinate system.

4. The spatial positioning method according to any one of claims 1 to 3, characterized in that, Also includes: The target matrix code block is positioned using calipers and sub-pixel corner points to obtain the positions of the four corner points of the target matrix code block. The image coordinates of the target matrix code block are obtained based on the positions of the four corner points of the target matrix code block.

5. The spatial positioning method according to claim 1, characterized in that, The step of locating the target matrix code block based on the data matrix code blocks in the camera's field of view includes: Based on the grayscale gradient changes in the camera's field of view, locate multiple data matrix code blocks in the camera's field of view; The code block to be located is determined from the plurality of data matrix code blocks; Decode the code block to be located to determine the type of the code block to be located; The target matrix code block is determined according to the type of the code block to be located.

6. The spatial positioning method according to claim 5, characterized in that, The step of determining the code block to be located from the plurality of data matrix code blocks includes: Scoring is performed on the edge lines of multiple data matrix code blocks; The data matrix code block with the highest score is selected as the code block to be located.

7. The spatial positioning method according to claim 5, characterized in that, Determining the target matrix code block based on the type of the code block to be located includes: When the type of the code block to be located is a single code, the code block to be located is used as the target matrix code block; When the type of the code block to be located is a code array or a code band, the sub-code in the code array or the code band that is closest to the center of the camera's field of view is taken as the target matrix code block.

8. The spatial positioning method according to claim 1, characterized in that, The step of calibrating camera parameters and obtaining a mapping table between image coordinates and physical coordinates includes: Identify the response code of the chessboard image and extract the physical parameters of the chessboard image; Based on the position of the response code in the chessboard image, corner detection is performed on the chessboard image, and sub-pixel optimization is performed on the chessboard image to obtain the image coordinates of the chessboard image. Based on the physical parameters of the chessboard image and the image coordinates, the physical coordinates corresponding to each pixel in the chessboard image are obtained, and a mapping table between the image coordinates and the physical coordinates is generated.

9. The spatial positioning method according to claim 8, characterized in that, The step of obtaining the physical coordinates corresponding to each pixel in the chessboard image based on the physical parameters of the chessboard image and the image coordinates, and generating a mapping table between the image coordinates and the physical coordinates, includes: The correspondence between pixels and physical coordinates is obtained based on the physical parameters and the image coordinates; The camera parameters are nonlinearly optimized based on the correlation coefficient in the correspondence between the pixel points and physical coordinates to generate a mapping table between the image coordinates and physical coordinates.

10. A spatial positioning device, characterized in that, include: The calibration module is used to calibrate camera parameters and obtain a mapping table between image coordinates and physical coordinates; The code block recognition module is used to locate the target matrix code block based on the data matrix code blocks in the camera's field of view; The positioning module is used to obtain the position of the camera in the target coordinate system according to the mapping table between the image coordinates and physical coordinates, the image coordinates of the target matrix code block, the actual size of the target matrix code block, and the preset conversion rule; the preset conversion rule is the conversion rule between the target coordinate system and the target matrix code block.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images