Positioning method and device based on multiple DM codes, autonomous mobile device and storage medium

Abstract

The embodiment of the invention provides a positioning method and device based on multiple DM codes, an autonomous mobile device and a storage medium, and relates to the technical field of mobile robots. The method comprises the steps of obtaining a first layer image, a second layer image and a third layer image obtained by performing continuous downsampling on a to-be-detected image; determining an actual position and an actual angle of at least one DM code in the to-be-detected image according to the first layer image, the second layer image and the third layer image; based on the actual position and the actual angle of the at least one DM code, determining the actual position and the actual angle of a graphic identification code comprising the at least one DM code; and positioning is carried out based on the actual position and the actual angle of the graphic identification code, so that the navigation positioning efficiency, accuracy and robustness based on the multi-DM code are improved.

Description

Technical Field

[0001] The present application relates to the technical field of mobile robots, and more specifically, to a positioning method and device based on multiple DM codes, an autonomous mobile device and a storage medium. Background Art

[0002] QR code navigation and positioning technology is one of the technologies used by mobile robots for navigation and positioning. The mobile robot is equipped with a downward-looking camera, which is used to capture and decode the QR code on the ground to achieve QR code positioning.

[0003] Data Matrix (DM) is a type of QR code. DM code is a graphic composed of black and white modules. DM code includes data area and positioning graphics.

[0004] See also Figure 1 , Figure 1 DM code is a schematic diagram of an exemplary embodiment of the present application. The data area of ​​the DM code is composed of regularly arranged square modules. The data area is used to store specific information. By decoding the data area, specific information can be obtained. The specific information is usually customized by the user according to actual needs.

[0005] See also Figure 2 , Figure 2 : is a schematic diagram of a locator pattern of a DM code provided by an exemplary embodiment of the present application. The locator pattern of the DM code is the boundary of the data area. The locator pattern is composed of a solid line L edge (called an "alignment pattern") and a dotted line L edge (called a "clock pattern"), wherein the solid line L edge is entirely composed of black code blocks, and the dotted line L edge is composed of alternating black code blocks and white code blocks.

[0006] At present, the navigation and positioning of mobile robots is usually performed based on a single DM code as a navigation graph, that is, However, the inventors found in their research that the technical robustness of using a single DM code as a navigation graph for navigation and positioning is low. In the case where the DM code is not captured or the DM code decoding fails, the position of the DM code cannot be known, which affects the accuracy and efficiency of navigation and positioning. In addition, if the spacing between DM codes is small, the image captured by the downward-looking camera may include multiple DM codes, and it takes a long time to locate the DM code to be located from multiple DM codes. Summary of the invention

[0007] The embodiments of the present application provide a positioning method, device, autonomous mobile device and storage medium based on multiple DM codes to improve the efficiency, accuracy and robustness of navigation and positioning of a mobile robot based on DM codes.

[0008] In a first aspect, an embodiment of the present application provides a positioning method based on multiple DM codes, the method comprising: acquiring a first layer image, a second layer image, and a third layer image obtained by continuously downsampling the image to be detected; determining the actual position and actual angle of at least one DM code in the image to be detected based on the first layer image, the second layer image, and the third layer image; determining the actual position and actual angle of a graphic identification code including the at least one DM code based on the actual position and actual angle of the at least one DM code; and performing positioning based on the actual position and actual angle of the graphic identification code.

[0009] In a second aspect, an embodiment of the present application provides a positioning device based on multiple DM codes, the device comprising: an image sampling module, used to obtain a first layer image, a second layer image and a third layer image obtained by continuously downsampling the image to be detected; an image determination module, used to determine the actual position and actual angle of at least one DM code in the image to be detected based on the first layer image, the second layer image and the third layer image; a position and angle determination module, used to determine the actual position and actual angle of a graphic identification code including the at least one DM code based on the actual position and actual angle of the at least one DM code; a navigation positioning module, used to perform positioning based on the actual position and actual angle of the graphic identification code.

[0010] In a third aspect, an embodiment of the present application provides an autonomous mobile device, which includes: a memory and a processor, wherein an application is stored in the memory, and the application is used to execute the method provided by the embodiment of the present application when called by the processor.

[0011] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program code stored thereon, wherein the program code is used to enable the processor to execute the method provided by the embodiment of the present application when called by the processor.

[0012] The positioning method, device, autonomous mobile device and storage medium based on multiple DM codes provided in the embodiments of the present application use a large graphic identification code including multiple DM codes as a navigation graphic for navigation positioning, rather than a single DM code as a navigation graphic for navigation positioning, which can ensure that the DM code can be photographed. As long as the position and angle of at least one DM code are determined, the position and angle of the large graphic identification code can be determined, and the problem of the inability to photograph the DM code and the failure of DM code decoding affecting the precision and accuracy of navigation positioning can be avoided when using a single DM code for positioning, thereby improving the robustness and accuracy of navigation positioning based on DM codes. In addition, the method determines the actual position and actual angle of at least one DM code in the image to be detected based on the first layer image, the second layer image and the third layer image obtained by continuously downsampling the image to be detected, and can quickly determine the position and angle of at least one DM code. As long as at least one DM code is determined, the position and angle of the large graphic identification code can be determined, so that when there are multiple DM codes in the captured image, the position of the graphic identification code can be quickly determined, thereby improving the accuracy of navigation positioning based on DM codes. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] Figure 1 is a schematic diagram of a DM code provided by an exemplary embodiment of the present application;

[0015] Figure 2 is a schematic diagram of a positioning pattern of a DM code provided by an exemplary embodiment of the present application;

[0016] Figure 3 is a schematic diagram of a graphic identification code provided by an exemplary embodiment of the present application;

[0017] Figure 4 is a flowchart of a positioning method based on multiple DM codes provided in one embodiment of the present application;

[0018] Figure 5 is a grayscale schematic diagram of an image to be detected provided by an exemplary embodiment of the present application;

[0019] Figure 6 is a flowchart of step S120 provided in an embodiment of the present application;

[0020] Figure 7An exemplary embodiment of the present application provides Figure 5 Grayscale schematic diagram of the corresponding corner map;

[0021] Figure 8 is a grayscale schematic diagram of an unfilled hole image provided by an exemplary embodiment of the present application;

[0022] Fig. 9 This is an exemplary embodiment of the present application. Figure 8 Grayscale diagram of an image with some holes filled;

[0023] Fig.10 This is an exemplary embodiment of the present application. Figure 8 Grayscale diagram of the image obtained after filling;

[0024] Fig.11 An exemplary embodiment of the present application provides Figure 7 Grayscale schematic diagram of the corresponding corner density map;

[0025] Fig.12 An exemplary embodiment of the present application provides Fig.11 Grayscale schematic diagram of the connected domain in ;

[0026] Fig.13 This is a deletion provided by an exemplary embodiment of the present application. Fig.12 Grayscale schematic diagram of the image obtained by the wrong connected domain in;

[0027] Fig.14 is a flowchart of step S122 provided in an embodiment of the present application;

[0028] Fig.15 is a schematic diagram of a line segment in a region of interest provided by an exemplary embodiment of the present application;

[0029] Fig.16 is a flowchart of step S123 provided in an embodiment of the present application;

[0030] Fig.17 is a grayscale schematic diagram of a measuring caliper on an alignment pattern provided by an exemplary embodiment of the present application;

[0031] Fig.18 This is an embodiment of the present application. Fig.17 A grayscale schematic diagram of an image obtained after correction of the alignment pattern;

[0032] Fig.19 is a grayscale schematic diagram of a measuring caliper on a clock pattern provided by an exemplary embodiment of the present application;

[0033] Fig. 20 is a schematic diagram of a DM code provided by another exemplary embodiment of the present application;

[0034] Fig.21 is a schematic diagram of a graphic identification code provided by another exemplary embodiment of the present application;

[0035] Fig. 22 is a schematic diagram of a graphic identification code provided by another exemplary embodiment of the present application;

[0036] Fig.23 is a structural block diagram of a positioning device based on multiple DM codes provided in one embodiment of the present application;

[0037] Fig.24 It is a structural block diagram of an autonomous mobile device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0039] The positioning method based on multiple DM codes in the embodiments of the present application can be applied to a positioning device based on multiple DM codes or an autonomous mobile device, and the positioning device based on multiple DM codes can be applied to an autonomous mobile device. The autonomous mobile device can be a mobile robot, and the mobile robot can include but is not limited to an autonomous mobile robot (Autonomous Mobile Robot, referred to as AMR) or an automated guided vehicle (Automated Guided Vehicle, referred to as AGV), etc. A downward-looking camera is installed on the autonomous mobile device, and the downward-looking camera is used to capture and identify the DM code on the ground.

[0040] The embodiment of the present application adopts a graphic identification code including multiple DM codes as a navigation graphic to perform navigation positioning based on the DM code. In actual application, multiple graphic identification codes can be deployed on the ground, and multiple DM codes can be deployed in each graphic identification code. For example, see Figure 3 , Figure 3 2 is a schematic diagram of a graphic identification code provided by an exemplary embodiment of the present application. The graphic identification code R1 includes a plurality of DM codes, and the plurality of DM codes are arranged in the graphic identification code R1 in a certain arrangement.

[0041] The output information of the graphic identification code is the identity document (ID) of the graphic identification code. The encoding information of each DM code in the graphic identification code includes the ID of the graphic identification code and the position of the DM code in the graphic identification code (referred to as the subcode position, which can also be understood as the number of the DM code).

[0042] For example, suppose Figure 3The ID of the graphic identification code R1 shown is 00001, then Figure 3 The coding information of the 1st to 8th DM codes shown are 000011, 000012, 000013, 000014, 000015, 000016, 000017, and 000018 respectively.

[0043] That is to say, in the embodiment of the present application, as long as at least one DM code is decoded, the ID of the graphic identification code and the subcode position of the DM code can be obtained, and the position of the graphic identification code can be obtained based on the subcode position, thereby realizing the navigation positioning of the mobile robot.

[0044] It should be noted that the graphic identification code specifically includes several DM codes and the arrangement of multiple DM codes can be set according to actual needs. Figure 3 This is only an example and is not intended to limit the number and arrangement of DM codes.

[0045] See also Figure 4 , Figure 4 1 is a flow chart of a positioning method based on multiple DM codes provided by an embodiment of the present application. The positioning method based on multiple DM codes may include steps S110 to S140. The following describes the positioning method based on multiple DM codes by taking a mobile robot as an example.

[0046] Step S110: Acquire a first layer image, a second layer image, and a third layer image obtained by continuously downsampling the image to be detected.

[0047] The image to be detected in the embodiment of the present application may refer to an image including a DM code. The downward camera of the mobile robot may capture the DM code on the ground by shooting a ground image, thereby obtaining the image to be detected including the DM code.

[0048] In some embodiments, the image captured by the downward-looking camera is referred to as the original image. In order to avoid wasting computing power for invalid DM code recognition, after obtaining the original image, a preliminary determination may be made as to whether the DM code exists in the original image. For example, it may be determined whether there is a rectangular area formed by a plurality of black and white code blocks in the original image. If there is a rectangular area formed by a plurality of black and white code blocks in the original image, the original image may be used as the image to be detected, and the image to be detected may be continuously downsampled to obtain the first to third layer images, that is, step S110 is executed. If there is no rectangular area formed by a plurality of black and white code blocks in the original image, in order to avoid wasting computing power for invalid recognition, it may be determined whether there is a DM code in the next frame of the original image until the image to be detected with the DM code is captured, and step S110 is executed.

[0049] After obtaining the image to be detected, the image to be detected can be continuously downsampled to obtain an image pyramid, which includes at least the first to third layers of images. The image pyramid can be understood as a set of images, and all images in the image pyramid are obtained by continuously downsampling the image to be detected. The size of the bottom layer of the image pyramid is the largest, and the size of the image becomes smaller and smaller from the bottom layer to the top. That is, the first layer of images in the embodiment of the present application refers to the bottom layer of the image pyramid, that is, the image with the largest size. The size of the first layer of images is usually the same as the size of the image to be detected. The second layer of images refers to the second to last layer of images of the image pyramid, and the third layer of images refers to the third to last layer of images of the image pyramid. If the image pyramid only includes three layers of images, the third layer of images is the topmost image of the image pyramid.

[0050] For example, a Gaussian pyramid can be used to continuously downsample the image to be detected to obtain a Gaussian pyramid. The Gaussian pyramid is the first to third layer images from the bottom up. The Gaussian pyramid is obtained by downsampling layer by layer from the bottom up (no layer skipping). The length and width of the image obtained by each downsampling of the Gaussian pyramid are half of the length and width of the original image.

[0051] For example, see Figure 5 , Figure 5 It is a grayscale schematic diagram of an image to be detected provided by an exemplary embodiment of the present application. If the size of the image to be detected is 1280*1024, the size of the first layer image can be 1280*1024, the size of the second layer image obtained by downsampling the first layer image can be 640*512, and the size of the third layer image obtained by downsampling the second layer image can be 320*256.

[0052] In the embodiment of the present application, the pyramid is used for image downsampling processing, which can improve the image processing speed and thus improve the overall efficiency of positioning based on multiple DM codes.

[0053] Step S120: determining the actual position and actual angle of at least one DM code in the image to be detected according to the first layer image, the second layer image and the third layer image.

[0054] See also Figure 6 , Figure 6 1 is a flow chart of step S120 provided in an embodiment of the present application. Step S120 may include steps S121 to S123.

[0055] Step S121: Determine at least one DM code candidate region from the third layer image.

[0056] The DM code candidate area in the embodiment of the present application refers to an area where the possibility or probability of the existence of the DM code is extremely high. Since the third layer image has the largest size among the first to third layer images, determining the DM code candidate area based on the third layer image can improve the accuracy and efficiency of determining the candidate area.

[0057] In some embodiments, after obtaining the third layer image, the third layer image can be denoised to reduce the interference of noise in the subsequent image processing process, facilitate the subsequent DM code area positioning, and thus improve the accuracy and efficiency of identifying the DM code. For example, the third layer image can be denoised by mean filtering, and the denoising process of the denoised third layer image can adopt the image mean filtering denoising method. Among them, the mean filtering denoising algorithm can include but is not limited to the weighted mean filtering algorithm, the median filtering algorithm or the Gaussian filtering algorithm.

[0058] The denoised third layer image is subjected to corner extraction processing to obtain a corner map. For example, the Fast (Features from accelerated segment test) corner extraction method can be used to extract corners from the third layer image. The principle of the Fast corner extraction method is to take a detection point in the image, and determine whether the detection point is a corner point based on the pixels in the surrounding neighborhood with the point as the center. That is to say, if a pixel has a certain number of pixels around it that have different pixel values ​​from the point, it is considered to be a corner point. Those skilled in the art should understand that, in addition to the Fast corner extraction method, other corner extraction methods can also be used to perform corner extraction operations on the third layer image, such as the Scale Invariant Feature Transform (SIFT) algorithm or the Harris algorithm. For example, see Figure 7 , Figure 7 An exemplary embodiment of the present application provides Figure 5 The grayscale diagram of the corresponding corner point map is Figure 5 After Fast corner point extraction, we can get Figure 7 The corner point diagram shown includes multiple corner points.

[0059] After obtaining the corner point map, at least one connected domain in the corner point map can be determined. The connected domain of the image refers to an area in the image composed of pixels with the same pixel value and adjacent positions. Connected domain analysis refers to finding and marking independent connected domains in the image. Generally, a connected domain contains only one pixel value. In order to prevent the influence of pixel value fluctuation on the extraction of different connected domains, the embodiment of the present application can first perform binary division on the pixels in the corner point map, and then use the image after binary division to perform connected domain analysis to improve the accuracy of the determined connected domain. For example, after obtaining the corner point map, the corner point map can be gridded, and the number of corner points included in each grid is determined as the gray value of each grid to form a corner point density map; the corner point density map is sequentially subjected to binary segmentation, hole filling and opening operations to form a binary corner point density map, and there are only two kinds of black and white pixels in the binary corner point density map; a connected domain analysis method can be used to perform connected domain segmentation on the binary corner point density map to obtain at least one connected domain. The connected domain analysis method can include but is not limited to a two-pass scanning method and a seed filling method. In the embodiment of the present application, the connected domain search based on the angle density map can effectively cope with the lighting changes that affect the captured images in the actual working scene and improve the robustness of DM code positioning.

[0060] The size of each grid may be determined according to the actual size of the DM code. For example, assuming that the actual size of each DM code is 25 pixels, in order to have sufficient corner point statistics for each DM code, the size of each grid may be defined as 4 pixels.

[0061] Among them, the binary segmentation method may include but is not limited to the Otsu's method (OTSU) threshold segmentation algorithm, the adaptive threshold segmentation algorithm, the maximum entropy threshold segmentation algorithm, and the iterative threshold segmentation algorithm. Among them, OTSU uses the idea of ​​clustering to divide the grayscale of the image into two parts according to the grayscale level, so that the grayscale value difference between the two parts is the largest, and the grayscale difference between each part is the smallest, and a suitable grayscale level is found for division by calculating the variance. The OTSU algorithm is simple in calculation and is not affected by the brightness and contrast of the image. Therefore, preferably, the OTSU algorithm can be used to perform binary segmentation on the corner density map with the lowest probability of misclassification.

[0062] Hole filling refers to the filling of holes. A hole refers to a background area that is included in the boundary connected by foreground pixels. Hole filling refers to the operation of closing the holes. For an example, see Figures 8 to 10 , Figure 8 Schematic diagram of grayscale of an unfilled hole image provided by an exemplary embodiment of the present application. Fig. 9 This is an exemplary embodiment of the present application. Figure 8Grayscale representation of an image with some of the holes filled. Fig.10 This is an exemplary embodiment of the present application. Figure 8 Grayscale diagram of the image obtained after filling. Figure 8 The process of hole filling is as follows: Fig. 9 , the hole filling result is Fig.10 The hole filling method may include but is not limited to the morphological closing operation method and the contour drawing method.

[0063] Among them, image opening operation refers to the process of image after being eroded and expanded in sequence. After the image is eroded, the noise is removed, but the image is also compressed. Then the eroded image is expanded to remove the noise and retain the original image. The above-mentioned hole filling operation is mostly closed operation, that is, the hole is closed. And denoising is generally an open operation, that is, filtering out small white noise points. Opening operation can be used to eliminate small objects, separate objects at thin points, and smooth the boundaries of larger objects without changing their area.

[0064] For example, see Figure 7 , Fig.11 as well as Fig.12 , Fig.11 An exemplary embodiment of the present application provides Figure 7 Grayscale diagram of the corresponding corner density map. Fig.12 An exemplary embodiment of the present application provides Fig.11 Grayscale diagram of the connected domain in . Figure 7 The corresponding corner point density map is Fig.11 , Fig.11 The connected domain in (the binary corner density map) can be expressed as Fig.12 As shown, Fig.12 The 10 connected domains shown in Fig.12 The area selected by the middle rectangle is the connected domain.

[0065] After obtaining at least one connected domain, at least one DM code candidate area can be determined from the at least one connected domain. Specifically, the ratio of the number of corners (corners) in each connected domain to the area (area) occupied by the connected domain can be determined as the average actual angle density of each connected domain (p=corners / area), wherein the area occupied by the connected domain can be measured by the number of pixels. For example, if there are 100 pixels in the connected domain, the area of ​​the connected domain is 100. All connected domains are sorted in descending order according to the average actual angle density corresponding to each connected domain, and the first N connected domains with the largest average actual angle density are extracted from at least one connected domain, wherein N is a positive integer, and N can be the number of DM codes actually included in a graphic identification code. For example, assuming that a graphic identification code includes 9 DM codes, N can be 9. Based on the centers of each of the N connected domains, a rectangle of a preset size is made to obtain N DM code candidate areas. The preset size can be set according to actual needs, and the preset size is greater than the size of a DM code. For example, assuming that the size of the DM code is 25 pixels, the preset size can be a rectangle of 44 pixels. In this embodiment, by calculating the average actual angle density of the connected domain, the connected domain in which the DM code is more likely to exist can be quickly screened out, so as to subsequently locate the alignment pattern based on the screened connected domain.

[0066] In some embodiments, to avoid incorrect connectivity domains (e.g. Fig.12 In order to prevent the interference of the connected domain P) which is obviously not a DM code, the connected domains can be preliminarily screened before or after all the connected domains are sorted in descending order according to the average actual angle density corresponding to each connected domain, that is, the wrong connected domain in at least one connected domain is deleted, thereby improving the connected domain screening speed and the connected domain screening accuracy, so as to improve the calculation speed and the accuracy of locating the DM code as a whole.

[0067] In some embodiments, the number of pixels in each connected domain can be counted as the area of ​​each connected domain. It is determined whether the area of ​​each connected domain is greater than or equal to the area threshold. If the area of ​​the connected domain is greater than or equal to the area threshold, it means that the possibility of the existence of the DM code in the connected domain is high, and it can be determined that the connected domain meets the requirements. If the area of ​​the connected domain is less than the area threshold, it means that the possibility of the existence of the DM code in the connected domain is small, and it can be determined that the connected domain does not meet the requirements. Based on this, the connected domain whose area is less than the area threshold in at least one connected domain can be deleted. Among them, the area threshold can be determined according to the number of pixels actually included in each DM code. For example, assuming that there are 25 pixels in the DM code, the area threshold can be pre-set to 100, so as to eliminate the wrong connected domain.

[0068] In some other embodiments, the length and width of the circumscribed rectangle of each connected domain may be generated, for example Fig.12 The rectangle shown is the circumscribed rectangle of the connected domain, and the ratio of the length and width of each circumscribed rectangle is calculated. Determine whether the length-to-width ratio of the circumscribed rectangle of each connected domain meets the ratio threshold. If the length-to-width ratio of the circumscribed rectangle meets the ratio threshold, that is, the length-to-width ratio of the circumscribed rectangle is equal to or substantially equal to the ratio threshold, then it means that there is a high possibility of a DM code in the connected domain, and it can be determined that the connected domain meets the requirements. If the length-to-width ratio of the circumscribed rectangle does not meet the ratio threshold, that is, the length-to-width ratio of the circumscribed rectangle is not equal to the ratio threshold, then it means that there is a low possibility of a DM code in the connected domain, and it can be determined that the connected domain does not meet the requirements. Based on this, the connected domain of the connected domain in which the length-to-width ratio of the circumscribed rectangle in at least one connected domain does not meet the ratio threshold can be deleted. Among them, the ratio threshold can be predetermined based on the actual length-to-width ratio of the circumscribed rectangle of the DM code. For example, assuming that the actual ratio of the circumscribed rectangle of the DM code is one-to-one, the ratio threshold can be pre-set to one-to-one, thereby eliminating erroneous connected domains. For example, Fig.12 If the length-to-width ratio of the circumscribed rectangle of the connected domain P in is obviously not one-to-one, then the erroneous connected domains can be accurately removed by judging the length-to-width ratio of the circumscribed rectangle to improve the accuracy of screening connected domains.

[0069] In some embodiments, in order to improve the accuracy of connected domain screening, a connected domain whose area is greater than or equal to an area threshold and whose length-to-width ratio of a circumscribed rectangle does not meet a ratio threshold may be deleted from at least one connected domain.

[0070] For example, see Fig.13 , Fig.13 This is a deletion provided by an exemplary embodiment of the present application. Fig.12 Grayscale schematic diagram of the image obtained by removing the wrongly connected domain in .

[0071] In the embodiment of the present application, a triple judgment is performed by combining the area of ​​the connected domain, the length-width ratio of the circumscribed rectangle of the connected domain, and the average angle density of the connected domain, so that incorrect connected domains (such as Fig.12 The interference of the connected domain P in the image is eliminated, so as to quickly and accurately screen out the connected domain that meets the requirements, and then quickly and accurately generate candidate regions based on the connected domain, for example, Fig.13 The area 0-8 selected by the black bold rectangle is the DM code candidate area.

[0072] Step S122: determining at least one DM code alignment pattern in the second layer image according to at least one DM code candidate region.

[0073] See also Figure 2Each DM code includes an alignment pattern and a clock pattern, and the alignment pattern and the clock pattern form the four sides of the DM code. The alignment pattern is a solid line L side formed by two solid line adjacent sides. The clock pattern is a dotted line L side formed by two dotted line adjacent sides. The alignment pattern and the clock pattern are used to locate the DM code.

[0074] See also Fig.14 , Fig.14 1 is a flow chart of step S122 provided in an embodiment of the present application. Step S122 may include steps S1221 to S1224.

[0075] Step S1221: extract at least one region of interest from the second layer image according to at least one DM code candidate region, each DM code candidate region corresponds to a region of interest.

[0076] See also Fig.13 , at least one DM code candidate region can be mapped one by one to the second layer image to obtain at least one region of interest 0-8, and the alignment pattern of each DM code can be roughly located based on each region of interest. After the above processing of the third layer image, the obtained region of interest is very likely to include a DM code, and usually there will be a DM code in each region of interest. It should be noted that although this application Figure 3 The regions of interest 0-8 shown are selected by rectangles, but in actual applications, other shapes can be selected according to actual needs. Other shapes may include but are not limited to squares, circles, ellipses, and irregular polygons.

[0077] The region of interest (ROI) refers to an area to be processed that is outlined in a processed image in the form of a box, a circle, an ellipse, an irregular polygon, etc. An ROI extraction algorithm can be used to extract at least one ROI from the second layer image according to at least one DM code candidate region. The ROI extraction algorithm may include but is not limited to a difference image algorithm, an interactive extraction algorithm, and an automatic image segmentation extraction algorithm.

[0078] Step S1222: extracting a line segment in at least one region of interest.

[0079] An edge detection (ED) operator may be used to extract line segments in at least one region of interest. The edge detection operator may include, but is not limited to, a Canny edge detection operator, a Roberts edge detection operator, a Sobel edge detection operator, a Prewitt edge detection operator, a Krisch edge detection operator, and a Gaussian-Laplacian (LoG-Laplacian) operator. In this embodiment, the edge detection operator is used to extract line segments, which can improve the stability of line segment extraction.

[0080] For example, see Fig.13 and Fig.15 , Fig.15 is a schematic diagram of a line segment in a region of interest provided by an exemplary embodiment of the present application. Fig.13 The line segments of the region of interest 0-8 shown in the figure can be extracted to obtain all the line segments in the region of interest 0-8. For example, Fig.15 A line segment in one of the regions of interest is shown.

[0081] After extracting the line segments, for each line segment in the region of interest, multiple conditions can be judged in parallel or in series to screen out line segment pairs that simultaneously meet multiple conditions and meet the alignment pattern, i.e., the solid line L edge requirement. In some embodiments, the multiple conditions may include a first condition, a second condition, a third condition, and a fourth condition. If a line segment pair simultaneously meets the first to fourth conditions, the line segment pair is determined to be the alignment pattern (solid line L edge) in the corresponding region of interest.

[0082] In some embodiments, the method for determining whether the first condition is satisfied is as follows: for each line segment in each region of interest, the length of each line segment is calculated, and it is determined whether the length of each line segment is greater than the first length and less than the second length. If the length of the line segment is greater than the first length and less than the second length, it is determined that the line segment satisfies the first condition. If the length of the line segment is less than the first length or greater than the second length, it is determined that the line segment does not satisfy the first condition. The first length and the second length can be preset according to the length of the actual DM code. For example, if the length of the DM code is code_len, the first length can be preset to a value greater than code_len / 3, and the second length can be preset to a value less than code_len*1.1.

[0083] In some embodiments, the method for determining whether the second condition is satisfied is as follows: for any two line segments in each region of interest, obtain the angle between the two line segments. Determine whether the angle between the two line segments is greater than or equal to the first angle and less than or equal to the second angle. If the angle between the two line segments is greater than or equal to the first angle and less than or equal to the second angle, determine that the two line segments satisfy the second condition. If the angle between the two line segments is less than the first angle or greater than the second angle, determine that the two line segments do not satisfy the second condition. Among them, the first angle and the second angle can be set according to the actual accuracy requirements for the positioning DM code. For example, the first angle can be preset to 85 degrees and the second angle can be preset to 95 degrees.

[0084] In some embodiments, the method for determining whether the third condition is satisfied is as follows: for any two line segments having an intersection in each region of interest, determine whether the two endpoints of each line segment are both located on one side of the intersection. If the two endpoints of each line segment of the two line segments are both located on one side of the intersection, it is determined that the two line segments satisfy the third condition. If the two endpoints of each line segment of the two line segments are respectively located on both sides of the intersection, it is determined that the two line segments do not satisfy the third condition.

[0085] In some other embodiments, the method for determining whether the third condition is satisfied is as follows: for any two line segments having an intersection in each region of interest, the distance between the endpoint closest to the intersection and the intersection is calculated. Determine whether the distance is less than a third length. If the distance is less than the third length, it is determined that the two line segments satisfy the third condition. If the distance is greater than or equal to the third length, it is determined that the two line segments do not satisfy the third condition. The third length can be preset according to the length of the black and white code blocks in the actual DM code. The length of each black code block in the DM code is code_point_len, and the third length can be preset to a value less than code_point_len / 2.

[0086] In some embodiments, in order to improve the accuracy of line segment screening and thus improve the accuracy of positioning and alignment patterns, two implementation methods for determining whether the third condition is satisfied may be combined, that is, when it is determined based on both implementation methods that two line segments satisfy the third condition, then it is determined that the two line segments satisfy the third condition.

[0087] In some other embodiments, in order to improve the efficiency of line segment screening and improve the overall efficiency of positioning and aligning patterns, two implementation methods for determining whether the third condition is satisfied may be executed in series. As an example, when it is determined according to one implementation that the two endpoints of each of the two line segments are located on both sides of the intersection, another implementation method is performed to determine the size of the distance between the endpoint closest to the intersection and the intersection and the third length. As another example, when it is determined according to one implementation that the distance between the endpoint closest to the intersection and the intersection is greater than or equal to the third length, another implementation method is performed to determine whether the two endpoints of each of the two line segments are both located on one side of the intersection.

[0088] In some embodiments, the method for determining whether the fourth condition is satisfied is as follows: for any two line segments having an intersection in each region of interest, the distances between the distal end points of the two line segments and the intersection are calculated respectively, wherein the distal end points are the end points of the line segments far from the intersection. It is determined whether the distances are both less than the fourth length. If the distances between the distal end points of the two line segments and the intersection are both less than the fourth length, it is determined that the two line segments satisfy the fourth condition. If the distance between the distal end point and the intersection of at least one of the two line segments is greater than or equal to the fourth length, it is determined that the two line segments do not satisfy the fourth condition. The fourth length can be preset according to the length of the actual DM code and the length of the black and white code blocks in the DM code. For example, the fourth length can be preset to a value less than code_len+code_point_len / 2.

[0089] In the embodiment of the present application, by performing logical judgments of multiple conditions on line segment pairs in parallel or in series to screen out line segment pairs that simultaneously meet multiple conditions, the efficiency and accuracy of positioning and aligning patterns can be improved.

[0090] Step S1223: Filter out at least one group of line segment pairs satisfying multiple conditions from the line segments in at least one region of interest, each group of line segment pairs including two line segments, and the length of each group of line segment pairs is the total length of the two line segments included in each group of line segment pairs.

[0091] After the above multiple conditions are judged, line segment pairs that simultaneously meet the multiple conditions may be screened out, so as to locate the DM code in the region of interest according to the screened line segment pairs.

[0092] Step S1224: extract the first N line segment pairs with the largest length from at least one group of line segment pairs as the alignment pattern of at least one DM code, where N is a positive integer.

[0093] As mentioned above, the embodiment of the present application combines a graphic identification code including multiple DM codes for navigation positioning. As long as the position and angle of at least one DM code are determined, the position and angle of the graphic identification code can be determined. N can be preset according to actual needs and the number of DM codes included in a graphic identification code.

[0094] In some embodiments, as more DM codes are identified, the angle and position of the graphic identification code can be calculated by combining the angles and positions of more DM codes. In order to improve the accuracy of locating the position and angle of the graphic identification code, N can be set to be the number of DM codes included in a graphic identification code. For example, Figure 3 The graphic identification code shown includes 9 DM codes, so N=9 can be preset.

[0095] In other embodiments, since the position and angle of the graphic identification code can be calculated as long as the position and angle of at least one DM code are determined, in order to simplify the calculation and reduce the hardware requirements of the present method, N can be set to be less than the number of DM codes included in a graphic identification code and greater than or equal to 1.

[0096] For each region of interest, the selected line segment pairs corresponding to the region of interest may be sorted from long to short according to the length of the line segment pairs, and the line segment pair with the largest length is selected as the alignment pattern of the DM code in the region of interest. Fig.15 The line segments L1 and L2 in Fig.15 Alignment pattern of DM code in the region of interest is shown.

[0097] Step S123: Determine the actual position and actual angle of at least one DM code in the first layer image according to at least one alignment pattern.

[0098] See also Fig.16 , Fig.16 1 is a flowchart of step S123 provided in an embodiment of the present application. Step S123 may include steps S1231 to S1233.

[0099] Step S1231: Map at least one alignment pattern into the first layer image.

[0100] In some embodiments, after at least one alignment pattern is mapped to the first layer image, the alignment pattern may be corrected first, and then the corrected alignment pattern may be used to calculate the clock pattern according to step S1232, thereby improving the accuracy of DM code decoding. Specifically, multiple measuring calipers are generated on the alignment pattern in the first layer image, and the caliper directions of the multiple measuring calipers are from the outside to the inside of the DM code. Based on the multiple measuring calipers, the alignment pattern in the first layer image is corrected. See Fig.17 , Fig.17FIG. 1 is a grayscale schematic diagram of a measuring caliper on an alignment pattern provided by an exemplary embodiment of the present application. Fig.17 As shown in the figure, an arrow represents a measuring caliper, and the direction of the arrow is the caliper direction of the measuring caliper. There are multiple measuring calipers on each alignment pattern, and the caliper direction of each measuring caliper is from the outside to the inside of the DM code. The measurement spacing between adjacent measuring calipers is the length of the DM code (code_len), and the caliper length of each measuring caliper is code_len*3.

[0101] In order to speed up the calculation, the graphics can be rotated. Fig.17 As shown, the directions of the multiple measuring calipers on each alignment pattern are horizontal or vertical, and the caliper directions are all from the outside of the DM code to the inside. Along the caliper direction of each measuring caliper, the difference between the front and rear pixels in each measuring caliper is calculated as the gradient of the next pixel. Wherein, the gradient is a vector (vector). In the embodiment of the present application, the gradient represents the pixel change along the caliper direction at the pixel position. The pixel changes fastest and the rate of change is the largest at the pixel with the largest gradient along the caliper direction. The pixel with the largest gradient in each measuring caliper, the previous pixel is black, and the pixel itself is white is determined as the edge point of each measuring caliper. Wherein, the edge point can be recorded as P1 = (x1, y1), x1 is the index position of the P1 point in the measuring caliper, and y1 is the gradient size. Linear fitting is performed on the edge points of all measuring calipers to obtain a corrected alignment pattern. Wherein, the linear fitting method may include but is not limited to the least squares method, the gradient descent method, the Gauss-Newton method, the Leh-Ma algorithm, etc. Among them, the effect of linear fitting using the least squares method is the best.

[0102] For example, see Fig.17 , an alignment image includes multiple horizontal measuring calipers and multiple vertical measuring calipers. The least square method can be used to perform linear fitting on the edge points of the multiple horizontal measuring calipers along the caliper direction to obtain the first boundary where the multiple horizontal measuring calipers are located. The least square method can be used to perform linear fitting on the edge points of the multiple vertical measuring calipers along the caliper direction to obtain the second boundary where the multiple vertical measuring calipers are located. The first boundary and the second boundary form a corrected solid line L edge, that is, a corrected alignment pattern. The linear fitting operations in the horizontal and vertical directions can be performed in parallel to increase the linear fitting speed.

[0103] In some embodiments, in order to improve the accuracy of DM code decoding, the sub-pixel position of the edge point can be used for straight line fitting to improve the accuracy of the correction alignment pattern. Specifically, after obtaining the edge point of each measuring caliper, the quadratic function can be solved according to the edge point "P1 = (x1, y1)", the previous pixel point of the edge point "P0 = (x0, y0)" and the next pixel point "P2 = (x2, y2)". The vertex of the quadratic function is determined as the sub-pixel position of the edge point, and the vertex can also be understood as the final edge point. The least squares method is used to perform straight line fitting based on the sub-pixel position of the edge point, thereby improving the accuracy of positioning the alignment pattern in the DM code.

[0104] For example, see Fig.18 , Fig.18 This is an embodiment of the present application. Fig.17 The grayscale diagram of the image obtained after the alignment pattern is corrected. Fig.17 By correcting the alignment pattern in Fig.18 The alignment pattern in .

[0105] In the embodiment of the present application, a quick caliper is used to measure and extract edge points, thereby achieving fast and accurate fitting and improving the accuracy of positioning and aligning patterns.

[0106] Step S1232: According to at least one alignment pattern in the first layer image, a clock pattern of at least one DM code is calculated to obtain at least one DM code, where each DM code includes an alignment pattern and a clock pattern.

[0107] For each alignment pattern, the clock pattern corresponding to the alignment pattern can be calculated according to the length of the DM code corresponding to the alignment pattern and the number of black and white code blocks included in the DM code.

[0108] After obtaining the clock pattern of the DM code, in order to improve the accuracy of DM code positioning, the above-mentioned measuring caliper correction method can be used to correct the clock pattern, and then the corrected clock pattern is used to calculate the position and angle of the DM code in step S1233.

[0109] It should be noted that the specific process of "adopting the above-mentioned measuring caliper correction method to correct the clock pattern" is similar to the specific process of "adopting the above-mentioned measuring caliper correction method to correct the alignment pattern". Therefore, for the specific description of "adopting the above-mentioned measuring caliper correction method to correct the clock pattern", please refer to the relevant part of "adopting the above-mentioned measuring caliper correction method to correct the alignment pattern". However, it should be noted that since the clock pattern is a dotted line with a white part on its edge, in order to improve the accuracy of correcting the clock pattern, refer to Fig.19 , Fig.19 It is a grayscale schematic diagram of a measuring caliper on a clock pattern provided by an exemplary embodiment of the present application. The measuring spacing of the measuring caliper on the clock pattern should be set smaller than the measuring spacing of the measuring caliper on the quasi-pattern. For example, the measuring spacing of the measuring caliper on the clock pattern may be code_len / 4.

[0110] Step S1233: Determine the actual position and actual angle of at least one DM code according to the alignment pattern and the clock pattern of at least one DM code.

[0111] For each DM code, calculate the angles of each side in the alignment pattern and the clock pattern of each DM code, and calculate the actual angle of each DM code based on the angles of each side. Specifically, the sum of the angles of the four sides and 180 can be calculated, and then the sum can be divided by 4 to get the actual angle of the DM code. The angle of each side can be calculated based on a preset reference. For example, it can be calculated in the horizontal direction or the vertical direction. As an example, taking the horizontal direction as 0, the angle between each side and the horizontal direction is the angle of each side. As another example, taking the vertical direction as 0, the angle between each side and the vertical direction is the angle of each side. For example, see Fig. 20 , Fig. 20 FIG. 1 is a schematic diagram of a DM code provided by another exemplary embodiment of the present application. Fig. 20 As shown, each DM code includes four sides, namely sides 1-4, and the actual angle sub_ang_n of each DM code can be calculated according to the following expression:

[0112] sub_ang_n=(a1+a3+a2+90+a4+90) / 4;

[0113] Among them, a1, a2, a3 and a4 are the angles of side 1, side 2, side 3 and side 4 of the DM code respectively, and n is the number of the DM code, that is, sub_ang_n represents the actual angle of the nth DM code in the graphic identification code.

[0114] For each DM code, the alignment pattern of each DM code and the intersection points of each edge in the clock pattern are calculated to obtain the actual positions of the four corner points of each DM code. For an example, see Fig.21 , Fig.21 FIG. 1 is a schematic diagram of a graphic identification code provided by another exemplary embodiment of the present application. Fig.21 As shown, the graphic identification code R1 includes 9 DM codes 0-8, and the actual positions of the four corner points of each DM code are P'm_0, P'm_2, P'm_3 and P'm_4, where m is the number 0-8 of the DM code, and the number of the DM code can be understood as the position of the DM code in the graphic identification code.

[0115] Step S130: Determine the actual position and actual angle of the graphic identification code including at least one DM code based on the actual position and actual angle of at least one DM code.

[0116] By identifying the data area of ​​each DM code, that is, the area surrounded by the alignment pattern and the clock pattern, each DM code can be decoded to obtain the reference position of each MD code and the reference position of the center DM code of the graphic identification code. The reference position is saved when the DM code is deployed, and the center DM code is the DM code located at the center of the graphic identification code.

[0117] Specifically, by identifying the data area of ​​each DM code, the coding information of each DM code can be obtained, and each DM code corresponds to a coding information. According to the coding information of each DM code, the reference position of each DM code is determined. According to the coding information of at least one DM code, the reference position of the central DM code is determined.

[0118] There is a mapping relationship between the coding information and the reference position of the DM code. The coding information of each DM code includes the identity identifier (ID) of the graphic identification code where the DM code is located and the position of the DM code in the graphic identification code (that is, the number of the DM code). For example, each coding information can be expressed as ID+n, where n is the number of the DM code. By identifying the data area in the DM code, the coding information ID+n of the DM code can be obtained. According to the ID, the graphic identification code corresponding to the ID can be found. According to n, the reference position of the nth DM code in the graphic identification code corresponding to the ID can be found (stored when the DM code is deployed). According to the number of the DM code located at the center of the graphic identification code, for example 0, the reference position of the center DM code in the graphic identification code corresponding to the ID can be found.

[0119] See also Fig. 22 , Fig. 22 FIG. 1 is a schematic diagram of a graphic identification code provided by another exemplary embodiment of the present application. Fig. 22 As shown, the graphic identification code R1 includes 9 DM codes 0-8, among which the DM code 0 located at the center of the graphic identification code R1 is called the central DM code. The position of the DM code of the entire graphic identification code is known. Taking the center of the graphic identification code as the coordinate origin, the reference position of the four corner points of each DM code is calculated, and the reference positions of the four corner points of each DM code are recorded as Pn_0, Pn_2, Pn_3 and Pn_4 respectively, wherein n is the number of the DM code 0-8.

[0120] According to the actual position and reference position of at least one DM code, the homography matrix H between the pixel coordinate system and the world coordinate system is determined. For example, based on the principle that the actual position P'm corresponds to the reference position Pn, a random sample consensus (RANSAC) algorithm can be used to solve the homography matrix H = P'm / Pn between Pn and P'm according to the actual position P'm and the reference position Pn. Among them, the homography matrix can be understood as a projection matrix from one plane to another plane. In the embodiment of the present application, the homography matrix H can be understood as a position mapping relationship between an object in the world coordinate system and the pixel coordinate system.

[0121] According to the homography matrix H and the reference position Pn of the central DM code, the actual position of the graphic identification code is determined. For example, the actual position of the graphic identification code can be calculated using the following expression:

[0122] P_cen=sum(H*P0_i) / 4, i=1,2,3,4;

[0123] Among them, P_cen represents the actual position of the graphic identification code, represents the homography matrix, and P0_i represents the reference position of the i-th corner point of the central DM code.

[0124] The average value of the actual angle of at least one DM code is determined as the actual angle of the graphic identification code. For example, the actual angle of the graphic identification code can be calculated using the following expression:

[0125] angle_cen=sum(sub_ang_n) / count;

[0126] Wherein, angle_cen represents the actual angle of the graphic identification code, sub_ang_n represents the angle of the i-th DM code in at least one DM code, and count represents the number of at least one DM code.

[0127] Step S140: Positioning is performed based on the actual position and actual angle of the graphic identification code.

[0128] After determining the graphic identification code, the current actual position and actual angle of the mobile robot can be inferred based on the actual position and actual angle of the graphic identification code, thereby realizing the navigation and positioning of the mobile robot. Specifically, after obtaining the actual position and actual angle of the graphic identification code, the navigation system of the mobile robot will match the actual position and actual angle of the graphic identification code with a pre-established global two-dimensional code map (including the map of the graphic identification code), thereby determining the current actual position and actual angle of the mobile robot. After determining the current actual position and actual angle of the mobile robot, the navigation system will guide the mobile robot to move and navigate along the path output by the path planning algorithm or navigation algorithm according to a preset path planning algorithm or navigation algorithm, thereby realizing navigation and positioning based on the graphic identification code.

[0129] It should be noted that the positioning precision and accuracy of the mobile robot based on the graphic identification code navigation are affected by factors such as the layout density of the graphic identification code and the DM code therein, the resolution of the downward-looking camera, and the scanning speed. Therefore, when designing and using the graphic identification code for navigation, it is necessary to reasonably arrange the density of the graphic identification code and the DM code therein. The density of the graphic identification code should not be too large to avoid that the graphic identification code cannot be scanned in the road section between the graphic identification codes. The density of the graphic identification code should not be too small to avoid scanning two graphic identification codes at the same time, which will increase the computational complexity. The number of DM codes included in the graphic identification code should not be too large to increase the computational complexity. The number of DM codes included in the graphic identification code should not be too small to reduce the accuracy of the position and angle of the decoded graphic identification code. In addition, according to the actual hardware requirements, a downward-looking camera with a higher scanning speed and a higher resolution can be selected to improve the accuracy and robustness of positioning.

[0130] Based on step S110 to step S140, the method can be applied to embedded devices, and a large graphic identification code including multiple DM codes is used as a navigation graphic for navigation positioning, rather than a single DM code as a navigation graphic for navigation positioning, so as to ensure that the DM code can be photographed. As long as the position and angle of at least one DM code are determined, the position and angle of the large graphic identification code can be determined, and the problem of the inability to photograph the DM code and the failure of DM code decoding affecting the precision and accuracy of navigation positioning can be avoided when a single DM code is used for positioning, thereby improving the robustness, accuracy and stability of the overall graphic positioning of navigation positioning based on the DM code. In addition, the method identifies the actual position and actual angle of at least one DM code based on the first to third images obtained by continuously downsampling the image to be detected, and can quickly determine the position and angle of at least one DM code. As long as at least one DM code is determined, the position and angle of the large graphic identification code can be determined, so that when there are multiple DM codes in the captured image, the position of the graphic identification code can be quickly determined, thereby improving the accuracy of navigation positioning based on the DM code.

[0131] See also Fig.23 , Fig.23 1 is a block diagram of a positioning device based on multiple DM codes provided in one embodiment of the present application. The positioning device based on multiple DM codes 100 can be applied to a mobile robot, which may include but is not limited to an AGV or an AMR. Fig.23 The positioning device 100 based on the multi-DM code may include an image sampling module 110 , an image determination module 120 , a position and angle determination module 130 , and a navigation positioning module 140 .

[0132] The image sampling module 110 is used to obtain the first layer image, the second layer image and the third layer image obtained by continuously downsampling the image to be detected. The specific working process of the image sampling module 110 is shown in step S110, which will not be repeated here.

[0133] The image determination module 120 is used to determine the actual position and actual angle of at least one DM code in the image to be detected according to the first layer image, the second layer image and the third layer image. The specific working process of the image determination module 120 is shown in step S120 and will not be repeated here.

[0134] The position and angle determination module 130 is used to determine the actual position and actual angle of the graphic identification code including the at least one DM code based on the actual position and actual angle of the at least one DM code. The specific working process of the position and angle determination module 130 is shown in step S130 and will not be repeated here.

[0135] The navigation and positioning module 140 is used to perform positioning based on the actual position and actual angle of the graphic identification code. The specific working process of the navigation and positioning module 140 is shown in step S140, which will not be described in detail here.

[0136] Those skilled in the art can clearly understand that the above device provided in the embodiment of the present application can implement the method provided in the embodiment of the present application. The specific working process of the above-described device and module can refer to the corresponding process of the method in the embodiment of the present application, which will not be repeated here.

[0137] In the embodiments provided in the present application, the coupling, direct coupling or communication connection between the modules shown or discussed may be indirect coupling or communication coupling through some interfaces, devices or modules, and may be electrical, mechanical or other forms, and the embodiments of the present application do not impose specific limitations on this.

[0138] In addition, each functional module in the embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0139] See also Fig.24 , Fig.24 The autonomous mobile device 200 may include a memory 210 and a processor 220. The memory 210 stores an application program, which is configured to execute the method provided in the embodiment of the present application when called by the processor 220.

[0140] The processor 220 may include one or more processing cores. The processor 220 uses various interfaces and lines to connect various parts of the entire autonomous mobile device 200, and is used to run or execute instructions, programs, code sets or instruction sets stored in the memory 210, and call to run or execute data stored in the memory 210, perform various functions of the autonomous mobile device 200, and process data.

[0141] The processor 220 can be implemented in at least one of the following hardware forms: digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 220 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 220, but may be implemented separately through a communication chip.

[0142] The memory 210 may include a random access memory (RAM) or a read-only memory (ROM). The memory 210 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 210 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may store data created by the autonomous mobile device 200 during use, etc.

[0143] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores program code, and the program code is configured to execute the method provided in the embodiment of the present application when called by a processor.

[0144] The computer-readable storage medium may be an electronic memory such as a flash memory, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a hard disk or a ROM.

[0145] In some embodiments, the computer readable storage medium includes a non-volatile computer readable medium (Non-Transitory Computer-Readable Storage Medium, referred to as Non-TCRSM). The computer readable storage medium has a storage space for the program code that executes any method step in the above method. These program codes can be read from or written into one or more computer program products. The program code can be compressed in an appropriate form.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A positioning method based on multiple DM codes, characterized in that: include: Acquire a first layer image, a second layer image, and a third layer image obtained by continuously downsampling the image to be detected; Determine an actual position and an actual angle of at least one DM code in the image to be detected according to the first layer of images, the second layer of images, and the third layer of images; Determining the actual position and actual angle of the graphic identification code including the at least one DM code based on the actual position and actual angle of the at least one DM code; Positioning is performed based on the actual position and actual angle of the graphic identification code.

2. The method according to claim 1, characterized in that The determining, according to the first layer of images, the second layer of images, and the third layer of images, an actual position and an actual angle of at least one DM code in the image to be detected comprises: Determining at least one DM code candidate region from the third layer image; determining, in the second layer image, an alignment pattern of at least one DM code according to the at least one DM code candidate region; According to the at least one alignment pattern, an actual position and an actual angle of at least one DM code are determined in the first layer image.

3. The method according to claim 2, characterized in that The step of determining at least one DM code candidate region from the third layer image comprises: Performing denoising on the third layer image; Performing corner point extraction processing on the denoised third layer image to obtain a corner point map; Determining at least one connected domain in the corner graph; At least one DM code candidate region is determined from the at least one connected region.

4. The method according to claim 3, characterized in that The determining of at least one connected domain in the corner point graph comprises: Dividing the corner point map into grids, determining the number of corner points included in each grid as the grayscale value of each grid, and forming a corner point density map; The corner point density map is sequentially subjected to binary segmentation, hole filling and opening operations to form a binary corner point density map; Perform connected domain segmentation on the binary corner point density map to obtain at least one connected domain.

5. The method according to claim 3, characterized in that: The step of determining at least one DM code candidate region from the at least one connected domain comprises: The ratio of the number of corner points in each connected domain to the area occupied by the connected domain is determined as the average actual angle density of each connected domain; Extracting first N connected domains with the largest average actual angle density from the at least one connected domain, where N is a positive integer; Based on the centers of the N connected domains, rectangles of preset sizes are made to obtain N DM code candidate regions.

6. The method according to claim 5, characterized in that Before extracting the top N connected domains with the largest average actual angle density from the at least one connected domain, the method further includes: Determine whether the area of ​​each connected domain is greater than or equal to the area threshold; Determine whether the length-to-width ratio of the circumscribed rectangle of each connected domain meets the ratio threshold; In the at least one connected domain, a connected domain whose area is greater than or equal to an area threshold and / or whose length-to-width ratio of a circumscribed rectangle does not meet a ratio threshold is deleted.

7. The method according to claim 2, characterized in that The step of determining, in the second layer image according to the at least one DM code candidate area, an alignment pattern of at least one DM code, comprises: Extracting at least one region of interest in the second layer image according to the at least one DM code candidate region, each DM code candidate region corresponding to a region of interest; extracting line segments in the at least one region of interest; At least one group of line segment pairs satisfying a plurality of conditions is screened out from the line segments in the at least one region of interest, each group of line segment pairs comprising two line segments, and the length of each group of line segment pairs is the total length of the two line segments included in each group of line segment pairs; From the at least one group of line segment pairs, first N line segment pairs with the largest length are extracted as alignment patterns of at least one DM code, where N is a positive integer.

8. The method according to claim 7, characterized in that The multiple conditions include a first condition, and before selecting at least one group of line segment pairs satisfying the multiple conditions from the line segments in the at least one region of interest, the method further includes: For each line segment in the at least one region of interest, determining whether a length of each line segment is greater than a first length and less than a second length; If the length of the line segment is greater than the first length and less than the second length, it is determined that the line segment satisfies the first condition.

9. The method according to claim 7, characterized in that: The multiple conditions include a second condition, and before selecting at least one group of line segment pairs satisfying the multiple conditions from the line segments in the at least one region of interest, the method further includes: For any two line segments in the at least one region of interest, determining whether an angle between the two line segments is greater than or equal to a first angle and less than or equal to a second angle; If the angle between the two line segments is greater than or equal to the first angle and less than or equal to the second angle, it is determined that the two line segments satisfy the second condition.

10. The method according to claim 7, characterized in that The multiple conditions include a third condition, and before screening out at least one group of line segment pairs satisfying the multiple conditions from the line segments in the at least one region of interest, the method further includes: For any two line segments having an intersection in the at least one region of interest, determining whether both endpoints of each line segment are located on one side of the intersection; If both endpoints of each of the two line segments are located on one side of the intersection, it is determined that any two line segments having the intersection satisfy the third condition.

11. The method according to claim 7, characterized in that The multiple conditions include a third condition, and before screening out at least one group of line segment pairs satisfying the multiple conditions from the line segments in the at least one region of interest, the method further includes: For any two line segments having an intersection in the at least one region of interest, determining whether a distance between an endpoint of the two line segments closest to the intersection and the intersection is less than a third length; If the distance between the endpoint closest to the intersection point among the endpoints of the two line segments and the intersection point is less than the third length, it is determined that the two line segments satisfy the third condition.

12. The method according to claim 7, characterized in that The multiple conditions include a fourth condition, and before selecting at least one group of line segment pairs satisfying the multiple conditions from the line segments in the at least one region of interest, the method further includes: For any two line segments having an intersection in the at least one region of interest, determining whether the distances between the distal end points of the two line segments and the intersection are both less than a fourth length, wherein the distal end points are end points of the line segments far away from the intersection; If the distances between the distal end points and the intersection point of the two line segments are both less than the fourth length, it is determined that the two line segments satisfy the fourth condition.

13. The method according to claim 2, characterized in that Determining the actual position and the actual angle of at least one DM code in the first layer image according to the at least one alignment pattern comprises: mapping the at least one alignment pattern into the first layer image; According to at least one alignment pattern in the first layer of images, a clock pattern of at least one DM code is calculated to obtain at least one DM code, each DM code including an alignment pattern and a clock pattern; According to the alignment pattern of at least one DM code and the clock pattern, the actual position and the actual angle of at least one DM code are determined.

14. The method according to claim 13, characterized in that Before calculating at least one clock pattern of a DM code according to at least one alignment pattern in the first layer image, the method further includes: Generating a plurality of measuring calipers on the alignment pattern in the first layer of images, wherein the caliper directions of the plurality of measuring calipers are from the outside to the inside of the DM code; Based on the plurality of measurement calipers, an alignment pattern in the first layer image is corrected.

15. The method according to claim 14, characterized in that The step of correcting the alignment pattern in the first layer of images based on the plurality of measuring calipers comprises: Along the caliper direction of each measuring caliper, the difference between the front and rear pixel points in each measuring caliper is calculated as the gradient of the rear pixel point; The pixel point with the largest gradient in each measuring caliper and whose previous pixel point is black and its own pixel point is white is determined as the edge point of each measuring caliper; Perform straight line fitting on the edge points of all measuring calipers to obtain the corrected alignment pattern.

16. The method according to claim 15, characterized in that After determining the pixel point with the largest gradient in each first measuring caliper and whose previous pixel point is black and whose own pixel point is white as the edge point of each first measuring caliper, the method further comprises: Solve the quadratic function based on the edge point, the previous pixel point and the next pixel point of the edge point; The vertices of the quadratic function are determined as the final edge points.

17. The method according to claim 13, characterized in that The determining the actual position and actual angle of at least one DM code according to the alignment pattern and the clock pattern of at least one DM code comprises: For each DM code of the at least one DM code, calculating the intersection points of the alignment pattern and the edges in the clock pattern to obtain the actual positions of the four corner points of each DM code; For each DM code in the at least one DM code, the angles of the alignment pattern and the edges in the clock pattern are calculated, and the actual angle of each DM code is calculated according to the angles of the edges.

18. The method according to any one of claims 1 to 17, characterized in that: The determining the actual position of the graphic identification code including the at least one DM code based on the actual position of the at least one DM code comprises: Decoding the at least one DM code to obtain a reference position of the at least one MD code and a reference position of a central DM code, wherein the reference position has been saved when the DM code is deployed, and the central DM code is a DM code located at the center of the graphic identification code; Determining a homography matrix according to an actual position and a reference position of the at least one DM code; The actual position of the graphic identification code is determined according to the homography matrix and the reference position of the central DM code.

19. The method according to claim 18, characterized in that The decoding of the at least one DM code to obtain a reference position of the at least one MD code and a reference position of a center DM code includes: Decoding the at least one DM code to obtain at least one coded information, each DM code corresponding to one coded information; According to the at least one coding information, a reference position of the at least one MD code and a reference position of the central DM code are determined, wherein a mapping relationship exists between the coding information and the reference position of the DM code.

20. The method according to any one of claims 1 to 17, characterized in that: The determining the actual angle of the graphic identification code including the at least one DM code based on the actual angle of the at least one DM code comprises: An average value of the actual angles of the at least one DM code is determined as the actual angle of the pattern identification code.

21. A positioning device based on multiple DM codes, characterized in that: include: An image sampling module is used to obtain a first layer image, a second layer image and a third layer image obtained by continuously downsampling the image to be detected; An image determination module, configured to determine an actual position and an actual angle of at least one DM code in the image to be detected according to the first layer image, the second layer image and the third layer image; A position and angle determination module, configured to determine the actual position and actual angle of the graphic identification code including the at least one DM code based on the actual position and actual angle of the at least one DM code; The navigation and positioning module is used for positioning based on the actual position and actual angle of the graphic identification code.

22. An autonomous mobile device, characterized in that: include: A memory and a processor, wherein the memory stores an application program, and the application program is used to execute the method according to any one of claims 1 to 20 when called by the processor.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes are used to execute the method according to any one of claims 1 to 20 when called by the processor.

Images