Robot positioning method, device, equipment, storage medium and program product

Abstract

The present application provides a robot positioning method, device, equipment, computer-readable storage medium and computer program product; the method includes: during the process of the robot moving from a first planar area where a graphic code is deployed to an inclined plane auxiliary area, determining a first positioning position of the robot at a first intersection position between the first planar area and the inclined plane auxiliary area based on the graphic code; controlling the robot to move from the first intersection position in the inclined plane auxiliary area to a second planar area; when the robot moves to a second intersection position between the inclined plane auxiliary area and the second planar area, determining a second positioning position of the robot at the second intersection position based on the first positioning position; during the process of the robot moving on the second planar area, determining a third positioning position of the robot when moving on the second planar area based on the second positioning position. Through the present application, the adaptability of robot positioning can be improved.

Description

Technical Field

[0001] The present application relates to robot technology in artificial intelligence, and particularly to a robot positioning method, device, equipment, storage medium and program product. Background Art

[0002] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless, autonomous driving, drones, robots, smart healthcare, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0003] Taking robots as an example, there is a wide range of application requirements for construction robots in construction projects such as building and demolishing objects. During the application process of construction robots, especially when climbing the superstructure, only when the construction robot accurately identifies its current position can it help the construction robot climb onto the building and position itself on the building.

[0004] In related technologies, the positioning of construction robots mostly relies on dynamic capture systems. This method requires multiple sensors to be arranged in advance in the site to capture the movement of the robot. For example, a typical optical motion capture system usually uses 6 - 8 cameras to surround the site, and the overlapping area of the camera fields of view is the movement range of the robot. However, this method not only has high costs but also limits the size of the site, and when the robot exceeds the movement range, it cannot be positioned. Summary of the Invention

[0005] Embodiments of the present application provide a robot positioning method, device, equipment, computer-readable storage medium and computer program product, which can improve the adaptability of robot positioning.

[0006] The technical solution of the embodiments of the present application is implemented as follows:

[0007] Embodiments of the present application provide a robot positioning method, including:

[0008] During the process of the robot moving from the first planar area where a graphic code is deployed to the inclined plane auxiliary area, determining a first positioning position of the robot at a first intersection position between the first planar area and the inclined plane auxiliary area based on the graphic code;

[0009] Controlling the robot to move from the first intersection position in the inclined plane auxiliary area to a second planar area;

[0010] When the robot moves to a second intersection position between the inclined plane auxiliary area and the second planar area, determining a second positioning position of the robot at the second intersection position based on the first positioning position;

[0011] During the movement of the robot on the second plane area, a third positioning position of the robot during movement on the second plane area is determined based on the second positioning position.

[0012] An embodiment of the present application provides a robot positioning device, including:

[0013] A first determination module, configured to determine a first positioning position of the robot at a first intersection position between the first plane area and the inclined plane auxiliary area based on the graphic code during the process of the robot moving from the first plane area where the graphic code is deployed to the inclined plane auxiliary area;

[0014] A movement control module, configured to control the robot to move from the first intersection position in the inclined plane auxiliary area to the second plane area;

[0015] A second determination module, configured to determine a second positioning position of the robot at the second intersection position when the robot moves to the second intersection position between the inclined plane auxiliary area and the second plane area based on the first positioning position;

[0016] A third determination module, configured to determine a third positioning position of the robot during movement on the second plane area based on the second positioning position during the process of the robot moving on the second plane area.

[0017] In the above solution, the first determination module is further configured to obtain a first image of the robot moving to the first intersection position; perform graphic code recognition processing on the first image to obtain a first graphic code corresponding to the first image; when the number of the first graphic codes is at least two, select a first number of first graphic codes from the at least two first graphic codes to form a graphic code array; and determine the first positioning position of the robot at the first intersection position based on the graphic code array.

[0018] In the above solution, a first indication identifier for identifying the starting position of the inclined plane auxiliary area is deployed in the inclined plane auxiliary area. Before obtaining the first image of the robot moving to the first intersection position, the device further includes: a fourth determination module, configured to obtain a second image during the process of the robot moving from the first plane area to the inclined plane auxiliary area; perform identifier recognition on the second image, and when the first indication identifier is recognized from the second image, determine that the robot moves to the starting position of the inclined plane auxiliary area, where the starting position is the first intersection position.

[0019] In the above solution, the first determination module is further configured to select a second quantity of first graphic codes located at vertices from the graphic code array, and use each of the second quantity of first graphic codes as vertex graphic codes; based on each of the vertex graphic codes, determine first reference positions corresponding to the robot at the first intersection positions respectively; based on each of the first reference positions, determine a first positioning position of the robot at the first intersection positions.

[0020] In the above solution, the first determination module is further configured to perform decoding processing on each of the vertex graphic codes to obtain the encoding information of each of the vertex graphic codes; based on the encoding information of each of the vertex graphic codes, determine the position information of the corresponding vertex graphic code in the graphic code array, and determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot; based on the relative position relationship and the position information of each of the vertex graphic codes in the graphic code array, determine first reference positions corresponding to the robot at the first intersection positions respectively.

[0021] In the above solution, the first determination module is further configured to obtain the mapping relationship between the encoding information of each of the vertex graphic codes and the position information of the corresponding vertex graphic code; based on the encoding information of each of the vertex graphic codes and the mapping relationship, determine the position information of the corresponding vertex graphic code in the graphic code array.

[0022] In the above solution, the first determination module is further configured to determine the two-dimensional coordinates of a third quantity of corner points corresponding to the vertex graphic code, as well as the size information of the vertex graphic code; perform an averaging process on the two-dimensional coordinates of the third quantity of corner points to obtain the two-dimensional coordinates of the center point of the vertex graphic code; according to the three-dimensional coordinates of the center point in the plane coordinate system corresponding to the first plane region and the size information of the vertex graphic code, determine the three-dimensional coordinates of the third quantity of corner points in the plane coordinate system; according to the matching pairs composed of the two-dimensional coordinates and the corresponding three-dimensional coordinates of the third quantity of corner points, determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot.

[0023] In the above solution, a first guiding line for connecting a first indication mark and a second indication mark is deployed in the inclined plane auxiliary area. The first indication mark is used to mark the first intersection position, and the second indication mark is used to mark the second intersection position. The movement control module is further configured to obtain a third image during the process of the robot moving from the first intersection position in the inclined plane auxiliary area to the second plane area. During the movement of the robot, the movement path of the robot is adjusted based on the third image to control the robot to move from the first intersection position along a first path indicated by the first guiding line to the second intersection position.

[0024] In the above solution, the movement control module is further configured to preprocess the third image to obtain a preprocessed third image. The preprocessing includes at least one of the following: grayscale conversion, binarization, and filtering. Edge detection is performed on the preprocessed third image to obtain a first reference line and a second reference line corresponding to the edge of the first guiding line in the preprocessed third image. The angle bisector of a first included angle between the first reference line and the second reference line, the center line, and a reference point of the third image are determined. A second included angle between the angle bisector and the center line, and a distance between the angle bisector and the reference point are obtained, so as to adjust the movement path of the robot based on the included angle and the distance.

[0025] In the above solution, the second determination module is further configured to determine the horizontal length and vertical height of the inclined plane auxiliary area, and the movement direction of the robot. Based on the first positioning position, the horizontal length, the vertical height, and the movement direction, the second positioning position of the robot at the second intersection position is determined.

[0026] In the above solution, a second guiding line for guiding the movement of the robot is deployed on the second plane area. Before determining the third positioning position of the robot when moving on the second plane area based on the second positioning position, the device further includes: a control module, configured to obtain a fourth image during the process of the robot moving on the second plane area. During the movement of the robot, the movement path of the robot is adjusted based on the fourth image to control the robot to move on the second plane area along a second path indicated by the second guiding line.

[0027] In the above solution, a third indication identifier for indicating the second planar region is deployed on the second planar region. The third determination module is further configured to obtain a fifth image during the movement of the robot on the second planar region; perform identifier recognition on the fifth image, and when the third indication identifier is recognized from the fifth image, determine the length of the second planar region and the movement direction of the robot; based on the second positioning position, the length of the second planar region, and the movement direction, determine a third positioning position of the robot when moving on the second planar region.

[0028] In the above solution, the third determination module is further configured to obtain a fifth image during the movement of the robot on the second planar region; perform graphic code recognition processing on the fifth image to obtain a second graphic code corresponding to the fifth image; determine a second reference position of the robot on the second planar region based on the second graphic code, and based on the second positioning position and the second reference position, determine a third positioning position of the robot when moving on the second planar region.

[0029] An embodiment of the present application provides an electronic device, including:

[0030] A memory for storing executable instructions;

[0031] A processor, when executing the executable instructions stored in the memory, implements the robot positioning method provided by the embodiment of the present application.

[0032] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to implement the robot positioning method provided by the embodiment of the present application when executed.

[0033] An embodiment of the present application provides a computer program product including a computer program or instruction, which when executed by a processor, implements the robot positioning method provided by the embodiment of the present application.

[0034] The embodiment of the present application has the following beneficial effects:

[0035] Applying the embodiment of the present application, when positioning a construction robot, such as when the robot moves from the first planar region to the second planar region through the inclined plane auxiliary region, positioning is performed only based on the graphic code deployed in the first planar region, without the need to deploy multiple sensors to capture the movement of the robot for positioning, which reduces costs and is not restricted by the site, improving the positioning applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a layout diagram of a modular structure block provided by an embodiment of the present application;

[0037] Figure 2 It is a schematic architecture diagram of the robot positioning system 100 provided by an embodiment of the present application;

[0038] Figure 3 It is a schematic structural diagram of the electronic device 500 provided by an embodiment of the present application;

[0039] Figure 4 It is a schematic flowchart of the robot positioning method provided by an embodiment of the present application;

[0040] Figure 5A It is a schematic diagram of the method for determining the first positioning position provided by an embodiment of the present application;

[0041] Figure 5B It is a schematic diagram of the method for determining the first positioning position provided by an embodiment of the present application;

[0042] Figure 6 It is a schematic diagram of the site deployment provided by an embodiment of the present application;

[0043] Figure 7 It is a schematic diagram of the moving direction provided by an embodiment of the present application;

[0044] Figure 8 It is a schematic diagram of visual line following provided by an embodiment of the present application. Detailed implementation manners

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0046] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0047] In the following description, the terms "first / second..." involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second..." can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0049] Most construction robots in the related art rely on dynamic capture systems and general templates for positioning. Among them, when using a dynamic capture system to capture the position of a robot, cameras need to be set in advance at key parts of the robot, and multiple sensors are arranged in the site to capture the movement of the robot. For example, a typical optical motion capture system usually uses 6 to 8 cameras to surround the site, and the overlapping area of the camera fields of view is the movement range of the robot. The positioning method based on a general template usually uses modular structural blocks. Refer to Figure 1 , Figure 1 which is a layout schematic diagram of the modular structural block provided in the embodiment of the present application. Each cube block has QR code labels pasted on six faces, and the robot needs to be equipped with a camera to identify the QR codes to obtain its position relative to the current block. The inventor found that the positioning method based on a dynamic capture system not only has high costs, but also limits the size of the site. When the robot exceeds the movement range, its position cannot be located; the positioning method based on a general template only relies on the QR code labels on the modular blocks for local positioning, and the robot needs to move randomly in the scene until a position where the positioning label can appear in the field of view. This kind of positioning method cannot obtain the global position of the robot, and when the scene is very large and the positioning labels are very sparse, the robot may lose its position and spend a lot of time looking for the labels, resulting in poor positioning efficiency.

[0050] Therefore, the embodiment of the present application provides a robot positioning method, device, equipment, computer storage medium and computer program product to at least solve the above problems. The following describes the exemplary application of the electronic device provided in the embodiment of the present application. The electronic device provided in the embodiment of the present application can be implemented as a server, or jointly implemented by a server and a terminal device. The following takes the joint implementation of the robot positioning method provided in the embodiment of the present application by a server and a terminal device as an example for description.

[0051] Exemplarily, refer to Figure 2 , Figure 2 which is an architecture schematic diagram of the robot positioning system 100 provided in the embodiment of the present application. To support an exemplary application, terminals (exemplarily showing terminals 400-1 and 400-2) are connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two, and uses a wireless link to implement data transmission.

[0052] The terminal can be a device with image acquisition and positioning functions, such as various types of user terminals like robots with image acquisition and positioning functions (such as industrial production robots, construction robots, etc.), smartphones, tablets, laptops, etc., and can also be a desktop computer, a television with positioning functions, or a combination of any two or more of these data processing devices; the server 200 can be either a single server configured to support various services, or configured as a server cluster, or a cloud server, etc.

[0053] In some embodiments, the robot positioning method provided by the embodiments of the present application can be executed by the terminal alone. Taking the terminal as a robot as an example, during the process of the robot moving from the first planar area where the graphic code is deployed to the inclined plane assistance area, the first positioning position of the first intersection position of the robot in the first planar area and the inclined plane assistance area is determined based on the graphic code; the robot is controlled to move from the first intersection position in the inclined plane assistance area to the second planar area; when the robot moves to the second intersection position of the inclined plane assistance area and the second planar area, based on the first positioning position, the second positioning position of the robot at the second intersection position is determined; during the movement of the robot on the second planar area, the third positioning position of the robot during the movement on the second planar area is determined based on the second positioning position; thus, a series of operations are realized in real time on the terminal side, improving the robot positioning efficiency.

[0054] In some embodiments, the robot positioning method provided by the embodiments of the present application can be coordinated and executed by a terminal and a server. Taking the terminal as the robot and the server 200 as the background server corresponding to the robot as an example, when the robot moves from the first planar area where the graphic code is deployed to the inclined plane assistance area, the robot sends the positioning images collected during the movement to the server 200; the server 200 performs graphic code recognition processing on the positioning images and performs positioning based on the recognized graphic code, such as determining the first positioning position of the robot at the first intersection position of the first planar area and the inclined plane assistance area based on the graphic code, and returning the first positioning position to the robot to control the robot to move from the first intersection position in the inclined plane assistance area to the second planar area, and sending the positioning images collected in real time during the movement to the server 200. The server 200 determines the second positioning position of the robot at the second intersection position based on the first positioning position and returns the second positioning position to the robot to control the robot to move on the second planar area from the second positioning position, and sends the positioning images collected in real time during the movement to the server 200. The server 200 determines the third positioning position of the robot when moving on the second planar area based on the second positioning position and returns it to the robot; thus, the determination of the first positioning position, the second positioning position, and the third positioning position is completed by the server, reducing the data processing pressure on the terminal side (the robot), and being applicable to the situation where the capacity of the positioning images is relatively large or the positioning requirements are relatively large.

[0055] See Figure 3 , Figure 3 is a schematic structural diagram of the electronic device 500 provided by the embodiments of the present application. In practical applications, the electronic device 500 can be Figure 2 the terminal or the server 200 in Figure 2 Taking the terminal shown as an example, the electronic device for implementing the robot positioning method of the embodiments of the present application will be described. Figure 3 The electronic device 500 shown includes: at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. Each component in the electronic device 500 is coupled together through a bus system 540. It can be understood that the bus system 540 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear description, in Figure 3 all kinds of buses are labeled as the bus system 540.

[0056] The processor 510 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0057] The user interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons, and controls.

[0058] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 550 optionally includes one or more storage devices that are physically located remotely from the processor 510. The memory 550 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 550 described in the embodiments of the present application is intended to include any suitable type of memory.

[0059] In some embodiments, the memory 550 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are exemplarily described below. The operating system 551 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks; the network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520. Exemplary network interfaces 520 include: Bluetooth, Wi-Fi (Wireless Fidelity), and USB (Universal Serial Bus), etc.; the presentation module 553 is used to enable the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 associated with the user interface 530 (such as a display screen, a speaker, etc.); the input processing module 554 is used to detect and translate one or more user inputs or interactions from one of the one or more input devices 532.

[0060] In some embodiments, the robot positioning device provided by the embodiments of the present application can be implemented in software. Figure 3 Shown is the robot positioning device 555 stored in the memory 550, which can be software in the form of programs and plugins, etc., including the following software modules: a first determination module 5551, a movement control module 5552, a second determination module 5553, and a third determination module 5554. These modules are logical, so they can be combined arbitrarily or further split according to the functions to be implemented. The functions of each module will be described below.

[0061] In other embodiments, the robot positioning device provided by the embodiments of the present application can be implemented in hardware. As an example, the robot positioning device provided by the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the robot positioning method provided by the embodiments of the present application. For example, a processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0062] In some embodiments, a terminal or a server can implement the robot positioning method provided by the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a positioning APP or an instant messaging APP; it can also be a small program, that is, a program that only needs to be downloaded into a browser environment to run; it can also be a small program that can be embedded into any APP. In short, the above computer program can be any form of application program, module, or plugin.

[0063] Based on the above description of the robot positioning system provided by the embodiments of the present application, the robot positioning method provided by the embodiments of the present application will be described below. In actual implementation, this method can be implemented independently by Figure 2 the terminal or server 200 shown, or can be implemented collaboratively by Figure 2 the terminal and server 200 shown. Next, it will be combined with Figure 2 and Figure 4 ,Figure 4 The following is a schematic flowchart of the robot positioning method provided by the embodiments of the present application. Taking Figure 2 the terminal shown (the terminal is taken as a robot for example) to implement the robot positioning method provided by the embodiments of the present application alone as an example for illustration.

[0064] Step 101: During the process of the robot moving from the first planar area where the graphic code is deployed to the inclined plane assistance area, determine the first positioning position of the first intersection position of the robot in the first planar area and the inclined plane assistance area based on the graphic code.

[0065] In practical applications, before the robot moves for positioning, it is necessary to deploy the site where the robot needs to move or pass through in advance. Taking the robot moving between different planar areas (such as the first planar area and the second planar area) located on different horizontal planes as an example, a certain number of graphic codes are densely deployed in the first planar area to ensure that when the robot equipped with a camera moves in the first planar area, the camera can capture an image including at least one graphic code. The inclined plane assistance area is an inclined plane, which refers to a plane with a certain angle (the angle can be 0 to 90 degrees) with the horizontal plane, and is used to assist the robot in moving between planar areas located on different horizontal planes, such as assisting the robot to move from the first planar area (such as the ground area) to the second planar area (such as building material blocks), or assisting the robot to move from the second planar area to the first planar area, or assisting the robot to move between the second planar areas (building material blocks) located on different horizontal planes. Indication signs (i.e., the first indication sign and the second indication sign mentioned below) and the first guiding line for guiding the robot to move are set at the starting position and the ending position of the inclined plane assistance area. An indication sign for identifying this plane (i.e., the third indication sign mentioned below) is set at the first position (optional, such as the middle position) in the second planar area (such as building material blocks) to identify whether the robot has reached this plane, and graphic codes for positioning are set at the second position (optional, such as the perimeter) in the second planar area (such as building material blocks) to perform local positioning on the robot in the second planar area.

[0066] Among them, the graphic code here can be a two-dimensional code, a bar code or other identification codes. The embodiments of the present application do not limit the form and type of the graphic code. Each graphic code corresponds to a unique position information, which is used to represent the specific position in the corresponding area of the graphic code.

[0067] The first intersection position involved in the embodiments of the present application is the starting position of the robot in the auxiliary inclined plane area and at the same time the ending position of the robot in the first planar area; the second intersection position is the ending position of the robot in the auxiliary inclined plane area and at the same time the starting position of the robot in the second planar area.

[0068] In some embodiments, a first indication mark for identifying the starting position of the inclined plane assistance area is deployed in the inclined plane assistance area. Before the robot acquires the first image of the robot moving to the first intersection position, the robot may also acquire a second image during the process of the robot moving from the first plane area to the inclined plane assistance area; perform mark recognition on the second image, and when the first indication mark is recognized from the second image, determine that the robot has moved to the starting position of the inclined plane assistance area.

[0069] Wherein, the starting position is the first intersection position. Since a first indication mark for identifying the starting position of the inclined plane assistance area is deployed in the inclined plane assistance area, during the process of the robot moving from the first plane area where the graphic code is deployed to the inclined plane assistance area, the camera installed in the robot captures positioning images (such as the second image) in real time for the positioning of the robot. For example, when performing mark recognition on the captured second image, when the recognition result indicates that the first indication mark is included in the second image, it is determined that the robot has moved to the starting position of the inclined plane assistance area indicated by the first indication mark (i.e., the first intersection position).

[0070] In some embodiments, refer to Figure 5A , Figure 5A which is a schematic diagram of the method for determining the first positioning position provided by the embodiment of the present application. Figure 4 The first positioning position for determining the first intersection position of the robot in the first plane area and the inclined plane assistance area based on the graphic code in step 101 can be realized through Figure 5A the steps 201 - 204 shown: In step 201, acquire the first image of the robot moving to the first intersection position; in step 202, perform graphic code recognition processing on the first image to obtain the first graphic code corresponding to the first image; in step 203, when the number of the first graphic codes is at least two, select the first number of first graphic codes from the at least two first graphic codes to form a graphic code array; in step 204, based on the graphic code array, determine the first positioning position of the robot at the first intersection position.

[0071] Here, during the process of the robot moving from the first plane area where the graphic code is deployed to the inclined plane assistance area, the camera installed in the robot captures positioning images in real time for the positioning of the robot. When the robot moves to the first intersection position of the first plane area and the inclined plane assistance area, the positioning image is the above-mentioned first image. Taking the positioning image as the first image as an example, in order to obtain the graphic code in the first image, it is necessary to detect the position of the graphic code in the first image, that is, the image area containing the graphic code. Since there is a set of black borders around each graphic code, the black borders can accelerate the detection speed of the graphic code. The edge detection algorithm (such as Sobel) can be used to perform segmentation processing on the image area to extract the outer contour of the graphic code and obtain the graphic code in the first image.

[0072] In some embodiments, the robot can perform graphic code recognition processing on the first image in the following manner to obtain the graphic code corresponding to the first image: preprocess the first image to obtain the preprocessed first image; detect the finder pattern in the preprocessed first image according to the contour detection algorithm to obtain the finder pattern corresponding to the first image; perform view correction processing on the finder pattern to obtain the target graphic code corresponding to the first image.

[0073] Among them, the preprocessing of the image usually includes at least one of the operations such as image grayscale conversion, image filtering, and image binarization. The purpose is to obtain a binarized image, which is convenient for subsequent detection of the finder pattern. In practical applications, the first image collected by the image acquisition device is often a color image, while the graphic code recognition processing of the first image mainly recognizes the arrangement of black and white squares in the first image. There is a large gray-scale difference between the two colors. Using a grayscale image can well perform graphic code recognition and improve the subsequent image processing efficiency without affecting the recognition accuracy.

[0074] After the grayscale conversion processing, in order to extract the graphic code information from the grayscale first image subsequently, a binarization operation needs to be performed on it, that is, the gray-scale values of all pixel points in the image need to be changed to 0 or 255, that is, there are only two cases of black and white. Filtering the first image mainly takes into account problems such as the environment during image acquisition, the sensor itself, and image transmission, resulting in certain noise in the collected first image. The filtering process is mainly to filter out the noise in the positioning image while retaining the characteristic details of the graphic code. In addition, it should be particularly noted that due to different shooting perspectives, the first image often undergoes a projective transformation. Before performing the next finder pattern detection, the positioning image after preprocessing can also be corrected for distortion.

[0075] Taking the graphic code as a two-dimensional code as an example, the two-dimensional code consists of an array of square modules, which is composed of identification function graphics such as an encoding area, a separator, a finder pattern, a calibration pattern, and a positioning pattern. Among them, the function identification graphics are not used for encoding, and the periphery of the two-dimensional code symbol is a blank area. At the upper left, lower left, and upper right positions in the effective area of the two-dimensional code symbol, there is a graphic with a common center respectively, and there is a light-colored interval with a unit width between their edges and the encoding area. These three graphics are the finder patterns. Each finder pattern is composed of three squares with a common center point. The width of the three squares is one unit module width. The width ratio of the black and white alternating graphics in the finder pattern is 1:1:3:1:1. Combining this proportional relationship, the finder pattern can be effectively identified. For example, when detecting the finder pattern, based on the "return" shape of the finder pattern and the proportional characteristics of the black and white pixel blocks, and based on contour detection, the finder pattern is detected in combination with the characteristics of the graphic code. Generally, the three finder patterns of the two-dimensional code should have a certain relative position relationship, and the graphics after the first two steps of processing may still have a rotation transformation. Therefore, it is necessary to correct the direction of the two-dimensional code graphic according to the detected relative position relationship of the finder patterns. The view correction is divided into projective transformation view correction and position correction of the finder pattern. On the one hand, since the image shooting angle is not necessarily a front view, the first image may undergo a projective transformation, and it is necessary to correct the first image to a front view as much as possible to facilitate the detection of the finder pattern. On the other hand, in the image after projective transformation view correction, the finder pattern may be in the wrong position, and it is necessary to perform a rotation transformation on the image. Through the above process, the first graphic code in the first image can be detected for subsequent positioning.

[0076] Here, when the number of the first graphic codes recognized from the first image is at least two, select the first number of the first graphic codes from the at least two first graphic codes to form a graphic code array. For example, select M*M (M is a positive integer greater than 2) first graphic codes that can form the largest square, and use the square formed by the selected M*M (M is a positive integer greater than 2) first graphic codes as the graphic code array, and determine the first positioning position of the robot on the first plane area according to the graphic code array.

[0077] In some embodiments, refer to Figure 5B , Figure 5B is a schematic diagram of the method for determining the first positioning position provided by the embodiment of the present application. Figure 5A In step 204, based on the graphic code array, determine the first positioning position of the robot at the first intersection position, which can be passed through Figure 5BThe steps 2041 - 2043 shown are implemented as follows: In step 2041, select a second quantity of first graphic codes located at the vertices from the graphic code array, and use each of the second quantity of first graphic codes as a vertex graphic code; in step 2042, based on each vertex graphic code, determine the first reference positions corresponding to the robot at the first intersection positions respectively; in step 2043, based on each of the first reference positions, determine the first positioning positions of the robot at the first intersection positions.

[0078] Taking the graphic code array as a square composed of M * M first graphic codes as an example, use the first graphic codes at the four vertices of the square as vertex graphic codes, and determine the first reference positions corresponding to the robot at the first intersection positions according to each vertex graphic code. In this way, four first reference positions are obtained, and then an averaging process is performed on these four first reference positions to obtain the first positioning position of the robot at the first intersection position.

[0079] In some embodiments, the robot can determine the first reference positions corresponding to the robot at the first intersection positions respectively based on each vertex graphic code in the following manner: perform a decoding process on each vertex graphic code to obtain the encoding information of each vertex graphic code; based on the encoding information of each vertex graphic code, determine the position information of the corresponding vertex graphic code in the graphic code array, and determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot; based on the relative position relationship and the position information of each vertex graphic code in the graphic code array, determine the first reference positions corresponding to the robot at the first intersection positions respectively.

[0080] In some embodiments, the robot can determine the position information of the corresponding vertex graphic code in the graphic code array based on the encoding information of each vertex graphic code in the following manner: obtain the mapping relationship between the encoding information of each vertex graphic code and the position information of the corresponding vertex graphic code; based on the encoding information of each vertex graphic code and the mapping relationship, determine the position information of the corresponding vertex graphic code in the graphic code array.

[0081] Here, for each graphic code (including the first graphic code and the vertex graphic code), configuration information related to each graphic code can be pre - stored, and a graphic code information list is generated according to the preset configuration information. The graphic code information list is set with the corresponding relationship between the encoding information of the graphic code and the position information of the graphic code. Among them, the preset configuration information is an information set that records the size information and position information of each graphic code in a predetermined format. For example, taking a robot moving in an elevator venue as an example, the elevators can be numbered according to their order in the building, record the positions of the elevators in the building, and record the corresponding graphic codes and the size information and position information of the graphic codes according to the elevator numbers.

[0082] The encoding information of the graphic code refers to the information integrated with the graphic code identifier. Each graphic code identifier uniquely corresponds to a graphic code. For example, the identifier information can be the number of the graphic code. The graphic code position information refers to the relative position information of the graphic code in the first plane area. For example, the position information of the graphic code can include the distances of each edge of the graphic code in the ground area.

[0083] In practical applications, the position information of the graphic code in the first plane area can be characterized by recording the coordinates of the graphic code in the plane coordinate system where the first plane area is located. Among them, the plane coordinate system can be a right-handed coordinate system with the upper left corner of the first plane area as the coordinate origin, and the positive direction of the z-axis of the plane coordinate system is the normal direction of the first plane area. The graphic code array coordinate system corresponding to the graphic code array can also be a right-handed coordinate system with the upper left corner of the first plane area as the coordinate origin. That is, a consistent plane coordinate system and graphic code array coordinate system can be initialized. In this case, the position information of the graphic code in the graphic code array is the position information of the graphic code in the plane coordinate system (world coordinate system).

[0084] The corresponding relationship between the encoding information and the position information of the graphic code is set in the graphic code information list. Through the graphic code information list, the corresponding relationship between the encoding information and the position information of each graphic code is established. In this way, the robot can obtain the position information of the graphic code in the corresponding graphic code array by searching the graphic code information list.

[0085] In some embodiments, the robot can determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot in the following way: determine the two-dimensional coordinates of the third number of corner points corresponding to the vertex graphic code and the size information of the vertex graphic code; perform an averaging process on the two-dimensional coordinates of the third number of corner points to obtain the two-dimensional coordinates of the center point of the vertex graphic code; determine the three-dimensional coordinates of the third number of corner points in the plane coordinate system according to the three-dimensional coordinates of the center point in the plane coordinate system corresponding to the first plane area and the size information of the vertex graphic code; determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot according to the matching pairs composed of the two-dimensional coordinates and the corresponding three-dimensional coordinates of the third number of corner points.

[0086] Here, after obtaining the encoding information of the vertex graphic code, since the world coordinates of each vertex graphic code are written into the configuration information in the graphic code information list in advance and the size information of the vertex graphic code is known, the relative position relationship (rotation matrix, translation matrix) between the graphic code array coordinate system and the camera coordinate system can be calculated according to the size and position of the vertex graphic code in the first image.

[0087] In practical applications, regardless of how the vertex graphic code is arranged, the vertex graphic code can be divided into a third number of corner points. Taking the two-dimensional code with the vertex graphic code as a square as an example, the third number is 4, and each corner point will be uniquely determined during the encoding process of the two-dimensional code. After the vertex graphic code is recognized, the positions of the 4 corner points of the vertex graphic code in the positioning image can be obtained, and their pixel coordinates (i.e., two-dimensional coordinates, 2D) are respectively denoted as (u1, v1), (u2, v2), (u3, v3), (u4, v4), and the two-dimensional coordinate of the center point of the vertex graphic code is denoted as (u, v), where u = (u1 + u2 + u3 + u4) / 4 and v = (v1 + v2 + v3 + v4) / 4.

[0088] Since the three-dimensional (3D) coordinates of the center point of the vertex graphic code and the size information of the vertex graphic code are known, based on this, the 3D coordinates of the 4 corner points of the vertex graphic code can be calculated, and several 3D-2D matching pairs are obtained. The relative position relationship between the graphic code array coordinate system and the camera coordinate system can be determined through the PnP solution method; thus, based on the determined relative position relationship between the graphic code array coordinate system and the camera coordinate system and the position information of the vertex graphic code in the graphic code array, the first reference position (including the rotation matrix and the translation matrix) of the robot in the graphic code array can be determined.

[0089] It can be understood that in practical applications, several 3D-2D matching pairs can also be constructed based on the first number of vertex graphic codes in the graphic code array, and the relative position relationship between the graphic code array coordinate system and the camera coordinate system can be determined through the PnP solution method. Moreover, when the robot moves to any position in the first plane area, the above positioning method can be used to position the robot.

[0090] Step 102: Control the robot to move from the first intersection position in the inclined plane auxiliary area to the second plane area.

[0091] In some embodiments, a first guiding line for connecting the first indication mark and the second indication mark is deployed in the inclined plane auxiliary area. The first indication mark is used to mark the first intersection position, and the second indication mark is used to mark the second intersection position; the robot can be controlled to move from the first intersection position in the inclined plane auxiliary area to the second plane area in the following manner: obtain a third image during the process of the robot moving from the first intersection position in the inclined plane auxiliary area to the second plane area; during the movement of the robot, adjust the movement path of the robot based on the third image to control the robot to move from the first intersection position along the first path indicated by the first guiding line to the second intersection position.

[0092] Here, the first guiding line deployed in the inclined plane auxiliary area is used for visual line following, so that during the movement of the robot on the inclined plane auxiliary area, the robot can be ensured to move along the center line of the inclined plane auxiliary area. During the movement of the robot on the inclined plane auxiliary area, the third image collected by the camera can be first identified and processed. When the first guiding line is recognized, the movement path of the robot is adjusted based on the first guiding line, so that the robot starts from the starting position of the inclined plane auxiliary area and always moves along the path executed by the first guiding line until it moves to the second intersection position of the inclined plane auxiliary area and the second plane area.

[0093] In some embodiments, the movement path of the robot can be adjusted based on the third image in the following manner: preprocess the third image to obtain the preprocessed third image, where the preprocessing includes at least one of the following: grayscale conversion, binarization, filtering; perform edge detection on the preprocessed third image to obtain the first reference line and the second reference line corresponding to the edge of the first guiding line in the preprocessed third image; determine the angle bisector of the first angle between the first reference line and the second reference line, as well as the center line and the reference point of the third image; obtain the second angle between the angle bisector and the center line, and the distance between the angle bisector and the reference point, so as to adjust the movement path of the robot based on the angle and the distance.

[0094] Here, during the movement of the robot on the inclined plane auxiliary area, the third image collected by the camera can be preprocessed such as grayscale conversion, binarization, and filtering. The preprocessed third image can enhance the features of the first guiding line compared with the third image before preprocessing; then, perform edge detection on the preprocessed third image to obtain the first reference line and the second reference line corresponding to the edge of the first guiding line, as Figure 8 shown, detect the two reference lines l0 and l1 of the edge of the first guiding line, and calculate the angle bisector l of l0 and l1; finally, calculate the angle α between l and the center line (i.e., the visual center line) l ref of the third image, and the distance d from a fixed position (such as the intersection point of the upper edge of the third image and the center line), and adjust the position of the construction robot according to these two values, so that the construction robot moves along the first guiding line.

[0095] Step 103: When the robot moves to the second intersection position of the inclined plane auxiliary area and the second plane area, determine the second positioning position of the robot at the second intersection position based on the first positioning position.

[0096] In some embodiments, the robot can determine the second positioning position of the robot at the second intersection position based on the first positioning position in the following manner: determine the horizontal length and vertical height of the inclined plane assistance area, and the moving direction of the robot; based on the first positioning position, the horizontal length, the vertical height, and the moving direction, determine the second positioning position of the robot at the second intersection position.

[0097] Here, when the robot moves to the second intersection position of the inclined plane assistance area and the second plane area (i.e., the end position of the inclined plane assistance area or the start position of the second plane area), since the first positioning position of the starting position (i.e., the first intersection position) of the robot in the inclined plane assistance area is known through the above method, the length, height of the inclined plane assistance area, and the moving direction of the robot can be obtained, and the second positioning position of the robot at the second intersection position can be determined in combination with the first positioning position.

[0098] Assume that the horizontal length of the slope is l s , the height is h, the first positioning position corresponding to the starting position of the robot entering the inclined plane assistance area is (x, y, z), and the second positioning position after the robot finishes walking through the inclined plane assistance area is (x m , y m , z m ), the moving direction of the robot is (a, b), as Figure 7 shown, the moving direction can be right (1, 0), left (-1, 0), up (0, 1), and down (0, -1). When the robot moves upward along the inclined plane assistance area, the robot recognizes the collected positioning image. When the first indication mark indicating the starting position of the inclined plane assistance area is recognized, it indicates that the robot will move upward along the inclined plane assistance area. During the movement, visual line following will be carried out relying on the first guiding line in the inclined plane assistance area until the robot recognizes the second indication mark indicating the end position of the inclined plane assistance area from the collected positioning image, indicating that the robot has finished walking through this inclined plane assistance area. At this time, the position of the robot is updated as: x m = x + l s * a, y m = y + l s * b, z m = z + h.

[0099] Step 104: During the movement of the robot on the second plane area, determine the third positioning position of the robot when moving on the second plane area based on the second positioning position.

[0100] In some embodiments, a second guiding line for guiding the movement of the robot is deployed on the second planar region. Before determining the third positioning position of the robot when moving on the second planar region based on the second positioning position, a fourth image of the robot during its movement on the second planar region can be acquired; during the movement of the robot, the movement path of the robot is adjusted based on the fourth image to control the robot to move along the second path indicated by the second guiding line on the second planar region.

[0101] It can be understood that when the robot is moving in the second planar region, the manner of adjusting the movement path of the robot based on the fourth image is the same as the manner of adjusting the movement path of the robot based on the third image during the movement of the robot in the inclined plane assisting region, which will not be elaborated here.

[0102] In some embodiments, a third indicating identifier for indicating the second planar region is deployed on the second planar region. The third positioning position of the robot when moving on the second planar region can be determined based on the second positioning position in the following manner: acquire a fifth image of the robot during its movement on the second planar region; perform identifier recognition on the fifth image, and when the third indicating identifier is recognized from the fifth image, determine the length of the second planar region and the movement direction of the robot; based on the second positioning position, the length of the second planar region, and the movement direction, determine the third positioning position of the robot when moving on the second planar region.

[0103] Here, when the robot starts to move on the second horizontal plane from the second intersection position (i.e., the starting position of the second planar region) of the second planar region, since the second positioning position of the robot at the second intersection position is known through the above method, the length of the second planar region and the movement direction of the robot can be acquired, and the third positioning position of the robot in the second planar region can be determined in combination with the second positioning position.

[0104] Assume that the side length of the second planar region is l b , when the second positioning position corresponding to the robot moving from the inclined plane assisting region to the second intersection position of the second planar region is (x m , y m , z m ), and the third positioning position when completing one second planar region is (x n , y n , z n) The moving direction of the robot is (c, d). Similarly, the moving directions of the robot can be right (1, 0), left (-1, 0), up (0, 1), and down (0, -1). When the robot moves on the second plane area, the robot still relies on the second guiding line in the middle of the second plane area for visual line following. When the robot recognizes the third indication mark for indicating the arrival at the second plane area from the collected positioning image, it means that the robot has passed through the second plane area. At this time, update the third positioning position of the robot to: x n = x m + l b * c, y n = y m + l b * d, z n = z m .

[0105] In some embodiments, a graphic code for positioning is deployed on the second plane area. The third positioning position of the robot when moving on the second plane area can be determined based on the second positioning position in the following manner: Obtain the fifth image during the movement of the robot on the second plane area; perform graphic code recognition processing on the fifth image to obtain the second graphic code corresponding to the fifth image; determine the second reference position of the robot on the second plane area based on the second graphic code, and determine the third positioning position of the robot when moving on the second plane area based on the second positioning position and the second reference position.

[0106] It can be understood that the graphic code for positioning deployed on the second plane area can be dense. In this case, the method for determining the second reference position of the robot on the second plane area based on the second graphic code in the fifth image is similar to the method for determining the second reference position of the robot on the first plane area based on the first graphic code in the first image during the movement of the robot in the first plane area, and will not be elaborated here.

[0107] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described. The robot positioning method provided by the embodiments of the present application can be applied to the positioning application of construction robots, such as Figure 6 shown Figure 6This is a schematic diagram of site deployment provided by an embodiment of the present application. Before the construction robot moves and locates, it is necessary to first deploy the site where the construction robot needs to move or pass through. Taking the construction robot moving from the ground area (i.e., the above-mentioned first planar area) to the building material block (i.e., the above-mentioned second planar area) with the help of a ramp (i.e., the above-mentioned inclined plane auxiliary area) as an example, a certain number of graphic codes (such as QR codes) are densely deployed in the ground area of Figure (a) to ensure that when the construction robot equipped with a camera moves in the ground area, the camera can capture an image including at least one graphic code; in Figure (b), the ramp is used to assist the construction robot to move from the ground area to the building material block, and indication signs (such as QR code labels with fixed numbers) and a first guiding line (such as the black line in the middle) for guiding the robot to move are deployed at the starting position and the ending position of the inclined plane auxiliary area; in the middle of the building material block in Figure (c), an indication sign (such as a QR code label with a fixed number) for identifying the plane is deployed to identify whether the construction robot has reached the current building material block, and graphic codes (such as QR codes) for positioning are deployed around for local high-precision positioning on the building material block, and the middle black line is the second guiding line for guiding the robot to move.

[0108] In addition, considering that the upper building is dynamically changing during the construction process, a counting and line-tracking strategy is designed according to the indication signs on the ramp and the building material block to assist in positioning on the ramp and the building material. Visual line-tracking is performed according to the black lines in the middle of the ramp and the building material to ensure that the robot travels in a straight line. This part of the positioning is discrete, and the position of the construction robot is updated when the robot sees an indication sign (such as a QR code with a fixed number).

[0109] Next, the positioning methods of the construction robot on the ground, ramp, and building material block will be described.

[0110] 1. Ground

[0111] When the construction robot moves in the ground area, it obtains a positioning image in real time during the movement (when the construction robot moves to the intersection position of the ground area and the slope, the positioning image is the above-mentioned first image), performs graphic code recognition processing on the positioning image, and obtains the corresponding graphic code (i.e., the above-mentioned first graphic code); when the number of graphic codes is at least two, select the first number of graphic codes from at least two graphic codes to form a graphic code array, and determine the positioning position of the robot on the ground according to the graphic code array. For example, select M*M (M is a positive integer greater than 2) graphic codes to form a graphic code array in the largest square form. First, determine the reference position of the construction robot on the ground according to each graphic code among the graphic codes at the four vertices of the square (i.e., the above-mentioned vertex graphic codes), and then combine the reference positions to determine the positioning position of the construction robot. When the construction robot moves to the intersection position of the ground area and the slope, this positioning position is the above-mentioned first positioning position.

[0112] In practical applications, if positioning is performed only based on one graphic code, when the camera is far from the graphic code, the graphic code in the camera's field of view is small, and the positioning accuracy is low. However, through the above method of joint positioning with multiple graphic codes, the construction robot can be positioned based on multiple vertex graphic codes, which has better robustness to the size of the graphic code and can also improve the positioning accuracy.

[0113] 2. Slope

[0114] Assume that the horizontal length of the slope is l s , the height is h, the position where the construction robot enters the slope (i.e., the first positioning position) is (x, y, z), and the second positioning position after walking the slope is (x m , y m , z m ). The moving direction of the construction robot is (a, b), as Figure 7 shown, Figure 7 is the schematic diagram of the moving direction provided by the embodiment of the present application. The moving direction of the construction robot can be right (1, 0), left (-1, 0), up (0, 1), and down (0, -1). When going uphill, when the construction robot recognizes the indication sign for indicating the starting position of the slope from the collected positioning image, it indicates that the construction robot will move up along the slope. During the movement, it will rely on the black line in the middle of the slope for visual line following until the construction robot recognizes the indication sign for indicating the end position of the slope from the collected positioning image, which means that the construction robot has finished walking this slope. At this time, update the position of the construction robot as: x m = x + l s * a, y m = y + l s * b, z m= z + h; Similarly, when the construction robot is going downhill, update the position of the construction robot as: x m = x + l s *a, y m = y + l s *b, z m = z - h.

[0115] 3. Building material block

[0116] Assume the side length of the building material block is l b , when the construction robot moves from the slope to the building material block, the position where it is located is the position where the mobile robot enters the building material block is (x m , y m , z m ), the third positioning position when walking through a building material block is (x n , y n , z n ), the moving direction of the robot is (c, d). Similarly, the moving direction of the construction robot can be right (1, 0), left (-1, 0), up (0, 1) and down (0, -1).

[0117] When the construction robot moves on the building material block, the construction robot still relies on the black line in the middle of the building material block for visual line following. When the construction robot recognizes the indication sign used to indicate reaching the building material block from the collected positioning image, it means that the robot has passed through a building material block. At this time, update the position of the construction robot as: x n = x m + l b *c, y n = y m + l b *d, z n = z m .

[0118] 4. Visual line following strategy

[0119] Visual line following is to ensure that the construction robot moves along the center line during the movement of the construction robot on the slope or building material block. In practical applications, as Figure 8 shown Figure 8 is the visual line following schematic diagram provided by the embodiment of the present application. During the movement of the construction robot on the slope or building material block, the positioning image collected by the camera can be first subjected to global binarization processing to enhance the characteristics of the black line in the center of the field of view; then detect two straight lines l0 and l1 at the edge of the black line in the binary image, and calculate the angular bisector l of l0 and l1; finally calculate the angle between l and the center line l of the field of view refThe included angle α and the distance d at a fixed position are used to adjust the position of the construction robot according to these two values, so that the construction robot moves along the middle black line.

[0120] In the above manner, when the construction robot moves on the ground, it relies on multiple graphic codes within its field of view for real-time positioning and is more robust to the size of the graphic codes. Moreover, in structured scenarios such as slopes or building material blocks, the construction robot uses a combination of visual line following and counting for positioning, which can well adapt to the constantly changing dynamic construction, help the construction robot climb onto the building and position itself on the building. The positioning method is convenient, efficient, and accurate, and does not require complex devices such as lidar and motion capture. It can achieve positioning only through graphic codes with low cost and easy layout and the strategy of line following and counting.

[0121] Next, the implementation of the robot positioning device 555 provided in the embodiments of the present application as an exemplary structure of software modules will be further described. In some embodiments, the software modules in the robot positioning device 555 stored in Figure 3 the memory 550 may include: a first determination module 5551, configured to, during the process of the robot moving from the first planar area deployed with graphic codes to the slope assistance area, determine a first positioning position of the robot at a first intersection position between the first planar area and the slope assistance area based on the graphic codes; a movement control module 5552, configured to control the robot to move from the first intersection position in the slope assistance area to a second planar area; a second determination module 5553, configured to, when the robot moves to a second intersection position between the slope assistance area and the second planar area, determine a second positioning position of the robot at the second intersection position based on the first positioning position; a third determination module 5554, configured to, during the process of the robot moving on the second planar area, determine a third positioning position of the robot when moving on the second planar area based on the second positioning position.

[0122] In some embodiments, the first determination module is further configured to obtain a first image of the robot moving to the first intersection position; perform graphic code recognition processing on the first image to obtain a first graphic code corresponding to the first image; when the number of the first graphic codes is at least two, select a first number of first graphic codes from the at least two first graphic codes to form a graphic code array; and determine the first positioning position of the robot at the first intersection position based on the graphic code array.

[0123] In some embodiments, a first indication identifier for identifying the starting position of the inclined plane assistance area is deployed in the inclined plane assistance area. Before obtaining the first image of the robot moving to the first intersection position, the device further includes: a fourth determination module, configured to obtain a second image during the process of the robot moving from the first plane area to the inclined plane assistance area; perform identification recognition on the second image, and when the first indication identifier is recognized from the second image, determine that the robot moves to the starting position of the inclined plane assistance area, where the starting position is the first intersection position.

[0124] In some embodiments, the first determination module is further configured to select a second quantity of first graphic codes located at the vertices from the graphic code array, and use each of the second quantity of first graphic codes as a vertex graphic code; based on each of the vertex graphic codes, determine the first reference positions respectively corresponding to the robot at the first intersection position; based on each of the first reference positions, determine the first positioning position of the robot at the first intersection position.

[0125] In some embodiments, the first determination module is further configured to perform decoding processing on each of the vertex graphic codes to obtain the encoding information of each of the vertex graphic codes; based on the encoding information of each of the vertex graphic codes, determine the position information of the corresponding vertex graphic code in the graphic code array, and determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot; based on the relative position relationship and the position information of each of the vertex graphic codes in the graphic code array, determine the first reference positions respectively corresponding to the robot at the first intersection position.

[0126] In some embodiments, the first determination module is further configured to obtain the mapping relationship between the encoding information of each of the vertex graphic codes and the position information of the corresponding vertex graphic code; based on the encoding information of each of the vertex graphic codes and the mapping relationship, determine the position information of the corresponding vertex graphic code in the graphic code array.

[0127] In some embodiments, the first determination module is further configured to determine the two-dimensional coordinates of the third number of corner points corresponding to the vertex graphic code and the size information of the vertex graphic code; perform an averaging process on the two-dimensional coordinates of the third number of corner points to obtain the two-dimensional coordinates of the center point of the vertex graphic code; determine the three-dimensional coordinates of the third number of corner points in the plane coordinate system according to the three-dimensional coordinates of the center point in the plane coordinate system corresponding to the first plane area and the size information of the vertex graphic code; determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot according to the matching pairs composed of the two-dimensional coordinates and the corresponding three-dimensional coordinates of the third number of corner points.

[0128] In some embodiments, a first guiding line for connecting the first indication mark and the second indication mark is deployed in the inclined plane auxiliary area, the first indication mark is used to mark the first intersection position, and the second indication mark is used to mark the second intersection position; the movement control module is further configured to obtain a third image during the process of the robot moving from the first intersection position in the inclined plane auxiliary area to the second plane area; during the movement of the robot, adjust the movement path of the robot based on the third image to control the robot to move from the first intersection position along the first path indicated by the first guiding line to the second intersection position.

[0129] In some embodiments, the movement control module is further configured to preprocess the third image to obtain a preprocessed third image, and the preprocessing includes at least one of the following: grayscale conversion, binarization, filtering; perform edge detection on the preprocessed third image to obtain a first reference line and a second reference line corresponding to the edge of the first guiding line in the preprocessed third image; determine the angular bisector of the first included angle between the first reference line and the second reference line, the center line and the reference point of the third image; obtain the second included angle between the angular bisector and the center line and the distance between the angular bisector and the reference point, so as to adjust the movement path of the robot based on the included angle and the distance.

[0130] In some embodiments, the second determination module is further configured to determine the horizontal length and vertical height of the inclined plane auxiliary area and the movement direction of the robot; determine the second positioning position of the robot at the second intersection position based on the first positioning position, the horizontal length, the vertical height and the movement direction.

[0131] In some embodiments, a second guiding line for guiding the movement of the robot is deployed on the second planar region. Before determining the third positioning position of the robot when moving on the second planar region based on the second positioning position, the device further includes: a control module, configured to obtain a fourth image of the robot during the movement on the second planar region; during the movement of the robot, adjust the movement path of the robot based on the fourth image to control the robot to move along a second path indicated by the second guiding line on the second planar region.

[0132] In some embodiments, a third indication identifier for indicating the second planar region is deployed on the second planar region. The third determination module is further configured to obtain a fifth image of the robot during the movement on the second planar region; perform identifier recognition on the fifth image, and when the third indication identifier is recognized from the fifth image, determine the length of the second planar region and the movement direction of the robot; based on the second positioning position, the length of the second planar region, and the movement direction, determine the third positioning position of the robot when moving on the second planar region.

[0133] In some embodiments, the third determination module is further configured to obtain a fifth image of the robot during the movement on the second planar region; perform graphic code recognition processing on the fifth image to obtain a second graphic code corresponding to the fifth image; determine a second reference position of the robot on the second planar region based on the second graphic code, and based on the second positioning position and the second reference position, determine the third positioning position of the robot when moving on the second planar region.

[0134] An embodiment of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the robot positioning method described above in the embodiments of the present application.

[0135] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, where the executable instructions are stored, and when the executable instructions are executed by a processor, the processor will be caused to execute the robot positioning method provided in the embodiments of the present application, for example, Figure 4 the method shown.

[0136] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0137] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0138] As an example, the executable instructions may or may not correspond to a file in the file system, may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program under discussion, or, stored in multiple cooperating files (for example, files that store one or more modules, subroutines, or portions of code).

[0139] As an example, the executable instructions may be deployed to execute on one computing device, or on multiple computing devices located at one location, or, on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0140] As described above, the above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are all included in the protection scope of the present application.

Claims

1. A robot positioning method, characterized in that, The method includes: During the process of the robot moving from the first planar area with a graphic code to the inclined plane assisting area, determining a first positioning position of the robot at a first intersection position between the first planar area and the inclined plane assisting area based on the graphic code; Controlling the robot to move from the first intersection position in the inclined plane assisting area to the second planar area; When the robot moves to a second intersection position between the inclined plane assisting area and the second planar area, determining a second positioning position of the robot at the second intersection position based on the first positioning position, the horizontal length and vertical height of the inclined plane assisting area, and the moving direction of the robot; During the process of the robot moving on the second planar area, determining a third positioning position of the robot when moving on the second planar area based on the second positioning position.

2. The method according to claim 1, characterized in that The determining the first positioning position of the robot at the first intersection position between the first planar area and the inclined plane assisting area based on the graphic code includes: Obtaining a first image of the robot when it moves to the first intersection position; Performing graphic code recognition processing on the first image to obtain a first graphic code corresponding to the first image; When the number of the first graphic codes is at least two, selecting a first number of the first graphic codes from the at least two first graphic codes to form a graphic code array; Based on the graphic code array, determining the first positioning position of the robot at the first intersection position.

3. The method according to claim 2, wherein A first indication mark for identifying the starting position of the inclined plane assisting area is deployed in the inclined plane assisting area. Before obtaining the first image of the robot when it moves to the first intersection position, the method further includes: Obtaining a second image during the process of the robot moving from the first planar area to the inclined plane assisting area; Performing identification mark recognition on the second image, and when the first indication mark is recognized from the second image, determining that the robot moves to the starting position of the inclined plane assisting area, where the starting position is the first intersection position.

4. The method according to claim 2, wherein The determining the first positioning position of the robot at the first intersection position based on the graphic code array includes: Selecting a second number of the first graphic codes located at vertices from the graphic code array, and using each of the second number of the first graphic codes as a vertex graphic code; Based on each of the vertex graphic codes, determining a first reference position corresponding to the robot at the first intersection position respectively; Based on each of the first reference positions, determining the first positioning position of the robot at the first intersection position.

5. The method according to claim 4, characterized in that The determining the first reference position corresponding to the robot at the first intersection position based on each of the vertex graphic codes includes: Performing decoding processing on each of the vertex graphic codes to obtain the encoding information of each of the vertex graphic codes; Based on the encoding information of each of the vertex graphic codes, determining the position information of the corresponding vertex graphic code in the graphic code array, and determining the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot; Based on the relative position relationship and the position information of each vertex graphic code in the graphic code array, determine the first reference positions corresponding to the robot at the first intersection positions respectively.

6. The method according to claim 5, wherein The determining the position information of the corresponding vertex graphic code in the graphic code array based on the encoding information of each vertex graphic code includes: Obtain the mapping relationship between the encoding information of each vertex graphic code and the position information of the corresponding vertex graphic code; Based on the encoding information of each vertex graphic code and the mapping relationship, determine the position information of the corresponding vertex graphic code in the graphic code array.

7. The method according to claim 4, wherein The determining the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot includes: Determine the two-dimensional coordinates of the third number of corner points corresponding to the vertex graphic code and the size information of the vertex graphic code; Perform an averaging process on the two-dimensional coordinates of the third number of corner points to obtain the two-dimensional coordinates of the center point of the vertex graphic code; According to the three-dimensional coordinates of the center point in the plane coordinate system corresponding to the first plane area and the size information of the vertex graphic code, determine the three-dimensional coordinates of the third number of corner points in the plane coordinate system; According to the matching pairs composed of the two-dimensional coordinates and the corresponding three-dimensional coordinates of the third number of corner points, determine the relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot.

8. The method according to claim 1, characterized in that, A first guiding line for connecting a first indication mark and a second indication mark is deployed in the inclined plane auxiliary area, the first indication mark is used to mark the first intersection position, and the second indication mark is used to mark the second intersection position; The controlling the robot to move from the first intersection position in the inclined plane auxiliary area to the second plane area includes: Obtain a third image during the process of the robot moving from the first intersection position in the inclined plane auxiliary area to the second plane area; During the movement of the robot, adjust the movement path of the robot based on the third image to control the robot to move from the first intersection position along the first path indicated by the first guiding line to the second intersection position.

9. The method according to claim 8, wherein The adjusting the movement path of the robot based on the third image includes: Perform preprocessing on the third image to obtain a preprocessed third image, and the preprocessing includes at least one of the following: grayscale conversion, binarization, filtering; Perform edge detection on the preprocessed third image to obtain a first reference line and a second reference line corresponding to the edge of the first guiding line in the preprocessed third image; Determine the angular bisector of the first included angle between the first reference line and the second reference line, and the center line and the reference point of the third image; Obtain the second included angle between the angular bisector and the center line, and the distance between the angular bisector and the reference point, so as to adjust the movement path of the robot based on the second included angle and the distance.

10. The method according to claim 1, characterized in that A second guiding line for guiding the movement of the robot is deployed on the second planar region. Before determining the third positioning position of the robot when moving on the second planar region based on the second positioning position, the method further includes: Obtaining a fourth image of the robot during the movement on the second planar region; During the movement of the robot, adjusting the movement path of the robot based on the fourth image to control the robot to move along a second path indicated by the second guiding line on the second planar region.

11. The method according to claim 1, characterized in that, A third indication identifier for indicating the second planar region is deployed on the second planar region. Determining the third positioning position of the robot when moving on the second planar region based on the second positioning position includes: Obtaining a fifth image of the robot during the movement on the second planar region; Performing identifier recognition on the fifth image, and when the third indication identifier is recognized from the fifth image, determining the length of the second planar region and the movement direction of the robot; Based on the second positioning position, the length of the second planar region, and the movement direction, determining the third positioning position of the robot when moving on the second planar region.

12. The method according to claim 1, wherein A graphic code for positioning is deployed on the second planar region. Determining the third positioning position of the robot when moving on the second planar region based on the second positioning position includes: Obtaining a fifth image of the robot during the movement on the second planar region; Performing graphic code recognition processing on the fifth image to obtain a second graphic code corresponding to the fifth image; Based on the second graphic code, determining a second reference position of the robot on the second planar region, and based on the second positioning position and the second reference position, determining the third positioning position of the robot when moving on the second planar region.

13. A robot positioning device, characterized in that, The device includes: A first determination module, configured to, during the movement of the robot from a first planar region where a graphic code is deployed to an inclined plane auxiliary region, determine a first positioning position of the robot at a first intersection position between the first planar region and the inclined plane auxiliary region based on the graphic code; A movement control module, configured to control the robot to move from the first intersection position in the inclined plane auxiliary region to a second planar region; A second determination module, configured to, when the robot moves to a second intersection position between the inclined plane auxiliary region and the second planar region, determine a second positioning position of the robot at the second intersection position based on the first positioning position, the horizontal length and vertical height of the inclined plane auxiliary region, and the movement direction of the robot; A third determination module, configured to, during the movement of the robot on the second planar region, determine a third positioning position of the robot when moving on the second planar region based on the second positioning position.

14. The device according to claim 13, characterized in that, The first determination module is further configured to: Obtain a first image of the robot when moving to the first intersection position; Perform graphic code recognition processing on the first image to obtain a first graphic code corresponding to the first image; When the number of the first graphic codes is at least two, select a first number of the first graphic codes from the at least two first graphic codes to form a graphic code array; Based on the graphic code array, determine a first positioning position of the robot at the first intersection position.

15. The device according to claim 14, characterized in that, A first indication identifier for identifying a starting position of the inclined plane auxiliary area is deployed in the inclined plane auxiliary area, and the device further includes: A fourth determination module, configured to obtain a second image during the process of the robot moving from the first plane area to the inclined plane auxiliary area; perform identifier recognition on the second image, and when the first indication identifier is recognized from the second image, determine that the robot moves to the starting position of the inclined plane auxiliary area, where the starting position is the first intersection position.

16. The device according to claim 14, wherein The first determination module is further configured to: Select a second number of the first graphic codes located at vertices from the graphic code array, and use each of the second number of the first graphic codes as a vertex graphic code; Based on each of the vertex graphic codes, determine first reference positions respectively corresponding to the robot at the first intersection position; Based on each of the first reference positions, determine a first positioning position of the robot at the first intersection position.

17. The device according to claim 16, characterized in that, The first determination module is further configured to: Perform decoding processing on each of the vertex graphic codes to obtain encoding information of each of the vertex graphic codes; Based on the encoding information of each of the vertex graphic codes, determine position information of the corresponding vertex graphic code in the graphic code array, and determine a relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot; Based on the relative position relationship and the position information of each of the vertex graphic codes in the graphic code array, determine first reference positions respectively corresponding to the robot at the first intersection position.

18. The device according to claim 16, characterized in that, The first determination module is further configured to: Determine two-dimensional coordinates of a third number of corner points corresponding to the vertex graphic code, and size information of the vertex graphic code; Perform an averaging process on the two-dimensional coordinates of the third number of corner points to obtain two-dimensional coordinates of a center point of the vertex graphic code; According to three-dimensional coordinates of the center point in a plane coordinate system corresponding to the first plane area and the size information of the vertex graphic code, determine three-dimensional coordinates of the third number of corner points in the plane coordinate system; According to a matching pair formed by the two-dimensional coordinates and corresponding three-dimensional coordinates of the third number of corner points, determine a relative position relationship between the graphic code coordinate system corresponding to the graphic code array and the camera coordinate system corresponding to the robot.

19. The device according to claim 13, wherein A first guiding line for connecting the first indication identifier and the second indication identifier is deployed in the inclined plane auxiliary area, the first indication identifier is used to identify the first intersection position, and the second indication identifier is used to identify the second intersection position; the movement control module is further configured to: Obtain a third image during the process of the robot moving from the first intersection position in the inclined plane auxiliary area to the second plane area; During the movement of the robot, adjust the movement path of the robot based on the third image to control the robot to move from the first intersection position along the first path indicated by the first guiding line to the second intersection position.

20. The device according to claim 19, characterized in that, The movement control module is further configured to: Preprocess the third image to obtain a preprocessed third image, and the preprocessing includes at least one of the following: grayscale conversion, binarization, filtering; Perform edge detection on the preprocessed third image to obtain a first reference line and a second reference line corresponding to the edge of the first guiding line in the preprocessed third image; Determine the angular bisector of the first included angle between the first reference line and the second reference line, as well as the center line and the reference point of the third image; Obtain the second included angle between the angular bisector and the center line, and the distance between the angular bisector and the reference point, so as to adjust the movement path of the robot based on the second included angle and the distance.

21. An electronic device, characterized in that, Comprising: A memory for storing executable instructions; A processor, when executing the executable instructions stored in the memory, implements the robot positioning method according to any one of claims 1 to 12.

22. A computer-readable storage medium, characterized in that, Stored with executable instructions, when being executed by a processor, implements the robot positioning method according to any one of claims 1 to 12.

23. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instruction is executed by a processor, it implements the robot positioning method according to any one of claims 1 to 12.

Images