Positioning method, device, storage medium and program product

Abstract

The present application provides a positioning method, device, storage medium, and program product, relating to the technical field of information processing. The method comprises: receiving a positioning mode start instruction for a target positioning marker; in response to the positioning mode start instruction, if the robot itself stores the dotting results of the target positioning marker, obtaining the current global absolute position of the robot itself, and starting the target sensor configured on the robot that is adapted to the type of the target positioning marker; and determining the current relative position between the robot and the target positioning marker based on the dotting results of the target positioning marker, the current global absolute position of the robot itself, and the data collected by the target sensor. The present application aims to improve the accuracy of determining relative position.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of information processing, and in particular, to a positioning method, device, storage medium, and program product. Background Art

[0002] With the rapid development of society, robots that can improve work efficiency have been widely used. During the movement of the robot, the robot needs to be positioned so that the robot reaches the position point where the target positioning mark is located, enters the target working state, and performs the target task at this position point. However, the robot cannot accurately identify the working environment. If the working environment changes, for example, if the target positioning mark is accidentally moved by a person and the position changes, the robot can only rely on the map for absolute positioning. It cannot accurately locate the current relative position between the robot itself and the target positioning mark, and therefore cannot reach the position point where the target positioning mark is located, cannot enter the target working state, and cannot perform the target task at this position point.

[0003] Therefore, how to accurately determine the current relative position between the robot itself and the target positioning marker is an urgent problem to be solved. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a positioning method, device, storage medium, and program product, which can further improve the accuracy of determining the current relative position between the robot itself and the target positioning mark.

[0005] According to a first aspect of an embodiment of the present invention, a positioning method is provided, the method comprising:

[0006] receiving a positioning mode start instruction for a target positioning identifier, wherein the positioning mode start instruction includes at least attribute information of the target positioning identifier;

[0007] In response to the positioning mode start instruction, querying whether the robot itself has stored the marking result of the target positioning mark, the marking result of the target positioning mark including: attribute information of the target positioning mark, global absolute pose and target relative pose;

[0008] In a case where the robot itself stores the marking result of the target positioning mark, obtaining the current global absolute position of the robot itself, and starting the target sensor configured on the robot and adapted to the type of the target positioning mark;

[0009] The current relative position between the robot and the target positioning mark is determined according to the marking result of the target positioning mark, the current global absolute position of the robot itself and the data collected by the target sensor.

[0010] Optionally, the method further includes:

[0011] Receive a first dotting mode start instruction;

[0012] In response to the first dotting mode start instruction, detecting the working state of the robot itself;

[0013] When the working state of the robot itself is the target working state, obtaining the target global absolute position of the robot itself, and starting each sensor configured by the robot itself for identifying the positioning mark;

[0014] Starting the threads corresponding to the respective sensors, and processing the data collected by the respective sensors according to the corresponding data processing algorithms through the started threads to obtain the attribute information of the positioning marker and the relative position and posture of the target;

[0015] Processing the target relative pose of the positioning marker and the target global absolute pose of the robot itself through the started thread to determine the global absolute pose of the positioning marker;

[0016] The marking result of the positioning mark is stored locally and / or sent to other devices, and the marking result includes: attribute information of the positioning mark, global absolute position and target relative position.

[0017] Optionally, the method further includes:

[0018] When the robot itself does not store the marking result of the target positioning mark, outputting a prompt message indicating that there is no positioning result;

[0019] receiving a second dotting mode start instruction for the target positioning identifier, wherein the second dotting mode start instruction includes a type of the target positioning identifier;

[0020] Detecting whether the target sensor is configured on the device;

[0021] When the target sensor is not configured for the robot itself, a prompt message is output, where the prompt message is used to prompt the user to configure the target sensor for the robot and configure a data processing algorithm for processing the data collected by the target sensor.

[0022] Optionally, the method further includes:

[0023] In the case where the robot is equipped with the target sensor, when the working state of the robot itself is the target working state, obtaining the target global absolute position of the robot itself, and starting the target sensor to collect data;

[0024] Starting a thread corresponding to the target sensor, and processing the data collected by the target sensor according to a corresponding data processing algorithm through the started thread to obtain a marking result of the target positioning mark;

[0025] The marking result of the target positioning mark is stored locally and / or sent to other devices, and the marking result of the target positioning mark includes: attribute information of the target positioning mark, global absolute position and target relative position.

[0026] Optionally, determining the current relative position between the robot and the target positioning marker according to the marking result of the target positioning marker, the current global absolute position of the robot itself, and data collected by the target sensor includes:

[0027] Starting a thread corresponding to the target sensor, and processing data collected by the target sensor according to a corresponding data processing algorithm through the started thread to obtain attribute information of a candidate positioning marker within a sensing range of the target sensor, and a first relative position between the robot and the candidate positioning marker;

[0028] Processing the current global absolute position of the robot and the first relative position through the started thread to obtain the first global absolute position of the candidate positioning marker;

[0029] When the first global absolute pose of the candidate positioning marker matches the global absolute pose in the dot-marking result of the target positioning marker, determining the candidate positioning marker as the target positioning marker;

[0030] Controlling the target sensor to collect data for the target positioning identifier;

[0031] The thread corresponding to the target sensor is started, and the data collected by the target sensor is processed according to the corresponding data processing algorithm to obtain the current relative position between the robot and the target positioning mark.

[0032] Optionally, starting a thread corresponding to the target sensor, processing data collected by the target sensor according to a corresponding data processing algorithm, and obtaining a current relative position between the robot and the target positioning marker includes:

[0033] Synchronize the time of multiple frames of data collected by the target sensor for the target positioning identifier into data at one moment;

[0034] Starting a thread corresponding to the target sensor, processing the data at multiple moments according to a corresponding data processing algorithm, and calculating a second relative pose corresponding to each of the data at the multiple moments;

[0035] A joint optimization thread is started, and the plurality of second relative poses are averaged to obtain a current relative pose between the robot and the target positioning marker.

[0036] Optionally, after obtaining the current relative position between the robot and the target positioning marker, the method further includes:

[0037] comparing the current relative pose with the target relative pose;

[0038] When the current relative posture is different from the target relative posture, adjusting the current relative posture to the target relative posture;

[0039] When the robot is detected to be in the target relative posture, it controls itself to enter the target working state.

[0040] Optionally, the robot has three working modes, including: idle mode, dot mode and positioning mode;

[0041] In response to the positioning mode start instruction, querying whether the robot itself stores the marking result of the target positioning mark, including:

[0042] In response to the positioning mode start instruction, the robot enters the positioning mode from the idle mode;

[0043] In the positioning mode, querying whether the robot itself has stored the marking result of the target positioning mark;

[0044] In response to the first dotting mode start instruction, detecting the working state of the robot itself includes:

[0045] In response to the first dotting mode start instruction, the robot enters the dotting mode from the idle mode;

[0046] In the dotting mode, the working status of the robot itself is detected.

[0047] According to a second aspect of an embodiment of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the positioning method according to the first aspect of the embodiment.

[0048] According to a third aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the positioning method described in the first aspect of the embodiment is implemented.

[0049] According to a fourth aspect of an embodiment of the present invention, a computer program product is provided, comprising a computer program / instruction, which implements the positioning method described in the first aspect of the embodiment when executed by a processor.

[0050] The embodiments of the present invention include the following advantages:

[0051] In response to the positioning mode start instruction, on the one hand, absolute positioning is started to obtain the current global absolute position of the robot under absolute positioning output by the absolute positioning system, and at the same time, relative positioning is started to check whether the robot itself stores the attribute information and marking results about the target positioning mark. If stored, the absolute positioning is combined with the relative positioning, and the current relative position between the robot itself and the target positioning mark is determined based on the stored marking results, the current global absolute position of the robot under absolute positioning, and the target sensor configured by the robot that is adapted to the type of the target positioning mark.

[0052] This application improves the accuracy of the current relative position between the robot itself and the target positioning mark by combining absolute positioning and relative positioning. At the same time, when responding to the positioning mode start command, the robot of this application starts relative positioning and absolute positioning and combines the two. Therefore, relative positioning can be started when positioning is needed, thereby reducing the resource consumption of the robot throughout the entire working process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 This is a flowchart of the steps of the positioning method proposed in one embodiment of the present application;

[0055] Figure 2 This is a flowchart of the steps of the positioning method proposed in one embodiment of the present application;

[0056] Figure 3 This is a flowchart of the steps for the robot to obtain the dotting results in the dotting mode in one embodiment of the present application;

[0057] Figure 4 This is a functional module diagram of a positioning device proposed in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0059] With the development of intelligent technologies such as the Internet of Things (IoT), artificial intelligence (AI), and big data, the demand for transforming and upgrading the traditional logistics industry through these technologies is growing stronger. Intelligent logistics (Intelligent Logistics System) has become a research hotspot in the logistics field. Intelligent logistics utilizes AI, big data, and IoT devices and technologies such as information sensors, radio frequency identification (RFID), and the Global Positioning System (GPS). These technologies are widely applied to fundamental activities such as material transportation, warehousing, distribution, packaging, loading and unloading, and information services. This enables intelligent analysis and decision-making, automated operations, and efficient optimization of material management processes. IoT technologies include sensing devices, RFID, laser infrared scanning, and infrared sensor recognition. The IoT effectively connects materials in logistics to the network, enabling real-time monitoring of materials. It also senses environmental data such as humidity and temperature in warehouses to ensure a safe and secure storage environment. Big data technologies can sense and collect all logistics data, upload it to the data layer of an information platform, and filter, mine, and analyze the data. Ultimately, this data provides accurate data support for business processes such as transportation, warehousing, storage and retrieval, picking, packaging, sorting, outbound delivery, inventory, and distribution. The application of artificial intelligence in logistics can be broadly categorized into two main areas: 1) AI-enabled intelligent devices such as unmanned trucks, AGVs, AMRs, forklifts, shuttle trucks, stackers, unmanned delivery vehicles, drones, service robots, robotic arms, and smart terminals are replacing some manual labor; 2) software systems such as transportation equipment management systems, warehouse management systems, equipment scheduling systems, and order distribution systems are driven by technologies or algorithms like computer vision, machine learning, and operations optimization to improve human efficiency. With the research and advancement of smart logistics, this technology has been applied in numerous fields, including retail and e-commerce, electronics, tobacco, pharmaceuticals, industrial manufacturing, footwear and apparel, textiles, and food.

[0060] When a robot is working, it needs to be absolutely positioned, that is, the robot is positioned according to its own stored map. However, in actual application scenarios, the robot may change due to changes in the working area or the posture of the target task object (for example, a charging pile with a positioning mark, which can be a QR code, barcode, etc.). For example, if someone accidentally moves the charging pile, the robot may not be able to accurately reach the location to perform the target task. For example, if the robot reaches a charging pile at a certain location to charge, but the charging pile is accidentally moved by someone and the posture or position changes slightly, the robot may not be able to successfully dock with the charging port of the charging pile for charging when it reaches the position where the charging pile is located, resulting in charging failure. Therefore, it is difficult to meet the "high-precision positioning" of the robot by simply using absolute positioning.

[0061] This application addresses the problem that absolute positioning alone is unable to meet the robot's "high-precision positioning" requirements. A positioning method that combines absolute positioning and relative positioning is applied to the robot, so that the robot can more accurately determine the current relative position between itself and the target positioning marker, and then reach the location to perform the target task and enter the target working state to perform the target task.

[0062] Reference Figure 1 , shows a flowchart of the steps of a positioning method in an embodiment of the present invention, which may specifically include the following steps S101-S104:

[0063] S101: Receive a positioning mode start instruction for a target positioning identifier, where the positioning mode start instruction at least includes attribute information of the target positioning identifier.

[0064] When this method is applied, the robot's positioning system includes an absolute positioning system and a relative positioning system. The absolute positioning system is continuously in a positioning state during the robot's operation. The relative positioning system has three working modes, including: idle mode, dot mode and positioning mode.

[0065] When the relative positioning system is in idle mode, the robot does not start relative positioning, that is, the threads of each data processing in relative positioning are in a dormant state, and the data in each data queue in relative positioning is cleared. The robot can be set to be in idle mode by default when it is turned on, which can clear the space for data storage and reduce the resource consumption of the robot.

[0066] When the relative positioning system is in the marking mode, the robot combines absolute positioning and relative positioning to perform marking, that is, it uses its own configured sensors to identify the identifiable positioning marks in the current working area, and obtains various information of the positioning marks in the target working state as the marking result.

[0067] When the relative positioning system is in the positioning mode, the robot combines absolute positioning and relative positioning for positioning, responds to the positioning mode start command for the target positioning mark, identifies the target positioning mark, and determines the current relative position of the robot and the target positioning mark. Compared with positioning using absolute positioning alone, the positioning accuracy is higher. The robot can also adjust the current relative position to be consistent with the target relative position of the robot and the target positioning mark in the marking result, thereby achieving high-precision positioning and allowing the robot to enter the target working state.

[0068] In a feasible embodiment, the switching between the dotting mode and the positioning mode of the relative positioning system must go through the idle mode, that is, in response to the positioning mode start instruction, the robot enters the positioning mode from the idle mode; in response to the first dotting mode start instruction, the robot enters the dotting mode from the idle mode.

[0069] For example, its working process can be: there are multiple positioning markers set in the current working area. When the robot works in the current working area for the first time, the robot is commanded to work in the dotting mode, obtain the dotting results of the multiple positioning markers, and then store the dotting results locally in the robot; the robot can also share the dotting results stored in the dotting mode with other robots for storage, and then multiple robots can share the dotting results of one robot, thereby improving work efficiency.

[0070] When switching from dotting mode to idle mode, the robot clears the threads and data queues used to process data during the dotting process, reducing resource consumption and storage space. When the robot needs to perform high-precision positioning later, it can combine absolute and relative positioning based on the stored dotting results.

[0071] If the current task does not require high-precision positioning, the relative positioning system can be operated in idle mode, and the robot can be positioned only through absolute positioning, which can further reduce resource consumption.

[0072] Since multiple types of positioning identifiers can be set in the working area, but only a few types of positioning identifiers may be set in a certain working state, the positioning mode start instruction includes the attribute information of the target positioning identifier, that is, in the currently executed work task, only the target positioning identifier needs to be obtained for high-precision positioning. During a high-precision positioning process, the type of target positioning identifier can be one or more, and the number of target positioning identifiers can be one or more.

[0073] S102: In response to the positioning mode start instruction, query the robot whether it has stored the marking result of the target positioning mark. The marking result of the target positioning mark includes: attribute information of the target positioning mark, global absolute pose and target relative pose.

[0074] According to the positioning mode startup instruction, the robot can obtain the attribute information of the target positioning mark that needs to be positioned with high precision this time, and then the robot queries whether it has stored the marking results of the target positioning mark.

[0075] The attribute information of the target positioning marker includes: the type of the target positioning marker, the location attribute of the target positioning marker and the unique attributes of the target positioning marker. For example, the type of the target positioning marker includes a visual marker or a laser marker, and the location attribute of the target positioning marker is such as a charging pile QR code or a shelf laser marker. The unique attributes of the target positioning marker can be set so that the gaps between multiple flat panels on the shelf are different, and the corresponding spacing between positioning markers is also different.

[0076] The global absolute position of the target positioning marker refers to the global absolute position of the target positioning marker determined when the robot is in the target working state. The position of the target positioning marker in absolute positioning is stored in the robot's map and can be used by the robot to determine the target positioning marker among multiple positioning markers.

[0077] The target relative posture specifically refers to the posture of the target positioning marker relative to the robot when the robot is normally in the target working state. For example, in the dotting mode, when the robot is successfully in the charging state, it remembers its posture relative to the target positioning marker set on the charging pile; in the positioning mode, high-precision positioning can be performed based on the target relative posture. In order to successfully enter the charging state, the robot must adjust the current relative posture between itself and the target positioning marker to be consistent with the target relative posture in the dotting result, so that the global absolute posture of the robot in absolute positioning can be combined with the relative positioning of the robot relative to the target positioning marker. The robot's posture can be positioned more accurately, so that the robot can successfully enter the target working state.

[0078] Being in the target working state specifically refers to the robot performing the task normally. For example, the robot is successfully docked with the charging port of the charging pile for charging, or the robot is successfully docked with the shelf or material platform for material transmission.

[0079] S103: When the robot itself stores the marking result of the target positioning mark, the robot's current global absolute position is obtained, and a target sensor configured on the robot and adapted to the type of the target positioning mark is started.

[0080] After the robot retrieves the dotting results for the target positioning marker stored within itself, it obtains its current global absolute position, i.e., its current position under absolute positioning. Based on the type of target positioning marker, the robot activates a target sensor configured for the target positioning marker to obtain the target positioning marker within the robot's current field of view and within its work area. The robot can be configured with multiple sensors to identify different types of positioning markers. For example, a QR code can be captured by a configured camera, and a barcode can be recognized by a barcode scanner.

[0081] S104: Determine the current relative position between the robot and the target positioning marker according to the marking result of the target positioning marker, the current global absolute position of the robot itself, and the data collected by the target sensor.

[0082] In a feasible embodiment, the present application further provides a method for determining the current relative position between the robot itself and the target positioning mark, which specifically includes the following steps S21-S25:

[0083] First, you need to determine the target location identifier from multiple location identifiers, including:

[0084] S21: Start the thread corresponding to the target sensor, and process the data collected by the target sensor according to the corresponding data processing algorithm through the started thread to obtain attribute information of the candidate positioning marker within the sensing range of the target sensor, and the first relative posture between the robot and the candidate positioning marker.

[0085] In this application, each sensor has its own corresponding thread, which can avoid the situation where one thread processes data from multiple sensors, resulting in data accumulation and inability to output calculation results normally due to processing power limitations; by setting a thread for each sensor, the data processing process of each sensor can be relatively independent or non-interfering with each other, thereby improving the robustness of the relative positioning process.

[0086] Taking the target positioning identifier as a QR code as an example, the target sensor obtains data within its own sensing range, the camera installed on the robot is started and obtains the environmental image of the working area, and the thread corresponding to the target sensor processes the environmental image to obtain the features of multiple candidate QR code positioning identifiers, and determines the first relative posture between the robot and the multiple candidate QR code identifiers respectively.

[0087] S22: Processing the current global absolute position and the first relative position of the robot itself through the started thread to obtain the first global absolute position of the candidate positioning marker.

[0088] The current global absolute position of the robot itself can be read from the map of the robot's absolute positioning. Knowing the current global absolute position of the robot itself and the first relative position between the robot and multiple candidate QR code identifiers, the first global absolute position of each of the multiple candidate QR code identifiers can be calculated. The calculation formula is:

[0089]

[0090] in, Indicates the global absolute position of the positioning marker in absolute positioning, Refers to the global absolute position of the robot in the robot coordinate system, Indicates the relative position of the robot and the positioning marker in the robot coordinate system.

[0091] Here, the first global absolute pose of the candidate two-dimensional code identifier=the current global absolute pose of the robot*the first relative pose between the robot and the candidate two-dimensional code identifier.

[0092] S23: When the first global absolute position of the candidate positioning marker matches the global absolute position in the marking result of the target positioning marker, the candidate positioning marker is determined as the target positioning marker.

[0093] After determining the first global absolute pose of multiple candidate QR code identifiers, they are compared with the global absolute pose in the dot-marking result of the target positioning identifier, and the positioning identifier whose first global absolute pose matches the global absolute pose in the dot-marking result of the target positioning identifier is determined as the target positioning identifier.

[0094] In other embodiments, if the global absolute posture of the target object in the working area moves slightly, the global absolute posture of the positioning marker set on the target object will also change. Therefore, a range threshold can be preset. If a positioning marker is detected within the range threshold of the global absolute posture in the dot result of the target positioning marker, the positioning marker can be determined as the target positioning marker. If no positioning marker appears or multiple positioning markers appear within the range threshold of the global absolute posture of the target positioning marker, it means that the movement change of the target object is large, and abnormal information is generated and reported to the robot control system so that the staff can handle it in time.

[0095] After the target location marker is determined, the following steps are also included:

[0096] S24: Control the target sensor to collect data based on the target positioning identifier.

[0097] After finding the QR code of the target positioning identifier among multiple candidate QR code features, it is necessary to further accurately calculate the current relative posture between the robot itself and the target positioning identifier. Therefore, in this step, the robot's target sensor re-collects the QR code of the target positioning identifier.

[0098] S25: Start the thread corresponding to the target sensor, process the data collected by the target sensor according to the corresponding data processing algorithm, and obtain the current relative position between the robot and the target positioning mark.

[0099] In actual implementation, step S25 may include the following sub-steps S251-S253:

[0100] S251: Synchronize the time of multiple frames of data collected by the target sensor for the target positioning identifier into data at one moment.

[0101] First, the target sensor is used to obtain multi-frame data of the target positioning mark, and multi-frame matching is performed on the multi-frame data, that is, the multi-frame data is synchronized into the data at one moment. The local-pose of the robot under the odometer can be read. The difference of local-pose in a short time is very small, so local-pose can be used to synchronize the multi-frame data into the data at one moment. Then, the amount of data within a moment can be increased by using multi-frame data. Taking laser marking as an example, 10 laser points can be obtained in one frame. 5 frames can be obtained continuously, that is, 50 laser points can be obtained. The total 50 laser points obtained by 5 frames of data are synchronized as the data obtained at the same moment. By increasing the amount of data at that moment, the accuracy of the output effect can be improved.

[0102] S252: Start the thread corresponding to the target sensor, process the data at multiple moments according to the corresponding data processing algorithm, and calculate the second relative posture corresponding to each of the data at the multiple moments.

[0103] Using the data at a moment after time synchronization, the second relative posture corresponding to the data of the target positioning identifier at that moment is extracted and stored in a queue. The second relative posture corresponding to the data at that moment carries the sensor ID, which is convenient for distinguishing the threads corresponding to different sensors and calculating the different relative postures of multiple target positioning identifiers when there are multiple target positioning identifiers.

[0104] S253: Start the joint optimization thread, average the multiple second relative postures, and obtain the current relative posture between the robot and the target positioning marker.

[0105] The second relative postures corresponding to the data at multiple moments are averaged to obtain the current relative posture between the robot itself and the target positioning mark.

[0106] After determining the second relative pose of the robot and the target positioning marker at each moment through multi-frame matching and storing it, the second relative poses corresponding to the data at multiple moments are further jointly optimized to average the errors, so as to obtain a more accurate current relative pose between the robot itself and the target positioning marker.

[0107] For example, the data of the second relative pose corresponding to the data at multiple moments stored in the queue are retrieved, and the data at multiple moments of the same target positioning marker are jointly optimized, and the optimization results are output through the following steps A1-A7:

[0108] A1: Convert the target positioning marker's pose data to the robot's base coordinate system.

[0109] A2: Based on the robot's current global absolute pose and the second relative pose corresponding to each moment obtained by multi-frame matching in step S252, further determine whether it is data of the same target positioning mark; although the second relative pose corresponding to each moment carries the sensor ID when stored, when there are multiple target QR code marks, a camera sensor may obtain data of multiple target QR code mark marks. Therefore, it is necessary to obtain data of the same target QR code mark from the stored data again for further optimization. The determination method is as follows:

[0110]

[0111] That is, the global absolute pose of the target QR code marker is determined through the current global absolute pose of the robot in the robot coordinate system and the relative pose of the target QR code marker in the robot coordinate system, and the second relative pose of the target QR code marker at multiple moments with the same global absolute pose is further optimized.

[0112] A3: The entire optimization process is performed in the local coordinate system, so the data is converted to the local coordinate system:

[0113] A4: Construct a cost_function and optimize the data of the same target positioning marker using two constraints: observation constraints and inter-frame constraints. The observation constraints are the pose constraints measured by the sensor to the positioning marker, and the inter-frame constraints are the pose constraints of the robot at different times in the local coordinate system.

[0114] Specifically, the optimization variables are the pose of the target positioning identifier at each moment in the local coordinate system obtained by data conversion, as well as the pose of the target positioning mark.

[0115] Taking time t1 and t2 as an example, the optimized variables are recorded as: and The pose in the local system is known to be t1: t2: (Local pose can be obtained through the Local queue). The observation at time t1, that is, the pose of the target positioning mark identified at time t1, is: The credibility is s1. When detecting the target positioning mark, not only the relative position of the robot and the target positioning mark is output, but also its credibility is output. The observation at time t2, that is, the position of the target positioning mark recognized at time t1, is: Its credibility is s2.

[0116] Then, the observation constraint error is:

[0117]

[0118]

[0119] Odometer error:

[0120]

[0121] The relative position of the target positioning marker and the robot is optimized based on the observation constraint error and the odometry error.

[0122] A5: Use the solver to optimize the solution and update the data after the optimization is completed.

[0123] A6: Optimize the calibration times for a positioning mark before output to improve output accuracy.

[0124] A7: After the optimization times reach the calibration times, the latest output is generated in the form of callback. As the current phase pose after improving the output accuracy.

[0125] This application improves the accuracy of the current relative posture between the robot itself and the target positioning marker by combining absolute positioning and relative positioning, and through multi-frame matching and joint optimization. After outputting a more accurate current relative posture, in order to achieve high-precision points of the robot, the posture of the robot can be adjusted according to the current relative posture and the target relative posture.

[0126] Reference Figure 2 , shows a flowchart of the steps of a positioning method in another embodiment of the present invention, wherein the positioning method is Figure 1 The steps after S104 shown include:

[0127] S105: Compare the current relative posture with the target relative posture; when the current relative posture is different from the target relative posture, adjust the current relative posture to the target relative posture; when detecting that the robot itself is in the target relative posture, control itself to enter the target working state.

[0128] After multi-frame matching and joint optimization, a more accurate current relative posture between the robot and the target positioning marker is obtained, and then the current relative posture is compared with the target relative posture in the target positioning marker marking result. If they are consistent, it means that the current robot posture is accurate and can enter the target working state; if they are inconsistent, the robot adjusts its own global absolute posture, and then adjusts the current relative posture to the target relative posture to enter the target working state.

[0129] Reference Figure 3 , shows a flowchart of steps for a robot to obtain a dotting result in a dotting mode in another embodiment of the present invention, which may include the following steps:

[0130] B1: Receive the first dotting mode start instruction.

[0131] When the robot is in a new working area or needs to re-dot dotting, a first dotting mode start instruction is sent to the robot to instruct the robot to work in the dotting mode.

[0132] B2: In response to the first dotting mode start instruction, detecting the working status of the robot itself.

[0133] In the dotting mode, the robot detects whether its working state is in the target working state. The target working state refers to passing through a certain target posture or reaching a certain target posture during the execution of the target work task.

[0134] B3: When the working state of the robot itself is the target working state, the target global absolute position of the robot itself is obtained, and each sensor configured by the robot itself for identifying the positioning mark is started.

[0135] When the robot's own working state is the target working state, that is, the robot is in a state of passing through a certain target posture or reaching a certain target posture, the robot's own target global absolute posture is determined, and all sensors configured by the robot itself for identifying positioning marks are started to start data collection.

[0136] B4: Start the threads corresponding to the respective sensors, and process the data collected by the respective sensors according to the corresponding data processing algorithms through the started threads to obtain the attribute information of the positioning mark and the relative position and posture of the target.

[0137] In the target working state, each sensor collects various positioning identifiers that can be identified within its sensing range, and then determines the attribute information of each of the multiple identifiable positioning identifiers. Each sensor is equipped with its own corresponding thread, which can not only improve the efficiency of data processing, but also prevent the data processing of each sensor from interfering with each other, thereby improving the robustness of the data processing process.

[0138] When determining the relative positions of the plurality of positioning markers relative to the robot, a more accurate marking result can be obtained through multi-frame matching and joint optimization according to steps 251-253.

[0139] B5: Processing the target relative position of the positioning marker and the target global absolute position of the robot itself through the started thread to determine the global absolute position of the positioning marker.

[0140] The global absolute pose of each positioning marker is determined according to the target global absolute pose of the robot itself and the target relative pose of each positioning marker relative to the robot.

[0141] B6: The marking result of the positioning mark is stored locally and / or sent to other devices, wherein the marking result includes: attribute information of the positioning mark, global absolute position and target relative position.

[0142] When the marking mode of the relative positioning system is switched to the positioning mode, the data processing thread and the data in the queue must be cleared through the idle mode. Therefore, the marking results of multiple positioning identifiers need to be stored locally. The marking results of multiple positioning identifiers can also be sent to other robots. There is no need for each robot to perform marking, which can improve work efficiency.

[0143] After the marking is completed, the robot can perform high-precision positioning on the target positioning mark in the positioning mode based on the marking results of multiple stored positioning marks.

[0144] In step S102, the robot responds to the positioning mode activation command and queries whether it has stored the marking results for the target positioning marker. If the robot has stored the marking results for the target positioning marker, it executes step S103. If the robot does not store the marking results for the target positioning marker, it indicates that the robot has not marked the target positioning marker before, and outputs a prompt message indicating that there is no positioning result. The robot may have an information output component (such as a display screen, a prompt light, or a speaker) to output the prompt message indicating that there is no positioning result to inform the robot user. Alternatively, the robot may return the prompt message indicating that there is no positioning result to the robot control system, which will then process the prompt message.

[0145] When the marking results of the target positioning mark are not stored, the robot can also be instructed to perform additional marking only on the target positioning mark that has not been marked. The embodiment of the present invention also provides a method for supplementary marking by a robot, which can specifically include the following steps:

[0146] C1: When the robot does not store the marking result of the target positioning mark, receiving a second marking mode start instruction for the target positioning mark, where the second marking mode start instruction includes the type of the target positioning mark.

[0147] When the robot itself does not store the marking result of the target positioning mark, sending a second marking mode start instruction for the target positioning mark to the robot can enable the robot to perform additional marking on the target positioning mark.

[0148] C2: Detect whether the target sensor is configured on the device.

[0149] After receiving the second dotting mode start instruction, the robot will detect whether there is a target sensor that can be used to detect the target positioning mark among its configured sensors.

[0150] C3: When the target sensor is not configured for the robot, a prompt message is output, where the prompt message is used to prompt the user to configure the target sensor for the robot and configure a data processing algorithm for processing the data collected by the target sensor.

[0151] If the robot detects that its configured sensor cannot detect the target positioning mark, it outputs a prompt message so that the staff can add a new sensor that can detect the target positioning mark and the data processing algorithm corresponding to the sensor.

[0152] Since the processing process of the joint optimization thread is independent of the sensor, the joint optimization thread only optimizes the relative poses corresponding to multiple moments of data calculated by multi-frame matching. Therefore, when adding a new sensor, it is only necessary to add the multi-frame matching data processing algorithm corresponding to the new sensor. The relative pose corresponding to the data at a certain moment obtained by the multi-frame matching data processing algorithm of the new sensor carries the ID of the new sensor.

[0153] C4: When the robot is equipped with the target sensor, when the working state of the robot is the target working state, the target global absolute position of the robot is obtained, and the target sensor is started to collect data.

[0154] If the robot itself is equipped with a target sensor that can detect the target positioning mark, the characterization robot can mark the target positioning mark. When the robot's own working state is in the target working state, the robot's own target global absolute position is obtained through the absolute positioning system, and data is collected through the target sensor.

[0155] C5: Start the thread corresponding to the target sensor, and process the data collected by the target sensor according to the corresponding data processing algorithm through the started thread to obtain the marking result of the target positioning mark.

[0156] The process of obtaining the marking result is consistent with the process of steps B4-B5 and will not be repeated here. The marking result includes the attribute information of the target positioning mark, the global absolute pose and the relative pose of the target.

[0157] C6: storing the marking result of the target positioning mark locally and / or sending it to other devices.

[0158] The robot will store the supplementary marking results of the target positioning mark, or share them with other devices. Other devices can directly perform high-precision positioning based on the target positioning mark based on the supplementary marking results of the target positioning mark.

[0159] During the high-precision positioning process of this embodiment, anomaly detection can also be performed, including detection of sensor data anomalies and detection of anomalies in the extraction and matching of positioning markers. Anomalies include: failure to receive sensor data within a preset time; sensor instability; excessive time required for positioning marker extraction and multi-frame matching; feature extraction failure; or a significant difference between the current relative pose and the target relative pose. Throughout the high-precision positioning process, the current relative pose between the robot and the target positioning marker, as well as the anomaly detection results, are reported to the robot control system for display.

[0160] This application has at least the following beneficial effects:

[0161] 1. This application combines absolute positioning with relative positioning, and determines the current relative position between the robot itself and the target positioning identifier based on the stored dot results, the current global absolute position of the robot under absolute positioning, and the target sensor configured on the robot that is adapted to the type of the target positioning identifier.

[0162] This application improves the accuracy of the current relative position between the robot itself and the target positioning mark by combining absolute positioning and relative positioning.

[0163] 2. In this application, the robot works in idle mode by default. Only when high-precision positioning is required and instructions for marking or positioning are received, will it work in marking mode or positioning mode, thereby reducing resource consumption.

[0164] 3. It can be adapted to a variety of sensors and perform high-precision positioning based on various types of positioning markers.

[0165] 4. New types of positioning identifiers and compatible sensors can be added, as well as data processing algorithms corresponding to the newly added sensors, which has the effect of adapting to more types of positioning identifiers and has higher adaptability.

[0166] 5. Each sensor has its own corresponding data processing algorithm, which does not interfere with each other while improving the efficiency of data processing and the robustness of the positioning process.

[0167] 6. Based on multi-frame matching and joint optimization, a more accurate relative pose between the robot itself and the target positioning marker can be calculated.

[0168] The present application also provides a positioning device, referring to Figure 4 , is a functional module diagram of a positioning device proposed in an embodiment of the present application, the device comprising:

[0169] The instruction receiving module 100 is configured to receive a positioning mode start instruction for a target positioning identifier, wherein the positioning mode start instruction includes at least attribute information of the target positioning identifier;

[0170] The information storage module 200 is configured to query the robot itself in response to the positioning mode start instruction whether it has stored the marking result of the target positioning mark, wherein the marking result of the target positioning mark includes: attribute information of the target positioning mark, global absolute pose, and target relative pose;

[0171] The control module 300 is configured to obtain the current global absolute position of the robot itself when the robot itself stores the marking result of the target positioning mark, and activate the target sensor configured on the robot and adapted to the type of the target positioning mark;

[0172] The relative posture determination module 400 is used to determine the current relative posture between the robot and the target positioning marker based on the marking result of the target positioning marker, the current global absolute posture of the robot itself, and the data collected by the target sensor.

[0173] Optionally, the device further comprises:

[0174] A first dotting instruction receiving module, configured to receive a first dotting mode start instruction;

[0175] A first working state determining module is configured to detect the working state of the robot itself in response to the first dotting mode starting instruction;

[0176] a sensor selection module, configured to obtain the target global absolute position of the robot itself and activate various sensors configured on the robot itself for identifying positioning marks when the robot itself is in the target working state;

[0177] A first dotting result determination module is configured to start a thread corresponding to each of the sensors, process the data collected by each of the sensors according to a corresponding data processing algorithm through the started thread, and obtain attribute information of the positioning marker and the target relative pose; and further process the target relative pose of the positioning marker and the target global absolute pose of the robot through the started thread to determine the global absolute pose of the positioning marker;

[0178] The first marking result storage module is used to store the marking result of the positioning mark locally and / or send it to other devices. The marking result includes: attribute information of the positioning mark, global absolute posture and target relative posture.

[0179] Optionally, the device further comprises:

[0180] A first prompt module is configured to output a prompt message indicating that there is no positioning result when the robot itself does not store the marking result of the target positioning identifier;

[0181] A second dotting instruction receiving module is configured to receive a second dotting mode start instruction for the target positioning identifier, wherein the second dotting mode start instruction includes the type of the target positioning identifier;

[0182] A target sensor detection module, configured to detect whether the module is equipped with the target sensor;

[0183] The second prompt module is used to output prompt information when the target sensor is not configured for the robot, and the prompt information is used to prompt the user to configure the target sensor for the robot and configure a data processing algorithm for processing the data collected by the target sensor.

[0184] Optionally, the device further comprises:

[0185] a target sensor selection module, configured to, when the robot is equipped with the target sensor and the robot's operating state is a target operating state, obtain the robot's target global absolute position and activate the target sensor for data acquisition;

[0186] a second marking result determination module, configured to start a thread corresponding to the target sensor, and process the data collected by the target sensor according to a corresponding data processing algorithm through the started thread to obtain a marking result of the target positioning identifier;

[0187] The second marking result storage module is used to store the marking result of the target positioning mark locally and / or send it to other devices. The marking result of the target positioning mark includes: attribute information of the target positioning mark, global absolute pose and target relative pose.

[0188] Optionally, the relative posture determination module includes:

[0189] a first processing unit, configured to start a thread corresponding to the target sensor, and process data collected by the target sensor according to a corresponding data processing algorithm through the started thread to obtain attribute information of a candidate positioning marker within a sensing range of the target sensor, and a first relative position between the robot and the candidate positioning marker;

[0190] A second processing unit is configured to process the current global absolute position of the robot and the first relative position through the started thread to obtain a first global absolute position of the candidate positioning marker;

[0191] a third processing unit, configured to determine the candidate positioning marker as the target positioning marker when the first global absolute pose of the candidate positioning marker matches the global absolute pose in the dot-marking result of the target positioning marker;

[0192] The fourth processing unit is used to control the target sensor to collect data for the target positioning identifier; start the thread corresponding to the target sensor, process the data collected by the target sensor according to the corresponding data processing algorithm, and obtain the current relative position between the robot and the target positioning identifier.

[0193] Optionally, the fourth processing unit includes:

[0194] a multi-frame matching subunit, configured to synchronize the time of multiple frames of data collected by the target sensor for the target positioning identifier into data at a single moment; start a thread corresponding to the target sensor, process the data at multiple moments according to a corresponding data processing algorithm, and calculate a second relative pose corresponding to each of the data at the multiple moments;

[0195] The joint optimization subunit is used to start the joint optimization thread, average the multiple second relative postures, and obtain the current relative posture between the robot and the target positioning mark.

[0196] Optionally, the device further comprises:

[0197] The posture adjustment module is used to compare the current relative posture with the target relative posture; when the current relative posture is different from the target relative posture, adjust the current relative posture to the target relative posture; when detecting that the robot itself is in the target relative posture, control itself to enter the target working state.

[0198] Optionally, the robot has three working modes, including: idle mode, dot mode and positioning mode;

[0199] The information storage module includes:

[0200] a first mode conversion unit, configured to cause the robot to enter a positioning mode from an idle mode in response to the positioning mode start instruction;

[0201] A first query unit is used to query whether the robot itself stores the marking result of the target positioning mark in the positioning mode;

[0202] The first working state determination module includes:

[0203] a second mode conversion unit, configured to cause the robot to enter the dotting mode from the idle mode in response to the first dotting mode start instruction;

[0204] The second detection unit is used to detect the working status of the robot itself in the dotting mode.

[0205] Optionally, the device also includes an anomaly detection module for performing anomaly detection and reporting the results of the anomaly detection to the robot control system for display; the contents of the anomaly detection include sensor data anomaly detection, positioning mark extraction and matching anomaly detection, failure to receive sensor collected data within a preset time, sensor instability, positioning mark extraction and multi-frame matching taking too long, feature extraction failure, and a large difference between the current relative posture and the target relative posture.

[0206] An embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the positioning method described in the embodiment.

[0207] An embodiment of the present application further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the positioning method described in the embodiment is implemented.

[0208] An embodiment of the present application further provides a computer program product, including a computer program / instruction, which implements the positioning method described in the embodiment when executed by a processor.

[0209] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0210] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0211] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0214] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0215] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0216] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A positioning method, characterized in that: The method comprises: receiving a positioning mode start instruction for a target positioning identifier, wherein the positioning mode start instruction includes at least attribute information of the target positioning identifier; In response to the positioning mode start instruction, the robot is queried whether it has stored the marking result of the target positioning marker, the marking result of the target positioning marker including: attribute information, global absolute position and target relative position of the target positioning marker; the global absolute position of the target positioning marker is the global absolute position of the target positioning marker determined when the robot is in the target working state; the target relative position of the target positioning marker is the position of the target positioning marker relative to the robot when the robot is normally in the target working state; In a case where the robot itself stores the marking result of the target positioning mark, obtaining the current global absolute position of the robot itself, and starting the target sensor configured on the robot and adapted to the type of the target positioning mark; Determine the current relative position between the robot and the target positioning marker according to the marking result of the target positioning marker, the current global absolute position of the robot itself, and the data collected by the target sensor; Determining a current relative position between the robot and the target positioning marker based on the marking result of the target positioning marker, the current global absolute position of the robot itself, and data collected by the target sensor, including: Starting a thread corresponding to the target sensor, and processing data collected by the target sensor according to a corresponding data processing algorithm through the started thread to obtain attribute information of a candidate positioning marker within a sensing range of the target sensor, and a first relative position between the robot and the candidate positioning marker; Processing the current global absolute position of the robot and the first relative position through the started thread to obtain the first global absolute position of the candidate positioning marker; When the first global absolute pose of the candidate positioning marker matches the global absolute pose in the dot-marking result of the target positioning marker, determining the candidate positioning marker as the target positioning marker; Controlling the target sensor to collect data for the target positioning identifier; The thread corresponding to the target sensor is started, and the data collected by the target sensor is processed according to the corresponding data processing algorithm to obtain the current relative position between the robot and the target positioning mark.

2. The method according to claim 1, characterized in that The method further comprises: Receive a first dotting mode start instruction; In response to the first dotting mode start instruction, detecting the working state of the robot itself; When the working state of the robot itself is the target working state, obtaining the target global absolute position of the robot itself, and starting each sensor configured by the robot itself for identifying the positioning mark; Starting the threads corresponding to the respective sensors, and processing the data collected by the respective sensors according to the corresponding data processing algorithms through the started threads to obtain the attribute information of the positioning marker and the relative position and posture of the target; Processing the target relative pose of the positioning marker and the target global absolute pose of the robot itself through the started thread to determine the global absolute pose of the positioning marker; The marking result of the positioning mark is stored locally and / or sent to other devices, and the marking result includes: attribute information of the positioning mark, global absolute position and target relative position.

3. The method according to claim 1, characterized in that The method further comprises: When the robot itself does not store the marking result of the target positioning mark, outputting a prompt message indicating that there is no positioning result; receiving a second dotting mode start instruction for the target positioning identifier, wherein the second dotting mode start instruction includes a type of the target positioning identifier; Detecting whether the target sensor is configured on the device; When the target sensor is not configured for the robot itself, a prompt message is output, where the prompt message is used to prompt the user to configure the target sensor for the robot and configure a data processing algorithm for processing the data collected by the target sensor.

4. The method according to claim 3, characterized in that The method further comprises: In the case where the robot is equipped with the target sensor, when the working state of the robot itself is the target working state, obtaining the target global absolute position of the robot itself, and starting the target sensor to collect data; Starting a thread corresponding to the target sensor, and processing the data collected by the target sensor according to a corresponding data processing algorithm through the started thread to obtain a marking result of the target positioning mark; The marking result of the target positioning mark is stored locally and / or sent to other devices, and the marking result of the target positioning mark includes: attribute information of the target positioning mark, global absolute position and target relative position.

5. The method according to claim 1, characterized in that Starting a thread corresponding to the target sensor, processing the data collected by the target sensor according to a corresponding data processing algorithm to obtain the current relative position between the robot and the target positioning marker, including: Synchronize the time of multiple frames of data collected by the target sensor for the target positioning identifier into data at one moment; Starting a thread corresponding to the target sensor, processing the data at multiple moments according to a corresponding data processing algorithm, and calculating a second relative pose corresponding to each of the data at the multiple moments; A joint optimization thread is started, and the plurality of second relative poses are averaged to obtain a current relative pose between the robot and the target positioning marker.

6. The method according to any one of claims 1 to 5, characterized in that: After obtaining the current relative position between the robot and the target positioning marker, the method further includes: comparing the current relative pose with the target relative pose; When the current relative posture is different from the target relative posture, adjusting the current relative posture to the target relative posture; When the robot is detected to be in the target relative posture, it controls itself to enter the target working state.

7. The method according to claim 2, characterized in that The robot has three working modes, including: idle mode, dot mode and positioning mode; In response to the positioning mode start instruction, querying whether the robot itself stores the marking result of the target positioning mark, including: In response to the positioning mode start instruction, the robot enters the positioning mode from the idle mode; In the positioning mode, querying whether the robot itself has stored the marking result of the target positioning mark; In response to the first dotting mode start instruction, detecting the working state of the robot itself includes: In response to the first dotting mode start instruction, the robot enters the dotting mode from the idle mode; In the dotting mode, the working status of the robot itself is detected.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the positioning method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the positioning method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the positioning method according to any one of claims 1 to 7 is implemented.

Images