A label mapping method, device, equipment and storage medium
By extracting attribute information and calculating confidence in the label image, the problem of inaccurate positioning of the robot is solved, the accuracy and efficiency of label mapping are improved, and labor costs are reduced.
Patent Information
- Application Number
- CN202110959317.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-20
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-08-20
AI Technical Summary
In the prior art, robots require manual proofreading when building or updating maps due to inaccurate positioning caused by label posting errors or misidentification, resulting in high cost and low efficiency.
By extracting attribute information in the label image, calculating the confidence of the label, and adding labels to the robot map based on the confidence, improving positioning accuracy and reducing human verification costs.
It improves the accuracy of label mapping, reduces the mapping errors caused by misidentification and label errors, reduces the cost of labor verification, and improves efficiency.
Smart Images

Figure CN113673499B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of wireless positioning technology, and in particular to a tag mapping method, device, equipment and storage medium. Background Art
[0002] With the rapid development of the field of artificial intelligence, robots are widely used in more and more scenarios, such as delivery robots, disinfection robots, reception robots, and sweeping robots.
[0003] Before the robot is officially used or when the operating environment changes, it is necessary to create or update a map so that it can be accurately positioned when used. Simultaneous localization and mapping (SLAM) is a method for realizing robot positioning or mapping in an unknown environment. Common methods in SLAM include visual SLAM and laser SLAM, among which visual SLAM is widely used due to its low cost advantage. For example, positioning tags can be set in the robot's operating environment, such as the ceiling and beams at the top of the restaurant, and a tag detector can be set on the top of the robot. By traversing the positioning marks on the tags and determining the coordinates corresponding to each positioning mark, the tag mapping in the operating environment can be completed.
[0004] In the above label mapping method, in order to reduce the problem of inaccurate positioning caused by incorrect label posting or robot misrecognition, it is often necessary to manually proofread the labels detected by the robot one by one, which has high labor costs and low mapping efficiency. Summary of the invention
[0005] The embodiments of the present invention provide a label mapping method, device, equipment and medium, which obtain the confidence of the label through the attribute information of the label contained in the label image, facilitate the addition of labels in the robot map according to the confidence, improve the accuracy of label mapping, and reduce the labor cost of label verification.
[0006] In a first aspect, an embodiment of the present invention provides a label mapping method, the method comprising:
[0007] Perform image acquisition in the label area to obtain at least one label image; the label image includes at least one label;
[0008] Recognize the label image to obtain attribute information of the label in the label image;
[0009] According to the attribute information, the target label and its confidence are obtained.
[0010] In a second aspect, an embodiment of the present invention provides a label mapping method, which is applied to a robot or a control terminal, and the method includes:
[0011] Acquire attribute information and confidence of tags collected by the robot in the tag area, wherein the attribute information includes tag position information;
[0012] When the robot drives out of the label area, the label to be added is determined according to the confidence level, and the attribute information of the label to be added is added to the robot map.
[0013] In a third aspect, an embodiment of the present invention further provides a label mapping system, the system comprising a robot and a control terminal, the robot and the control terminal being in communication connection;
[0014] The robot is used to execute the label mapping method provided by any embodiment of the present invention;
[0015] The control terminal is used to execute the label mapping method provided in the fourth embodiment of the present invention.
[0016] In a fourth aspect, an embodiment of the present invention further provides a label mapping device, the device comprising:
[0017] A label image acquisition module, used for performing image acquisition in the label area to acquire at least one label image; the label image includes at least one label;
[0018] An attribute determination module, used to identify the label image and obtain attribute information of the label in the label image;
[0019] The confidence calculation module is used to obtain the target label and its confidence according to the attribute information.
[0020] In a fifth aspect, an embodiment of the present invention further provides a label mapping device, the device comprising:
[0021] A confidence acquisition module, used to acquire attribute information and confidence of the tags collected by the robot in the tag area, wherein the attribute information includes tag position information;
[0022] The label adding module is used to determine the label to be added according to the confidence level when the robot drives out of the label area, and add the attribute information of the label to be added to the robot map.
[0023] In a sixth aspect, an embodiment of the present invention further provides an electronic device, including:
[0024] one or more processors;
[0025] A memory for storing one or more programs;
[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the label mapping method provided by any embodiment of the present invention.
[0027] In a seventh aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the label mapping method provided by any embodiment of the present invention.
[0028] The technical solution of the embodiment of the present invention performs image acquisition in the label area to obtain at least one label image, then recognizes the label image, obtains the attribute information of the label in the label image, and finally obtains the target label and its confidence level based on the attribute information, so as to facilitate adding the label to the robot map according to the label confidence level, reduce the mapping errors caused by robot misidentification or label posting errors, improve the accuracy of label mapping, and reduce the labor cost of label verification. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 is a flow chart of a label mapping method in Embodiment 1 of the present invention;
[0030] Figure 2a is a flow chart of a label mapping method in Embodiment 2 of the present invention;
[0031] Figure 2b is a schematic diagram of tag identification in Embodiment 2 of the present invention;
[0032] Figure 3 is a flow chart of a label mapping method in Embodiment 3 of the present invention;
[0033] Figure 4 is a schematic diagram of a label shape in the fourth embodiment of the present invention;
[0034] Figure 5 It is a structural schematic diagram of a label mapping system in Embodiment 5 of the present invention;
[0035] Figure 6 It is a structural schematic diagram of a label mapping device in Embodiment 6 of the present invention;
[0036] Figure 7 It is a structural schematic diagram of a label mapping device in Embodiment 7 of the present invention;
[0037] Figure 8 It is a structural diagram of a device provided in Embodiment 8 of the present invention. DETAILED DESCRIPTION
[0038] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0039] Embodiment 1
[0040] Figure 1 This is a flow chart of a label mapping method in Embodiment 1 of the present invention. The technical solution of this embodiment is applicable to the case where the label confidence is obtained based on the attribute information of the label contained in the label image. The method can be executed by a label mapping device, which can be implemented by software and / or hardware and can be integrated in various general-purpose computer devices. The label mapping method in this embodiment specifically includes the following steps:
[0041] Step 110: perform image acquisition in the label area to obtain at least one label image; the label image includes at least one label.
[0042] Among them, the tag is used for wireless positioning of the robot. Specifically, the tag can be a marker containing specific identification information pre-set in the area where the robot is located. The robot can locate itself by identifying the tag during operation, for example, circular tags made of special reflective materials with different numbers and positions. Exemplarily, the robot captures images through a camera during operation and determines its own position information by identifying the tags in the captured images; the tag image is an image containing the tag collected by the robot; the tag area is the area where the robot can collect the tag image. Generally, different tag areas correspond to different tags. For example, tag A can be a set of circular tags with a specific arrangement pasted on the ceiling. The robot can collect images through the camera it carries. The area where the robot can collect the tag image containing tag A is the tag area corresponding to tag A.
[0043] In this embodiment, the robot needs to move in a positioning environment with pre-set labels, and collect images during the movement. Finally, the labels contained in the collected images are added to the robot map to complete the construction of the environmental map based on the labels, so as to facilitate the subsequent movement and positioning of the robot in the above positioning environment according to the labels. Specifically, the robot starts to move in the positioning environment according to the user's instructions, and collects images at a set frequency during the movement. When it is detected that the robot enters the label area, it continues to collect images in the label area, and determines the images containing labels collected in the label area as label images, and each label image contains at least one label. Among them, when the robot collects images containing labels for a set number of consecutive times, it can be determined that the robot has entered the label area to avoid misjudgment that may be caused by determining based on a single collection. The set number of times can be set as needed, such as 3 times, 5 times, etc.
[0044] Exemplarily, when the robot starts to continuously collect 10 frames of images containing labels near point A, it is considered that the robot has entered the label area, and the images containing labels that the robot starts to collect near point A are determined as label images for label mapping.
[0045] Step 120: Identify the label image and obtain the attribute information of the label in the label image.
[0046] Among them, the attribute information of the tag may include one or more of the tag ID (Identity Document, identification number), tag shape, tag location information, and tag color, as long as the corresponding tag can be identified. For example, when the shape corresponding to each tag is different, the attribute information may only include the tag shape. The different shapes may be that some tags are square, some are rectangular, or all are circular, but the number and relative position of the circular logos included are different; when there are multiple tags with the same shape but different corresponding colors, the attribute information includes the tag shape and the tag color; it can be understood that the tag shape and / or the tag color can correspond to the tag ID, which is used to distinguish and represent each tag, so that the user can view it clearly.
[0047] In this embodiment, after the label image is collected in the label area, image recognition is performed on the label image to obtain the attribute information of the label image. Specifically, by recognizing the label image, information such as the label ID and label position information in the label image is obtained. It is worth noting that the label position information can be calculated based on the acquisition angle and acquisition range of the image acquisition unit carried by the robot and the position information of the robot. The position information of the robot can be obtained based on the initial position of the robot, mileage data, IMU (Inertial Measurement Unit) data, etc.
[0048] Exemplarily, by identifying the collected label image, the label ID contained in the label image is obtained as 1200, and the label position information is calculated according to the camera angle on the top of the robot, the camera acquisition range, and the position information of the robot when it collects the current label image.
[0049] Step 130: Obtain the target label and its confidence level according to the attribute information.
[0050] The confidence of a label refers to the probability that the label obtained by the robot through label image analysis is the same as the actual label. The higher the confidence of the identified label, the higher the probability that the identified label is the same as the actual label.
[0051] In this embodiment, in order to avoid misidentification of labels due to accidental factors such as light or acquisition angle, the target label and its confidence are obtained according to the attribute information of the label, so that the label is added according to the confidence to reduce the label recognition error rate. Specifically, the label images containing the same label in the label area can be classified into one category according to the label ID, and the number of label images of each category is counted. When the number of label images of a certain category is greater than the set number threshold, the label corresponding to the label image of this category is determined as the target label. By setting the number threshold, the noise interference caused by a single acquisition error can be avoided. The number threshold can be 10 frames, 20 frames, etc. In the actual positioning process, a label area should only collect one label, and the robot positioning can be achieved through this label. However, due to problems such as light or acquisition angle, two or more target labels may be identified in a label area during the label mapping process. In order to improve the accuracy of label recognition, the correctly identified label is added to the robot map while the incorrectly identified label is discarded. The confidence of the target label can be further calculated so that the control terminal can determine the unique label matching the current label area according to the confidence and add the unique label to the robot map. The confidence of the label can be obtained based on the proportion of each category of label images in all label images corresponding to the label area. The higher the proportion, the higher the confidence of the label. It is understandable that when the robot collects images directly below the label, the collected image is relatively complete, contains less noise, and has higher accuracy. Therefore, when the attribute information includes label position information, the confidence of the label can be obtained based on the relative position information between the label and the robot. The closer to the label directly below, the higher the confidence of the label. Of course, the confidence of the label can also be obtained based on the relative position information between the label and the robot and the proportion of each category of label images in the total number of label images.
[0052] It is worth noting that the target label and its confidence can be obtained when the robot walks out of the label area and obtains all the label images in the label area, or when the robot is driving in the label area. Specifically, the robot collects label images according to the set frequency during driving, and each time a label image is collected, the attribute information of the label contained in the label image is recognized. After the robot travels a set distance, a set time, or collects a set number of label images (the robot has not walked out of the label area at this time), the label images collected from the time the robot entered the label area to the current time are statistically processed to obtain the target label and its confidence. Compared with data processing after walking out of the label area, the pressure of data processing can be dispersed.
[0053] Exemplarily, every 0.5 meters the robot travels, it will classify or update the label images collected since it entered the label area, and classify the label images with the same label ID into one category of labels. When the number of label images of category A (including label images with label ID 1200) exceeds 30, label 1200 corresponding to the label images of category A is determined as the target label, and the confidence of label 1200 is determined based on the proportion of label images of category A in the label images collected in the current label area. The higher the above proportion, the higher the accuracy of label 1200.
[0054] The technical solution of the embodiment of the present invention performs image acquisition in the label area to obtain at least one label image, then recognizes the label image, obtains the attribute information of the label in the label image, and finally obtains the target label and its confidence level based on the attribute information, so as to facilitate adding the label to the robot map according to the label confidence level, reduce the mapping errors caused by robot misidentification or abnormal label posting, improve the accuracy of label mapping, and reduce the labor cost of label verification.
[0055] Embodiment 2
[0056] Figure 2a This is a flow chart of a label mapping method in Embodiment 2 of the present invention. This embodiment further refines the above embodiment and provides specific steps for obtaining a target label and its confidence level based on attribute information. Figure 2a A label mapping method provided in the second embodiment of the present invention is described, which includes the following steps:
[0057] Step 210: perform image acquisition in the label area to obtain at least one label image; the label image includes at least one label.
[0058] Step 220: Identify the label image and obtain the attribute information of the label in the label image.
[0059] Step 230: Classify the label images according to the attribute information to obtain the number of label images in each category.
[0060] In this embodiment, after acquiring the attribute information of the label in the label image, in order to avoid label recognition errors caused by shooting light or shooting angle factors and improve the accuracy of adding labels to the robot map, the label image is further classified according to the attribute information, such as the label ID, and the number of label images in each category is obtained, so as to determine whether the identified label meets the preliminary conditions for adding to the robot map according to the number of labels in each category. Specifically, label images containing the same label can be classified into the same category, and the number of label images in each category can be counted.
[0061] Classify the label images according to the attribute information and obtain the number of label images in each category, including:
[0062] In the label area, according to the preset statistical rules and the attribute information, the label images in the label area are classified to obtain the number of label images in each category.
[0063] In this embodiment, a method is provided for classifying label images according to attribute information to obtain the number of label images of each category. Specifically, in the label area, according to the preset statistical rules, according to the attribute information of the label, such as the label ID, the label images in the label area are classified, and the number of label images of each category is obtained, wherein the preset statistical rules refer to a preset statistical period. For example, the robot classifies and counts the label images every time it advances 0.5 meters in the label area, or the robot classifies and counts the label images every time it travels in the label area for 1 second, or the robot classifies and counts the label images every time it collects 50 frames of label images. The statistical range of the label images is the label images from the time the robot enters the label area to the end of the above period.
[0064] Step 240: Obtain the confidence of the target label according to the number of label images of the target category and the attribute information of the target label in the label images of the target category.
[0065] In this embodiment, after the number of label images of each category is obtained by counting, a category of label images with a number greater than a set threshold can be used as a category of label images of a target category, and the confidence of the target label can be obtained according to the number of label images of each target category and the attribute information of the target label in the label image of the target category. Specifically, the confidence of the target label can be obtained together according to the proportion of the number of label images of the target category in the total number of label images collected in the corresponding label area, and the label position information of the target label in the label image of the target category, and finally the label is added according to the confidence to improve the accuracy of label mapping. Among them, the label position information of the target label can be estimated based on the acquisition angle of each target label image, the acquisition range of the robot, the position information of the robot, and the area where the label is located in the label image. For example, the area range of the corresponding ceiling of the corresponding image collected can be estimated according to the position information of the robot, the height of the image acquisition device, the corresponding acquisition angle, and the height of the label posting, and the coordinate range of the area corresponding to the robot map can be obtained to obtain the approximate location information of the label. Then, based on the area of the label in the label image, such as left, middle, right, top, bottom, etc., the label position information is estimated. Furthermore, in order to improve the accuracy of the position information, the label position information of the target label estimated based on each target label image is averaged, and the average position information is used as the final position information of the target label. The Kmeans clustering algorithm can also be used to determine whether a label appears, identify the label ID, and determine the label information. Among them, the current position information of the robot can be obtained based on the IMU data; the label position information of the label can be roughly estimated based on the camera angle, camera visual range, robot position and other information. The position x1, x2... of the label in each picture. Then, the roughly estimated position is clustered according to the Kmeans clustering algorithm to obtain the precise position corresponding to the label, and then the relative position relationship between the label and the robot is obtained.
[0066] Optionally, the attribute information also includes tag location information;
[0067] According to the number of label images of the target category and the attribute information of the target label in the label image of the target category, the confidence of the target label is obtained, including:
[0068] When the number of label images of at least one category is greater than a set number threshold, the label image of the category is used as the label image of the target category, and the label corresponding to the label image of the target category is determined as the target label;
[0069] The confidence of the target label is calculated based on the proportion of the label images of the target category in the collected label images in the label area, and the relative position relationship between the robot and the target label; the relative position relationship is determined based on the position information of the robot and the label position information of the target label.
[0070] In this optional embodiment, a specific method for obtaining the confidence of the target label is provided according to the number of label images of the target category and the attribute information of the target label in the label image of the target category. First, when the number of label images of at least one category is greater than the set number threshold, the label image of the category is used as the label image of the target category, and the label contained in the label image of the target category is determined as the target label. Only the labels corresponding to one or more categories of label images whose number is greater than the set number threshold are determined as target labels, so as to achieve pre-screening of labels, reduce the data processing pressure of subsequent calculation of confidence, and improve the efficiency of label mapping. Then, the confidence of the target label is calculated according to the proportion of the label image of the target category in the label images collected in the corresponding label area, and the relative position relationship between the robot and the target label. Among them, the relative position relationship is determined according to the position information of the robot and the label position information of the target label, which can characterize the orientation of the label in which the image acquisition unit carried by the robot is collected. The label image collected closer to the label directly below has a higher label confidence. For example, a vertical line can be drawn from the center point of the label position to the ground, intersecting the ground at a ground point, and the confidence of the label can be obtained according to the angle formed by the center point of the robot image acquisition device, the center point of the label position, and the ground point. For example, if the angle is less than or equal to 10°, the confidence is high; if the angle is greater than 10° and less than or equal to 20°, the confidence is medium; if the angle is greater than 20°, the confidence is low. The confidence level can also be numerically quantified according to the size of the angle.
[0071] For example, Figure 2bAs shown, tags A, B and C are set on the ceiling of the robot's environment in advance, where the ID of tag A is 1000, the ID of tag B is 1200, and the ID of tag C is 1400. The robot starts to move at a constant speed of 1 meter per second from point P1. During the movement, the camera set on the top of the robot takes pictures at a frequency of 50 Hz. During the movement of the robot from P1 to P2, the collected images do not contain tags. From P2, the robot collects images containing tags, and when 10 frames of images containing tags are collected continuously, it is determined that the robot enters the tag area and collects tag images. The robot counts the tag images every 2 meters, that is, the image tags are classified according to the tag IDs containing tags in the tag images to obtain various types. The number of labels, and when the number of label images containing label A is greater than 30, it is determined as the label image of the target category. At the same time, the label A contained in the label image of the target category is used as the target label. Further, since a label area only contains a unique label, in order to avoid the situation where the robot mistakenly recognizes multiple labels in a label area (for example, recognizes two target labels, label A and label B), which leads to the addition of wrong labels in the robot map, it is necessary to calculate the confidence of the recognized target label. Specifically, first, according to the proportion of the label image of the target category in the label image that has been collected in the label area, the number score of the target label is obtained or updated. For example, if the proportion of label A is 65% and the proportion of label B is 30%, the number score of label A is higher. According to the relative position relationship between the robot and the target label, the position score of the target label is obtained (the closer the label image is to the label directly below, the higher the confidence of the label), and finally the number score and the position score are weighted and summed to obtain the confidence of the target label.
[0072] The technical solution of the embodiment of the present invention performs image acquisition in the label area, obtains at least one label image, and recognizes the label image to obtain the attribute information of the label in the label image, and then classifies the label image according to the attribute information to obtain the number of label images of each category, and finally obtains the confidence of the target label according to the number of label images of the target category and the attribute information of the target label in the label image of the target category. The confidence of the label is obtained by comprehensively considering the number of label images corresponding to each category of labels and the attribute information of the label. The labels with higher confidence in the label area can be added to the robot map. Compared with the prior art of adding all recognized labels to the robot map, this avoids the situation of label addition errors caused by robot misrecognition and improves the accuracy of label mapping.
[0073] Embodiment 3
[0074] Figure 3This is a flowchart of a label mapping method in Embodiment 3 of the present invention. This embodiment further refines the above embodiment and provides specific steps for collecting images in the label area and obtaining at least one label image. Figure 3 A label mapping method provided in the third embodiment of the present invention is described, which includes the following steps:
[0075] Step 310: perform image acquisition based on a preset image acquisition frequency, and determine whether the acquired image contains a label.
[0076] In this embodiment, the robot uses the image acquisition unit carried on the top to acquire images according to a preset image acquisition frequency, recognizes the image every time a frame of image is acquired, and determines whether the image contains a label, or uniformly recognizes a set number of images every time a set number of images are acquired to determine whether the acquired images contain a label.
[0077] Step 320: When a set number of images containing labels are continuously collected, it is determined that the robot enters the label area, and collects at least one label image containing the label in the label area; the label image includes at least one label.
[0078] In this embodiment, when the robot identifies the collected images and determines that a set number of images containing labels are collected continuously, it is determined that the robot enters the label area, and then collects label images containing labels in the label area according to the set image collection frequency, wherein the label image includes at least one label. Exemplarily, when it is detected that the robot collects 10 frames of images containing labels continuously, it is determined that the robot enters the label area, and then collects label images containing labels in the label area, until it is detected that the 10 consecutive frames of images collected by the robot do not contain labels, and it is determined that the robot has left the current label area.
[0079] Optionally, when the label image contains at least two labels, label position alarm information is sent to the control terminal, so that the control terminal displays the label position alarm information to the user.
[0080] In this optional embodiment, when the robot collects a label image containing at least two labels in the label area, the robot sends label position alarm information to the control terminal in real time, so that the control terminal displays the label position alarm information to the user, and can remind the user that multiple labels are set too close to each other, resulting in them appearing in the same label image. Specifically, the label position alarm information may include attribute information of at least two labels in the label image, such as the label ID of the label, etc. Providing the above information to the user can remind the user that the above two labels are set too close to each other, resulting in the two labels appearing in the same label image, so that the user can find the corresponding label according to the label information and make position corrections.
[0081] Exemplarily, when it is detected that a label image contains both label A and label B, it means that the setting distance between label A and label B is too close, which will affect the actual positioning of the robot during operation, resulting in the robot being unable to determine whether to perform its own positioning based on the position information corresponding to label A in advance or based on label B during operation. Therefore, the label position alarm information can be sent to the control terminal, wherein the label position alarm information contains the attribute information of label A and the attribute information of label B. For example, the ID of label A is 1000, the label shape is three circular marks in the first row, the first two positions of the second row correspond to circular marks, and the third row has three circular marks. The ID of label B is 1200, and the label shape is a circular mark in the first two positions of the first row, a circular mark in the first two positions of the second row, and three circular marks in the third row. The label attribute information can help users quickly determine the labels that are set too close and correct the label positions. The attribute information can also include label position information, which is convenient for users to quickly reach the corresponding labels for correction.
[0082] Step 330: When a set number of images are continuously collected that do not contain a label, it is determined that the robot has driven out of the label area.
[0083] In this embodiment, after determining that the robot has entered the label area, image acquisition is performed according to the set image acquisition frequency to obtain label images containing labels in the label area, until the set number of images continuously acquired do not contain labels, and the robot is determined to have left the label area. In this way, the route for the robot to build a map can be divided into multiple sections, and each section of the route corresponds to a label area. In other words, the labels are set so that there is an area between adjacent label areas where label images cannot be collected, so as to avoid confusion in robot positioning caused by setting labels too close.
[0084] Step 340: Determine whether the number of label images of at least one category collected by the robot during the process from entering the label area to driving out of the label area is greater than a set number threshold.
[0085] In this embodiment, after the robot drives out of the label area, it is determined whether the number of label images of at least one category collected by the robot from the time it enters the label area to the time it drives out of the label area is greater than a set threshold, that is, whether at least one target label is determined in a label area. If so, it is considered that there are target labels that can be added to the label area. Through the above method, the robot performs an initial screening of the identified labels, and no subsequent processing is performed on the labels corresponding to the label images whose number is less than the set threshold. This improves the accuracy of label recognition while reducing the computational workload and improving the efficiency of label mapping.
[0086] Step 350: When the number of label images of at least one category that the robot fails to collect during the process from entering the label area to driving out of the label area is greater than a set number threshold, a label abnormality alarm message is sent to the control terminal so that the control terminal displays the label abnormality alarm message to the user.
[0087] In this embodiment, when the number of label images of at least one category that are not collected by the robot from entering the label area to driving out of the label area is greater than the set quantity threshold, it means that no target label that can be added to the label area is collected, that is, any target label that reaches the label addition quantity is not collected. Therefore, it is necessary to send a label abnormality alarm information to the control terminal so that the control terminal displays the label abnormality alarm information to the user, prompting the user to check whether the label has abnormal conditions such as occlusion.
[0088] The technical solution of the embodiment of the present invention performs image acquisition based on a preset image acquisition frequency, and determines whether the acquired image contains a label. When a set number of images containing labels are continuously acquired, it is determined that the robot enters the label area, and at least one label image containing the label is acquired in the label area. When a set number of images that do not contain labels are continuously acquired, it is determined that the robot exits the label area, and it is determined whether the number of label images of at least one category acquired by the robot from entering the label area to exiting the label area is greater than a set number threshold. When the number of label images of at least one category that are not acquired by the robot from entering the label area to exiting the label area is greater than a set number threshold, label abnormality alarm information is sent to the control terminal, so that the control terminal displays the label abnormality alarm information to the user. It can be determined whether to add labels according to the number of label images of each category. When the number of label images of any category that are not acquired in the label area is greater than the set number threshold, no label is added, and a label abnormality alarm information is initiated to prompt the user to check whether abnormal situations such as label obstruction occur, thereby reducing manual verification of posted labels, reducing labor costs, and improving the efficiency and accuracy of label mapping.
[0089] Embodiment 4
[0090] The technical solution of this embodiment is applicable to the case where labels are added to the robot map according to the confidence of the labels. The method can be executed by a label mapping device, which can be implemented by software and / or hardware and can be integrated in various general-purpose computer devices. The label mapping method in this embodiment specifically includes the following steps:
[0091] Step 410: Obtain attribute information and confidence of the tags collected by the robot in the tag area, wherein the attribute information includes tag position information.
[0092] In this embodiment, the control terminal receives the attribute information and confidence of the label collected by the robot in the label area. During the driving process, the robot determines the target label and determines and updates the confidence corresponding to the target label according to a pre-set statistical period. The control terminal receives the relevant information and confidence of the target label sent by the robot in each statistical period. If the current target label is received for the first time, the target label and the confidence are stored accordingly. If the target label has already been stored, the confidence of the stored target label is updated according to the confidence corresponding to the target label currently received.
[0093] Step 420: When the robot drives out of the label area, the label to be added is determined according to the confidence level, and the attribute information of the label to be added is added to the robot map.
[0094] In this embodiment, when it is determined that the robot has driven out of the label area, it is determined that the label image in the label area has been collected. At this time, the label to be added can be determined in the at least one target label according to the confidence corresponding to the at least one target label currently stored, and finally the attribute information of the label to be added is added to the robot map. Specifically, the label with the highest confidence obtained in the same label area can be determined as the label to be added corresponding to the label area, and the label position information corresponding to the label is added to the robot map so as to be displayed at the corresponding coordinate position on the map.
[0095] Optionally, the attribute information includes a tag ID and a tag shape;
[0096] Before adding the attribute information of the tag to be added to the robot map, it also includes:
[0097] Compare the tag ID of the tag to be added with the tag ID of the existing tag in the robot map;
[0098] When the tag ID of the tag to be added is the same as the tag ID of the tag already existing in the robot map, the tag to be added is updated according to the tag shape of the unused tags in the tag database and the tag shape of the tag to be added, and the updated attribute information of the tag to be added is displayed to the user.
[0099] In this optional embodiment, in order to avoid repeated addition of labels due to repeated label settings, before adding the attribute information of the label to be added to the robot map, the label ID of the label to be added is first compared with the label ID of the label already existing in the robot map. When the label ID of the label to be added is the same as the label ID of the label already existing in the robot map, it means that the current label to be added has been added to the robot map before, that is, there is a situation where the same label is set repeatedly. Therefore, the user can be prompted for duplicate labels to facilitate the user to handle exceptions in time. Further, in order to facilitate the user to correct the label, the label to be added can be updated according to the label shape of the unused label in the label database and the label shape of the label to be added, and the attribute information of the updated label to be added is displayed to the user to prompt the user that the currently determined label to be identified should be changed to the updated label to be added. Among them, the shape of the tag to be added can be compared with the shape of the unlabeled shape in the tag database to calculate the similarity, and the unlabeled tag with a similarity higher than a set threshold is determined as the updated tag to be added. For example, the similarity is determined according to the number of identifiers that need to be modified when the tag to be added is changed to the unlabeled tag. The more identifiers that need to be modified, the lower the similarity, thereby reducing the workload of the user in subsequent modifications. For example, Figure 4 As shown, the shape information of the label can include multiple circular logos. Figure 4 The label shapes in the first row have circular labels for the first three positions, the second row has circular labels for the first three positions, the third row has circular labels for the first and third positions, and the fourth row has circular labels for the first, second, and fourth positions.
[0100] The technical solution of the embodiment of the present invention first obtains the attribute information and confidence of the labels collected by the robot in the label area, and then when the robot drives out of the label area, determines the labels to be added according to the confidence, and adds the attribute information of the labels to be added to the robot map, thereby realizing label addition according to the label confidence, avoiding the misrecognition of labels caused by factors such as light or collection angle, and improving the accuracy of label mapping.
[0101] Embodiment 5
[0102] Figure 5 The present invention provides a structural diagram of a label mapping system provided in Embodiment 5 of the present invention. The label mapping system includes: a robot 1 and a control terminal 2, wherein the robot 1 and the control terminal 2 are connected in communication; the robot 1 includes an image acquisition unit 11 and a data processing unit 12. The robot 1 and the control terminal 2 can be connected in communication directly or indirectly through a cloud server.
[0103] The robot 1 is used to collect images in the label area, obtain at least one label image, identify the label image, obtain attribute information of the label in the label image, and obtain the target label and its confidence according to the attribute information.
[0104] The image acquisition unit 11 is used to perform image acquisition in the label area to obtain at least one label image.
[0105] The data processing unit 12 is used to identify the label image, obtain the attribute information of the label in the label image, and obtain the target label and its confidence according to the attribute information.
[0106] In this embodiment, the robot 1 starts to move in the positioning environment according to the user's instructions, and during the movement, the image acquisition unit 11 performs image acquisition at a set frequency. When it is detected that the robot 1 enters the label area, the image acquisition unit 11 continues to perform image acquisition in the label area, and the image containing the label acquired in the label area is determined as a label image, and each label image contains at least one label. Among them, when the robot 1 acquires images containing labels for a set number of consecutive times, it is determined that the robot 1 has entered the label area.
[0107] After the label image is collected in the label area, the label image is recognized by the data processing unit 12 to obtain the attribute information of the label image. Specifically, by recognizing the label image, the label image contains information such as label ID, label shape, label position information and label color. Finally, the data processing unit 12 can determine the target label that meets the label adding conditions according to the attribute information of the label, and obtain the confidence corresponding to the target label. Compared with the control terminal 2 or the cloud server, the label image recognition by the data processing unit 12 of the robot 1 can reduce the pressure of concurrent data processing. Finally, the robot 1 can send the target label, the attribute information of the target label and the confidence corresponding to the target label to the control terminal 2, so that the control terminal 2 can determine the target label that needs to be added to the robot map according to the confidence of the target label sent by the robot 1, avoiding the label addition error caused by the misidentification of the robot 1, improving the accuracy of label mapping, and no need to spend a lot of manpower costs to verify the labels in the robot map, improving the efficiency of label mapping.
[0108] In this embodiment, the robot 1 performs pre-identification and processing during the process of collecting label images, screens out at least one target label that meets the set label adding conditions, and only sends the attribute information and confidence of the target label to the control terminal 2. Compared with the method in which the robot 1 sends all images to the control terminal 2 for unified processing after completing the image collection in the mapping environment, the efficiency of label mapping is improved.
[0109] The control terminal 2 is used to obtain the attribute information and confidence of the tags collected by the robot 1 in the tag area. When the robot 1 drives out of the tag area, the tag to be added is determined according to the confidence, and the attribute information of the tag to be added is added to the robot map.
[0110] In this embodiment, the control terminal 2 receives the attribute information and confidence of the label collected by the robot 1 in the label area. The robot 1 determines the target label and determines and updates the confidence corresponding to the target label according to the pre-set statistical period during the driving process. The control terminal 2 receives the relevant information and confidence of the target label sent by the robot in each statistical period. If the current target label is received for the first time, the target label and the confidence are stored accordingly. If the target label has been stored, the confidence of the stored target label is updated according to the confidence corresponding to the target label currently received. When it is determined that the robot 1 has driven out of the label area, it is determined that the label image in the label area has been collected. At this time, the label to be added can be determined in at least one target label according to the confidence corresponding to at least one currently stored target label, and finally the attribute information of the label to be added is added to the robot map.
[0111] The technical solution of the embodiment of the present invention performs image acquisition in the label area to obtain at least one label image, then recognizes the label image, obtains the attribute information of the label in the label image, and finally obtains the target label and its confidence level based on the attribute information. The label can be added to the robot map based on the label confidence level, thereby avoiding mapping errors caused by robot misidentification or abnormal label posting, improving the accuracy of label mapping, and reducing the labor cost of label verification.
[0112] Embodiment 6
[0113] Figure 6 This is a structural schematic diagram of a label mapping device provided in Embodiment 6 of the present invention. The label mapping device includes: a label image acquisition module 610, an attribute determination module 620 and a confidence calculation module 630.
[0114] The label image acquisition module 610 is used to collect images in the label area to obtain at least one label image; the label image includes at least one label;
[0115] The attribute determination module 620 is used to identify the label image and obtain the attribute information of the label in the label image;
[0116] The confidence calculation module 630 is used to obtain the target label and its confidence according to the attribute information.
[0117] The technical solution of the embodiment of the present invention performs image acquisition in the label area to obtain at least one label image, then recognizes the label image, obtains the attribute information of the label in the label image, and finally obtains the target label and its confidence level based on the attribute information, so as to facilitate adding the label to the robot map according to the label confidence level, reduce the mapping errors caused by robot misidentification or abnormal label posting, improve the accuracy of label mapping, and reduce the labor cost of label verification.
[0118] Optionally, the confidence calculation module 630 includes:
[0119] A quantity acquisition unit, used to classify the label images according to the attribute information to obtain the quantity of label images of each category;
[0120] The confidence calculation unit is used to obtain the confidence of the target label according to the number of label images of the target category and the attribute information of the target label in the label images of the target category.
[0121] Optionally, the label image acquisition module 610 includes:
[0122] A label determination unit, used to perform image acquisition based on a preset image acquisition frequency and determine whether the acquired image contains a label;
[0123] The label image acquisition unit is used to determine that the robot enters the label area when a set number of images containing labels are continuously acquired, and to acquire at least one label image containing the label in the label area.
[0124] The quantity acquisition unit includes:
[0125] The quantity acquisition subunit is used to classify the label images in the label area according to the preset statistical rules and the attribute information in the label area to obtain the quantity of label images of each category.
[0126] Optionally, the attribute information also includes tag location information;
[0127] Confidence calculation unit, including:
[0128] a target label determination subunit, configured to, when the number of label images of at least one category is greater than a set number threshold, use the category label image as the label image of the target category, and determine the label corresponding to the label image of the target category as the target label;
[0129] The confidence calculation subunit is used to calculate the confidence of the target label according to the proportion of the label images of the target category in the label images collected in the label area, and the relative position relationship between the robot and the target label; the relative position relationship is determined according to the position information of the robot and the label position information of the target label.
[0130] Optionally, the label mapping device further includes:
[0131] The tag position alarm information sending module is used to send the tag position alarm information to the control terminal when the tag image contains at least two tags, so that the control terminal displays the tag position alarm information to the user.
[0132] Optionally, the label mapping device further includes:
[0133] A label area determination module, used for determining that the robot has left the label area when a set number of images continuously collected do not contain labels;
[0134] A label image judgment module is used to judge whether the number of label images of at least one category collected by the robot during the process of entering the label area and driving out of the label area is greater than a set number threshold;
[0135] The label abnormality alarm information sending module is used to send label abnormality alarm information to the control terminal when the number of label images of at least one category that the robot fails to collect during the process from entering the label area to driving out of the label area is greater than a set number threshold, so that the control terminal displays the label abnormality alarm information to the user.
[0136] The label mapping device provided in the embodiment of the present invention can execute the label mapping method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0137] Embodiment 7
[0138] Figure 7 This is a structural schematic diagram of a label mapping device provided in Embodiment 7 of the present invention. The label mapping device includes: a confidence acquisition module 710 and a label adding module 720.
[0139] A confidence acquisition module 710 is used to acquire attribute information and confidence of tags collected by the robot in the tag area, wherein the attribute information includes tag position information;
[0140] The label adding module 720 is used to determine the label to be added according to the confidence level when the robot drives out of the label area, and add the attribute information of the label to be added to the robot map.
[0141] The technical solution of the embodiment of the present invention first obtains the attribute information and confidence of the labels collected by the robot in the label area, and then when the robot drives out of the label area, determines the labels to be added according to the confidence, and adds the attribute information of the labels to be added to the robot map, thereby realizing label addition according to the label confidence, avoiding the misrecognition of labels caused by factors such as light or collection angle, and improving the accuracy of mapping.
[0142] Optionally, the attribute information includes a tag ID and a tag shape;
[0143] Optionally, the label mapping device further includes:
[0144] An identification information comparison module, used for comparing the tag ID of the tag to be added with the tag ID of the tag already existing in the robot map before adding the attribute information of the tag to be added to the robot map;
[0145] The module for updating the label to be added is used to update the label to be added according to the label shape of the unused label in the label database and the label shape of the label to be added when the label ID of the label to be added is the same as the label ID of the label already existing in the robot map, and to display the updated attribute information of the label to be added to the user.
[0146] The label mapping device provided in the embodiment of the present invention can execute the label mapping method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0147] Embodiment 8
[0148] Figure 8 A schematic diagram of the structure of an electronic device provided in Embodiment 8 of the present invention is shown in FIG. Figure 8 As shown, the electronic device includes a processor 80, a memory 81, an input device 82 and an output device 83; the number of processors 80 in the device can be one or more. Figure 8 A processor 80 is taken as an example; the processor 80, memory 81, input device 82 and output device 83 in the device can be connected by a bus or other means. Figure 8 The example of connecting through bus is taken in the following.
[0149] The memory 81 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the label mapping method in the embodiment of the present invention (for example, the label image acquisition module 610, the attribute determination module 620 and the confidence calculation module 630 in the label mapping device, or the confidence acquisition module 710 and the label adding module 720). The processor 80 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 81, that is, realizing the above-mentioned label mapping method, including:
[0150] Perform image acquisition in the label area to obtain at least one label image; the label image includes at least one label;
[0151] Recognize the label image to obtain attribute information of the label in the label image;
[0152] According to the attribute information, the target label and its confidence are obtained.
[0153] Alternatively, implement the label mapping method, including:
[0154] Obtain the attribute information and confidence of the tags collected by the robot in the tag area;
[0155] When the robot drives out of the label area, the label to be added is determined according to the confidence level, and the attribute information of the label to be added is added to the robot map.
[0156] The memory 81 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 81 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 81 may further include a memory remotely arranged relative to the processor 80, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0157] Embodiment 9
[0158] Embodiment 9 of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to execute a label mapping method when executed by a computer processor, the method comprising:
[0159] Perform image acquisition in the label area to obtain at least one label image; the label image includes at least one label;
[0160] Recognize the label image to obtain attribute information of the label in the label image;
[0161] According to the attribute information, the target label and its confidence are obtained.
[0162] Alternatively, implement the label mapping method, including:
[0163] Acquire attribute information and confidence of tags collected by the robot in the tag area, wherein the attribute information includes tag position information;
[0164] When the robot drives out of the label area, the label to be added is determined according to the confidence level, and the attribute information of the label to be added is added to the robot map.
[0165] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present invention is not limited to the method operations described above, and the computer executable instructions can also execute related operations in the label mapping method provided in any embodiment of the present invention.
[0166] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, an application server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0167] It is worth noting that in the embodiment of the above-mentioned label mapping device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0168] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A label mapping method, characterized in that: include: Perform image acquisition in the label area to obtain at least one label image; wherein the label image includes at least one label; Recognize the label image to obtain attribute information of the label in the label image; wherein the attribute information includes label position information; Classify the label images according to the attribute information to obtain the number of label images in each category; When the number of label images of at least one category is greater than a set number threshold, the label image of the category is used as the label image of the target category, and the label corresponding to the label image of the target category is determined as the target label; The confidence of the target label is calculated based on the proportion of the label images of the target category in the label images collected in the label area, and the relative position relationship between the robot and the target label; wherein the relative position relationship is determined based on the position information of the robot and the label position information of the target label.
2. The method according to claim 1, characterized in that The step of performing image acquisition in the label area to obtain at least one label image includes: Capturing images based on a preset image acquisition frequency, and determining whether the acquired images contain labels; When a set number of images containing the label are continuously collected, it is determined that the robot enters the label area, and collects at least one label image containing the label in the label area.
3. The method according to claim 1, characterized in that Also includes: When the label image contains at least two labels, label position alarm information is sent to the control terminal, so that the control terminal displays the label position alarm information to the user.
4. The method according to claim 1, characterized in that: Also includes: When a set number of images are continuously collected without the label, it is determined that the robot has left the label area; Determine whether the number of label images of at least one category collected by the robot from the time the robot enters the label area to the time it exits the label area is greater than a set number threshold; If not, the tag abnormality alarm information is sent to the control terminal, so that the control terminal displays the tag abnormality alarm information to the user.
5. A label mapping method, characterized in that: include: Acquire attribute information and confidence of the tags collected by the robot in the tag area; wherein the attribute information includes tag location information; and the confidence is determined by the tag mapping method according to any one of claims 1 to 4; When the robot drives out of the label area, the label to be added is determined according to the confidence level, and the attribute information of the label to be added is added to the robot map.
6. The method according to claim 5, characterized in that The attribute information includes tag ID and tag shape; Before adding the attribute information of the to-be-labeled tag to the robot map, the method further includes: Compare the tag ID of the tag to be added with the tag ID of the tag already existing in the robot map; When the tag ID of the tag to be added is the same as the tag ID of the tag already existing in the robot map, the tag to be added is updated according to the tag shape of the unused tags in the tag database and the tag shape of the tag to be added, and the attribute information of the updated tag to be added is displayed to the user.
7. A label mapping system, comprising a robot and a control terminal, wherein the robot and the control terminal are in communication connection; The robot is used to perform the label mapping method described in any one of claims 1 to 4; The control terminal is used to execute the label mapping method described in claim 5 or 6.
8. A label mapping device, characterized in that: include: A label image acquisition module, used to collect images in the label area to acquire at least one label image; wherein the label image includes at least one label; An attribute determination module, used to identify the label image and obtain attribute information of the label in the label image; wherein the attribute information includes label position information; A confidence calculation module, including a quantity acquisition unit and a confidence calculation unit; The quantity acquisition unit is used to classify the label images according to the attribute information to obtain the quantity of label images of each category; The confidence calculation unit includes a target tag determination subunit and a confidence calculation subunit; The target label determination subunit is used to, when the number of label images of at least one category is greater than a set number threshold, use the category label image as the label image of the target category, and determine the label corresponding to the label image of the target category as the target label; The confidence calculation subunit is used to calculate the confidence of the target label based on the proportion of the label images of the target category in the label images collected in the label area, and the relative position relationship between the robot and the target label; wherein the relative position relationship is determined based on the position information of the robot and the label position information of the target label.
9. A label mapping device, characterized in that: include: A confidence acquisition module, used to acquire attribute information and confidence of the labels collected by the robot in the label area; wherein the attribute information includes label position information; and the confidence is determined by the label mapping method according to any one of claims 1 to 4; The label adding module is used to determine the label to be added according to the confidence level when the robot drives out of the label area, and add the attribute information of the label to be added to the robot map.
10. An electronic device, characterized in that: The device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the label mapping method as described in any one of claims 1-6.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the label mapping method as described in any one of claims 1 to 6 is implemented.
