Robot positioning method and device

Abstract

The invention provides a robot positioning method and device, and the method comprises the steps: obtaining a to-be-processed panoramic image which is obtained based on a panoramic camera of a robot; determining at least one initial detection area in the to-be-processed panoramic image; determining a target detection object corresponding to each initial detection region according to the region space information of each initial detection region and the image space information of the panoramic image to be processed; identifying content information corresponding to each target detection object; and matching is carried out according to the content information and preset map information, the position information of the robot is determined, and the preset map information is created based on a panoramic image shot by a panoramic camera of the robot. According to the robot positioning method, the around view camera is used for positioning the robot, the implementation cost of the positioning method is reduced, the recognition result of the at least one target object is used for positioning the robot, the accuracy of the positioning result is guaranteed, and then the high-precision and low-cost robot positioning method is achieved.

Description

Technical Field

[0001] The present application relates to the field of panoramic camera technology, and in particular to a robot positioning method, a robot positioning device, a computing device, and a computer-readable storage medium. Background Art

[0002] With the development of artificial intelligence technology, the field of autonomous robots has gradually become a hot research direction. Among them, the robot's self-positioning ability is the basis for service robots to achieve motion planning tasks, so high-precision and low-cost positioning methods have always been an important research issue in the field of robotics.

[0003] Currently, the field of mobile robots widely uses multiple sensors to perceive the environment around the robot in complex scenes. The use of multiple sensors will not only increase the manufacturing cost of the robot, but multiple cameras will also need to run multiple algorithms, which will affect the operating efficiency of the robot and further increase the cost of the robot. Therefore, a high-precision and low-cost robot positioning method is needed. Summary of the invention

[0004] In view of this, the embodiment of the present application provides a robot positioning method. The present application also relates to a robot positioning device, a computing device, and a computer-readable storage medium to solve the above problems existing in the prior art.

[0005] According to a first aspect of an embodiment of the present application, a robot positioning method is provided, comprising:

[0006] Acquire a panoramic image to be processed, wherein the panoramic image to be processed is acquired based on a surround-view camera of the robot;

[0007] Determining at least one initial detection area in the panoramic image to be processed;

[0008] Determining the target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed;

[0009] Identify content information corresponding to each target detection object;

[0010] The position information of the robot is determined by matching each content information with preset map information, wherein the preset map information is created based on a panoramic image taken by a surround-view camera of the robot.

[0011] According to a second aspect of an embodiment of the present application, a robot positioning device is provided, comprising:

[0012] An acquisition module is configured to acquire a panoramic image to be processed, wherein the panoramic image to be processed is acquired based on a surround-view camera of the robot;

[0013] an initial determination module, configured to determine at least one initial detection area in the panoramic image to be processed;

[0014] A target determination module is configured to determine a target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed;

[0015] An identification module is configured to identify content information corresponding to each target detection object;

[0016] The positioning module is configured to determine the position information of the robot by matching each content information with preset map information, wherein the preset map information is created based on a panoramic image taken by a surround-view camera of the robot.

[0017] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the robot positioning method when executing the computer instructions.

[0018] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed by a processor, the steps of the robot positioning method are implemented.

[0019] The robot positioning method provided in the present application obtains a panoramic image to be processed, wherein the panoramic image to be processed is obtained based on the surround-view camera of the robot; determines at least one initial detection area in the panoramic image to be processed; determines the target detection object corresponding to each initial detection area based on the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed; identifies the content information corresponding to each target detection object; and determines the position information of the robot by matching each content information with preset map information, wherein the preset map information is created based on the panoramic image taken by the surround-view camera of the robot.

[0020] An embodiment of the present application realizes that by confirming at least one initial detection area in the panoramic image acquired by the surround-view camera, information around the robot is acquired only by the surround-view camera, and then a recognizable target object is obtained by processing the initial detection area, and then the target object is identified and the position of the robot is determined according to the identification result, so that the robot can be positioned only by the surround-view camera, which reduces the implementation cost compared to positioning the robot through multiple sensors; and considering that a large amount of computing resources are required to process the panoramic image acquired by the surround-view camera to obtain the target object, by processing the detection frame, computing resources are saved when positioning the robot, thereby reducing the implementation cost of the positioning method, and at the same time, the use of identifying at least one target object and positioning the robot according to the identification result also ensures the accuracy of the positioning result, thereby realizing a high-precision and low-cost robot positioning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a robot positioning method provided by an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a detection frame rotation provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of a detection frame rotation provided by an embodiment of the present application;

[0024] Figure 4 is a processing flow chart of a robot positioning method applied to an automatic robot in a shopping mall provided by an embodiment of the present application;

[0025] Figure 5 It is a trademark space topology structure provided by an embodiment of the present application;

[0026] Figure 6 is a structural schematic diagram of a robot positioning device provided by an embodiment of the present application;

[0027] Figure 7 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0029] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] First, the terms involved in one or more embodiments of the present application are explained.

[0033] Object detection: Object detection is a key task in computer vision. Its purpose is to locate and identify multiple objects in an image or video. Unlike traditional image classification tasks (which only tell you whether an object exists in the image), object detection aims to provide an accurate bounding box for each object in the image and give its category.

[0034] YOLO (You Only Look Once): YOLO is a real-time object detection system that treats the object detection task as a regression problem, directly from image pixels to bounding box coordinates and category probabilities.

[0035] Optical Character Recognition (OCR): used to convert text from paper documents, photos, or screen captures into editable and searchable data.

[0036] Face recognition technology: Face recognition technology is a computer vision technology that authenticates or identifies a person based on their facial features. It can automatically detect and identify a face, then compare it to a known facial database to determine the individual's identity.

[0037] Simultaneous localization and mapping (SLAM) technology: is a method widely used in fields such as robots and self-driving cars, which allows a robot to perform localization and mapping in an unknown environment at the same time. In complex and repetitive scenarios, robots can easily lose their positions if they rely solely on SLAM technology. At the same time, in the constructed map, it is necessary to manually assign semantic information to different modules in the map (that is, assign semantic information such as real store names to different location modules in the established map), which will greatly reduce the operating efficiency of the machine and make the robot less intelligent. At the same time, in large commercial scenarios, the surrounding environment is complex and changeable. Robots need to be able to more comprehensively perceive dynamic and static objects in the surrounding environment in order to execute different obstacle avoidance and cleaning strategies. Most existing solutions use multiple sensors to perceive the environment around the robot, which will not only greatly increase the cost, but also require multiple cameras to run multiple algorithms, which will affect the operating efficiency of the robot and increase the cost.

[0038] In the present application, a robot positioning method is provided. The present application also relates to a robot positioning device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0039] Figure 1 A flowchart of a robot positioning method provided according to an embodiment of the present application is shown, which specifically includes the following steps:

[0040] Step 102: Acquire a panoramic image to be processed, wherein the panoramic image to be processed is acquired based on a surround-view camera of the robot.

[0041] Specifically, the panoramic image to be processed is used to extract content information of the target detection object, classification information of the target detection object and spatial information of the target detection object, and the surround view camera is used to obtain the panoramic image to be processed.

[0042] Furthermore, the panoramic image to be processed can be understood as an image taken by a surround-view camera, including the scene within 360° of the surround-view camera, the shape of the image is circular, and the position of the surround-view camera is the center of the image.

[0043] It should be noted that since the panoramic image has the scene within 360° of the surround camera, the scene therein is often inverted, tilted, etc., and the inverted and tilted image may cause, for example, an inverted "6" to be recognized as a "9", reducing the accuracy of subsequent image recognition. Therefore, the image to be recognized needs to be subsequently rotated to ensure the accuracy of image recognition.

[0044] The robot is positioned by using the to-be-processed panoramic image captured by the surround-view camera, which effectively avoids the problem of significantly increasing costs caused by using multiple sensors to perceive the environment around the robot.

[0045] Step 104: Determine at least one initial detection area in the panoramic image to be processed.

[0046] Specifically, the initial detection area is used to identify the initial detection object contained in the panoramic image and carries the category of the corresponding initial detection object to facilitate subsequent different recognition operations on detection objects of different categories. The initial detection area contains the detection category information of the target detection object.

[0047] Furthermore, the initial detection area can be understood as using target detection technology to process the target detection object to obtain an upright rectangular target recognition frame. Since the targets contained in the panoramic image are often inverted, tilted, etc., the upright rectangular target recognition frame cannot correctly match the initial recognition object in the panoramic image, and thus the initial detection area needs to be rotated to a certain extent to obtain a target detection area that matches the panoramic image in order to facilitate subsequent operations on the initial detection object included therein. In addition, the initial detection area also has category information of the objects contained. For example, when a trademark appears in the image, an upright rectangular target recognition frame with the detection category of the positioning mark category is marked at the location of the trademark.

[0048] like Figure 2 As shown, Figure 2 is a schematic diagram of a detection frame rotation provided by an embodiment of the present application, Figure 2 The circle in is a schematic diagram of a panoramic image to be processed taken by a surround-view camera, wherein 202 is an initial detection area.

[0049] By performing preliminary target detection on the image to be processed to obtain the initial detection area, the category information of the target detection object can be accurately obtained as well as the rough position of the target detection object.

[0050] Step 106: Determine the target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed.

[0051] Specifically, the target detection object is used to obtain its corresponding content information and position information to facilitate the subsequent positioning and adjustment of the robot.

[0052] Furthermore, the target detection object can be understood as an upright image, and its corresponding content information can be obtained from the image. For example, by performing text recognition on an upright trademark image, the text information corresponding to the trademark image can be accurately obtained, and by performing face recognition on an upright user image, the facial feature information corresponding to the user can be accurately obtained.

[0053] Because the surround view camera causes the detection objects in the image to be tilted left and right, inverted, straight, and other angles when shooting, and recognizing images at multiple angles will result in low accuracy. By processing the initial detection area, upright target detection objects are obtained, thereby improving the accuracy of target detection object content recognition.

[0054] Specifically, determining the target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed includes:

[0055] Determine the detection area center point position information of each initial detection area based on the spatial information of each area;

[0056] Determine the center point position information of the panoramic image based on the image spatial information;

[0057] Determine the target detection area corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area;

[0058] The target detection object corresponding to each target detection area is determined in the panoramic image to be processed.

[0059] Specifically, the detection area center point position information and the panoramic image center point position information are used to determine the target detection area, and the target detection area is used to obtain the target detection object.

[0060] Further, such as Figure 2 As shown, the center point position of the detection area is 202, which can be understood as the center point coordinates of the initial detection area. The center point position of the panoramic image is Figure 2 The center point coordinates of the circle in can be understood as the center point coordinates of the panoramic image to be processed, and the target detection area is Figure 2 The 204 in the figure can be understood as the detection area after the initial detection area is rotated.

[0061] The target detection area is obtained by rotating the initial detection area through the position information of the center point of the detection area and the center point of the panoramic image, which solves the problem of mismatch between the original upright rectangular target recognition frame and the object in the image caused by the left and right tilt of the detection object in the image when shooting by the surround-view camera.

[0062] Further, according to the panoramic image center point position information and the center point position information of each detection area, determining the target detection area corresponding to each initial detection area includes:

[0063] Generating an initial detection area rotation angle corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area;

[0064] Each initial detection area is rotated based on the initial detection area rotation angle to determine each target detection area.

[0065] Specifically, the initial detection area rotation angle is used to rotate the initial detection area to obtain the target detection area.

[0066] Furthermore, the initial detection area rotation angle is determined by the panoramic image center point position information and the detection area center point position, such as Figure 2 As shown, the angle α is the rotation angle of the initial detection area.

[0067] The rotation angle of the initial detection area is obtained through the position information of the center point of the panoramic image and the center point of the detection area, and the initial detection area is rotated according to the rotation angle. Compared with the existing method of rotating the initial detection area using a deep learning model, this saves computing resources for rotating the target detection area, thereby reducing the operating cost of robot positioning.

[0068] Further, generating an initial detection area rotation angle corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area includes:

[0069] Determine the first horizontal coordinate information and the first vertical coordinate information corresponding to the panoramic image center point position information, and determine the second horizontal coordinate information and the second vertical coordinate information corresponding to the target detection area center point position information, wherein the target detection center point position information is any one of the center point position information of each detection area;

[0070] Obtaining horizontal coordinate difference information according to the first horizontal coordinate information and the second horizontal coordinate information, and obtaining vertical coordinate difference information according to the first vertical coordinate information and the second vertical coordinate information;

[0071] A detection area rotation angle is generated based on the horizontal coordinate difference information and the vertical coordinate difference information.

[0072] Specifically, the first horizontal coordinate information and the first vertical coordinate information are obtained according to the center point position of the panoramic image, the second horizontal coordinate information and the second vertical coordinate information are obtained according to the center point position of the detection area, and the horizontal coordinate difference and the vertical coordinate difference are obtained according to the first horizontal coordinate information and the second horizontal coordinate information and the first vertical coordinate information and the second vertical coordinate information, respectively.

[0073] Furthermore, the first horizontal coordinate information and the first vertical coordinate information can be understood as the horizontal coordinate and the vertical coordinate of the panoramic image center position information, such as Figure 2 As shown, Figure 2 (x0, y0) in is the first horizontal coordinate information and the first vertical coordinate information; the second horizontal coordinate information and the second vertical coordinate information can be understood as the horizontal coordinate and the vertical coordinate of the central position information of the detection area, such as Figure 2 As shown, Figure 2 Where (x1, y1) is the first horizontal coordinate information and the first vertical coordinate information; the horizontal coordinate difference can be understood as the value obtained by subtracting the first horizontal coordinate information from the second horizontal coordinate information; the vertical coordinate difference can be understood as the value obtained by subtracting the first vertical coordinate information from the second vertical coordinate information, and the specific formula is shown in Formula 1:

[0074]

[0075] Among them, α is the rotation angle of the initial detection area, x0 is the first horizontal coordinate information, that is, the horizontal coordinate of the center position information of the panoramic image, y0 is the first vertical coordinate information, that is, the vertical coordinate of the center position information of the panoramic image, x1 is the second horizontal coordinate information, that is, the horizontal coordinate of the center position information of the detection area, and y1 is the second vertical coordinate information, that is, the vertical coordinate of the center position information of the detection area.

[0076] Furthermore, determining the target detection object corresponding to each target detection area in the panoramic image to be processed includes:

[0077] Determine an initial detection object of each target detection area in the panoramic image to be processed;

[0078] Dividing the panoramic image to be processed into at least two panoramic image areas;

[0079] A target rotation rule corresponding to a target panoramic image area is determined, and initial detection objects in the target panoramic image area are adjusted based on the target rotation rule to obtain target detection objects corresponding to each initial detection object.

[0080] Specifically, the initial detection object is an object contained in the target detection area, the panoramic image area is an area divided according to the panoramic image to be processed, and the target rotation rule is a rule preset in each panoramic image area.

[0081] Furthermore, the initial detection object can be understood as an image carrying information contained in the panoramic image to be processed, such as a trademark image, a user image, a billboard image, etc.; the panoramic image area can be understood as an area obtained by dividing the panoramic image to be processed according to preset rules; the target rotation rule can be understood as a rotation rule preset in different panoramic image areas, which is used to rotate the originally tilted initial detection object in the corresponding panoramic image area to an upright image; the detection area to which the initial detection object belongs can be determined according to the coordinate information, for example, the center of the panoramic image to be processed is regarded as the origin to establish a coordinate system, the horizontal and vertical coordinates of the center point of the initial detection object have the same sign as the first area, and the horizontal and vertical coordinates of the center point of the initial detection object have different signs as the second area.

[0082] like Figure 3 As shown, Figure 3 is a schematic diagram of a detection frame rotation provided by an embodiment of the present application. Figure 3 There are four panoramic image areas 302 to 308, of which 3022 and 3024 are two initial detection areas in area 302, and the values ​​"123" are initial detection objects, of which 3022 is in a tilted and inverted state and 3024 is in a tilted state.

[0083] By dividing the panoramic image to be processed into different areas and presetting different rotation rules according to different areas, the tilted and inverted initial detection objects in the panoramic image can be effectively processed into upright target detection objects, so as to facilitate subsequent content recognition.

[0084] Specifically, adjusting the initial detection object in the target panoramic image area based on the target rotation rule includes:

[0085] Obtaining object space information corresponding to each initial detection object;

[0086] The initially detected objects in the target panoramic image area are adjusted based on the target rotation rule and the spatial information of each object.

[0087] Specifically, the object space information is used to adjust the initial detection object, and the object space information includes the width and height of the initial detection object and the position of the initial detection object in the panoramic image.

[0088] Furthermore, the object spatial information may be understood as spatial information of the initially detected object in the panoramic image, including the width and height of the initially detected object and the panoramic image region to which the initially detected object belongs.

[0089] Considering that the images collected by the surround view camera at different angles may be images of multiple angles such as left and right tilt, inverted, and straight, the object space information includes the height information, width information, and horizontal angle of the initial detection object;

[0090] Further, adjusting the initial detection objects in the target panoramic image area based on the target rotation rule and the spatial information of each object includes:

[0091] Determining shape information according to a relationship between height information and width information corresponding to an initial detection object, wherein the initial detection object is any one of the initial detection objects in the target panoramic image area;

[0092] Determine an initial positive rotation angle according to the target rotation rule and shape information, and add the object horizontal angle and the initial positive rotation angle to obtain a target positive rotation angle;

[0093] The initial detection object is rotated based on the target positive rotation angle.

[0094] Specifically, the height information is the length of the side adjacent to the initial detection height information, the width information is the length of any side of the initial detection object, the horizontal angle is the angle between the side corresponding to the width information of the initial detection object and the horizontal line, the shape information is the relationship between the width and height of the initial detection object, the initial rotation angle is the rotation angle determined by the target rotation rule and the shape information, and the target rotation angle is the rotation angle obtained by adding the horizontal angle and the initial rotation angle.

[0095] Furthermore, the height information, width information and horizontal angle can be understood as the spatial state of the initial detection object in the panoramic image to be processed. The edge corresponding to the width information can be the first edge encountered when the horizontal line is rotated along the point with the smallest vertical coordinate that intersects the initial detection object, and the edge corresponding to the height information can be the edge adjacent to the above-mentioned width information.

[0096] like Figure 3 As shown in the figure, the side corresponding to w in the figure is the side corresponding to the width information of the initial detection object, the side corresponding to h is the side corresponding to the height information of the initial detection area, and the angle between the side corresponding to w and the horizontal line is the horizontal angle. Specifically, the target rotation angle is obtained as shown in formula 2-5:

[0097]

[0098]

[0099]

[0100]

[0101] Among them, β is the target rotation angle, γ is the horizontal angle, w is the width information of the initial detection object, and h is the height information of the initial detection object. The w and h in the formula can be understood as length information. Interchanging w and h only interchanges the corresponding values ​​of w and h. This is to facilitate the subsequent program to identify the content of the target detection object, and does not interchange the sides corresponding to the width and height.

[0102] By rotating the initial detection object and identifying the image after the rotation, errors such as identifying an inverted "6" as a "9" are effectively avoided, thereby improving the accuracy of identifying the content information of the detection object.

[0103] Step 108: Identify content information corresponding to each target detection object.

[0104] Specifically, the content information is obtained by identifying the target detection object.

[0105] Furthermore, the content information can be understood as characteristic information of the target detection object. For example, if the detection type of the target detection object is a trademark, then the content information of the target detection object is positioning representation information, which is used to locate the position of the robot; if the detection type of the target detection object is a user, then the content information of the target detection object is user characteristic information, etc.

[0106] It should be noted that the method for identifying the target detection object can be understood as performing image recognition on the target detection object according to different categories to obtain different types of features corresponding to different types of objects. For example, OCR technology can be used to identify the text in the target detection object to obtain the corresponding text feature information of the target detection object, or face recognition technology can be used to identify the face feature information contained in the target detection object.

[0107] Because the objects in the panoramic images taken by the surround-view camera are tilted, inverted, and non-upright images may cause an inverted "6" to be recognized as "9" and a tilted "8" to be recognized as "00". Therefore, non-upright images may cause the accuracy of image recognition to be too low. The previous method of rotating the initial detection frame to obtain the target detection frame and then uprighting the initial detection object in the target detection frame can effectively improve the accuracy of image recognition, and thus the robot can be accurately positioned.

[0108] Step 110: Determine the location information of the robot by matching each content information with preset map information, wherein the preset map information is created based on a panoramic image taken by a surround-view camera of the robot.

[0109] Specifically, the preset map information is used to match the content information to determine the location information of the robot.

[0110] Furthermore, the preset map information can be understood as the map feature information contained in the automatically created digital map containing map feature information. For example, the map feature information at a certain location includes the feature information of the trademarks around the location and the spatial information between all the surrounding trademarks.

[0111] By using the positioning information detected and identified by the surround view camera during the mapping process, the map is automatically given certain semantic information, reducing human intervention and improving the robot's operating efficiency and intelligence level.

[0112] Because the images collected by the surround view camera at different angles contain objects of various categories such as trademarks, users, billboards, etc., the initial detection area includes detection category information, the detection category information includes a location identification category, and the content information includes location identification information;

[0113] According to the matching of each content information and the preset map information, the position information of the robot is determined, including:

[0114] In the case where the detection category information of the target initial detection area is a positioning identification category, the content information corresponding to the target initial detection area is positioning identification information, wherein the target initial detection area is any one of the initial detection areas;

[0115] Acquire the information position relationship between each positioning mark information in the image to be processed;

[0116] The position information of the robot is confirmed based on each positioning identification information and each information position relationship.

[0117] Specifically, the detection category information is the category information of the initial detection area, the content information of the target object whose positioning identification category is used for positioning detection category information, the positioning identification information is the content information corresponding to the detection area when the detection category information is the positioning identification category, and the information position relationship is the spatial information of all target detection objects whose detection category is the positioning identification category in the panoramic image to be processed obtained by the surround view camera.

[0118] Further, the detection category information can be understood as the category to which the target detection object belongs, which can be obtained when the OCR recognition is performed on the panoramic image to be processed to obtain the initial detection area. For example, if the target detection object is a trademark, then its corresponding detection category information is the positioning identification category; if the target object is a pedestrian, then its corresponding detection category information is the user category, etc. The positioning identification category can be understood as the detection category to which the target detection object used for robot positioning belongs, such as trademarks in shopping malls, characteristic QR codes in factories, and roadside buildings. The positioning identification information can be understood as the content information of the target detection object whose detection category information is the positioning identification category, such as text information corresponding to trademarks in shopping malls, characteristic information corresponding to characteristic QR codes in factories, and picture characteristic information corresponding to roadside buildings. The information position relationship can be understood as the spatial relationship between all target detection objects of the positioning identification category in the panoramic image to be processed obtained by the surround view camera, such as the topological relationship formed by the positions of all trademarks in the panoramic image to be processed captured by the surround view camera at a certain position in the shopping mall, and the graphic structure formed by the characteristic QR codes in the factory.

[0119] In one embodiment provided in the application, there are three trademarks in the panoramic image to be processed acquired by the surround-view camera, and the corresponding text information is "AA", "BB" and "CC" respectively. The three trademarks form a star topology structure. After comparing with the preset map information, the position of the robot equipped with the surround-view camera is obtained as A; at another moment, there are three trademarks in the panoramic image to be processed acquired by the surround-view camera, and the corresponding text information is "AA", "BB" and "CC" respectively. The three trademarks form a ring topology structure. After comparing with the preset map information, the position of the robot equipped with the surround-view camera is obtained as B.

[0120] All target detection objects of the positioning identification category around the robot are identified by the surround camera. After the positioning identification information is identified, the inherent position of the target detection object in the map is used to relocate the robot, with short detection time and high accuracy. At the same time, in order to prevent the existence of multiple target detection objects with repeated positioning identification information in the map, the method can collect 360° target detection object information around the robot by using the surround camera, and a spatial structure can be formed between the multiple target detection objects. When the target detection object and its surrounding target detection objects are identified, the spatial structure of the target detection object in the panoramic image to be processed can be retrieved in the database in combination with multiple positioning identification information, so that the position of the robot can be accurately confirmed. The method has high running speed and high accuracy, which improves the operating efficiency of the robot and the accuracy of relocation.

[0121] Specifically, the preset map information is obtained through the following steps:

[0122] Acquire at least one panoramic image for mapping, wherein the panoramic image for mapping is acquired based on a surround-view camera of the robot;

[0123] Determine at least one target detection object in each mapping panoramic image, and identify positioning identification information corresponding to each target detection object;

[0124] Acquire the information position relationship between each positioning mark information according to each mapping panoramic image;

[0125] The preset map information is generated based on each positioning identification information and each information position relationship.

[0126] Specifically, the panoramic image for mapping is obtained by the robot's surround camera.

[0127] Furthermore, the panoramic image can be understood as an image taken by a surround camera, which includes the scene within 360° of the robot. The shape of the image is circular, and the position of the robot is the center of the image.

[0128] By using the surround camera to detect and identify the surrounding categories of target detection objects as positioning identification categories during the mapping process, the map is automatically given certain semantic information, reducing human intervention and improving the robot's operating efficiency and intelligence level.

[0129] Furthermore, after identifying the content information corresponding to each target detection object, it also includes:

[0130] Determining a behavior strategy of the robot according to the content information of each area, and generating a behavior instruction of the robot based on the behavior strategy;

[0131] The robot is adjusted based on the behavioral instructions.

[0132] Specifically, the robot's behavior strategy is used to generate the robot's behavior instructions, and the robot's behavior instructions are used to adjust the corresponding robot.

[0133] Furthermore, a behavioral strategy can be understood as an instruction generating signal generated by the robot through the content information of the target detection object, and a behavioral instruction can be understood as an instruction generated corresponding to the behavioral strategy that can enable the robot to perform corresponding actions. For example, when it is detected that a user with user characteristic information of a tourist passes in front of the robot, a behavioral strategy is generated to cause the robot to stop moving and use a loudspeaker to send a welcome message to the user. According to this strategy, a corresponding behavioral instruction is generated, and according to this instruction, the robot stops moving and uses a loudspeaker to send a welcome message to the user.

[0134] Specifically, determining the behavior strategy of the robot according to the content information of each area includes:

[0135] The robot's behavior strategy is determined based on the detection category information and area content information of each initial detection area.

[0136] Specifically, the robot's behavior strategy can also be determined based on the detection category information and area content information of the initial detection area. For example, when the detection category information of the initial detection area is user, the robot's behavior strategy is determined based on the user feature information of the user; when the detection category information of the initial detection area is billboard, the robot's behavior strategy is determined based on the preset billboard size information and the billboard's spatial information, etc.

[0137] Further, the detection category information includes user category, and the area content information includes user feature information;

[0138] The robot's behavior strategy is determined based on the detection category information and area content information of each initial detection area, including:

[0139] In the case where the detected category information is a user category, the user type is determined according to the user feature information, and the behavior strategy of the robot is determined based on the user type.

[0140] Specifically, the content information of the target object with user category is used to determine the detection category information of different behavior strategies according to different users, the user feature information is the content information corresponding to the detection area when the detection category information is the user category, and the user type is the user type determined according to the user features.

[0141] Furthermore, user feature information can be understood as user facial feature information obtained based on the user's facial features and spatial location information of the user's map, wherein the facial feature information is obtained from the user's face image contained in the target detection object whose detection category is the user, and the user type can be understood as the category to which the user belongs, for example, employee, visitor, administrator, child, etc.

[0142] Selecting different behavior strategies according to different user categories allows the robot to make decisions quickly based on the needs and characteristics of the user, thereby completing tasks more efficiently. For example, for regular customers who are familiar with the layout of the mall, the robot can provide quick guidance; for first-time customers, the robot will provide more detailed guided tours. The robot can also adjust its behavior according to the behavior and needs of different users to ensure safety in the mall. For example, for children, the robot may adopt a more cautious behavior strategy.

[0143] By applying the scheme of the above-mentioned embodiment, by confirming at least one angle-free rectangular detection frame in the panoramic image acquired by the surround-view camera, it is possible to acquire information around the robot only through the surround-view camera, and then obtain a target detection frame that matches the target object in the panoramic image by rotating the angle-free rectangular detection frame, and then rotate the target object included in the target detection frame to obtain an upright target object, and then identify the target object and determine the position of the robot and the subsequent behavior of the robot based on the identification result, thereby realizing the positioning of the robot and the realization of the robot's autonomous behavior only through the surround-view camera, which reduces the implementation cost compared to realizing the positioning and autonomous behavior of the robot through multiple sensors.

[0144] Taking into account that processing the panoramic image acquired by the surround-view camera to obtain the target object will require a lot of computing resources, by processing the detection frame instead of the entire panoramic image, the computing resources used in robot positioning are saved, thereby reducing the implementation cost of the positioning method. At the same time, the target object after being erected is identified, which preliminarily improves the accuracy of target object identification. Subsequently, multiple target objects are identified and the robot is positioned according to the identification results, which further improves the accuracy of the positioning results, thereby realizing a high-precision and low-cost robot positioning method.

[0145] The following combination Figure 4 Taking the application of the robot positioning method provided by the present application to an automatic robot in a shopping mall as an example, the robot positioning method is further described. Figure 4 A processing flow chart of a robot positioning method for an automatic robot in a shopping mall provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0146] Step 402: The panoramic camera acquires a panoramic image to be processed.

[0147] Step 404: Perform target detection on the panoramic image to be processed to obtain a non-angled regular rectangular detection frame corresponding to the target object and a target category corresponding to the non-angled regular rectangular detection frame.

[0148] Step 406: Obtain the matching angles of each of the angle-free regular rectangular detection frames according to the center point of the panoramic image to be processed and the center point of each of the angle-free regular rectangular detection frames.

[0149] Step 408: Rotate each non-angled rectangular detection frame by a corresponding matching angle to obtain a target detection frame that matches the target object.

[0150] Step 410: Divide the panoramic image to be processed into four areas of equal size.

[0151] Step 412: Based on the position of the target detection frame in the panoramic image to be processed and the relationship between the width and height of the target detection frame itself, obtain a rotation angle for rotating the target object.

[0152] Step 414: Rotate the target object contained in the target detection frame by a corresponding rotation angle to obtain an upright target object.

[0153] Step 416: Identify the text features corresponding to all target objects whose target categories are trademarks in the panoramic image to be processed.

[0154] Step 418: Retrieve the trademark's topological structure from the database based on the identified multiple text features to confirm the actual position of the robot.

[0155] Step 420: Identify the facial features corresponding to the target object that is closest to the center point in the panoramic image to be processed and whose target object category is human.

[0156] Step 422: Compare the recognized different facial features with the facial features in the preset action table, and perform different actions according to the comparison results.

[0157] Step 424: When the target category is a billboard, perform an evasive action.

[0158] By applying the scheme of the above-mentioned embodiment, by confirming at least one angle-free rectangular detection frame in the panoramic image acquired by the surround-view camera, it is possible to acquire information around the robot only through the surround-view camera, and then obtain a target detection frame that matches the target object in the panoramic image by rotating the angle-free rectangular detection frame, and then rotate the target object included in the target detection frame to obtain an upright target object, and then identify the target object and determine the position of the robot and the subsequent behavior of the robot based on the identification result, thereby realizing the positioning of the robot and the realization of the robot's autonomous behavior only through the surround-view camera, which reduces the implementation cost compared to realizing the positioning and autonomous behavior of the robot through multiple sensors.

[0159] Taking into account that processing the panoramic image acquired by the surround-view camera to obtain the target object will require a lot of computing resources, by processing the detection frame instead of the entire panoramic image, the computing resources used in robot positioning are saved, thereby reducing the implementation cost of the positioning method. At the same time, the target object after being erected is identified, which preliminarily improves the accuracy of target object identification. Subsequently, multiple target objects are identified and the robot is positioned according to the identification results, which further improves the accuracy of the positioning results, thereby realizing a high-precision and low-cost robot positioning method.

[0160] Embodiment 1

[0161] In the first embodiment provided in the present application, taking the positioning of a service robot in a shopping mall based on the surround-view camera it carries as an example, the robot positioning method provided in the present application is further explained.

[0162] First, the map needs to be constructed. Map construction requires a robot with a surround-view camera to construct a map based on the images it obtains in real time. This can be understood as making the robot run in the mall and process the images obtained in real time and store the results obtained in real time until the map is constructed and stopped. The following takes the processing of an image at a certain moment as an example:

[0163] A panoramic image taken by its surround camera is obtained, and the obtained panoramic image is processed using the YOLO model to obtain each detection object in the panoramic image and the initial recognition frame corresponding to the detection object. In this embodiment, the panoramic image contains 7 detection objects, namely, trademark A containing the text "123", trademark B containing the text "Zhang San Food", trademark C with a white rabbit logo, trademark D with a white tiger logo, trademark E with a black rabbit logo, a pedestrian and a chair. Then, the lower left corner of the circumscribed rectangle of the panoramic image is regarded as the origin of the coordinate axis, and the central coordinates (x0, y0) of the center point of the panoramic image and the central coordinates (x1, y2) of all the initial recognition frames are obtained. * ,y * ).

[0164] Taking the initial recognition frame of brand C as an example, its center coordinates are (4, 6). The center coordinates (3, 3) of the center point of the panoramic image and the center coordinates (4, 6) of the initial recognition frame are substituted into the above formula 1 to calculate the rotation angle α=333° of the recognition frame. Then, the recognition frame is rotated 333° clockwise around the center point to obtain the target recognition frame corresponding to the initial recognition frame.

[0165] The initial recognition frame of the panoramic image is calculated to obtain the initial recognition frame rotation angle corresponding to each initial recognition frame, and each initial recognition frame is rotated according to the initial recognition frame rotation angle to obtain the target recognition frame corresponding to the detection object in the panoramic image. Then, the regional image in the target recognition frame is obtained and regarded as the initial detection object. The width w, height h and horizontal angle γ of the initial detection object are obtained, where the width w is the first edge of the detection object encountered by the horizontal line of the initial detection object rotating clockwise, h is another edge adjacent to the width w, and the video angle γ is the angle between the width w and the horizontal line. The line segment passing through the center of the panoramic image, with an angle of 45° to the horizontal and the end point at the edge of the panoramic image is taken as the first diameter, and the line segment perpendicular to the first diameter and with the end point at the edge of the panoramic image is taken as the second diameter. The panoramic image is divided into 4 areas according to the first diameter and the second diameter, where the first area is the area on the right side of the first diameter and the left side of the second diameter, and the next area 90° clockwise from the center point of the panoramic image is the second area. All 4 areas are determined by analogy, and the division results are as follows: Figure 3 As shown in the figure, 302 is the first area, 304 is the second area, 306 is the third area, and 308 is the fourth area. Then, the corresponding formula for calculating the rotation angle β is determined according to the area where each initial detection object is located, and then the initial detection object is rotated clockwise by β to obtain the target detection object after rotation.

[0166] Taking the initial detection object of trademark C as an example, its width w=2, height h=4, horizontal angle γ=63°, and the area it is located in is area 1, so it is substituted into formula 2 to obtain the rotation angle β=-27°, and then the initial detection object is rotated 27° counterclockwise to obtain the rotated target detection object.

[0167] Since the robot needs to be positioned, target detection objects of the trademark category are subsequently obtained in the panoramic image, which are Trademark A containing the text "123", Trademark B containing the text "Zhang San Food", Trademark C with a white rabbit logo, Trademark D with a white tiger logo, and Trademark E with a black rabbit logo. Subsequently, OCR image recognition is performed on each trademark object after the conversion to obtain its corresponding text features, which are the text feature "123" corresponding to Trademark A, the text feature "Zhang San Food" corresponding to Trademark B, the text feature "white rabbit" corresponding to Trademark C, the text feature "white tiger" corresponding to Trademark D, and the text feature "black rabbit" corresponding to Trademark E. Subsequently, the trademark spatial topological structure contained in the panoramic image is obtained according to the position of each trademark in the panoramic image. The spatial topological structure formed by the five trademarks in this embodiment is as shown in FIG. Figure 5 As shown, Figure 5A trademark space topology structure provided by an embodiment of the present application is shown, wherein 501 is the center point position of the panoramic image, that is, the position of the robot. The trademark space topology structure and the position of the robot acquired at each moment are saved in the map construction database to complete the map construction.

[0168] When the robot needs to obtain the real-time position, the acquired panoramic image is processed by the above-mentioned image processing method to obtain the trademark spatial topological structure at the current moment, and then the spatial topological structure at the current moment is compared with the spatial topological structure existing in the mapping database to obtain the current position of the robot.

[0169] By applying the scheme of the above-mentioned embodiment, by confirming at least one angle-free rectangular detection frame in the panoramic image acquired by the surround-view camera, it is possible to acquire information around the robot only through the surround-view camera, and then obtain a target detection frame that matches the target object in the panoramic image by rotating the angle-free rectangular detection frame, and then rotate the target object included in the target detection frame to obtain an upright target object, and then identify the target object and determine the position of the robot and the subsequent behavior of the robot based on the identification result, thereby realizing the positioning of the robot and the realization of the robot's autonomous behavior only through the surround-view camera, which reduces the implementation cost compared to realizing the positioning and autonomous behavior of the robot through multiple sensors.

[0170] Taking into account that processing the panoramic image acquired by the surround-view camera to obtain the target object will require a lot of computing resources, by processing the detection frame instead of the entire panoramic image, the computing resources used in robot positioning are saved, thereby reducing the implementation cost of the positioning method. At the same time, the target object after being erected is identified, which preliminarily improves the accuracy of target object identification. Subsequently, multiple target objects are identified and the robot is positioned according to the identification results, which further improves the accuracy of the positioning results, thereby realizing a high-precision and low-cost robot positioning method.

[0171] Embodiment 2

[0172] In the second embodiment provided in the present application, taking the navigation of a service robot in a shopping mall according to navigation instructions issued by a customer as an example, the robot positioning method provided in the present application is further explained.

[0173] A route is preset for the service robot so that it runs according to the preset route to enable the robot to cruise in the shopping mall. The acquired panoramic image is processed with detection objects as provided in the above embodiment every 0.5 seconds, and the real-time position of the robot is determined using detection objects of the trademark category. During operation, the robot avoids detection objects of categories that need to be avoided, such as chairs, billboards, users, walls, etc.

[0174] After receiving the voice command "Take me to Zhang San Food", the acquired panoramic image is processed by the detection object of the above embodiment, and then the detection object of the user category is processed, and the detection object of the closest target category is recognized by face recognition. When the user type detected by the user is a customer, the current user's face feature is recorded as customer A and stored in the navigation customer group. Then, the voice command "Take me to Zhang San Food" is parsed, and the next behavior strategy of the robot is determined to lead customer A to Zhang San Food, that is, to lead customer A to the location closest to "Zhang San Food" in the map database, such as Figure 5 As shown, 501 is the location of the robot. Figure 5 The corresponding position is the position closest to "Zhang San Food".

[0175] Go to the corresponding location according to the search results in the mapping database, and perform the detection object processing provided in the above embodiment on the acquired panoramic image every 0.5 seconds during the process of going, and use the detection object of the trademark category to determine the current position of the robot to ensure the correct path. If there is no customer A in the acquired panoramic image, stop moving and wait for 10 seconds. During the waiting process, perform the detection object processing provided in the above embodiment on the acquired panoramic image every 0.5 seconds. When customer A is detected in the panoramic image, continue the navigation task. If customer A is not detected within 10 seconds, delete customer A from the navigation customer group and continue the scheduled route.

[0176] The trademark space topology to be obtained is Figure 5 When shown, the navigation task is completed, and a voice prompt "Your navigation destination has been reached, I wish you a happy life!" is issued, and then the temporarily stored customer A is deleted from the navigation customer group, and the vehicle continues to travel along the preset route.

[0177] Considering that traditional robots need multiple sensors for navigation, and multiple sensors require complex algorithms, the use of the robot is improved. By applying the solution of the above embodiment, the panoramic image acquired by the surround-view camera is processed to complete the navigation of the user according to the user's instructions, thereby reducing the implementation cost of the robot.

[0178] Corresponding to the above method embodiment, the present application also provides a robot positioning device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a robot positioning device provided by an embodiment of the present application. Figure 6 As shown, the device comprises:

[0179] An acquisition module 602 is configured to acquire a panoramic image to be processed, wherein the panoramic image to be processed is acquired based on a surround view camera of the robot;

[0180] The initial determination module 604 is configured to determine at least one initial detection area in the panoramic image to be processed.

[0181] The target determination module 606 is configured to determine the target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed.

[0182] The identification module 608 is configured to identify content information corresponding to each target detection object.

[0183] The positioning module 610 is configured to determine the position information of the robot by matching each content information with preset map information, wherein the preset map information is created based on a panoramic image taken by a surround-view camera of the robot.

[0184] The target determination module 606 is further configured to:

[0185] Determine the detection area center point position information of each initial detection area based on the spatial information of each area;

[0186] Determine the center point position information of the panoramic image based on the image spatial information;

[0187] Determine the target detection area corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area;

[0188] The target detection object corresponding to each target detection area is determined in the panoramic image to be processed.

[0189] The target determination module 606 is further configured to:

[0190] Generating an initial detection area rotation angle corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area;

[0191] Each initial detection area is rotated based on the initial detection area rotation angle to determine each target detection area.

[0192] The target determination module 606 is further configured to:

[0193] Determine the first horizontal coordinate information and the first vertical coordinate information corresponding to the panoramic image center point position information, and determine the second horizontal coordinate information and the second vertical coordinate information corresponding to the target detection area center point position information, wherein the target detection center point position information is any one of the center point position information of each detection area;

[0194] Obtaining horizontal coordinate difference information according to the first horizontal coordinate information and the second horizontal coordinate information, and obtaining vertical coordinate difference information according to the first vertical coordinate information and the second vertical coordinate information;

[0195] A detection area rotation angle is generated based on the horizontal coordinate difference information and the vertical coordinate difference information.

[0196] The target determination module 606 is further configured to:

[0197] Determine an initial detection object of each target detection area in the panoramic image to be processed;

[0198] Dividing the panoramic image to be processed into at least two panoramic image areas;

[0199] A target rotation rule corresponding to a target panoramic image area is determined, and initial detection objects in the target panoramic image area are adjusted based on the target rotation rule to obtain target detection objects corresponding to each initial detection object.

[0200] The target determination module 606 is further configured to:

[0201] Obtaining object space information corresponding to each initial detection object;

[0202] The initially detected objects in the target panoramic image area are adjusted based on the target rotation rule and the spatial information of each object.

[0203] Optionally, the object space information includes height information, width information, and horizontal angle of the initially detected object;

[0204] Adjusting the initial detection objects in the target panoramic image area based on the target rotation rule and the spatial information of each object includes:

[0205] Determining shape information according to a relationship between height information and width information corresponding to an initial detection object, wherein the initial detection object is any one of the initial detection objects in the target panoramic image area;

[0206] Determine an initial positive rotation angle according to the target rotation rule and shape information, and add the object horizontal angle and the initial positive rotation angle to obtain a target positive rotation angle;

[0207] The initial detection object is rotated based on the target positive rotation angle.

[0208] Optionally, the initial detection area includes detection category information, the detection category information includes a positioning identifier category, and the content information includes positioning identifier information;

[0209] The positioning module 610 is further configured to:

[0210] In the case where the detection category information of the target initial detection area is a positioning identification category, the content information corresponding to the target initial detection area is positioning identification information, wherein the target initial detection area is any one of the initial detection areas;

[0211] Acquire the information position relationship between each positioning mark information in the image to be processed;

[0212] The position information of the robot is confirmed based on each positioning identification information and each information position relationship.

[0213] Optionally, the robot positioning method further includes:

[0214] A mapping module is configured to obtain at least one mapping panoramic image, wherein the mapping panoramic image is obtained based on a surround view camera of the robot;

[0215] Determine at least one target detection object in each mapping panoramic image, and identify positioning identification information corresponding to each target detection object;

[0216] Acquire the information position relationship between each positioning mark information according to each mapping panoramic image;

[0217] The preset map information is generated based on each positioning identification information and each information position relationship.

[0218] Optionally, the robot positioning method further includes:

[0219] An instruction generation module is configured to determine a behavior strategy of the robot according to the content information of each area, and generate a behavior instruction of the robot based on the behavior strategy;

[0220] The robot is adjusted based on the behavioral instructions.

[0221] The instruction generation module is further configured to:

[0222] The robot's behavior strategy is determined based on the detection category information and area content information of each initial detection area.

[0223] The instruction generation module is further configured to:

[0224] The robot's behavior strategy is determined based on the detection category information and area content information of each initial detection area, including:

[0225] In the case where the detected category information is a user category, the user type is determined according to the user feature information, and the behavior strategy of the robot is determined based on the user type.

[0226] By applying the scheme of the above-mentioned embodiment, by confirming at least one angle-free rectangular detection frame in the panoramic image acquired by the surround-view camera, it is possible to acquire information around the robot only through the surround-view camera, and then obtain a target detection frame that matches the target object in the panoramic image by rotating the angle-free rectangular detection frame, and then rotate the target object included in the target detection frame to obtain an upright target object, and then identify the target object and determine the position of the robot and the subsequent behavior of the robot based on the identification result, thereby realizing the positioning of the robot and the realization of the robot's autonomous behavior only through the surround-view camera, which reduces the implementation cost compared to realizing the positioning and autonomous behavior of the robot through multiple sensors.

[0227] Taking into account that processing the panoramic image acquired by the surround-view camera to obtain the target object will require a lot of computing resources, by processing the detection frame instead of the entire panoramic image, the computing resources used in robot positioning are saved, thereby reducing the implementation cost of the positioning method. At the same time, the target object after being erected is identified, which preliminarily improves the accuracy of target object identification. Subsequently, multiple target objects are identified and the robot is positioned according to the identification results, which further improves the accuracy of the positioning results, thereby realizing a high-precision and low-cost robot positioning method.

[0228] The above is a schematic scheme of a robot positioning device of this embodiment. It should be noted that the technical scheme of the robot positioning device and the technical scheme of the robot positioning method described above are of the same concept, and the details not described in detail in the technical scheme of the robot positioning device can be referred to the description of the technical scheme of the robot positioning method described above.

[0229] Figure 7 The structure block diagram of a computing device 700 provided according to an embodiment of the present application is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0230] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0231] In one embodiment of the present application, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0232] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 may also be a mobile or stationary server.

[0233] Among them, when the processor 720 executes the computer instructions, the steps of the robot positioning method are implemented.

[0234] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the robot positioning method described above are of the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the robot positioning method described above.

[0235] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the robot positioning method as described above.

[0236] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the robot positioning method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the robot positioning method described above.

[0237] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0238] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0239] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0240] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0241] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A robot positioning method, characterized in that: include: Acquire a panoramic image to be processed, wherein the panoramic image to be processed is acquired based on a surround-view camera of the robot; Determining at least one initial detection area in the panoramic image to be processed; Determining the target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed; Identify content information corresponding to each target detection object; The position information of the robot is determined by matching each content information with preset map information, wherein the preset map information is created based on a panoramic image taken by a surround-view camera of the robot.

2. The method according to claim 1, characterized in that The initial detection area includes detection category information, the detection category information includes a positioning identifier category, and the content information includes positioning identifier information; According to the matching of each content information and the preset map information, the position information of the robot is determined, including: In the case where the detection category information of the target initial detection area is a positioning identification category, the content information corresponding to the target initial detection area is positioning identification information, wherein the target initial detection area is any one of the initial detection areas; Acquire the information position relationship between each positioning mark information in the image to be processed; The position information of the robot is confirmed based on each positioning identification information and each information position relationship.

3. The method according to claim 1, characterized in that Determining the target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed, including: Determine the detection area center point position information of each initial detection area based on the spatial information of each area; Determine the center point position information of the panoramic image based on the image spatial information; Determine the target detection area corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area; The target detection object corresponding to each target detection area is determined in the panoramic image to be processed.

4. The method according to claim 3, characterized in that Determining target detection areas corresponding to the initial detection areas according to the panoramic image center position information and the center position information of each detection area includes: Generating an initial detection area rotation angle corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area; Each initial detection area is rotated based on the initial detection area rotation angle to determine each target detection area.

5. The method according to claim 4, characterized in that Generating an initial detection area rotation angle corresponding to each initial detection area according to the panoramic image center point position information and the center point position information of each detection area, including: Determine the first horizontal coordinate information and the first vertical coordinate information corresponding to the panoramic image center point position information, and determine the second horizontal coordinate information and the second vertical coordinate information corresponding to the target detection area center point position information, wherein the target detection center point position information is any one of the center point position information of each detection area; Obtaining horizontal coordinate difference information according to the first horizontal coordinate information and the second horizontal coordinate information, and obtaining vertical coordinate difference information according to the first vertical coordinate information and the second vertical coordinate information; A detection area rotation angle is generated based on the horizontal coordinate difference information and the vertical coordinate difference information.

6. The method according to claim 3, characterized in that Determining the target detection object corresponding to each target detection area in the panoramic image to be processed includes: Determine an initial detection object of each target detection area in the panoramic image to be processed; Dividing the panoramic image to be processed into at least two panoramic image areas; A target rotation rule corresponding to a target panoramic image area is determined, and initial detection objects in the target panoramic image area are adjusted based on the target rotation rule to obtain target detection objects corresponding to each initial detection object.

7. The method according to claim 6, characterized in that Adjusting the initial detection object in the target panoramic image area based on the target rotation rule includes: Obtaining object space information corresponding to each initial detection object; The initially detected objects in the target panoramic image area are adjusted based on the target rotation rule and the spatial information of each object.

8. The method according to claim 7, characterized in that The object space information includes height information, width information, and horizontal angle of the initially detected object; Adjusting the initial detection objects in the target panoramic image area based on the target rotation rule and the spatial information of each object includes: Determining shape information according to a relationship between height information and width information corresponding to an initial detection object, wherein the initial detection object is any one of the initial detection objects in the target panoramic image area; Determine an initial positive rotation angle according to the target rotation rule and shape information, and add the object horizontal angle and the initial positive rotation angle to obtain a target positive rotation angle; The initial detection object is rotated based on the target positive rotation angle.

9. The method according to claim 1, characterized in that The preset map information is obtained by the following steps: Acquire at least one panoramic image for mapping, wherein the panoramic image for mapping is acquired based on a surround-view camera of the robot; Determine at least one target detection object in each mapping panoramic image, and identify positioning identification information corresponding to each target detection object; Acquire the information position relationship between each positioning mark information according to each mapping panoramic image; The preset map information is generated based on each positioning identification information and each information position relationship.

10. The method according to claim 2, characterized in that After identifying the content information corresponding to each target detection object, it also includes: Determining a behavior strategy of the robot according to the content information of each area, and generating a behavior instruction of the robot based on the behavior strategy; The robot is adjusted based on the behavioral instructions.

11. The method according to claim 10, characterized in that Determining the behavior strategy of the robot according to the content information of each area includes: The robot's behavior strategy is determined based on the detection category information and area content information of each initial detection area.

12. The method according to claim 11, characterized in that The detection category information includes user category, and the area content information includes user feature information; The robot's behavior strategy is determined based on the detection category information and area content information of each initial detection area, including: In the case where the detected category information is a user category, the user type is determined according to the user feature information, and the behavior strategy of the robot is determined based on the user type.

13. A robot positioning device, characterized in that: include: An acquisition module is configured to acquire a panoramic image to be processed, wherein the panoramic image to be processed is acquired based on a surround-view camera of the robot; an initial determination module, configured to determine at least one initial detection area in the panoramic image to be processed; A target determination module is configured to determine a target detection object corresponding to each initial detection area according to the regional spatial information of each initial detection area and the image spatial information of the panoramic image to be processed; An identification module is configured to identify content information corresponding to each target detection object; The positioning module is configured to determine the position information of the robot by matching each content information with preset map information, wherein the preset map information is created based on a panoramic image taken by a surround-view camera of the robot.

14. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer instructions, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 12 are implemented.

Images