Robot mapping method, device, equipment and medium

Abstract

The application is suitable for the technical field of map construction, and provides a robot mapping method, device, equipment and medium. The method comprises the following steps: in response to a mapping instruction, a robot sensor scans a target environment to obtain a scanned map; at least a part of the scanned map is rendered and displayed in real time according to a map rendering parameter in a front-end map space, the map space is an adjustable space, and the map space is determined according to the map rendering parameter and the scanned map; and in response to a completion instruction, a robot map is generated according to the scanned map after editing. The application can realize rendering only for necessary parts (i.e. entity parts of the scanned map) by using a map space with a dynamic size, thereby saving unnecessary calculation resource consumption; meanwhile, the map space with an adjustable size can adapt to robot map construction tasks of various scene sizes, and has better applicability.

Description

Technical Field

[0001] The present application belongs to the field of map construction technology, and in particular relates to robot mapping methods, devices, equipment and media. Background Art

[0002] Before robots can operate in their application scenarios, they must first map the scene. Using devices like lidar and cameras, they scan the scene and construct a scene map. Only after this map is constructed can the robot perform tasks based on the scene map. While real-time map construction and navigation technologies are becoming increasingly mature, the large environmental areas of large-scale scenarios still pose significant challenges to mapping.

[0003] In order to adapt to robot mapping in large-scale environments, existing technologies provide some robot mapping solutions based on mapping path optimization, which can overcome the difficulty of large-scale map construction to a certain extent, but there is still the inevitable problem of poor accuracy.

[0004] Therefore, how to provide a robot mapping method, device, equipment and medium with higher mapping accuracy has become a technical problem that urgently needs to be solved in the industry. Summary of the Invention

[0005] The embodiments of the present application provide a robot mapping method, apparatus, device, and medium, which can solve the problems of high resource consumption and poor scene size compatibility during robot mapping.

[0006] In a first aspect, an embodiment of the present application provides a robot mapping method, comprising:

[0007] In response to the mapping instruction, the robot sensor scans the target environment to obtain a scanned map;

[0008] Rendering and displaying at least a portion of the scanned map in real time in a map space of a front end according to map rendering parameters, wherein the map space is an adjustable space and is determined according to the map rendering parameters and the scanned map;

[0009] In response to the completion instruction, a robot map is generated according to the edited scan map.

[0010] The beneficial effects of the above method are:

[0011] By rendering the scanned map obtained by the robot's sensors into the map space, a real-time visualization output image of the map construction progress is provided, which provides a good foundation for secondary corrections in the robot's map construction process based on staff or processing correction modules (for example, computers with stronger computing power than the robot).

[0012] Furthermore, as the robot scans, the size of the scanned map continues to increase, and the map space also increases accordingly. Therefore, the above method can use the map space of dynamic size to render only the necessary parts (i.e., the physical part of the scanned map), thereby saving unnecessary computing resources (on the robot side); at the same time, the map space with adjustable size can adapt to robot map construction tasks of various scene sizes and has better applicability.

[0013] In a possible implementation of the first aspect, before rendering the scanned map in real time according to the map rendering parameters in the front-end map space, the method includes:

[0014] Obtaining map rendering parameters, and obtaining the size of the complete rendered map according to the map rendering parameters and the size of the scanned map;

[0015] If it is determined that the size of the complete rendered map exceeds the maximum boundary of the map space, the scanned map is divided into blocks to obtain block maps;

[0016] Accordingly, the map space on the front end renders and displays the scanned map in real time according to the map rendering parameters, including:

[0017] Determining a central block map to be rendered according to the central coordinates of a visible area of ​​the map space, wherein the visible area is an area in the map space displayed on the robot screen;

[0018] The central block map and its adjacent block maps are rendered according to the map rendering parameters, and the central block map is displayed in the visible area.

[0019] The above method achieves local rendering of maps with extremely large scene sizes by setting an upper limit on the size of the map space, that is, the maximum boundary. This converts the rendering of extremely large maps based on high computing resource consumption into low-resource consumption rendering for multiple block maps, ensuring the upper limit of the robot's computing resource usage at any time. No matter how large the scene size is, the robot can build maps smoothly and without lag.

[0020] In a possible implementation of the first aspect, the step of cutting the scanned map into blocks to obtain the block maps includes:

[0021] Get the size of the visible area;

[0022] The scanned map is divided into blocks and cut into a plurality of block maps having the same size as the visible area.

[0023] The method associates the size of the visible area with the size of the block map, so that the picture obtained by local rendering can better meet the display requirements of the user when dragging the robot screen, that is, only the center block map and its adjacent block maps in the visible area need to be rendered to meet the display requirements of the user when dragging the robot screen, and the rendering and display efficiency balance is better.

[0024] In a possible implementation of the first aspect, the rendering of the center block map and the block maps adjacent to the center block map according to the map rendering parameters, and the display of the center block map in the visible area, comprises:

[0025] The rendered block map is saved in a two-dimensional array with block map labels.

[0026] The method saves the real-time rendered block map, after the robot completes the map scanning, directly exports the rendered block map and splices it to obtain a complete map, so that the real-time rendering of the block map can be used for display on one hand and output of the complete map on the other hand, which is equivalent to real-time block rendering of the complete map, and realizes real-time rendering and display according to the scanning progress of the robot without increasing the total consumption of computing resources.

[0027] In a possible implementation of the first aspect, the real-time rendering and display of at least part of the scanned map in the map space according to the map rendering parameters, comprises:

[0028] The current coordinates of the robot are located in a preset following area of the map space, and at least part of the scanned map is displayed on the map space.

[0029] The at least part of the scanned map is rendered to obtain a map.

[0030] A preset grid scale; and

[0031] An actual area of the scanned scene.

[0032] The method displays the current position of the robot in the preset area of the map space and adjusts following, so that the user can easily locate the current position of the robot, and also facilitates the calculation of the offset amount of the map space, to further realize the size adjustment of the map space, and has better mapping efficiency.

[0033] In a possible implementation of the first aspect, before the robot sensor scans the target environment to obtain a scanned map in response to a mapping instruction, the method comprises:

[0034] Obtaining the category of the target environment, and determining a feature target of the target environment according to the category.

[0035] Correspondingly, the robot sensor scans the target environment to obtain a scan map in response to the mapping instruction, including:

[0036] In response to the mapping instruction, the robot sensor scans a feature region of the target environment at a first frequency and scans a non-feature region of the target environment at a second frequency to obtain a scan map; the feature region is a preset region including the feature target; and the non-feature region is a preset region not including the feature target.

[0037] The above method introduces a feature target and a feature region, so that the robot performs more detailed scanning on the region of interest during map construction to obtain a more accurate map, thereby facilitating the work process after the mapping is completed.

[0038] In a possible implementation of the first aspect, after the robot sensor scans the target environment to obtain a scan map in response to the mapping instruction, the method further includes:

[0039] Performing low-pixelization processing on the scan map to obtain a plurality of thumbnails with different numbers of pixels; the number of pixels of the thumbnail is less than that of the scan map.

[0040] In response to a zoom display signal, selecting a zoom display image from the thumbnails according to a zoom ratio of the zoom display signal and displaying the zoom display image.

[0041] The above method introduces a plurality of low-pixel thumbnails, so that the user can display thumbnails with different numbers of pixels according to the zoom ratio when viewing the global scan state, thereby reducing unnecessary interference of the performance overhead of global image rendering on the robot mapping work and improving the efficiency of robot mapping.

[0042] In a second aspect, the embodiments of the present application provide a robot mapping device, including:

[0043] A scan map module, configured to scan a target environment by a robot sensor to obtain a scan map in response to a mapping instruction.

[0044] A rendering display module, configured to render and display at least part of the scan map in a map space of a front end in real time according to a map rendering parameter, the map space being an adjustable space and being determined according to the map rendering parameter and the scan map.

[0045] A map generation module, configured to generate a robot map according to the edited scan map in response to a completion instruction.

[0046] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot mapping method described in any one of the first aspects above is implemented.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the robot mapping method described in any one of the first aspects above is implemented.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the robot mapping method described in any one of the first aspects above.

[0049] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0051] Figure 1 Schematic diagram of the process of the robot mapping method provided in the embodiment of the present application;

[0052] Figure 2 Schematic diagram of the structure of the robot mapping device provided in an embodiment of the present application;

[0053] Figure 3 This is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0055] Figure 5 This is a schematic diagram of a first display structure of a robot screen provided in an embodiment of the present application;

[0056] Figure 6 This is a schematic diagram of the structure of the block map provided in the embodiment of the present application;

[0057] Figure 7 2 is a schematic diagram of a second display structure of a robot screen provided in an embodiment of the present application;

[0058] Figure 8 This is a schematic diagram of the third display structure of the robot screen provided in an embodiment of the present application.

[0059] Reference numerals:

[0060] Scanning map module 201;

[0061] Rendering and display module 202;

[0062] Map generation module 203;

[0063] Terminal device 30;

[0064] Processor 301;

[0065] Memory 302;

[0066] Computer Program 303;

[0067] Robot screen 400;

[0068] Map space 410;

[0069] Interaction button 401;

[0070] Prompt message 402;

[0071] Robot current position 403;

[0072] Map information 404;

[0073] Feature area 405. DETAILED DESCRIPTION

[0074] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0075] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0076] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0077] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0078] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0079] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0080] In some embodiments of this application, a mobile service robot will be used as an example to illustrate the solution in detail. It is worth noting that the mobile service robot is used as an example. Mobile service robots have diverse application scenarios, such as hotels, restaurants, airports, hospitals, exhibition halls, etc. The area between different scenarios varies greatly, and the terrain of most application scenarios is irregular. Therefore, the example based on the mobile service robot can better illustrate these complex situations and does not constitute a limitation on the scope of protection of this application. The various embodiments of this application can also be applied to other types of robot mapping methods.

[0081] The following will take a mobile service robot as an example to illustrate the inventor's inventive concept.

[0082] Mobile service robots are a typical type of robotic product that can be applied in many industries, such as catering services, hotel services, airport services, etc. In specific applications, service robots, similar to autonomous driving, need to have the ability to stably locate and navigate in complex indoor environments. On this basis, the accuracy of the map used by the robot will greatly improve its service quality and operational reliability.

[0083] Therefore, a better robot mapping method needs to be provided.

[0084] There are currently some robot mapping methods based on algorithm improvements. However, these methods still cannot overcome the mapping difficulties caused by error accumulation (for example, error accumulation of sensors such as gyroscopes, odometers, and positioning and navigation units).

[0085] Furthermore, to address the error accumulation problem, an alternative approach is to provide a solution for building robot maps based on small-scale closed-loop paths. However, this solution requires repeated use of small-scale closed-loop paths to overcome the error accumulation problem, resulting in a time-consuming and performance-intensive mapping process.

[0086] In other words, the inventors noticed that before artificial intelligence has achieved a qualitative breakthrough, the above-mentioned algorithm-based improved robot mapping solution is still unable to more efficiently build a high-precision robot map while balancing computing resource consumption (i.e., performance overhead).

[0087] Therefore, in order to improve the accuracy and efficiency of map construction, a better solution is to introduce a solution in which the robot body scans once and technicians or related algorithms make secondary corrections. This secondary correction solution requires the scanned map obtained by the robot's primary scan to be output and rendered for secondary correction.

[0088] In order to achieve the above-mentioned better solution, a more direct idea is to build a canvas (similar to the map space of each embodiment of the present application), and render the scanned map obtained by the robot in real time onto the canvas for output so as to facilitate subsequent secondary correction operations. In order to adapt to larger scenes, the size of the canvas should be set to correspond to a larger actual space, such as 200 meters by 200 meters, which can basically cover most situations. The map of a single floor of most hotels, restaurants and shopping malls is Figure 1 Generally, it will not exceed this range. In other words, under this approach, the scanned map will be added with a background and rendered to a fixed size. If the map pixel ratio is 0.05, the actual image to be rendered is 4000*4000 pixels.

[0089] However, the above ideas still have some problems, specifically:

[0090] 1. Due to the fixed canvas size, most scenes in actual applications do not reach 200m*200m. However, in these scenes, a 4000*4000 pixel map background still needs to be rendered. This results in a lot of meaningless content being rendered during the map construction process, which consumes a lot of performance.

[0091] 2. Incompatible with the rendering of some extremely large scenes and special terrains. In actual needs, some customers encounter shopping malls, airports, or hotels where the length or width of a single layer map exceeds 200 meters. The solution based on the above ideas is still not compatible with such map rendering.

[0092] Therefore, the inventors have made further improvements to the above problems and have developed the present application. In summary, one of the purposes of the embodiments of the present application is to:

[0093] In actual applications, robot deployment scenarios such as restaurants, hotels, and shopping malls generally have different terrains and areas. To improve the compatibility of mapping for different terrains and cope with the rendering of some very large maps, a solution that supports dynamic map rendering is provided. The various embodiments of this application mainly focus on improving functions such as rendering dynamically sized maps, slicing rendering of very large maps, and coordinate conversion of dynamically sized maps.

[0094] Figure 4 An optional application scenario of each embodiment of the present application is shown. Figure 4 The map shown in is the scene map that the catering service robot needs to construct. In an optional embodiment, as Figure 4 As shown, multiple robots ( Figure 4 Two robots are shown in the figure) to collaboratively execute or separately execute the methods of each embodiment of the present application to further improve the efficiency of map construction.

[0095] Figure 1 A robot mapping method provided in an embodiment of the present application is shown, including:

[0096] Step 102: In response to the mapping instruction, the robot sensor scans the target environment to obtain a scanned map;

[0097] Step 104: Rendering and displaying at least a portion of the scanned map in real time in a map space 410 of the front end according to map rendering parameters, wherein the map space 410 is an adjustable space and is determined according to the map rendering parameters and the scanned map;

[0098] Step 106 : In response to the completion instruction, a robot map is generated according to the edited scan map.

[0099] In this embodiment, the mapping instruction can be triggered by manually clicking a GUI interactive button; the robot sensor can be any one or a combination of multiple sensors including a laser radar, an infrared sensor, and an ultrasonic sensor.

[0100] As an example and not a limitation, the following describes the installation and scanning method of the sensor in an embodiment using a lidar.

[0101] The laser radar system can be set in a robotic device; the laser radar system can rotate along a set plane so that the photoelectric receiving array of the laser radar system forms a scanning cylinder.

[0102] The laser radar system can be set at the slit of the robot shell, so that it is easy to emit laser signals to detect surrounding objects. The above-mentioned laser radar system includes a photoelectric receiving array and a laser emitting unit array. When the laser radar system rotates along the set plane, the photoelectric receiving array can form a scanning cylinder, thereby increasing the scanning area, making it easier to obtain the details of the object shape and avoid the robot equipment from colliding with obstacles. If the laser radar system only includes a single photoelectric receiving unit and a single laser emitting unit, the laser radar system can only measure the shape of an object in one circle after rotating along the set plane, and cannot obtain the shape of complex objects in time, which is prone to collision and endangering personal and property safety. Optionally, the above-mentioned set plane can be a horizontal plane to facilitate the robot equipment to detect objects during movement. In addition, other set planes, such as vertical planes, can be selected according to user needs. This is not limited in this embodiment.

[0103] Furthermore, in an optional embodiment, step 102 includes:

[0104] The robot is equipped with acquisition sensors and a modeling processor. The modeling processor uses the environmental data collected by the acquisition sensors to build a model and construct an environmental map. In this embodiment, the acquisition sensors include lidar, ultrasonic sensors, and infrared sensors. The lidar, ultrasonic sensors, and infrared sensors collect data about the robot's working area. The modeling processor uses the data collected by these sensors to create a map. During the map creation process, different map layers are generated using different sensors, such as a static layer, a dynamic obstacle layer, an ultrasonic layer, and a visual layer. These layers are then fused to produce a positioning map for the robot's positioning and navigation.

[0105] Illustratively, the map rendering parameters in step 104 of this embodiment may include pixel ratio, sampling rate, noise filtering parameters, etc., that is, the map rendering parameters are configuration parameters of the rendering process based on scanning the map to obtain the rendered image.

[0106] It is worth noting that the rendering action and the display action in step 104 are not necessarily completely corresponding. In some optional implementations, a larger-sized image may be obtained through the rendering action, and a part of the larger-sized image may be displayed through the display action.

[0107] In addition, the concept of the map space 410 in step 104 is similar to the canvas described above, but in this embodiment, the map space 410 is a space with adjustable size, and during the rendering process, only the scanned map part is rendered, and there is no need to render too much background (for example, the gray canvas background beyond the outline of the scanned map is not rendered).

[0108] In step 106, in response to a completion instruction, such as a user clicking a completion button on a GUI interaction page, or when the robot determines that mapping of the current scene is completed, a robot map can be generated based on the edited scan map, such as adjusting the boundaries of the scan map, manually adding target points, setting speed zones, etc., for use by the robot in navigation during operation.

[0109] In an optional implementation, the map is scanned with the robot's initial position as the origin of the coordinate system. By moving the robot's position, sensors such as lidar scan the terrain to generate a new map, which is then pushed to the front end for rendering. The pushed map no longer needs to be supplemented with a gray background to form a 4000*4000 (pixel) image. Instead, the map actually scanned by the current robot can be transmitted. Map parameters (for example, the coordinates of the lower left corner of the current scanned map, the pixel ratio of the scanned map image, etc.) can also be added during map transmission. In other words, the pixel ratio can be determined by ROS and transmitted to the front end.

[0110] The map's pixel ratio is the actual length corresponding to one pixel. For example, if the pixel ratio is 0.1, one pixel corresponds to 10 cm in real life. Optionally, after receiving the map, the frontend instantiates a base64 Image object with the map image as its source (image file path) to obtain the image's width and height. Using the pixel ratio, the actual length, width, and area of ​​the corresponding site can be calculated. Mapping can then be performed based on the pixel ratio, rendering the scanned map image pixel by pixel.

[0111] The coordinates of the lower left corner of the image can be used to determine the coordinate system to complete the mutual conversion between the front-end pixel coordinates and the actual coordinates of ROS, and perform point rendering and interaction.

[0112] Optionally, during the map scanning process, the robot transmits the latest map and map parameters to the front end at a frequency of 2 times per second, and continuously builds a complete map as the robot moves in the scene.

[0113] Figure 5 The robot screen 400 is shown as the display content of the robot screen in the above embodiment; the robot screen 400 is provided with interactive buttons 401 and prompt information 402 to meet the interactive needs and information prompt needs in actual applications. The robot screen 400 also displays a map space 410. Figure 5In FIG. 4 , the map space 410 shows a portion of the map rendered and displayed during the robot scanning process. The parameters of this portion of the map can be displayed through the map information 404 , and the current position of the robot 403 is marked on the map.

[0114] It is worth noting that the size of the map space 410 displayed on the robot screen 400 may be fixed in some cases (for example, the size of the map space 410 has reached the maximum size allowed by the screen 400 to display the map space 410 at the current scale). In this case, it can be understood that a part of the map space 410 (visible area) is displayed on the robot screen 400, and the rest is not displayed through the robot screen 400. The actual size of the map space 410 is still adjusted according to the map rendering parameters and the scanned map.

[0115] In addition, for the portion of the map space 410 displayed on the robot screen 400, by adjusting the scale, the size of the scene actually displayed can be changed to meet the secondary correction requirements of the staff or related algorithms.

[0116] Furthermore, the shape of map space 410 can be either a rectangular space with adjustable size or a space with irregular boundaries that changes with the shape of the scanned map. In the latter embodiment, map space 410 can be understood as a space that is consistent with the shape of the scanned map and has a size corresponding to a preset scale (pixel ratio).

[0117] The beneficial effects of this embodiment are:

[0118] By rendering the scanned map obtained by the robot sensor into the map space 410, a real-time visualization output image of the map construction progress is provided, which provides a good basis for secondary corrections in the robot map construction process based on staff or processing correction modules (for example, a computer with stronger computing power than the robot).

[0119] Furthermore, the size of the map space 410 matches the map rendering parameters and the scanned map. As the robot scans, the size of the scanned map continues to increase, and the map space 410 also increases accordingly. Therefore, the solution of this embodiment can use the dynamic size of the map space 410 to render only the necessary parts (i.e., the physical part of the scanned map) without rendering the background canvas of a fixed size, thereby saving unnecessary rendering resource consumption; at the same time, the size-adjustable map space 410 can adapt to robot map construction tasks of various scene sizes and has better applicability.

[0120] According to the above embodiment, another embodiment of the present application provides a solution for constructing a robot map with an extremely large scene size.

[0121] For robot map construction with extremely large scene sizes, such as a 10,000*10,000 pixel scene determined based on pixel ratio, especially in complex environments such as restaurants, more details need to be rendered. Directly rendering the entire map (for example, rendering to a canvas) requires a very large memory usage, which can cause the entire system to lag.

[0122] To solve this problem, this embodiment uses a local rendering solution for rendering such a large map by setting an upper limit on the size of the map space (i.e., the maximum boundary).

[0123] Local rendering means rendering only the part the user is currently viewing and its surrounding area, and dividing the map into units based on the maximum size of the current map space display part. For example, if the map space display part is 1000*500 in size, the 10000*10000 original map image is divided into 200 small blocks based on this unit. These 200 small blocks are stored in separate canvas instances and saved in an array.

[0124] The content that the user can currently see through the robot screen 400 is only one tile map (the size is equal to the maximum size of the map space display part), and the distance the user can drag is also limited to the size of the tile map. Therefore, as long as the currently displayed tile map and the 8 tile maps surrounding the currently displayed tile map are rendered, it can be ensured that the user can see the map normally (even if dragging and rotation occur, the map content within the visible range can be displayed normally).

[0125] According to the user's operation, when the map is dragged and translated, and the dragging and translation distance exceeds half the width or height of the tile map, it is necessary to calculate the new 9 tile maps to be displayed, retain the tile maps that still need to be displayed in the new 9 tile maps to avoid repeated rendering, and perform new rendering for the newly added tile maps.

[0126] Specifically, this embodiment includes:

[0127] Before the front-end map space 410 renders the scanned map in real time according to the map rendering parameters, the process includes:

[0128] Obtaining map rendering parameters, and obtaining the size of the complete rendered map according to the map rendering parameters and the size of the scanned map;

[0129] If it is determined that the size of the complete rendered map exceeds the maximum boundary of the map space 410 , the scanned map is divided into blocks to obtain block maps;

[0130] Accordingly, the map space 410 on the front end renders and displays the scanned map in real time according to the map rendering parameters, including:

[0131] Determine the central block map to be rendered according to the central coordinates of the visible area of ​​the map space 410, where the visible area is the area of ​​the map space 410 displayed on the robot screen;

[0132] The central block map and its adjacent block maps are rendered according to the map rendering parameters, and the central block map is displayed in the visible area.

[0133] In a preferred embodiment, the size of the viewable area is consistent with the maximum size of the map space 410;

[0134] As an example and not a limitation, if it is determined that the size of the complete rendered map exceeds the maximum boundary of the map space 410, the step of cutting the scanned map into blocks to obtain block maps can be understood as a state switching step, that is, for the entire process of robot mapping, before the condition of "the size of the complete rendered map exceeds the maximum boundary of the map space 410" is met, the rendering is still based on the complete map rendering performed in the map space 410. As the scanned map increases, after the condition of "the size of the complete rendered map exceeds the maximum boundary of the map space 410" is met, local rendering is performed based on the central block map and its critical block maps.

[0135] In addition, in the above example, each rendered tile map is saved with a preset number (for example, for a rectangular tile map, the number may be a row and column number) so that it can be read when called in response to a user operation without having to render repeatedly.

[0136] Figure 6 Shown is a local rendering structure based on a rectangular block map. In an optional embodiment, as Figure 6 As shown, the central oblique line filled block represents the visible area of ​​the robot screen 400, the cross line filled block is the part that needs to be actually rendered, and the excess part (ie, the block without filling) is not rendered to optimize performance.

[0137] The beneficial effects of this embodiment are:

[0138] By setting the upper limit of the size of the map space 410, that is, the maximum boundary, local rendering of ultra-large scene size maps is achieved, so that the ultra-large map rendering based on high computing resource consumption is converted into low-resource consumption rendering for multiple block maps, ensuring the upper limit of the robot's computing resource usage at any time. No matter how large the scene size is, the robot can build maps smoothly without lag.

[0139] Furthermore, according to the above embodiment, in yet another embodiment:

[0140] The step of cutting the scanned map into blocks to obtain a block map includes:

[0141] Get the size of the visible area;

[0142] The scanned map is divided into blocks and cut into a plurality of block maps having the same size as the visible area.

[0143] Still Figure 6 The structure shown is used to illustrate this embodiment. When the tile map is a rectangular map, the tile cutting step is to divide the image into visible areas. That is, when the pushed map (i.e., the robot's scanned map) is received, it is first determined whether the length or width of the map exceeds the range of the diagonal fill block. If so, the map is divided.

[0144] In another optional embodiment, a set of oblique line filling blocks and intersection line filling blocks may be used as the determination range for segmentation, that is, the size of a single visible area and a set of adjacent areas having the same size as the visible area are consistent with the maximum size of the map space 410.

[0145] In this embodiment, before the condition that "the size of the complete rendered map exceeds the maximum boundary of the map space 410" is met, rendering is based on the complete map rendering performed in the map space 410, but the complete map may be partially displayed. After the condition that "the size of the complete rendered map exceeds the maximum boundary of the map space 410" is met, local rendering is performed based on the central block map and its critical block maps.

[0146] The beneficial effects of this embodiment are:

[0147] By associating the size of the visible area with the size of the block map, the image obtained by local rendering can better meet the display requirements of the user when dragging the robot screen 400. That is, only rendering the central block map and its adjacent block maps in the visible area can meet the display requirements of the user when dragging the robot screen 400, which has a better balance between rendering and display efficiency.

[0148] According to any of the above embodiments, in yet another embodiment:

[0149] The step of rendering the central block map and its adjacent block maps according to the map rendering parameters and displaying the central block map in the visible area includes:

[0150] The rendered tile map is saved in a two-dimensional array with a tile map tag.

[0151] In an optional implementation, after the received large map base64 is loaded, the image is divided into pieces using the canvas's drawImage method and saved in a two-dimensional array, with the subscript being the row and column number of the segmented piece. Furthermore, after completing the map scan, it is necessary to splice all the cut pieces (i.e., the segmented map) into a canvas when exporting the base map to export the complete map. This ensures that no matter how large the map is, it has the same performance consumption, ensuring that the machine can run smoothly, and that there is no difference in perception for the user, resulting in a better user experience.

[0152] The beneficial effects of this embodiment are:

[0153] By saving the real-time rendered tiled maps, after the robot completes the map scan, the rendered tiled maps can be directly exported and spliced ​​together to obtain the complete map. This allows the real-time rendering of the tiled maps to be used for display on the one hand, and to output the complete map on the other hand. This is equivalent to rendering the complete map in real time in blocks. Without increasing the overall consumption of computing resources, real-time rendering and display based on the robot's scanning progress are achieved.

[0154] According to any of the above embodiments, in yet another embodiment:

[0155] The map space 410 on the front end renders and displays at least a portion of the scanned map in real time according to map rendering parameters, including:

[0156] With the current coordinates of the robot being located in the preset following area of ​​the map space 410 as a constraint, the following is displayed on the map space 410:

[0157] at least a portion of the map obtained by rendering the scanned map;

[0158] Preset grid scales; and,

[0159] The actual area of ​​the scanned scene.

[0160] In a preferred implementation of this embodiment, the preset following area can be the center of the robot screen 400 or the map space 410. This setting of the map display following the robot ensures that the robot is always centered during scanning. After obtaining the robot's coordinates, the offset in the map space when the robot is centered is calculated. This effectively improves mapping efficiency, ensuring that the user can see changes in the scene currently being scanned by the robot in real time and determine whether the robot's positioning is offset through the laser point cloud.

[0161] It is worth noting that the scanned map needs to draw the target points, the robot and the laser point cloud. In the implementation of the dynamic map, it is necessary to convert the pixel coordinates and the actual coordinates according to the origin and pixel ratio of the map.

[0162] As an example and not a limitation, since the size of a dynamic map image changes in real time during scanning, and the map can expand in any direction, the position of the origin in the image is uncertain. The underlying ROS layer needs to transmit the coordinates corresponding to the lower-left corner of the map each time the image size changes. Upon receiving the image and the coordinates of the lower-left corner, the origin's position can be determined. The pixel ratio can then be used to convert ROS coordinates to front-end pixel coordinates, determining the exact position of the target point, robot, and laser point cloud.

[0163] In another optional implementation based on a rectangular block map, when the user drags the robot screen 400 or the robot moves away from the preset following area, the position of the center point of the current visible area is obtained based on the offset obtained according to the user's dragging or the robot's movement, and the numbers of the 9 block maps currently required to be rendered are calculated in real time, and the map is rendered dynamically.

[0164] In this embodiment, the preset grid scale may be grid lines whose intervals between adjacent lines are determined according to the scale, that is, the grid line intervals may be different depending on the scale, but regardless of the scale, adjacent grid lines correspond to the same real space distance.

[0165] The reason for setting a grid scale is that when viewing a large scene, users generally only observe a small part, and it is sometimes difficult to grasp the size of the scene. By adding a grid as a scale to the map, users can have a more accurate judgment of the scanned scene. For example, the grid can be 100*100 pixels, corresponding to 5m*5m.

[0166] Figure 7 and Figure 8 The exemplary pictures of the grid scales in different robot scanning stages are shown respectively. It can be seen that Figure 7 The grid lines are spaced farther apart, and Figure 8 The grid line spacing is small. Since the adjacent grid lines have the same real space distance, it is easy to know that Figure 8 The map shown in covers a larger area of ​​real space.

[0167] In addition, the actual area of ​​the currently scanned scene can be displayed in real time in the upper right corner of the map, i.e., at map information 404. Due to the diversity of robot application scenarios and large differences in area, the actual area is displayed in real time in the upper right corner of the map to facilitate the user to understand the scanning progress and whether the scan is complete. Optionally, based on the size of the area to be scanned for the current scene input by the user, it can be compared with the size of the currently scanned map, and then the mapping percentage can be displayed in real time. More specifically, the user can also enter the maximum length and width of the current scene to facilitate the automatic determination of the boundary mapping progress and whether the scan is complete based on the scan map.

[0168] The beneficial effects of this embodiment are:

[0169] By displaying the robot's current position in a preset area of ​​the map space and adjusting it accordingly, users can more easily locate the robot's current position. This also facilitates the calculation of the offset in the map space, further enabling the resizing of the map space and achieving better mapping efficiency.

[0170] According to any of the above embodiments, in yet another embodiment:

[0171] Before the robot sensor scans the target environment to obtain a scanned map in response to the mapping instruction, the process includes:

[0172] Obtaining a category of the target environment, and determining a characteristic target of the target environment according to the category;

[0173] Accordingly, in response to the mapping instruction, the robot sensor scans the target environment to obtain a scanned map, including:

[0174] In response to the mapping instruction, the robot sensor scans the characteristic area of ​​the target environment at a first frequency and scans the non-characteristic area of ​​the target environment at a second frequency to obtain a scanned map; the characteristic area is a preset area including the characteristic target; the non-characteristic area is a preset area not including the characteristic target.

[0175] In this embodiment, the category of the target environment is at least one of a plurality of predefined categories. In some implementations, the category of the target environment may include an airport, a hotel, a restaurant, a factory, and the like.

[0176] The characteristic target is a position and / or article of interest predefined based on a specific category of target environment, for example, for a restaurant category of target environment, a dining table and a dining chair are articles that need to be focused on and avoided, for a food service robot, a serving port is a position frequently used in the working process, therefore, in the restaurant category of target environment, the characteristic target can include the dining table, the dining chair and the serving port; for example, for an airport category of target environment, the characteristic target can include a boarding gate, a waiting hall and a waiting seat; for example, for a hotel category of target environment, the characteristic target can include a room door and a corridor.

[0177] Figure 8 The characteristic region 405 is also shown, for the process of robot scanning, the sampling density can be increased for the environment near the characteristic target, to achieve more accurate map construction for these areas of interest, in an optional embodiment of the present embodiment, the execution of step 102 can include:

[0178] After the robot collects the terrain information at the preset scanning point, it is judged whether the terrain information contains the characteristic target, if not, the original path is scanned; if so, the scanning points in the characteristic region and on the original path are increased.

[0179] The beneficial effects of the present embodiment are:

[0180] By introducing the characteristic target and the characteristic region, the robot performs more detailed scanning for the areas of interest in the map construction process, to obtain a more accurate map, thereby providing convenience for the working process after the map construction is completed.

[0181] According to any of the above embodiments, in yet another embodiment:

[0182] After the robot sensor scans the target environment to obtain a scan map in response to the mapping instruction, the method further includes:

[0183] Performing low-pixelization processing on the scan map to obtain a plurality of thumbnails with different numbers of pixels; the number of pixels of the thumbnail is less than that of the scan map;

[0184] In response to a zoom display signal, selecting a zoom display image from the thumbnails according to a zoom ratio of the zoom display signal and displaying the zoom display image.

[0185] When scanning a large scene, users often need to zoom out to view the overall scanning state, and rendering and displaying the original image will consume a lot of performance. However, when viewing the overall situation and the zoom ratio is small, users are more concerned about details, and therefore there is no need to display high-definition images.

[0186] Exemplarily, in a preferred embodiment, when a scan map is received and the scan map is greater than a set threshold, two low-pixel version thumbnails of the original scan map are synchronously generated, and the number of pixels can be 1 / 4 and 1 / 100 of the original map, that is, the original map is scaled to 0.5 times and 0.1 times. When the user reduces the zoom ratio to 0.5-1, the 1 / 4 thumbnail is switched to, and when the zoom ratio is 0.1-0.5, the 1 / 100 thumbnail is switched to.

[0187] In this way, the performance can be effectively improved when the full map is displayed at a small zoom ratio. For example, for a scan map picture with an original map of 10000*10000 pixels, only 1000*1000 pixels need to be displayed.

[0188] Optionally, when the map is rendered, the rendering can not be performed according to the pixel ratio, but a rendering ratio is set instead of the pixel ratio as one of the map rendering parameters, and the rendering ratio represents the ratio between the pixels rendered by the front end and the size of the real environment. The rendering ratio can be dynamically changed. For example, when the length and width of the scan map are less than or equal to a set threshold, the rendering ratio can be a1, a1 is preferably equal to the pixel ratio, which is convenient for direct rendering. When any one of the length or width of the scan map exceeds the set threshold, the pixel ratio can be a2, a2 is less than a1. For example, the set threshold can be 150 m, a1=0.1, and a2=0.05. Optionally, the pixel ratio is provided with a minimum value to ensure that the map display clarity meets the minimum requirement. Optionally, the set threshold includes a width threshold and a length threshold, which are determined according to the width and length of the maximum boundary of the map space and the rendering ratio a1.

[0189] The beneficial effects of the embodiment are as follows:

[0190] By introducing multiple low-pixel thumbnails, the user can display different pixel thumbnails according to the zoom ratio when viewing the global scan state, so as to reduce the unnecessary interference of the performance overhead of global image rendering on the robot mapping work, and improve the efficiency of robot mapping.

[0191] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0192] Corresponding to the robot mapping method described in the above embodiment, Figure 2 A structural block diagram of a robot mapping device provided by an embodiment of the application is shown, and only parts related to the embodiment of the application are shown for ease of illustration.

[0193] With reference to Figure 2 The device comprises:

[0194] A scanning map module 201 is configured to respond to a mapping instruction and use a robot sensor to scan a target environment to obtain a scanned map;

[0195] a rendering and display module 202 configured to render and display at least a portion of the scanned map in a front-end map space 410 in real time according to map rendering parameters, wherein the map space 410 is an adjustable space and is determined according to the map rendering parameters and the scanned map;

[0196] The map generation module 203 is configured to generate a robot map according to the edited scan map in response to the completion instruction.

[0197] Furthermore, the device further comprises:

[0198] A complete rendering size module is used to obtain map rendering parameters and obtain the size of the complete rendered map according to the map rendering parameters and the size of the scanned map;

[0199] A block cutting module is used to determine that the size of the complete rendered map exceeds the maximum boundary of the map space, and then cut the scanned map into blocks to obtain block maps;

[0200] Accordingly, the rendering and display module 202 includes:

[0201] a central rendering unit, configured to determine a central block map to be rendered based on the central coordinates of a visible area of ​​the map space, wherein the visible area is an area in the map space displayed on the robot screen;

[0202] An adjacent rendering unit is used to render the central block map and its adjacent block maps according to the map rendering parameters, and display the central block map in the visible area.

[0203] The block cutting module includes:

[0204] The visible area size unit is used to obtain the visible area size;

[0205] The size segmentation unit is used to segment the scanned map into a plurality of segmented maps having the same size as the visible area.

[0206] Furthermore, the device further comprises:

[0207] The storage module is used to save the rendered block map in a two-dimensional array with block map marks.

[0208] In some optional implementations, the rendering and display module 202 includes:

[0209] A constraint display unit is configured to display on the map space:

[0210] at least a portion of the map obtained by rendering the scanned map;

[0211] Preset grid scales; and,

[0212] The actual area of ​​the scanned scene.

[0213] In some other optional embodiments, the device further comprises:

[0214] A category module, configured to obtain the category of the target environment and determine a characteristic target of the target environment according to the category;

[0215] Accordingly, the map scanning module 201 includes:

[0216] A feature target scanning unit is used to respond to the mapping instruction, wherein the robot sensor scans the feature area of ​​the target environment at a first frequency and scans the non-feature area of ​​the target environment at a second frequency to obtain a scan map; the feature area is a preset area including the feature target; the non-feature area is a preset area not including the feature target.

[0217] In some further optional embodiments, the device further comprises:

[0218] a low-pixel module, configured to perform low-pixel processing on the scanned map to obtain a plurality of thumbnails with different numbers of pixels; the number of pixels of the thumbnails is less than that of the scanned map;

[0219] The zoom display module is configured to select and display a zoom display image from the thumbnail in response to a zoom display signal according to a zoom ratio of the zoom display signal.

[0220] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0221] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0222] The present application also provides a terminal device, such as Figure 3 As shown, the terminal device 30 includes: at least one processor 301, a memory 302, and a computer program 303 stored in the memory and executable on the at least one processor, and the processor implements the steps of any of the above method embodiments when executing the computer program.

[0223] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0224] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned various method embodiments when executing the computer program product.

[0225] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0226] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0227] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0228] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0229] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0230] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A robot mapping method, characterized in that: include: In response to the mapping instruction, the robot sensor scans the target environment to obtain a scanned map; Rendering and displaying at least a portion of the scanned map in real time in a map space of a front end according to map rendering parameters, wherein the map space is an adjustable space and is determined according to the map rendering parameters and the scanned map; In response to the completion instruction, generating a robot map based on the edited scan map; Before rendering the scanned map in real time according to the map rendering parameters in the map space of the front end, the method further includes: Obtaining the size of the complete rendered map according to the acquired map rendering parameters and the size of the scanned map; if it is determined that the size of the complete rendered map exceeds the maximum boundary of the map space, cutting the scanned map into blocks to obtain block maps; Accordingly, the map space on the front end renders and displays the scanned map in real time according to the map rendering parameters, including: The central block map to be rendered is determined according to the center coordinates of the visible area of ​​the map space, where the visible area is the area in the map space displayed on the robot screen; the central block map and its adjacent block maps are rendered according to the map rendering parameters, and the central block map is displayed in the visible area.

2. The robot mapping method according to claim 1, wherein: The step of cutting the scanned map into blocks to obtain a block map includes: Get the size of the visible area; The scanned map is divided into blocks and cut into a plurality of block maps having the same size as the visible area.

3. The robot mapping method according to claim 1 or 2, wherein: The step of rendering the central block map and its adjacent block maps according to the map rendering parameters and displaying the central block map in the visible area includes: The rendered tile map is saved in a two-dimensional array with a tile map tag.

4. The robot mapping method according to claim 1 or 2, wherein: The map space on the front end renders and displays at least a portion of the scanned map in real time according to map rendering parameters, including: With the current coordinates of the robot being within the preset following area of ​​the map space as a constraint, the following is displayed on the map space: at least a portion of the map obtained by rendering the scanned map; Preset grid scales; and, The actual area of ​​the scanned scene.

5. The robot mapping method according to claim 1 or 2, wherein: Before the robot sensor scans the target environment to obtain a scanned map in response to the mapping instruction, the process includes: Obtaining a category of the target environment, and determining a characteristic target of the target environment according to the category; Accordingly, in response to the mapping instruction, the robot sensor scans the target environment to obtain a scanned map, including: In response to the mapping instruction, the robot sensor scans the characteristic area of ​​the target environment at a first frequency and scans the non-characteristic area of ​​the target environment at a second frequency to obtain a scanned map; the characteristic area is a preset area including the characteristic target; the non-characteristic area is a preset area not including the characteristic target.

6. The robot mapping method according to claim 2, wherein: After the robot sensor scans the target environment to obtain a scanned map in response to the mapping instruction, the method includes: Performing low-pixel processing on the scanned map to obtain a plurality of thumbnails with different numbers of pixels; the number of pixels of the thumbnails is less than that of the scanned map; In response to the zoom display signal, a zoom display image is selected from the thumbnail images according to the zoom ratio of the zoom display signal and displayed.

7. A robot mapping device, characterized in that: include: A scanning map module is used to respond to a mapping instruction, and the robot sensor scans the target environment to obtain a scan map; a rendering and display module, configured to render and display at least a portion of the scanned map in real time in a map space of a front end according to map rendering parameters, wherein the map space is an adjustable space and is determined according to the map rendering parameters and the scanned map; A map generation module is used to generate a robot map based on the edited scan map in response to the completion instruction. Wherein, the device is also used for: Obtaining the size of the complete rendered map according to the acquired map rendering parameters and the size of the scanned map; if it is determined that the size of the complete rendered map exceeds the maximum boundary of the map space, cutting the scanned map into blocks to obtain block maps; Correspondingly, the rendering and display module is also used to: determine the central block map to be rendered according to the center coordinates of the visible area of ​​the map space, where the visible area is the area in the map space displayed on the robot screen; render the central block map and its adjacent block maps according to the map rendering parameters, and display the central block map in the visible area.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images