Map display method, autonomous mobile robot and storage medium

Abstract

The invention discloses a map display method, an autonomous mobile robot and a storage medium, and belongs to the technical field of robots. The method comprises the steps that a main processor obtains first coordinate information of each element in one or more elements in a map and sends the first coordinate information to a display card, wherein the first coordinate information of the elements is used for indicating the position of each preset graph in one or more preset graphs forming the elements in a map coordinate system; the display card determines second coordinate information according to the first coordinate information of each element in the one or more elements, wherein the second coordinate information of the element is used for indicating the position of each preset graph in one or more preset graphs forming the element in the cutting coordinate system. And the display card displays each element according to the second coordinate information of each element in the one or more elements. When the map is displayed, the main processor and the display card are respectively responsible for a part of processing tasks, so that the calculation pressure of the main processor and the display card can be balanced, and the smoothness of map display can be improved to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a map display method, an autonomous mobile robot and a storage medium. Background Art

[0002] With the continuous development of science and technology, robots have gradually been applied to various industries, such as food delivery robots, hotel or hospital delivery robots, etc. Before the robot works, it needs to scan the map, edit the map and other mapping processes so that the robot can move safely according to the map. The robot can display the map in real time during the movement.

[0003] In the related art, when displaying a map, the robot calculates the positions of all elements in the map by calling the canvas interface of the browser, and then displays these elements accordingly.

[0004] However, in the above method, the operation of calculating the position of the elements in the map is performed by the central processing unit (CPU) of the robot. As the number of elements in the map increases, the calculation pressure of the CPU will increase, which will affect the smoothness of the map display and the user experience of the robot. Summary of the invention

[0005] The present application provides a map display method, an autonomous mobile robot and a storage medium, which can improve the smoothness of map display. The technical solution is as follows:

[0006] In a first aspect, a map display method is provided, which is applied to an autonomous mobile robot, wherein the autonomous mobile robot includes a main processor and a graphics card, and the method includes:

[0007] The main processor acquires first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card, wherein the first coordinate information of the element is used to indicate the position of each of the one or more preset graphics constituting the element in the map coordinate system of the map;

[0008] The graphics card determines the second coordinate information of each element according to the first coordinate information of each element in the one or more elements, wherein the second coordinate information of the element is used to indicate the position of each preset graphic in the one or more preset graphics constituting the element in the clipping coordinate system;

[0009] The graphics card displays each element of the one or more elements according to the second coordinate information of each element.

[0010] In the present application, when the autonomous mobile robot displays a map, the main processor in the autonomous mobile robot obtains the first coordinate information of each element in one or more elements in the map to be displayed, and then the graphics card in the autonomous mobile robot determines each second coordinate information according to the first coordinate information of each element in the one or more elements, and displays each element accordingly. In this way, when the autonomous mobile robot displays a map, the main processor and the graphics card are each responsible for a part of the processing task, so that the computing pressure of the main processor and the graphics card can be balanced. In addition, since the computing power of the main processor and the graphics card is relatively strong, the efficiency of the main processor in obtaining the first coordinate information of each element is relatively high, and then the efficiency of the graphics card in determining the second coordinate information of each element is also relatively high. In this way, the smoothness of the map display can be improved to a certain extent, and the user experience of the autonomous mobile robot can be guaranteed.

[0011] In a second aspect, an autonomous mobile robot is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the map display method described in the first aspect.

[0012] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the map display method described in the first aspect is implemented.

[0013] In a fourth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the steps of the map display method described in the first aspect.

[0014] It can be understood that the beneficial effects of the second, third and fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a structural schematic diagram of an autonomous mobile robot provided in an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of a map provided in an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of an autonomous mobile robot provided in an embodiment of the present application;

[0019] Figure 4 is a flow chart of a map display method provided in an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a circular element provided in an embodiment of the present application;

[0021] Figure 6 is a schematic diagram of a rectangular element provided in an embodiment of the present application;

[0022] Figure 7 is a schematic diagram of a polygonal element provided in an embodiment of the present application;

[0023] Figure 8 is a schematic diagram of another polygonal element provided in an embodiment of the present application;

[0024] Fig. 9 is a schematic diagram of a width line element provided in an embodiment of the present application;

[0025] Fig.10 It is a structural schematic diagram of an autonomous mobile robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0027] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0028] The phrases such as "one embodiment" or "some embodiments" described in the present application mean that the specific features, structures or characteristics described in the embodiment are included in one or more embodiments of the present application. Therefore, the phrases such as "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments" that appear in the differences in the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. In addition, the terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0029] The autonomous mobile robot provided in the embodiments of the present application is described below.

[0030] Autonomous mobile robots can be used in goods delivery scenarios, such as room delivery services in hotel scenarios. Among them, delivery scenarios can include airports or hotels. The delivery task can be a task to provide users with delivery of goods. The first delivery area refers to a pre-designated area in the delivery scenario. The autonomous mobile robot can move to the first delivery area and notify the user, and then wait for pickup. For example, the first delivery area can be a specific location in the delivery scenario, such as the door of a room in a hotel. The second delivery area is the specific address that the delivery task requires delivery. For example, the delivery task provides express delivery service for Room B in Area A. The autonomous mobile robot can move to the second delivery area, that is, the door of Room B in Area A, and call the fixed telephone in the room to notify the user to pick up the goods at the door; for another example, the delivery task provides food delivery service for Door D of Hotel C. The autonomous mobile robot can move to the second delivery area, that is, Door D of Hotel C, and call the phone in the door to notify the user to pick up the food at the door.

[0031] For example, assuming that the distribution scene is a hotel, the autonomous mobile robot can be a delivery robot of the hotel, and each delivery robot has identification information, such as an identity document (ID), which is used to identify the identity of the robot. The delivery area of ​​the delivery robot of the hotel includes the room door area inside the hotel. After checking in to the hotel, the user can use the hotel's service system to order food or daily necessities, wherein the service system can be a telephone order or a mobile phone application (application, APP) order. For example, when ordering food, the user can reserve food and beverage in the APP in the mobile phone and enter the room number of the user. The APP sends the reserved food and beverage information entered by the user to the platform system of the hotel, and the platform system can generate a target delivery task according to the room number of the user and the reserved food and beverage information of the user and send it to the delivery robot in the hotel. After the delivery robot obtains the user's reserved food and beverage from the hotel staff, it moves to the target delivery area, and the target delivery area can be the area in front of the room where the user stays.

[0032] The delivery robot can send a pickup prompt to the APP in the user's mobile phone, prompting the user to move to the target delivery area to pick up the item. After the user learns of the pickup prompt from the delivery robot, he can talk to the delivery robot, and the delivery robot can inform the user of the delivered item information and the location of the target delivery area in the form of voice. After the user confirms that the item delivered by the delivery robot is the item he has reserved, he arrives at the target delivery area to obtain the item delivered by the delivery robot. For example, the delivery robot can trigger the generation of a key, which is used to confirm the pickup, such as a pickup code.

[0033] Figure 1 is a schematic diagram of the structure of an autonomous mobile robot provided in an embodiment of the present application. Figure 1 The autonomous mobile robot can be a delivery robot, and the autonomous mobile robot can include a mobile chassis 10, a shell 20 and a laser radar system 103, wherein a turn signal unit 101 and a driving wheel 102 can be provided on the mobile chassis 10, a plurality of turn signals 1011 are provided in the turn signal unit 101, and a display screen 201 can be provided on the shell 20.

[0034] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the autonomous mobile robot. In some embodiments, the autonomous mobile robot may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0035] The mobile chassis 10 is used to control the autonomous mobile robot to move.

[0036] The turn signal unit 101 is used to indicate that the autonomous mobile robot is turning. The number of the turn signal unit 101 can be one or more. The one or more turn signal units 101 can be arranged at the front end of the mobile chassis 10. For example, see Figure 1 The turn signal unit 101 can be arranged on the left and right sides of the front end of the mobile chassis 10 .

[0037] The driving wheel 102 is used to drive the autonomous mobile robot to move. The number of the driving wheel 102 can be one or more. The one or more driving wheels 102 can be arranged at the bottom of the mobile chassis 10. For example, see Figure 1 The driving wheels 102 can be arranged on the left and right sides of the bottom of the mobile chassis 10 .

[0038] Optionally, one or more driven wheels may be further provided at the bottom of the mobile chassis 10. The one or more driven wheels are used to assist the driving wheel 102 in driving the housing 20 and the chassis 10 of the autonomous mobile robot to move, thereby reducing the load pressure of the driving wheel 102. For example, the one or more driven wheels may be provided at the rear side of the driving wheel 102.

[0039] The housing 20 is used to carry items.

[0040] The display screen 201 is used to provide functional operations for the user. For example, the display screen 201 can be a touch screen. The display screen 201 can be arranged at the front end of the housing 20, or can be arranged at the top of the housing 20, which is not limited in the embodiment of the present application.

[0041] Optionally, a bottom layer controller, a component controller for controlling the moving unit, and a component controller for the environment detection unit may also be disposed in the housing 20 .

[0042] The underlying controller is used to control map generation and path planning.

[0043] The element controller of the control mobile unit is used to control the speed of the driving wheel 102 and control the turn signal unit 101 to light up in a preset sequence when the autonomous mobile robot turns. The preset sequence can be pre-set. For example, the preset sequence can be set by the user according to needs.

[0044] In some cases, if it is detected that the speed difference between the driving wheels 102 is greater than a preset value, the element controller of the control moving unit controls one or more turn lights 1011 in the turn light unit 101 to light up in a preset order.

[0045] The element controller of the environment detection unit is used to connect and process the data collected by the laser radar.

[0046] The laser radar system 103 is used to collect environmental information, perform environmental perception, obstacle identification, and self-positioning. Figure 1 As shown, the laser radar system 103 can be arranged between the mobile chassis 10 and the shell 20.

[0047] Optionally, the laser radar system 103 can rotate along a preset plane. The preset plane can be preset. For example, the preset plane can be set to a horizontal plane, a vertical plane, etc., which is not limited in the embodiment of the present application.

[0048] Optionally, the autonomous mobile robot may further include a voice module, which may be connected to the element controller of the control mobile unit. The voice module is used to receive instructions sent by the element controller of the control mobile unit, and issue a voice prompt message accordingly. For example, the voice module may receive a control instruction sent by the element controller of the control mobile unit when the autonomous mobile robot turns, so as to issue a voice prompt message of turning.

[0049] Optionally, before the autonomous mobile robot delivers items, a map may be obtained, which is a map of the area where the autonomous mobile robot can move. Specifically, the autonomous mobile robot can move, and during the movement, the laser radar system 103 obtains the position information of the autonomous mobile robot and collects environmental information, and then constructs a map of the area based on the position information and the environmental information and stores it.

[0050] It should be noted that the above is only an example of the autonomous mobile robot being applied to the item delivery scenario. In actual applications, the autonomous mobile robot can also be applied to other scenarios, and the embodiments of the present application do not limit this. For example, the autonomous mobile robot can be applied to the cleaning scenario. In this case, the autonomous mobile robot is a cleaning robot.

[0051] In some embodiments, a mapping application (APP) may be installed in the autonomous mobile robot, and the mapping APP is used for map scanning and map editing.

[0052] For example, the map building APP can continuously display the constructed map during the process of building a map of a certain area. In this case, the displayed map may include elements such as laser point cloud and stereoscopic point cloud.

[0053] For example, the map building APP can display a map of a certain area. Users can edit the map through the map building APP. During the process of editing the map, the map building APP can refresh the map based on the user's editing operation. For example, users can add target points, target point names, path points, label points, prohibited lines, speed limit areas and other elements to the map through the map building APP, and finally get the following: Figure 2 Map shown.

[0054] For example, the mapping APP can continuously display a map of the area near the autonomous mobile robot based on the position of the autonomous mobile robot during the movement of the autonomous mobile robot.

[0055] Since many elements need to be rendered in the map, and all types of elements are graphics, a large amount of graphics drawing is required in a complete robot map scene. In addition, all elements in the map may need to be refreshed according to the real-time status of the robot and user operations. For users, a smooth APP experience requires maintaining a frame rate of around 30. If you want to run the map-building APP smoothly on low-cost hardware, you need to improve the rendering performance as much as possible and reduce the amount of element drawing calculations.

[0056] In the related art, when displaying a map, the positions of all elements in the map are calculated by calling the browser's canvas interface, and then these elements are displayed accordingly. However, this method requires the CPU to calculate the element positions, and cannot perform fine-grained control over rendering. When the screen needs to be refreshed, it can only be fully refreshed, resulting in a large loss in computing performance. In addition, as the number of elements in the map increases, the computing pressure on the CPU will increase, which will affect the smoothness of the map display.

[0057] To this end, the present application proposes a map display method that can reduce the computing pressure of the CPU and ensure the smoothness of map display.

[0058] Figure 3 is a schematic diagram of an autonomous mobile robot provided in an embodiment of the present application. Figure 3 , the autonomous mobile robot may include a main processor 301 and a graphics card 302 .

[0059] The main processor 301 can be communicatively connected with the graphics card 302 .

[0060] The main processor 301 is used to obtain the target coordinate information of each of the one or more elements in the map, determine the first coordinate information of the element according to the target coordinate information of each of the one or more elements, and send the first coordinate information of each of the one or more elements to the graphics card 302. By way of example, the main processor 301 may be a CPU or the like.

[0061] For example, the elements in the map may include one or more of a laser point cloud, a stereoscopic vision point cloud, a target point, a target point name, a path point, a label point, a prohibited line, a speed limit area, a robot icon, and the like.

[0062] Each of the one or more elements in the map has a graphic type. For example, the graphic type of the one or more elements may include one or more of a circle, a rectangle, a polygon, a straight line, a polyline, a width line, etc. For example, the graphic type of a laser point cloud, a stereoscopic point cloud, or a target point may be a circle, the graphic type of a target point name may be a width line, the graphic type of a speed limit area or a label point may be a rectangle or a polygon, and the graphic type of a prohibited line may be a straight line, a polyline, or a width line.

[0063] The target coordinate information of an element is used to indicate the position of the element in the map coordinate system of the map.

[0064] By way of example, the target coordinate information of a circular element may include the coordinates of the center and radius of the circle; the target coordinate information of a rectangular element may include the coordinates of the four vertices of the rectangle; the target coordinate information of a polygonal element may include the coordinates of each of the multiple vertices of the polygon; the target coordinate information of a straight line element may include the coordinates of each of the two endpoints of the straight line; the target coordinate information of a polyline element may include the coordinates of each of the multiple endpoints of the polyline (including a starting point, one or more inflection points, and an end point); the target coordinate information of a width line element may include the coordinates of each of the multiple endpoints of the width line.

[0065] The map coordinate system can be preset. For example, the map coordinate system can be constructed with the upper left corner of the map as the origin, the right as the positive direction of the x-axis (i.e., the horizontal axis), and the downward as the positive direction of the y-axis (i.e., the vertical axis). Alternatively, the map coordinate system can be constructed with the lower right corner of the map as the origin, the left as the positive direction of the x-axis, and the upward as the positive direction of the y-axis. Of course, it is not limited to this, and the map coordinate system can also be constructed by other methods, which is not limited in the embodiments of the present application.

[0066] The first coordinate information of an element is used to indicate the position of each preset graphic in the map coordinate system of one or more preset graphics constituting the element. For example, the first coordinate information of the element may be vertex data that can be rendered by the graphics card 302 .

[0067] The preset graphics may be pre-set. The preset graphics may be graphics that can be rendered by the graphics card 302. For example, the preset graphics may include one or more of a point, a straight line, a triangle, etc.

[0068] For example, the preset graphics constituting the circular element may be points, the preset graphics constituting the rectangular element, the polygonal element, and the width line element may be triangles, and the preset graphics constituting the straight line element and the polyline element may be straight lines.

[0069] Optionally, for any element in the map, the main processor 301 may execute a renderer program according to the target coordinate information of the element to determine the first coordinate information of the element.

[0070] For example, the renderer program may include one or more of a point renderer program, a line renderer program, and a triangle renderer program.

[0071] For example, the main processor 301 may execute a point renderer program according to the target coordinate information of the circular element to determine the first coordinate information of the circular element.

[0072] For example, the main processor 301 may execute a line renderer program according to the target coordinate information of the straight line element to determine the first coordinate information of the straight line element.

[0073] For example, the main processor 301 may execute a line renderer program according to the target coordinate information of the polyline element to determine the first coordinate information of the polyline element.

[0074] For example, the main processor 301 may execute a triangle renderer program according to the target coordinate information of the rectangular element to determine the first coordinate information of the rectangular element.

[0075] For example, the main processor 301 may execute a triangle renderer program according to the target coordinate information of the polygon element to determine the first coordinate information of the polygon element.

[0076] For example, the main processor 301 may execute a triangle renderer program according to the target coordinate information of the width line element to determine the first coordinate information of the width line element.

[0077] The graphics card 302 is used to convert the first coordinate information of each of the one or more elements in the map into the second coordinate information, and display each element in the display screen 201 according to the second coordinate information of each of the one or more elements.

[0078] The second coordinate information of an element is used to indicate the position of each of the one or more preset graphics constituting the element in the clipping coordinate system.

[0079] The clipping coordinate system can be preset. For example, the clipping coordinate system can be a web graphics library (WebGl) coordinate system. For example, the clipping coordinate system can be constructed with the center of the display screen 201 as the origin, the lower left corner of the display screen 201 as the coordinate (-1, -1), and the upper right corner of the display screen 201 as the coordinate (1, 1). Of course, it is not limited to this, and the clipping coordinate system can also be constructed by other methods, which is not limited in the embodiments of the present application.

[0080] For example, for any element in the map, the graphics card 302 can execute a shader program for the element to convert the first coordinate information of the element into second coordinate information and draw the element according to the second coordinate information of the element, and then display the element through the display screen 201.

[0081] By way of example, the shader program may include a vertex shader program and a fragment shader program.

[0082] For example, the graphics card 302 can execute a vertex shader program based on the first coordinate information of the element to determine the second coordinate information of the element, and then execute a fragment shader program based on the second coordinate information of the element to draw the element, and finally display the element through the display screen 201.

[0083] In an embodiment of the present application, when the autonomous mobile robot displays a map, the main processor 301 and the graphics card 302 are each responsible for a part of the processing tasks, thereby balancing the computing pressure of the main processor 301 and the graphics card 302, thereby improving the smoothness of the map display to a certain extent.

[0084] The map display method provided in the embodiment of the present application is explained in detail below.

[0085] Figure 4 is a flowchart of a map display method provided in an embodiment of the present application. The method can be applied to Figure 1 or Figure 3 The autonomous mobile robot in the embodiment can be specifically applied to the main processor and graphics card in the autonomous mobile robot. Figure 4 , the method comprises the following steps:

[0086] Step 401: The main processor obtains the first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card.

[0087] The first coordinate information of an element is used to indicate the position of each of the one or more preset graphics constituting the element in the map coordinate system of the map. That is, for any element in the map, the element can be split into one or more preset graphics.

[0088] It should be noted that the graphic type of the element in the map in the embodiment of the present application may include one or more of a circle, a straight line, a polyline, a rectangle, a polygon, and a width line (including a non-closed width line and a closed width line). In this case, the main processor may split the element into one or more preset graphics according to the graphic type of the element to determine the first coordinate information of the element.

[0089] For example, the polygon in the embodiment of the present application is a simple polygon, which refers to a closed polygon in which the edges do not intersect and do not overlap. The straight lines and broken lines described in the embodiment of the present application can be regarded as lines without width, such as the graphic type of the robot's driving path element can be a straight line or a broken line. The width line described in the embodiment of the present application can be regarded as a line with width, such as the graphic type of a rectangular border (such as a text box) element can be a width line.

[0090] The preset graphic may be set in advance. For example, the preset graphic may include one or more of a point, a straight line, and a triangle.

[0091] For example, a circle element can be split into points; a line element can be split into lines; a polyline element can be split into lines; a rectangle element can be split into triangles; a polygon element can be split into triangles; and a width line element can be split into triangles. This is described in detail below:

[0092] In some embodiments, if the graphic type of an element is a circle, the target coordinate information of the element may include the center coordinates and radius of the circle. In this case, the main processor may execute a point renderer program on the circular element to determine the first coordinate information of the circular element.

[0093] Specifically, the main processor can use the circular element as the target point and the target coordinate information of the circular element (ie, the center coordinates and radius) as the first coordinate information of the circular element. The first coordinate information of the circular element is used to indicate the position of the target point in the map coordinate system.

[0094] For example, Figure 5 As shown, the target coordinate information of the circular element may include the center coordinate E(3, 3) and the radius 2, and the first coordinate information of the circular element may include the center coordinate E(3, 3) and the radius 2.

[0095] In some embodiments, if the graphic type of an element is a straight line, the target coordinate information of the element may include the coordinates of the two endpoints of the straight line. In this case, the main processor may execute a line renderer program on the straight line element to determine the first coordinate information of the straight line element.

[0096] Specifically, the main processor can use the straight line element as the target straight line and the target coordinate information (i.e., the two endpoint coordinates) of the straight line element as the first coordinate information of the straight line element. The first coordinate information of the straight line element is used to indicate the position of the target straight line in the map coordinate system.

[0097] In some embodiments, if the graphic type of an element is a polyline, the target coordinate information of the element may include multiple endpoint coordinates of the polyline. In this case, the main processor may execute a line renderer program on the polyline element to determine the first coordinate information of the polyline element.

[0098] Specifically, the main processor can use the polyline element as the target polyline and the target coordinate information (i.e., multiple endpoint coordinates) of the polyline element as the first coordinate information of the polyline element. The first coordinate information of the polyline element is used to indicate the position of the target polyline in the map coordinate system.

[0099] In some embodiments, if the graphic type of an element is a rectangle, the target coordinate information of the element may include the coordinates of four vertices of the rectangle. In this case, the main processor may execute a triangle renderer program on the rectangle element to determine the first coordinate information of the rectangle element.

[0100] Specifically, the main processor can split the rectangular element into two triangles, and use the target coordinate information of the rectangular element (i.e., four vertex coordinates) and the vertex sequence of each of the two triangles as the first coordinate information of the rectangular element. The first coordinate information of the rectangular element includes four vertex coordinates and two vertex sequences. The first coordinate information of the rectangular element is used to indicate the position of each of the two triangles in the map coordinate system.

[0101] The vertex sequence of a triangle may also be referred to as a vertex index, which is used to indicate which three vertices of the four vertices of the rectangular element constitute the triangle.

[0102] For example, Figure 6As shown, the target coordinate information of rectangle ABCD includes A(1, 5), B(1, 1), C(5, 1), and D(5, 5). The index order of the four vertices of rectangle ABCD is 1, 2, 3, and 4. The rectangle ABCD is split into triangle ABC and triangle ADC, and the vertex sequence of triangle ABC and the vertex sequence of triangle ADC are (1, 3, 4). Then the four vertex coordinates of rectangle ABCD and the vertex sequence of each triangle in the two triangles are the first coordinate information of the rectangle element.

[0103] In some embodiments, if the graphic type of an element is a polygon, the target coordinate information of the element may include multiple vertex coordinates of the polygon. In this case, the main processor may execute a triangle renderer program on the polygon element to determine the first coordinate information of the polygon element.

[0104] Specifically, the main processor can split the polygon element into one or more triangles, and use the target coordinate information of the polygon element (i.e., multiple vertex coordinates) and the vertex sequence of each triangle in the one or more triangles as the first coordinate information of the polygon element. The first coordinate information of the polygon element includes multiple vertex coordinates and one or more vertex sequences. The first coordinate information of the polygon element is used to indicate the position of each triangle in the one or more triangles in the map coordinate system.

[0105] The vertex sequence of a triangle may also be referred to as a vertex index, which is used to indicate which three vertices of the multiple vertices of the polygon element constitute the triangle.

[0106] Optionally, the polygons may include convex polygons and concave polygons.

[0107] In this case, the convex polygonal elements can be directly split in the order of their multiple vertices to obtain one or more triangles; the concave polygonal elements can be split using the ear shearing method to obtain one or more triangles.

[0108] For example, the concave polygon element splitting process may include the following steps (1) to (4):

[0109] (1) Polygon vertex traversal: Starting from the vertices of the polygon, check each vertex and its adjacent vertices one by one.

[0110] (2) Find the internal angles: For each vertex (assuming it is point B), calculate the angle formed by its two adjacent vertices (point A and point C). If this angle is less than 180 degrees, point B is considered to be an "ear" vertex. In other words, the angle B between AB and BC is an internal angle.

[0111] For example, the cross product of vectors can be used to find vertices whose interior angles are less than 180 degrees.

[0112] In this example, the cross product of vector AB and vector BC can be calculated as follows: Cross product = (-1*2) - (-6*4) = -2 + 24 = 22. Since the cross product is positive, it means that angle B is an interior angle and is less than 180 degrees.

[0113] (3) Cutting off the ear: If the internal angle is less than 180 degrees and the corresponding triangle ABC (composed of points A, B, and C) does not overlap or intersect with other sides of the polygon, the triangle can be cut off. At this time, the remaining polygon will have one less vertex, and the next ear cutting operation can be performed.

[0114] (4) Iteration process: Repeat the above process until only one triangle remains.

[0115] In this way, the concave polygon element can be split into multiple triangles.

[0116] For example, Figure 7 As shown, the target coordinate information of the convex polygon ABCDEF includes A(-1, 4), B(0.5, 5.9), C(2.5, 5), D(3.1, 2.4), E(2.5, 0.9), and F(0.8, 0.7). The index order of the six vertices of the convex polygon ABCDEF is 1, 2, 3, 4, 5, and 6. The convex polygon ABCDEF is split in sequence according to the order of the six vertices to obtain triangle ABC, triangle ACD, triangle ADE, and triangle AEF. The vertex sequence of triangle ABC is (1, 2, 3), the vertex sequence of triangle ACD is (1, 3, 4), the vertex sequence of triangle ADE is (1, 4, 5), and the vertex sequence of triangle AEF is (1, 5, 6). Then the six vertex coordinates of the convex polygon ABCDEF and the vertex sequence of each of the four triangles are the first coordinate information of the convex polygon element.

[0117] For example, Figure 8As shown, the target coordinate information of the concave polygon ABCDEF includes A(0.5, 3.5), B(2, 6), C(7, 3.8), D(6, 1), E(3, 3), and F(2, 0.4). The index order of the six vertices of the concave polygon ABCDEF is 1, 2, 3, 4, 5, and 6. The concave polygon ABCDEF is split according to the ear shearing method to obtain triangle AEF, triangle AEB, triangle BEC, and triangle EDC. The vertex sequence of triangle AEF is (1, 5, 6), the vertex sequence of triangle ABE is (1, 2, 5), the vertex sequence of triangle BCE is (2, 3, 5), and the vertex sequence of triangle EDC is (4, 3, 5). Then the six vertex coordinates of the concave polygon ABCDEF and the vertex sequence of each of the four triangles are the first coordinate information of the polygon element.

[0118] In some embodiments, if the graphic type of the element is a width line, the target coordinate information of the element may include multiple endpoint coordinates of the width line. In this case, the main processor may execute a triangle renderer program on the width line element to determine the first coordinate information of the width line element.

[0119] Assume that the multiple endpoints of the width line element are: P0, P1, P2, ..., P m , m is a positive integer.

[0120] For example, Fig. 9 As shown in Figure (a), the main processor can first determine the target coordinate information of the polygon corresponding to the width line element (i.e., multiple vertex coordinates of the polygon) based on the target coordinate information of the width line element (i.e., multiple endpoint coordinates), and then split the polygon into multiple triangles, and use the target coordinate information of the polygon and the vertex sequence of each triangle in the multiple triangles as the first coordinate information of the width line element. The first coordinate information of the width line element includes multiple vertex coordinates and multiple vertex sequences. The first coordinate information of the width line element is used to indicate the position of each triangle in the multiple triangles in the map coordinate system.

[0121] The vertex sequence of a triangle may also be referred to as a vertex index, which is used to indicate which three vertices of the multiple vertices of the polygon constitute the triangle.

[0122] It should be noted that the polygon corresponding to the width line element is a polygon consisting of two vertices obtained by extending each end point of the width line element to both sides by a line width d, wherein the line width d can be set in advance.

[0123] In some embodiments, the operation of the main processor determining the target coordinate information of the polygon corresponding to the width line element according to the target coordinate information of the width line element may be as follows: Fig. 9 As shown in Figure (b), for the starting point P0 of the width line element, the perpendicular vector of the vector P0P1 is calculated, and the perpendicular vector is normalized to a unit vector and multiplied by the line width d and -d respectively to obtain two first vectors. The end point coordinates of each of the two first vectors are used as the two vertex coordinates corresponding to the starting point P0. n , the end point P is obtained by a similar method n The corresponding two vertex coordinates. For any inflection point P of the width line element i , calculate the vector P i-1 P i The corresponding unit vector and vector P i P i+1 The sum of the corresponding unit vectors is used to obtain the second vector, and then the perpendicular vector of the second vector is calculated. The perpendicular vector is normalized to a unit vector and multiplied by the line width d and -d respectively to obtain two third vectors. The end point coordinates of each of the two third vectors are used as the two vertex coordinates corresponding to the inflection point Pi. In this way, the coordinates of multiple vertices of the polygon corresponding to the line element of this width are obtained. Among them, i is an integer greater than 0 and less than m.

[0124] For example, Fig. 9 As shown in Figure (c), assume that vector AB is vector P0P1, and vector BE is vector P1P2. Add vector AB (that is, vector BC) and vector BE to obtain vector BD. Then determine the unit vectors corresponding to the perpendicular vector of vector BD, that is, vector BF and vector BG. Multiply vector BF and vector BG by the line width d respectively, and you can get two first vectors. The coordinates of the end point of each of the two first vectors are used as the coordinates of the two vertices corresponding to the starting point P0.

[0125] In some cases, when drawing a width line, the width of the width line at the corner is relatively large. For example, the line width of the width line is preset to d, but the line width at the corner of the width line is actually larger than d. Therefore, when determining the inflection point P i The corner ratio should be considered when matching the coordinates of two vertices.

[0126] Specifically, we can use the vector P i-1 P i The corresponding unit vector and vector P i P i+1 The corresponding unit vector dot product obtains the target cosine value, the corner ratio is determined according to the target cosine value, and d is divided by the corner ratio to obtain the corner width d1. Then for the above inflection point P i , in calculating the vector P i-1 P i The corresponding unit vector and vector P i P i+1The corresponding unit vectors are summed to obtain the second vector, and then the perpendicular vector of the second vector is calculated. The perpendicular vector is normalized to a unit vector and multiplied by d1 and -d1 respectively to obtain two third vectors. The end point coordinates of each of the two third vectors are taken as the inflection point P. i The corresponding two vertex coordinates.

[0127] For example, the corner ratio can be determined by the following formula:

[0128]

[0129] in, is the corner ratio, and cosα is the target cosine value.

[0130] For example, Fig. 9 As shown in Figure (d), the width line is IJK, and point J is the inflection point. Line segments DH and BA are obtained by extending line segment IJ to both sides by line width d, and line segments AC and GF are obtained by extending line segment JK to both sides by line width d. Line segment AH is perpendicular to line segment DH and line segment BA, and line segment AG is perpendicular to line segment AC and line segment GF. Angle HAG is angle ɑ.

[0131] The target cosine value, i.e., cosα, is obtained by multiplying the unit vector corresponding to vector IJ with the unit vector corresponding to vector JK, and then the corner ratio is determined based on the above formula Divide d by the corner ratio to get the corner width d1. The length of line segment JA and line segment JE is d1. In this way, the coordinates of the two vertices corresponding to point J, namely point A and point E, can be determined.

[0132] In some cases, if the sum of the target cosine value and 1 is less than a preset value, the corner ratio can be directly determined to be 1, so that an excessively large corner width can be avoided. The preset value can be pre-set. For example, the preset value can be set to 0.01.

[0133] Optionally, if the number of endpoints of the width line element is relatively large, or if the average distance between adjacent endpoints in the width line element is relatively small, that is, the endpoints in the width line element are relatively dense, then the width line element can be determined to be a continuous width curve, for example, the points when the user drags the mouse brush to draw a line are quite dense. In this case, the corner ratio can be directly determined to be 1 to obtain a smooth curve effect.

[0134] In some embodiments, the host processor includes a target tree, and the map corresponds to the target tree.

[0135] The target tree is a tree structure, which includes at least one branch node and multiple leaf nodes, wherein a node with 0 child nodes is a leaf node, and a node with 1 child nodes is a branch node. The topmost branch node in the target tree can be called a root node, which is a branch node without a parent node in the target tree.

[0136] Each of the multiple leaf nodes in the target tree includes target coordinate information and an update flag for each element in at least one element in the map, the target coordinate information of an element is used to indicate the position of the element in the map coordinate system, and the update flag of the element is used to indicate whether the element is updated, and each of the at least one branch node includes an update flag, and the update flag in a branch node is used to indicate whether there is an updated element in the leaf nodes under the branch node.

[0137] The correspondence between the map and the target tree means that the target tree covers the entire map space.

[0138] In this case, when the main processor needs to obtain the first coordinate information of each of one or more elements in the map, it can read the target coordinate information of each of the one or more elements from the target tree, and then determine the first coordinate information of each element based on the target coordinate information.

[0139] In some embodiments, for any element in the map, the main processor sets the update flag of the element in the target tree to logical true when the element is updated, and logical true means that the element is updated. The element being updated means that the target coordinate information of the element is updated, that is, the position of the element in the map coordinate system is updated.

[0140] In some embodiments, for a newly added element in the map, the main processor stores the target coordinate information and update flag of the element in a leaf node in the target tree, and sets the update flag of the element to logical true.

[0141] Optionally, the main processor obtains the first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card. The operation may be as follows: if the update flag of the root node of the target tree is logically true, then for any element in the target tree whose update flag is logically true, the main processor obtains the target coordinate information of the element from the target tree; the main processor determines the first coordinate information of the element based on the target coordinate information of the element; the main processor sends the first coordinate information of the element to the graphics card, and stores the first coordinate information of the element in the corresponding leaf node in the target tree, and sets the update flag of the element in the leaf node to logically false.

[0142] If the update flag of the root node of the target tree is logically true, it means that there are updated elements in the leaf nodes under the root node of the target tree, so it is necessary to traverse the elements in each leaf node of the multiple leaf nodes in the target tree to determine the elements in the target tree whose update flag is logically true. Then, for any element in the target tree whose update flag is logically true, the main processor determines the first coordinate information of the element based on the target coordinate information of the element and sends it to the graphics card, and stores the first coordinate information of the element in the corresponding leaf node in the target tree, and sets the update flag of the element in the leaf node to logically false. In this way, the first coordinate information of the element is calculated when the target tree is traversed for the first time, and subsequently, when the update flag of the element is logically false, the first coordinate information of the element can be directly sent to the graphics card without repeated calculation.

[0143] In some embodiments, if the update flag of the root node of the target tree is logically false, the main processor obtains the first coordinate information of each of one or more elements in the map from the target tree; each time the main processor obtains the first coordinate information of an element, it sends the first coordinate information of the element to the graphics card.

[0144] If the update flag of the root node of the target tree is logically false, it means that no element in the target tree has been updated, and the first coordinate information of all elements in the target tree is valid. Therefore, there is no need to recalculate, and the first coordinate information of all elements stored in the target tree can be directly sent to the graphics card.

[0145] In some embodiments, if the update flag of the root node of the target tree is logically true, then for any element in the target tree whose update flag is logically false, the main processor obtains the first coordinate information of the element from the target tree; the main processor sends the first coordinate information of the element to the graphics card.

[0146] If the update flag of the root node of the target tree is logically true, it means that there are updated elements in the leaf nodes under the root node of the target tree, so it is necessary to traverse the elements in each leaf node of the multiple leaf nodes in the target tree. In this case, if any element whose update flag is logically false is traversed, it means that the target coordinate information of the element has not been updated, that is, the first coordinate information of the element is valid, so the first coordinate information of the element can be sent to the graphics card.

[0147] In some embodiments, each leaf node in the target tree includes target coordinate information and an update flag for each element of at least one element of the same graphic type in the map.

[0148] In this case, when the user needs to display or hide an element of a graphic type in the map, all elements related to the graphic type can be quickly found in the target tree, thereby improving the query efficiency of the elements.

[0149] In some embodiments, the target tree includes a first target tree and a second target tree, the first target tree corresponds to all elements of a first element type in the map, the second target tree corresponds to all elements of a second element type in the map, and the element update frequency of the first element type is higher than or equal to the element update frequency of the second element type.

[0150] The first element type is a type of element with a higher update frequency. For example, the first element type may include one or more of a target point, a target point name, a path point, a label point, a prohibited line, a speed limit zone, etc. The first target tree is used to store elements of the first element type.

[0151] The second element type is a type of element with a lower update frequency. By way of example, the second element type may include one or more of a laser point cloud, a stereoscopic point cloud, a robot icon, etc. The second target tree is used to store elements of the second element type.

[0152] Since the update frequency of the second element type is low, and the branch nodes of the first target tree and the second target tree include an update flag indicating whether there is an updated element in the leaf node under the branch node, the probability that the update flag in the root node of the second target tree is logically false is relatively high. When the main processor needs to obtain the first coordinate information of each of one or more elements in the map, there is a high possibility that the first coordinate information of all elements in the second target tree can be directly sent to the graphics card, so that the amount of calculation when determining the first coordinate information of the element can be reduced.

[0153] Optionally, for the elements in the first target tree and the second target tree, a set of renderers can be loaded respectively for rendering, so that the elements in the first target tree and the second target tree can be rendered independently, thereby optimizing the number of elements that need to be calculated for each frame.

[0154] In some embodiments, the main processor obtains the first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card. The operation may be: the main processor obtains the first coordinate information of each of the one or more elements in the map when receiving the nth frame rendering signal, where n is a positive integer; the main processor stores the first coordinate information of each of the one or more elements in the memory in the graphics card.

[0155] The frame rendering signal can be generated based on the refresh rate of the display screen. The frame rendering signal can be generated by hardware or simulated by software. The frame rendering signal is used to synchronize the workflow of the main processor and the graphics card. In this way, the map display method described in the embodiment of the present application is cyclically called according to the refresh rate of the display screen to continuously display each frame image of the map, so as to realize the display of the map.

[0156] The nth frame rendering signal is a time for the main processor to obtain the first coordinate information of each of the one or more elements in the map.

[0157] After the main processor stores the first coordinate information of each of the one or more elements in the memory in the graphics card, the graphics card can subsequently perform processing based on the first coordinate information stored in the memory.

[0158] It should be noted that in the embodiment of the present application, when the main processor receives the nth frame rendering signal, it can perform the operation of step 401 to determine the first coordinate information of each element in the map and send it to the graphics card. Afterwards, after receiving the n+1th frame rendering signal, the graphics card can perform the operations of steps 402 and 403 to display the map based on the first coordinate information of each element in the map.

[0159] In some embodiments, the main processor generates a unit matrix as a first transformation matrix after the map is displayed; if the main processor detects a target operation acting on the map, the second transformation matrix is ​​determined based on operation information of the target operation; the main processor multiplies the current first transformation matrix by the second transformation matrix to obtain a new first transformation matrix.

[0160] The first transformation matrix is ​​used to indicate the mapping relationship from the screen coordinate system to the map coordinate system.

[0161] The screen coordinate system may be referred to as a canvas coordinate system. For example, the screen coordinate system may be constructed with the upper left corner of the screen as the origin, the right as the positive direction of the horizontal axis, and the downward as the positive direction of the vertical axis. Of course, the screen coordinate system is not limited thereto, and may also be constructed in other ways, which is not limited in the embodiments of the present application.

[0162] The first transformation matrix has an initial value. For example, the initial value of the first transformation matrix may be a value in the following identity matrix:

[0163]

[0164] The target operation is an operation performed by the user on the displayed map. The target operation can adjust the area of ​​the map displayed on the screen.

[0165] For example, the target operation may be one or more of a translation operation, a zoom operation, a rotation operation, etc. on the map. For example, the user's target operation on the displayed map may be two-finger zooming, dragging, two-finger rotation, etc., which is not limited in the present embodiment.

[0166] For example, the operation information of the translation operation may be the translation distance, the operation information of the zoom operation may be the zoom ratio, and the operation information of the rotation operation may be the rotation angle.

[0167] The second transformation matrix is ​​a matrix generated according to the operation information of the target operation, and can indicate the affine transformation caused by the target operation.

[0168] For example, the second transformation matrix determined according to the operation information of the translation operation may be as follows:

[0169]

[0170] Wherein, x is the translation distance of the map in the screen coordinate system in the horizontal direction caused by the translation operation, and y is the translation distance of the map in the screen coordinate system in the vertical direction caused by the translation operation.

[0171] If x is greater than 0, it means translation in the positive direction of the horizontal axis; if x is equal to 0, it means no movement in the horizontal direction; if x is less than 0, it means translation in the negative direction of the horizontal axis. If y is greater than 0, it means translation in the positive direction of the vertical axis; if y is equal to 0, it means no movement in the vertical direction; if y is less than 0, it means translation in the negative direction of the vertical axis.

[0172] For example, the second transformation matrix determined according to the operation information of the scaling operation may be as follows:

[0173]

[0174] Among them, scale x is the scaling ratio of the map in the screen coordinate system in the horizontal direction of the zoom operation, and x is the scaling factor in the horizontal direction; scale y is the scaling ratio of the map in the screen coordinate system in the vertical direction of the zoom operation, and y is the scaling factor in the vertical direction.

[0175] If x is greater than 1, it means enlarging in the horizontal direction; if x is equal to 1, it means not scaling in the horizontal direction; if x is less than 1, it means shrinking in the horizontal direction. If y is greater than 1, it means enlarging in the vertical direction; if y is equal to 1, it means not scaling in the vertical direction; if y is less than 1, it means shrinking in the vertical direction.

[0176] For example, the second transformation matrix determined according to the operation information of the rotation operation may be as follows:

[0177]

[0178] Among them, θ is the rotation angle of the rotation operation; cosθ represents the scaling factor, which is used to adjust the horizontal and vertical components after rotation; -sinθ represents the influence factor in the horizontal direction after rotation, which is used to adjust the horizontal component after rotation; sinθ represents the influence factor in the vertical direction, which is used to adjust the vertical component after rotation.

[0179] If the main processor detects a target operation acting on the map, it means that the user is currently adjusting the displayed map, then the elements in the displayed map may change, and the mapping relationship between the screen coordinate system and the map coordinate system will also change accordingly. In this case, a second transformation matrix for representing the operation information of the target operation can be determined first. Since the second transformation matrix can represent the specific operation performed by the user on the map in the screen coordinate system, the current first transformation matrix can be multiplied by the second transformation matrix to obtain a new first transformation matrix to indicate the mapping relationship between the screen coordinate system and the map coordinate system of the adjusted map.

[0180] Step 402: The graphics card determines second coordinate information of each element according to the first coordinate information of each element in the one or more elements.

[0181] The second coordinate information of an element is used to indicate the position of each of the one or more preset graphics constituting the element in the clipping coordinate system.

[0182] For example, for any circular element among the one or more elements, the graphics processor in the graphics card can execute a vertex shader program for the circular element to convert the first coordinate information of the circular element into the second coordinate information of the circular element. Specifically, the center coordinates in the first coordinate information of the circular element are converted. The second coordinate information of the circular element is used to indicate the position of the target point corresponding to the circular element in the clipping coordinate system. The second coordinate information of the circular element includes the center coordinates and the radius.

[0183] For example, for any straight line element among the one or more elements, the graphics processor in the graphics card can execute a vertex shader program for the straight line element to convert the first coordinate information of the straight line element into the second coordinate information of the straight line element, specifically converting the two endpoint coordinates in the first coordinate information of the straight line element. The second coordinate information of the straight line element is used to indicate the position of the target straight line corresponding to the straight line element in the clipping coordinate system. The second coordinate information of the straight line element may include two endpoint coordinates.

[0184] For example, for any one of the one or more elements, the graphics processor in the graphics card can execute a vertex shader program for the polyline element to convert the first coordinate information of the polyline element into the second coordinate information of the polyline element. Specifically, multiple endpoint coordinates in the first coordinate information of the polyline element are converted. The second coordinate information of the polyline element is used to indicate the position of the target polyline corresponding to the polyline element in the clipping coordinate system. The second coordinate information of the polyline element may include multiple endpoint coordinates.

[0185] For example, for any rectangular element among the one or more elements, the graphics processor in the graphics card can execute a vertex shader program for the rectangular element to convert the first coordinate information of the rectangular element into the second coordinate information of the rectangular element. Specifically, the four vertex coordinates in the first coordinate information of the rectangular element are converted. The second coordinate information of the rectangular element is used to indicate the position of each of the two triangles into which the rectangular element is split in the clipping coordinate system. The second coordinate information of the rectangular element may include four vertex coordinates and two vertex sequences.

[0186] For example, for any polygonal element among the one or more elements, the graphics processor in the graphics card can execute a vertex shader program for the polygonal element to convert the first coordinate information of the polygonal element into the second coordinate information of the polygonal element. Specifically, multiple vertex coordinates in the first coordinate information of the polygonal element are converted. The second coordinate information of the polygonal element is used to indicate the position of each triangle in the multiple triangles into which the polygonal element is split in the clipping coordinate system. The second coordinate information of the polygonal element may include multiple vertex coordinates and multiple vertex sequences.

[0187] For example, for any width line element among the one or more elements, the graphics processor in the graphics card can execute a vertex shader program for the width line element to convert the first coordinate information of the width line element into the second coordinate information of the width line element. Specifically, multiple vertex coordinates in the first coordinate information of the width line element are converted. The second coordinate information of the width line element is used to indicate the position of each triangle in the multiple triangles into which the polygon corresponding to the width line element is split in the clipping coordinate system. The second coordinate information of the width line element may include multiple vertex coordinates and multiple vertex sequences.

[0188] In an embodiment of the present application, when the autonomous mobile robot displays a map, the main processor in the autonomous mobile robot obtains the first coordinate information of each element in one or more elements in the map to be displayed, and then the graphics card in the autonomous mobile robot determines each second coordinate information according to the first coordinate information of each element in the one or more elements, and displays each element accordingly. In this way, when the autonomous mobile robot displays a map, the main processor and the graphics card are each responsible for a part of the processing task, so that the computing pressure of the main processor and the graphics card can be balanced. In addition, since the computing power of the main processor and the graphics card is relatively strong, the efficiency of the main processor in obtaining the first coordinate information of each element is relatively high, and then the efficiency of the graphics card in determining the second coordinate information of each element is also relatively high. In this way, the fluency of the map display can be improved to a certain extent, and the user experience of the autonomous mobile robot can be guaranteed.

[0189] In some embodiments, the operation of step 402 may be: for any one of the one or more elements, the graphics card transforms the first coordinate information of the element according to the first transformation matrix to obtain the screen coordinate information of the element, and transforms the screen coordinate information of the element into the second coordinate information of the element. The screen coordinate information of the element is used to indicate the position of the element in the screen coordinate system.

[0190] For example, for any coordinate in the first coordinate information of the element (hereinafter referred to as the first coordinate), the graphics card can convert the first coordinate into a first matrix, the value of the first row of the first matrix is ​​the horizontal coordinate value in the first coordinate, the value of the second row is the vertical coordinate value in the first coordinate, and the value of the third row is 1; the first transformation matrix is ​​multiplied by the first matrix to obtain the second matrix; the value of the first row and the value of the second row in the second matrix are used as the horizontal coordinate value and the vertical coordinate value, respectively, to obtain the second coordinate, which is the coordinate in the screen coordinate information of the element.

[0191] For example, the graphics card converts the coordinates (ie, the second coordinates) in the screen coordinate information of the element into the coordinates in the second coordinate information of the element by using the following formula:

[0192] xClip=-((x÷canvasWidth)×2-1);

[0193] yClip=-((y÷canvasHeight)×2-1);

[0194] Among them, xClip is the horizontal coordinate value of the coordinate in the second coordinate information of the element, x is the horizontal coordinate value of the second coordinate, canvasWidth is the screen width value, yClip is the vertical coordinate value of the coordinate in the second coordinate information of the element, y is the vertical coordinate value of the second coordinate, and canvasHeight is the screen height value.

[0195] In some embodiments, the operation of step 402 may be: the graphics processor in the graphics card reads the first coordinate information of each of the one or more elements from the memory in the graphics card when receiving the n+1th frame rendering signal; the graphics processor determines the second coordinate information of each element based on the first coordinate information of each of the one or more elements.

[0196] The first coordinate information of each of the one or more elements stored in the memory in the graphics card is sent by the main processor after receiving the n-th frame rendering signal.

[0197] Step 403: The graphics card displays each element according to the second coordinate information of each element in the one or more elements.

[0198] Since the second coordinate information of an element is used to indicate the position of each of the one or more preset graphics constituting the element in the clipping coordinate system, the graphics card can draw and display each of the one or more preset graphics accordingly. After the drawing of each of the one or more elements is completed, a frame image of the map is obtained, and then this frame image can be sent to the display screen for display.

[0199] For example, for any circular element among the one or more elements, the graphics processor in the graphics card can execute a fragment shader program for the circular element to draw pixels whose center coordinates are smaller than the radius as target points according to the center coordinates and coordinates in the second coordinate information of the circular element. The graphics processor in the graphics card can then display the drawn target points on a display screen to display the circular element.

[0200] For example, for any straight line element among the one or more elements, the graphics processor in the graphics card can execute a fragment shader program for the straight line element to draw a straight line with a width of 1 pixel as a target straight line according to the two endpoint coordinates in the second coordinate information of the straight line element. Then, the graphics processor in the graphics card can display the drawn target straight line through the display screen to realize the display of the straight line element.

[0201] For example, for any one of the one or more elements, the graphics processor in the graphics card can execute a fragment shader program for the polyline element to draw a polyline with a width of 1 pixel as a target polyline according to the multiple endpoint coordinates in the second coordinate information of the polyline element. Then, the graphics processor in the graphics card can display the drawn target polyline on the display screen to display the polyline element.

[0202] For example, for any rectangular element among the one or more elements, the graphics processor in the graphics card can execute a fragment shader program for the rectangular element to draw two triangles corresponding to the two vertex sequences according to the four vertex coordinates and the two vertex sequences in the second coordinate information of the rectangular element. Then, the graphics processor in the graphics card can display the two drawn triangles through the display screen to realize the display of the rectangular element.

[0203] For example, for any polygonal element among the one or more elements, the graphics processor in the graphics card can execute a fragment shader program for the polygonal element to draw a plurality of triangles corresponding to the plurality of vertex sequences according to the plurality of vertex coordinates and the plurality of vertex sequences in the second coordinate information of the polygonal element. Then, the graphics processor in the graphics card can display the drawn plurality of triangles through a display screen to realize the display of the polygonal element.

[0204] For example, for any one width line element among the one or more elements, the graphics processor in the graphics card can execute a fragment shader program for the width line element to draw a plurality of triangles corresponding to the plurality of vertex sequences according to the plurality of vertex coordinates and the plurality of vertex sequences in the second coordinate information of the width line element. Then, the graphics processor in the graphics card can display the drawn plurality of triangles through a display screen to realize the display of the width line element.

[0205] In some embodiments, after the graphics card displays each element according to the second coordinate information of each element of the one or more elements, the graphics processor in the graphics card can delete the first coordinate information of all elements stored in the memory in the graphics card so that the memory can continue to store the next frame of image data.

[0206] In an embodiment of the present application, the main processor obtains the first coordinate information of each of the one or more elements in the map and sends it to the graphics card. The first coordinate information of an element is used to indicate the position of each of the one or more preset graphics constituting the element in the map coordinate system of the map. The graphics card determines its second coordinate information according to the first coordinate information of each of the one or more elements, and the second coordinate information of an element is used to indicate the position of each of the one or more preset graphics constituting the element in the clipping coordinate system. Finally, the graphics card displays each element according to the second coordinate information of each of the one or more elements. In this way, when the autonomous mobile robot displays the map, the main processor and the graphics card are each responsible for a part of the processing tasks, so that the computing pressure of the main processor and the graphics card can be balanced. In addition, since the computing power of the main processor and the graphics card is relatively strong, the efficiency of the main processor in obtaining the first coordinate information of each element is relatively high, and then the efficiency of the graphics card in determining the second coordinate information of each element is also relatively high. In this way, the smoothness of the map display can be improved to a certain extent, and the user experience of the autonomous mobile robot can be guaranteed.

[0207] Fig.10 This is a schematic diagram of the structure of an autonomous mobile robot provided in an embodiment of the present application. Fig.10 As shown, the autonomous mobile robot 10 includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program 102, the steps in the map display method in the above embodiment are implemented.

[0208] The processor 100 may include a CPU and a graphics processing unit (GPU). The processor 100 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0209] In some embodiments, the memory 101 may be an internal storage unit of the autonomous mobile robot 10, such as a hard disk or memory of the autonomous mobile robot 10. In other embodiments, the memory 101 may also be an external storage device of the autonomous mobile robot 10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the autonomous mobile robot 10. Further, the memory 101 may also include both an internal storage unit of the autonomous mobile robot 10 and an external storage device. The memory 101 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0210] An embodiment of the present application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and when the processor executes the computer program, the steps in any of the above-mentioned method embodiments are implemented.

[0211] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0212] An embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the steps in the above-mentioned method embodiments.

[0213] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above method embodiments, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, it can be a non-transient storage medium.

[0214] It should be understood that all or part of the steps to implement the above embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-mentioned computer readable storage medium.

[0215] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0216] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0217] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0218] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0219] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0220] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A map display method, characterized in that: Applied to an autonomous mobile robot, the autonomous mobile robot includes a main processor and a graphics card, and the method includes: The main processor acquires first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card, wherein the first coordinate information of the element is used to indicate the position of each of the one or more preset graphics constituting the element in the map coordinate system of the map; The graphics card determines the second coordinate information of each element according to the first coordinate information of each element in the one or more elements, wherein the second coordinate information of the element is used to indicate the position of each preset graphic in the one or more preset graphics constituting the element in the clipping coordinate system; The graphics card displays each element of the one or more elements according to the second coordinate information of each element.

2. The method according to claim 1, characterized in that The map corresponds to a target tree, the target tree includes at least one branch node and multiple leaf nodes, each of the multiple leaf nodes includes target coordinate information and an update flag bit for each element of at least one element in the map, the target coordinate information of the element is used to indicate the position of the element in the map coordinate system, the update flag bit of the element is used to indicate whether the element is updated, each of the at least one branch node includes an update flag bit, and the update flag bit in the branch node is used to indicate whether there is an updated element in the leaf node under the branch node; The main processor acquires first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card, including: If the update flag of the root node of the target tree is logically true, then for any element in the target tree whose update flag is logically true, the main processor obtains the target coordinate information of the element from the target tree; The main processor determines the first coordinate information of the element according to the target coordinate information of the element; The main processor sends the first coordinate information of the element to the graphics card, stores the first coordinate information of the element in a corresponding leaf node in the target tree, and sets the update flag of the element in the leaf node to logic false; And / or, the main processor obtains first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card, including: If the update flag of the root node of the target tree is logically true, then for any element in the target tree whose update flag is logically false, the main processor obtains the first coordinate information of the element from the target tree; The main processor sends the first coordinate information of the element to the graphics card.

3. The method according to claim 2, characterized in that The main processor acquires first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card, including: If the update flag of the root node of the target tree is logically false, the main processor obtains first coordinate information of each of the one or more elements in the map from the target tree; The main processor sends the first coordinate information of an element to the graphics card every time it obtains the first coordinate information of the element.

4. The method according to claim 2, characterized in that The method further comprises: For any element in the map, the main processor sets an update flag bit of the element in the target tree to logic true when the element is updated; and / or, For a newly added element in the map, the main processor stores the target coordinate information and update flag of the element in a leaf node in the target tree, and sets the update flag of the element to logical true.

5. The method according to claim 2, characterized in that Each leaf node in the target tree includes target coordinate information and an update flag bit for each element in at least one element of the same graphic type in the map; And / or, the target tree includes a first target tree and a second target tree, the first target tree corresponds to all elements of a first element type in the map, the second target tree corresponds to all elements of a second element type in the map, and the element update frequency of the first element type is higher than or higher than the element update frequency of the second element type.

6. The method according to claim 1, characterized in that The graphics card determines the second coordinate information of each element according to the first coordinate information of each element of the one or more elements, including: For any one of the one or more elements, the graphics card converts first coordinate information of the element according to a first transformation matrix to obtain screen coordinate information of the element, wherein the first transformation matrix is ​​used to indicate a mapping relationship from a screen coordinate system to the map coordinate system, and the screen coordinate information of the element is used to indicate a position of the element in the screen coordinate system; The graphics card converts the screen coordinate information of the element into second coordinate information of the element; And / or, the method further comprises: The main processor generates a unit matrix as the first transformation matrix after the map is displayed; If the main processor detects a target operation acting on the map, determining a second transformation matrix according to operation information of the target operation; The main processor multiplies the current first transformation matrix by the second transformation matrix to obtain a new first transformation matrix.

7. The method according to any one of claims 1 to 6, characterized in that: The main processor acquires first coordinate information of each of one or more elements in the map, and sends the first coordinate information of each of the one or more elements to the graphics card, including: The main processor obtains first coordinate information of each of the one or more elements in the map when receiving the nth frame rendering signal, where n is a positive integer; The main processor stores the first coordinate information of each element of the one or more elements in a memory in the graphics card; The graphics card determines the second coordinate information of each element according to the first coordinate information of each element of the one or more elements, including: The graphics processor in the graphics card reads the first coordinate information of each of the one or more elements from the memory in the graphics card when receiving the n+1th frame rendering signal; The graphics processor determines second coordinate information of each element according to the first coordinate information of each element of the one or more elements.

8. The method according to claim 7, characterized in that After the graphics card displays each element according to the second coordinate information of each element of the one or more elements, the further comprising: The graphics processor deletes the first coordinate information of all elements stored in the memory in the graphics card.

9. An autonomous mobile robot, characterized in that: The autonomous mobile robot comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 8 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images