Processing method and device for three-dimensional map of cleaning robot, equipment and medium
By providing cleaning robots with a variety of three-dimensional map display methods, the problem of single three-dimensional map display method in the prior art is solved, and a richer and more flexible user experience is achieved.
Patent Information
- Application Number
- CN202411145182.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-22
AI Technical Summary
The existing three-dimensional map display method of cleaning robot operating environment is single and cannot meet the diverse viewing needs of users.
Provides a three-dimensional map processing method for cleaning robots, including obtaining three-dimensional maps, receiving and processing different display types, such as overlay, segmentation, blocking, color discoloration, dynamics, job paths and three-dimensional element superposition displays, to meet the diverse displays of users need.
Through different display types, three-dimensional maps can be processed and displayed according to users' needs, improving the user experience and meeting diverse viewing needs.
Smart Images

Figure CN119006761A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robots, and in particular to a method, device, equipment and medium for processing a three-dimensional map of a cleaning robot. Background Art
[0002] With the development of computer technology and artificial intelligence technology, various cleaning robots with intelligent systems have emerged, such as sweeping machines, mopping robots, vacuum cleaners, lawn mowers, etc. These robots can automatically move in a certain area and perform operations such as cleaning or clearing without user operation. At present, the three-dimensional map display method of the cleaning robot's working environment is single and cannot meet the viewing needs of users. Summary of the invention
[0003] A series of simplified concepts are introduced in the Summary of the Invention, which will be further described in detail in the Detailed Description of the Invention. The Summary of the Invention does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.
[0004] In a first aspect, an embodiment of the present invention provides a method for processing a three-dimensional map of a cleaning robot, comprising:
[0005] Acquire a three-dimensional map of the working environment of the cleaning robot;
[0006] Receiving at least one of the display types of the three-dimensional map;
[0007] Based on at least one of the display types, processing the three-dimensional map;
[0008] Display the processed 3D map.
[0009] Optionally, at least one of the display types includes overlay display, and the processing of the three-dimensional map based on the display type includes:
[0010] Obtaining a two-dimensional map corresponding to the three-dimensional map;
[0011] The three-dimensional map is superimposed on the two-dimensional map to obtain a superimposed map.
[0012] Optionally, after superimposing the three-dimensional map on the two-dimensional map to obtain the superimposed map, the method further comprises:
[0013] In response to the display mode selected by the user, the superimposed map is displayed according to the display mode.
[0014] Optionally, at least one of the display types includes segmented display, and the processing of the three-dimensional map based on the display type includes:
[0015] The objects on the three-dimensional map are segmented.
[0016] Optionally, at least one of the display types includes block display, and the processing of the three-dimensional map based on the display type includes:
[0017] Acquire the clustering information, where the clustering information is one of the preset clustering information or is customized by a user;
[0018] Clustering the geometric primitives according to the clustering information to obtain block information;
[0019] The block information is added to the three-dimensional map.
[0020] Optionally, the clustering information is one of preset clustering information, or is customized by a user.
[0021] Optionally, at least one of the display types includes color changing display, and the processing of the three-dimensional map based on the display type includes:
[0022] Obtaining transparency data and color characteristics of at least one geometric primitive or a point cloud constituting the geometric primitive;
[0023] Corresponding transparency data is superimposed on the color feature of at least one of the geometric primitives or the point cloud constituting the geometric primitive.
[0024] Optionally, at least one of the display types includes three-dimensional photo display, and the processing of the three-dimensional map based on the display type includes:
[0025] In response to the user's selection operation, a three-dimensional photo corresponding to the portion selected by the user on the three-dimensional map is found.
[0026] Optionally, at least one of the display types includes dynamic display, and the processing of the three-dimensional map based on the display type includes:
[0027] The three-dimensional map is dynamically processed.
[0028] Optionally, the dynamically processing the three-dimensional map includes:
[0029] identifying dynamic objects on the three-dimensional map;
[0030] Acquiring real-time status information of the dynamic object;
[0031] Based on the real-time status information, the dynamic object is dynamically displayed.
[0032] Optionally, the dynamically processing the three-dimensional map includes:
[0033] Each geometric primitive of the three-dimensional map is disturbed within a corresponding disturbance range.
[0034] Optionally, the dynamically processing the three-dimensional map includes:
[0035] Obtaining the lightmap and particle dynamic effects corresponding to the three-dimensional map;
[0036] The light map and the particle dynamic effect are rendered onto the three-dimensional map.
[0037] Optionally, at least one of the display types includes a work path display, and the processing of the three-dimensional map based on the display type includes:
[0038] Obtaining an operation path, model or location identifier of the cleaning robot;
[0039] The operation path, model or location mark is added to the three-dimensional map.
[0040] Optionally, displaying the processed three-dimensional map includes:
[0041] When an instruction for modifying at least one of the geometric primitives in the three-dimensional map is received, at least one of the geometric primitives is split and displayed so that the user can modify the at least one of the geometric primitives.
[0042] Optionally, after displaying the processed three-dimensional map, the step further includes:
[0043] When a semantic information change instruction of the three-dimensional map is received, the semantic information of the three-dimensional map is changed so that the cleaning robot regenerates a cleaning strategy according to the new semantic information.
[0044] Optionally, at least one of the display types includes three-dimensional element overlay display, and the processing of the three-dimensional map based on the display type includes:
[0045] Acquire the three-dimensional elements corresponding to the three-dimensional map and the two-dimensional map constructed by the cleaning robot;
[0046] The three-dimensional elements are superimposed on the two-dimensional map to obtain a superimposed map.
[0047] Optionally, after obtaining the three-dimensional elements corresponding to the three-dimensional map, the step further includes:
[0048] When an instruction to modify at least a portion of the three-dimensional map is received, the at least a portion of the three-dimensional map is enlarged to allow the user to modify the at least a portion of the three-dimensional map.
[0049] In a second aspect, an embodiment of the present invention provides a three-dimensional map processing device of a cleaning robot, comprising:
[0050] An acquisition module, used to acquire a three-dimensional map of the working environment of the cleaning robot;
[0051] A receiving module, configured to receive at least one display type of the three-dimensional map;
[0052] A processing module, configured to process the three-dimensional map based on at least one of the display types;
[0053] The display module is used to display the processed three-dimensional map.
[0054] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the above-mentioned method for processing a three-dimensional map of a cleaning robot.
[0055] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when called and executed by a processor, implement the steps of the above-mentioned method for processing a three-dimensional map of a cleaning robot.
[0056] According to an embodiment of the present invention, a method, device, equipment and medium for processing a three-dimensional map of a cleaning robot are provided. The method obtains a three-dimensional map of the working environment of the cleaning robot; receives at least one display type of the three-dimensional map; processes the three-dimensional map based on at least one display type; and displays the processed three-dimensional map. Thus, the three-dimensional map of the working environment of the cleaning robot can be displayed through different display types to meet the viewing needs of users. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] The following drawings of the present invention are used as part of the embodiments of the present invention for understanding the present invention. The embodiments of the present invention are shown in the drawings and the description thereof is used to explain the principle of the present invention.
[0058] In the attached figure:
[0059] Figure 1 A schematic diagram of a system structure according to an optional embodiment of the present invention;
[0060] Figure 2 is a three-dimensional schematic diagram of a cleaning robot according to an optional embodiment of the present invention;
[0061] Figure 3 A bottom view of a cleaning robot according to an optional embodiment of the present invention;
[0062] Figure 4 is a perspective view of a wet cleaning system according to an optional embodiment of the present invention;
[0063] Figure 5 It is a flowchart of a method for processing a three-dimensional map of a cleaning robot according to an optional embodiment of the present invention;
[0064] Figure 6 is a flowchart of step S503 according to an optional embodiment of the present invention;
[0065] Figure 7 is a flowchart of step S503 according to another optional embodiment of the present invention;
[0066] Figure 8 is a flowchart of step S503 according to another optional embodiment of the present invention;
[0067] Fig. 9 A flowchart of dynamically processing a three-dimensional map according to an optional embodiment of the present invention;
[0068] Fig.10 A flowchart of dynamically processing a three-dimensional map according to another optional embodiment of the present invention;
[0069] Fig.11 is a flowchart of step S503 according to another optional embodiment of the present invention;
[0070] Fig.12 is a flowchart of step S503 according to another optional embodiment of the present invention;
[0071] Fig.13 It is a structural diagram of a three-dimensional map processing device of a cleaning robot according to an optional embodiment of the present invention. DETAILED DESCRIPTION
[0072] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.
[0073] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of the features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0074] Now, exemplary embodiments according to the present invention will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments set forth herein. It should be understood that these embodiments are provided to make the disclosure of the present invention thorough and complete, and to fully convey the concept of these exemplary embodiments to those of ordinary skill in the art.
[0075] Figure 1 An exemplary system architecture of the three-dimensional map construction method of the cleaning robot disclosed in the present invention is shown. Figure 1 As shown, the system architecture may include terminal devices, networks, servers, and cleaning robots. The terminal device may be a variety of electronic devices with a display screen and supporting input and output, including but not limited to smartphones, tablet computers, laptops, desktop computers, AR headsets, and the like. The network is a medium used to provide a communication link between the terminal device and the server. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, and the like. The server may be a server or server cluster that provides various services, such as a background processing server that provides support for map modeling of data sent by a sweeping robot to perceive the working environment.
[0076] The user can use the terminal device to interact with the server through the network to receive or send data, etc. For example, the server sends the constructed three-dimensional map to the mobile phone through the network, and the user can view the three-dimensional map through the mobile phone.
[0077] It should be understood that Figure 1 The number of terminal devices 2, networks 4 and servers 3 in the embodiment is only illustrative. According to the implementation requirements, there may be any number of terminal devices 2, networks 4 and servers 3. The cleaning robot may be a sweeping robot, a mopping robot, a floor polishing robot or a weeding robot. For ease of description, the present embodiment takes the sweeping robot 10 as an example to describe the technical solution of the present disclosure.
[0078] like Figure 2 and Figure 3As shown, the cleaning robot 10 may include a machine body 110, a sensing module 120, a controller, a driving module, a cleaning system 150, an energy system and a human-machine interaction module 130. Figure 1 As shown, the machine body 110 includes a forward portion 111 and a rearward portion 112, and has an approximately circular shape (both the front and the rear are circular), and may also have other shapes, including but not limited to an approximately D-shape with a front and rear circle and a rectangular or square shape with a front and rear.
[0079] like Figure 2 As shown, the perception module 120 includes a position determination device 121 located on the machine body 110, a collision sensor disposed on a front collision structure 122 of the forward portion 111 of the machine body 110, a short-distance sensor (wall sensor) located on the side of the machine, a carpet detection device disposed at the lower portion of the machine body 110, and a magnetometer, accelerometer, gyroscope, odometer and other sensing devices disposed inside the machine body 110, which are used to provide the controller with various position information and motion state information of the machine. The position determination device 121 includes but is not limited to a camera and a laser distance measuring device (LDS, full name Laser Distance Sensor). In some preferred implementations, the position determination device 121 (such as a camera, a laser sensor) is located on the front side of the body 110, that is, at the very front end of the forward portion 111, so as to be able to more accurately sense the environment in front of the cleaning robot and achieve precise positioning.
[0080] like Figure 2 As shown, the forward portion 111 of the machine body 110 can carry a front collision structure 122. During the cleaning process, when the driving wheel module 141 propels the cleaning robot 10 to walk on the ground, the front collision structure 122 detects one or more events in the driving path of the cleaning robot 10 via a sensor system arranged thereon, such as a collision sensor or a proximity sensor (infrared sensor). The cleaning robot 10 can control the driving module to enable the cleaning robot 10 to respond to the event, such as performing an obstacle avoidance operation away from the obstacle, through the event detected by the front collision structure 122, such as an obstacle or a wall.
[0081] The controller is arranged on the circuit board in the machine body 110, and includes a computing processor, such as a central processing unit, an application processor, which communicates with a non-temporary memory, such as a hard disk, a flash memory, and a random access memory. The application processor uses a positioning algorithm, such as SLAM (Simultaneous Localization And Mapping), to draw a real-time map of the environment where the cleaning robot 10 is located based on the obstacle information fed back by the laser ranging device. In addition, the distance information and speed information fed back by the sensors, carpet detection devices, magnetometers, accelerometers, gyroscopes, odometers and other sensor devices arranged on the front collision structure 122 are combined to comprehensively judge the current working state and position of the cleaning robot 10, as well as the current posture of the cleaning robot 10, such as passing the threshold, getting on the carpet, the dust box is full, being picked up, etc., and specific next action strategies are given for different situations, so that the cleaning robot 10 has better cleaning performance and user experience.
[0082] like Figure 3 As shown, the driving module can manipulate the machine body 110 to travel across the ground based on a driving command with distance and angle information. The driving module includes a main driving wheel module, which can control the left wheel 140 and the right wheel 141. In order to more accurately control the movement of the machine, it is preferred that the main driving wheel module includes a left driving wheel module and a right driving wheel module respectively. The left and right driving wheel modules are arranged along the transverse axis defined by the machine body 110. In order for the cleaning robot 10 to be able to move more stably or have stronger movement ability on the ground, the cleaning robot 10 may include one or more driven wheels 142, and the driven wheels 142 include but are not limited to universal wheels. The main driving wheel module includes a driving motor and a control circuit for controlling the driving motor, and the main driving wheel module can also be connected to a circuit and an odometer for measuring the driving current. And the left wheel 140 and the right wheel 141 may have a biased drop suspension system, which is fastened in a movable manner, for example, attached to the machine body 110 in a rotatable manner, and receives a spring bias biased downward and away from the machine body 110. The spring bias allows the drive wheels to maintain contact and traction with the ground with a certain ground force, while the cleaning elements of the cleaning robot 10 also contact the ground with a certain pressure.
[0083] The energy system includes rechargeable batteries, such as nickel-metal hydride batteries and lithium batteries. The rechargeable batteries can be connected to a charging control circuit, a battery pack charging temperature detection circuit, and a battery undervoltage monitoring circuit, which are then connected to a single-chip microcomputer control circuit. The host is charged by connecting to a charging pile through a charging electrode 160 disposed on the side or below the fuselage.
[0084] The human-computer interaction module 130 includes buttons on the host panel, which are used by the user to select functions; it may also include a display screen and / or indicator lights and / or speakers, which display the current mode of the machine or the function selection items to the user; it may also include a mobile client program. For the path navigation type automatic cleaning robot 10, the mobile client can show the user a map of the environment where the equipment is located, as well as the location of the machine, and can provide the user with more abundant and humanized function items. Specifically, the cleaning robot has multiple modes, such as working mode, self-cleaning mode, etc. Among them, the working mode refers to the mode in which the cleaning robot performs automatic cleaning operations, and the self-cleaning mode refers to the mode in which the cleaning robot removes dirt from the roller brush and side brush 152 on the base, and automatically collects dirt, and / or automatically washes and dries the mop.
[0085] The cleaning system 150 may be a dry cleaning system 151 and / or a wet cleaning system 153 .
[0086] like Figure 3 As shown, the dry cleaning system 151 provided in the embodiment of the present disclosure may include a roller brush, a dust box, a fan, and an air outlet. The roller brush that has a certain interference with the ground sweeps up the garbage on the ground and rolls it to the front of the dust suction port between the roller brush and the dust box, and then the suction gas generated by the fan and passing through the dust box is sucked into the dust box. The dry cleaning system 151 may also include a side brush 152 with a rotating shaft, and the rotating shaft is at a certain angle relative to the ground to move debris into the roller brush area of the cleaning system 150.
[0087] like Figure 3 and Figure 4 As shown, the wet cleaning system 153 provided in the embodiment of the present disclosure may include: a cleaning head 1531, a driving unit 1532, a water supply mechanism, a liquid storage tank, etc. Among them, the cleaning head 1531 can be arranged below the liquid storage tank, and the cleaning liquid inside the liquid storage tank is transmitted to the cleaning head 1531 through the water supply mechanism, so that the cleaning head 1531 performs wet cleaning on the surface to be cleaned. In other embodiments of the present disclosure, the cleaning liquid inside the liquid storage tank can also be directly sprayed onto the surface to be cleaned, and the cleaning head 1531 cleans the surface by evenly applying the cleaning liquid.
[0088] The cleaning head 1531 is used to clean the surface to be cleaned, and the driving unit 1532 is used to drive the cleaning head 1531 to reciprocate along the target surface, which is a part of the surface to be cleaned. The cleaning head 1531 reciprocates along the surface to be cleaned, and a mop is provided on the contact surface between the cleaning head 1531 and the surface to be cleaned. The mop of the cleaning head 1531 is driven by the driving unit 1532 to reciprocate and generate high-frequency friction with the surface to be cleaned, thereby removing stains on the surface to be cleaned; or the mop can be arranged to float and always keep in contact with the cleaning surface during the cleaning process, without the need for the driving unit 1532 to drive its reciprocating motion.
[0089] like Figure 4 As shown, the driving unit 1532 may also include a driving platform 1533 and a supporting platform 1534. The driving platform 1533 is connected to the bottom surface of the machine body 110 for providing driving force. The supporting platform 1534 is detachably connected to the driving platform 1533 for supporting the cleaning head 1531 and can be lifted and lowered under the drive of the driving platform 1533.
[0090] The wet cleaning system 153 can be connected to the machine body 110 through an active lifting module. When the wet cleaning system 153 is not temporarily involved in the work, for example, the cleaning robot 10 stops at the base station to clean the cleaning head 1531 of the wet cleaning system 153 and fill the liquid storage tank with water; or when encountering a surface to be cleaned that cannot be cleaned by the wet cleaning system 153, the wet cleaning system 153 is raised through the active lifting module.
[0091] Figure 5 FIG. 1 is a flow chart of a method for processing a three-dimensional map of a cleaning robot according to an exemplary embodiment. Figure 5 The method shown can be applied to the server side of the above system, or to the terminal device of the above system. Specifically, the method includes the following steps:
[0092] Step S501: Acquire a three-dimensional map of the working environment of the cleaning robot.
[0093] The process of constructing a three-dimensional map is as follows: first, a dense point cloud of the working space is obtained through the sensing system of the cleaning robot, and then a 3D mesh model of the space is generated based on the point cloud. Finally, based on the generated mesh model, model mapping is completed, that is, the texture is mapped to the 3DMesh model of the space through coordinates to obtain a three-dimensional map.
[0094] The construction of the three-dimensional map may be performed in advance, and the obtained map may be stored in a pre-terminal device or a terminal device, etc. In addition, the construction of the three-dimensional map may be performed simultaneously with the subsequent construction of the two-dimensional map, which is not limited in this embodiment.
[0095] Step S502: Receive at least one display type of a three-dimensional map.
[0096] The selection of the display type can be freely selected by the user on the terminal device according to his own needs. The display type can be one or more, and this embodiment does not make a strict limitation.
[0097] Step S503: Process the three-dimensional map based on at least one display type.
[0098] Step S504: display the processed three-dimensional map.
[0099] In this embodiment, the three-dimensional map of the cleaning robot's working environment can be displayed through different display types to meet the viewing needs of users.
[0100] The following is a detailed description of the display types and the process of processing three-dimensional maps.
[0101] Specifically, at least one display type includes an overlay display, such as Figure 6 As shown, step S503 specifically includes:
[0102] Step S601: Obtain a two-dimensional map corresponding to a three-dimensional map.
[0103] The cleaning robot can construct a two-dimensional map in real time during movement. For example, the sweeping robot measures the distance between itself and various obstacles in the working area through the installed LDS during the cleaning process, thereby drawing a real-time map of the area. A variety of Lidar simultaneous localization and mapping (SLAM) methods can be used to draw real-time maps, such as the HectorSLAM algorithm based on optimization (solving the least squares problem) and the Gmapping algorithm based on particle filtering. Step S602: Superimpose the three-dimensional map on the two-dimensional map to obtain the superimposed map.
[0104] Adjust the sizes of the two-dimensional map and the three-dimensional map so that they have the same size, then align the three-dimensional map with the two-dimensional map, and then overlay the three-dimensional map on the two-dimensional map. In this way, each object originally displayed in a plane on the two-dimensional map will have a corresponding three-dimensional image that is the same as the entity, so that the user can understand the working environment of the cleaning robot more intuitively and realistically.
[0105] Furthermore, in the above embodiment, after step S602, the following steps are further included:
[0106] In response to the display mode selected by the user, the superimposed map is displayed according to the display mode.
[0107] The display modes include but are not limited to displays in different colors and displays in different forms, such as surface mode, wireframe mode, and point mode.
[0108] In specific applications, users can select the desired display mode on the terminal device so that the map is displayed according to the selected display mode, which not only improves the flexibility of map display, but also meets user needs and improves user experience.
[0109] Furthermore, each display mode can be switched freely by the user, that is, it is possible to switch from one display mode to any other display mode. For example, the user can switch from the facet mode to the wireframe mode, so that the map can be switched from facet display to wireframe display.
[0110] Specifically, at least one display type includes segmented display, and step S503 specifically includes:
[0111] Segment objects on a 3D map.
[0112] Object segmentation refers to marking different objects on a three-dimensional map separately through image segmentation, so that users can better understand the distribution of each object.
[0113] This embodiment does not strictly limit the method of object segmentation. Exemplarily, in one implementation method, the depth information of the scene is used to roughly segment objects with larger planes that may exist in the scene (such as tables, computer screens, walls, floors, washing machines, etc.). For objects with discontinuous planes but that may belong to the same object (such as chairs and curved planes), their corresponding RGB images are identified and segmented by using a recognizer trained by deep learning and other methods.
[0114] In another implementation, semantic segmentation is used. Specifically, semantic segmentation is performed on the three-dimensional map through a trained classification model (such as a fully connected convolutional neural network), and / or semantic segmentation is performed on the three-dimensional map through an image segmentation algorithm.
[0115] Specifically, at least one display type includes block display, such as Figure 7 As shown, step S503 specifically includes:
[0116] Step S701: Obtain clustering information.
[0117] The clustering information includes, but is not limited to, geometric information of geometric primitives, surface normal information, or topological connection information.
[0118] Specifically, the clustering information is one of the preset clustering information or is customized by the user.
[0119] The clustering information may be one of the preset clustering information, and the user may select the preset clustering information according to his / her needs, or the clustering information may be customized by the user, that is, the user may create the clustering information by himself / herself according to the needs.
[0120] Step S702: clustering geometric primitives according to clustering information to obtain block information.
[0121] Based on the clustering information, we can obtain block information based on distance, surface normal or topological relationship.
[0122] Step S703: adding block information on the three-dimensional map.
[0123] The block information enables the user to understand the layout of the three-dimensional map. For example, taking the block information based on distance as an example, the area where the object is 0.5m away from the sweeping robot, the area where the object is 1m away from the sweeping robot, and the area where the object is 1.5m away from the sweeping robot are displayed on the three-dimensional map.
[0124] In particular, at least one display type includes a color changing display, such as Figure 8 As shown, step S503 specifically includes:
[0125] Step S801: Obtain transparency data and color features of at least one geometric primitive or a point cloud constituting the geometric primitive.
[0126] The geometric primitives are composed of point clouds, which can be obtained when the cleaning robot detects the surrounding environment.
[0127] The transparency data of the geometric primitive or the transparency data of the point cloud constituting the geometric primitive can be set by the user according to the requirements. The color feature can be determined by the texture feature of the geometric primitive.
[0128] Step S802: superimposing corresponding transparency data on the color feature of at least one geometric primitive or a point cloud constituting the geometric primitive.
[0129] Specifically, the method of superimposing transparency can be implemented by using a weighted or alpha method.
[0130] Taking alpha overlay as an example, transparency data includes alpha value, which ranges from 0 to 1. If the alpha value is 0, it means it is completely transparent, and 1 means it is completely opaque, that is, the larger the alpha value, the lower the transparency. The original color features of geometric primitives or point clouds, namely R (red), G (green), and B (blue), are obtained, that is, the three primary color pixel parameters, and then the RGB parameters of the original image and the alpha value embedded in the information to be transmitted are merged together to regenerate a new map.
[0131] In this embodiment, the user can change the coloring of the three-dimensional map according to his or her needs, thereby making the display of the three-dimensional map more diverse and improving the user experience.
[0132] Specifically, at least one display type includes a three-dimensional photo display, and step S503 specifically includes:
[0133] In response to the user's selection operation, a three-dimensional photo corresponding to the portion selected by the user on the three-dimensional map is displayed.
[0134] In specific applications, users can select the part of interest on the 3D map to display the corresponding 3D photos according to their viewing needs. In this way, users can only see the real pictures they are interested in, so that users can more intuitively understand the real scenes of the area of interest.
[0135] Furthermore, the perspective of the three-dimensional photo may be the same as or different from the perspective of the area of interest on the three-dimensional map, or may be displayed from multiple perspectives, so that the user can have a more comprehensive understanding.
[0136] The user's selection operation may be to click on the portion of interest on the three-dimensional map, or to delineate the portion of interest on the three-dimensional map. Of course, other forms may also be used, which are not strictly limited in this embodiment.
[0137] Specifically, at least one display type includes dynamic display, and step S503 specifically includes:
[0138] Dynamically process three-dimensional maps.
[0139] Through the dynamic display of the three-dimensional map, the three-dimensional map can be made more vivid and more realistically reflect the working environment of the cleaning robot.
[0140] In specific applications, the dynamic display of the three-dimensional map can be implemented in different ways. Specifically, in some possible implementations, such as Fig. 9 As shown, the dynamic processing of the three-dimensional map includes:
[0141] Step S901: Identify dynamic objects on the three-dimensional map.
[0142] Dynamic objects can be identified by determining whether the objects on the three-dimensional map have changed in position.
[0143] Step S902: Acquire real-time status information of dynamic objects.
[0144] The real-time status information of the dynamic object includes but is not limited to the current position information and current posture information of the dynamic object, etc. The real-time status information of the dynamic object can be obtained by sensing of the sensing system of the cleaning robot.
[0145] Step S903: Dynamically display the dynamic object based on the real-time status information.
[0146] The previous state information is replaced with the current state information, that is, the real-time state information, so that the dynamic object is presented in the three-dimensional map based on the real-time state information, so as to realize the dynamic display of the dynamic object.
[0147] In some other possible implementations, the dynamic processing of the three-dimensional map includes:
[0148] Each geometric primitive of the three-dimensional map is perturbed within a corresponding perturbation range.
[0149] The disturbance range of each geometric primitive can be set by the staff. Generally speaking, the disturbance range should not be too large to avoid deformation of the geometric primitive.
[0150] In some other possible implementations, such as Fig.10 As shown, the dynamic processing of the three-dimensional map includes:
[0151] Step S1001: Obtain the light map and particle dynamic effects corresponding to the three-dimensional map.
[0152] In a specific application, the type of lighting effect corresponding to the current time can be determined based on the real-time meteorological data of the area corresponding to the three-dimensional map, and then based on the real-time meteorological data. The lighting effect types are pre-classified according to the meteorological data, and specifically may include: cloudy, sunny, etc., and pre-classified according to the time, and specifically may include: early morning, noon (afternoon), evening and night, etc.; according to the lighting effect type, the light map corresponding to the lighting effect type is obtained from the database, and the corresponding particle animation is obtained according to the real-time meteorological data and the current time, and the particle animation is stored in the database; the corresponding light map can be pre-set according to the light map and stored in the database, and the light map can specifically include cloudy maps, sunny maps, soft light maps, bright light maps, pale light maps and dim light maps, etc.
[0153] Step S1002: Rendering the light map and particle dynamic effects onto the three-dimensional map.
[0154] The corresponding light map and particles obtained are dynamically rendered to the current area in the 3D map, so that the current area displays the corresponding dynamic lighting effect.
[0155] Through the above three methods, the three-dimensional map can be dynamically displayed, thus giving people an intuitive visual experience and greatly improving the visual effect.
[0156] It is understandable that the above three dynamic display methods can be implemented separately or in combination, and this embodiment does not make strict limitations.
[0157] Specifically, at least one display type includes a job path display, such as Fig.11 As shown, step S503 specifically includes:
[0158] Step S1101: Obtain the operation path, model or location identification of the cleaning robot.
[0159] The operation path of the cleaning robot may be set by the user, or may be automatically generated by the cleaning robot during the process of drawing a map. This embodiment does not strictly limit the method for obtaining the operation path.
[0160] Step S1102: Add the operation path, model or location mark on the three-dimensional map.
[0161] Adding the operation path to the map can allow users to understand the operation status of the cleaning robot more intuitively.
[0162] The cleaning robot model and the position mark may be displayed using specific graphics (such as triangles, circles), and this embodiment does not impose strict limitations on the representation of the cleaning robot model and the position mark.
[0163] The location mark of the cleaning robot is displayed on the three-dimensional map, so that the user can intuitively understand the location of the cleaning robot, thereby improving the user experience.
[0164] Specifically, at least one display type includes a three-dimensional element superimposed display, such as Fig.12 As shown, step S503 specifically includes:
[0165] Step S1201: Obtain the three-dimensional elements corresponding to the three-dimensional map and the two-dimensional map constructed by the cleaning robot.
[0166] The three-dimensional elements of the three-dimensional map include but are not limited to voxels, grids, grids with textures, three-dimensional maps, etc. The three-dimensional map can be displayed in a variety of display methods, which may include but are not limited to three-dimensional map display, model point display, model wireframe display, model surface mode, etc. Further, each display method can be displayed in color, transparently, or in black and white, etc., which is not strictly limited in this embodiment.
[0167] The construction method of the two-dimensional map can be referred to step S601, which will not be described in detail here. Step S1202: superimposing three-dimensional elements on the two-dimensional map to obtain a superimposed map.
[0168] Three-dimensional elements will be superimposed on the two-dimensional map, so that each object originally displayed in a flat surface on the two-dimensional map will have a corresponding three-dimensional image, allowing users to understand the working environment of the cleaning robot more intuitively and realistically.
[0169] In the above embodiment, after obtaining the three-dimensional elements corresponding to the three-dimensional map, the method further includes:
[0170] When an instruction to modify at least a portion of the three-dimensional map is received, at least a portion of the three-dimensional map is enlarged to allow the user to modify at least a portion of the three-dimensional map.
[0171] The modification operation instructions include but are not limited to adding or deleting three-dimensional elements (such as voxels, grids, grids with textures, etc.) of the three-dimensional map. The user can modify the three-dimensional map according to the needs.
[0172] After receiving the change operation, the primitive of the change operation is enlarged and displayed, so as to facilitate the user to perform related operations.
[0173] Furthermore, in the above embodiment, after step S504, the following steps are further included:
[0174] When a modification operation instruction of at least one geometric primitive in the three-dimensional map is received, the at least one geometric primitive is split and displayed so that the user can perform a modification operation on the at least one geometric primitive.
[0175] The modification operation instructions include but are not limited to adding and deleting geometric primitives, editing the clustering results of geometric primitives (such as manual creation, deletion, merging, partitioning, etc.), and editing the semantic information after clustering, etc. Users can modify geometric primitives according to their needs.
[0176] After receiving the change operation, the primitive of the change operation is split and displayed, that is, it is separated from the three-dimensional map, so as to facilitate the user to perform related operations.
[0177] Furthermore, in the above embodiment, after step S504, the following steps are further included:
[0178] When a semantic information change instruction of the three-dimensional map is received, the semantic information of the three-dimensional map is changed so that the cleaning robot regenerates a cleaning strategy according to the new semantic information.
[0179] The semantic information includes but is not limited to the name, display color, transparency, cleaning mode of the cleaning robot (such as sweeping mode, etc.), obstacle avoidance selection information, etc. The change instruction may include adding, deleting and changing, etc. The change instruction may be edited by the user and sent through the terminal device, so that the user can change the semantic information according to his own habits and needs.
[0180] Exemplarily, when a user clicks on any named label in the map display area, the label is displayed as selected (for example, the selected state is indicated by highlighting or changing color). If the user does not modify or replace the content of the selected label, but clicks on the item named label corresponding to another item, the selected state is switched to the most recently clicked item, that is, the most recently clicked item named label becomes highlighted or changes display color. If the user does not modify or replace the content of the selected label, but clicks on other areas of the map other than the item named label, the selected state of the item named label corresponding to the current item will be cancelled.
[0181] When the object naming label is in the selected state, if the user clicks on any object name the user wants in the editing tool area, for example, the user clicks on the object name "table", the object naming label in the map display area is changed to "table", and a cancel button is displayed on the border of the label. If the user clicks the cancel button, the naming is abandoned; if the user clicks on other object naming labels or other areas, the corresponding operation is performed, and the content in the object naming label will be saved and displayed.
[0182] The cleaning robot will generate cleaning strategies based on the new semantic information, such as determining the cleaning mode and planning the path, so that the user can interact with the 3D map to edit the properties of the 3D map, and the edited properties can affect the cleaning strategy of the cleaning robot, thereby meeting the user's cleaning needs and improving the user experience. Fig.13 As shown, an embodiment of the present invention provides a three-dimensional map processing device of a cleaning robot, comprising:
[0183] An acquisition module 1301 is used to acquire a three-dimensional map of the working environment of the cleaning robot;
[0184] A receiving module 1302, configured to receive at least one display type of the three-dimensional map;
[0185] A processing module 1303, configured to process the three-dimensional map based on at least one of the display types;
[0186] The display module 1304 is used to display the processed three-dimensional map.
[0187] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the above-mentioned method for processing a three-dimensional map of a cleaning robot.
[0188] The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0189] The memory is used to store programs. The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.
[0190] The computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
[0191] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when called and executed by a processor, perform the steps of the above-mentioned method for processing a three-dimensional map of a cleaning robot.
[0192] The present invention has been described by the above embodiments, but it should be understood that the above embodiments are only for the purpose of example and description, and are not intended to limit the present invention to the scope of the described embodiments. In addition, it can be understood by those skilled in the art that the present invention is not limited to the above embodiments, and more variations and modifications can be made according to the teachings of the present invention, and these variations and modifications all fall within the scope of the protection claimed by the present invention. The protection scope of the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for processing a three-dimensional map of a cleaning robot, characterized in that: include: Acquire a three-dimensional map of the working environment of the cleaning robot; Receiving at least one of the display types of the three-dimensional map; Based on at least one of the display types, processing the three-dimensional map; Display the processed 3D map.
2. The method according to claim 1, characterized in that: At least one of the display types includes an overlay display, and the processing of the three-dimensional map based on the display type includes: Obtaining a two-dimensional map corresponding to the three-dimensional map; The three-dimensional map is superimposed on the two-dimensional map to obtain a superimposed map.
3. The method according to claim 2, characterized in that After superimposing the three-dimensional map on the two-dimensional map to obtain the superimposed map, the method further includes: In response to the display mode selected by the user, the superimposed map is displayed according to the display mode.
4. The method according to claim 1, characterized in that: At least one of the display types includes segmented display, and the processing of the three-dimensional map based on the display type includes: The objects on the three-dimensional map are segmented.
5. The method according to claim 1, characterized in that At least one of the display types includes block display, and the processing of the three-dimensional map based on the display type includes: Acquire the clustering information, where the clustering information is one of the preset clustering information or is customized by a user; Clustering the geometric primitives according to the clustering information to obtain block information; The block information is added to the three-dimensional map.
6. The method according to claim 5, characterized in that: The clustering information is one of the preset clustering information or is customized by the user.
7. The method according to claim 1, characterized in that At least one of the display types includes a color-changing display, and the processing of the three-dimensional map based on the display type includes: Obtaining transparency data and color characteristics of at least one geometric primitive or a three-dimensional map constituting the geometric primitive; Corresponding transparency data is superimposed on at least one of the geometric primitives or a color feature of the three-dimensional map constituting the geometric primitive.
8. A three-dimensional map processing device for a cleaning robot, characterized in that: include: An acquisition module, used to acquire a three-dimensional map of the working environment of the cleaning robot; A receiving module, configured to receive at least one display type of the three-dimensional map; A processing module, configured to process the three-dimensional map based on the at least one display type; The display module is used to display the processed three-dimensional map.
9. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the steps of the method for processing a three-dimensional map of a cleaning robot as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: Computer program instructions are stored, and when the computer program instructions are called and executed by a processor, the steps of the method for processing a three-dimensional map of a cleaning robot according to any one of claims 1 to 7 are implemented.
