Method and device for constructing 3D color maps, storage medium, and electronic device

Abstract

This application discloses a method and apparatus for constructing a 3D color map, a storage medium, and an electronic device. The method includes the steps of obtaining at least one 3D object model and obtaining a 3D blank scene, placing the 3D object model in the 3D blank scene, coloring the 3D object model, and obtaining a 3D color map. The method of this application solves the problem that maps created by existing mobile smart devices are not intuitive.

Description

Related Applications

[0001] This application claims priority to a Chinese patent application filed with the China Patent Office on March 2, 2023, application number 202310203606.4, entitled "Method and apparatus for constructing three-dimensional color maps, storage medium, and electronic device," the entire contents of which are incorporated herein by reference. [Technical Field]

[0002] The present application relates to the technical field of smart home appliances, and in particular to a method and apparatus for constructing a three-dimensional color map, a storage medium and an electronic device. [Background technology]

[0003] With the development of computer technology and artificial intelligence technology, mobile smart devices equipped with smart systems, such as cleaning robots, have emerged. These mobile smart devices can autonomously navigate within a specific area without user interaction and perform corresponding operations according to user commands or pre-programmed programs. Mobile smart devices are typically equipped with sensors such as grayscale cameras, color cameras, depth cameras, or laser radar. During operation, the cleaning robot acquires corresponding information within the working area, such as color images, grayscale images, or point clouds, through the sensors, draws a map of the area, and provides the map as feedback to the user, allowing the user to understand the map information of the area where the mobile smart device is located.

[0004] However, in related technologies, maps created by mobile smart devices are typically two-dimensional flat maps or colorless three-dimensional maps that include models of walls, floors, furniture, etc., which means that the maps presented to users are difficult to intuitively understand. Summary of the Invention

[0005] In view of this, the present application provides a method and apparatus for constructing a three-dimensional color map, a storage medium and an electronic device that solve the problem that maps created on existing mobile smart devices are not intuitive.

[0006] According to one aspect of the present application, there is provided a method for constructing a three-dimensional color map, the method comprising: obtaining at least one three-dimensional object model and obtaining a three-dimensional blank scene; placing the three-dimensional object model in the three-dimensional blank scene, coloring the three-dimensional object model, and obtaining the three-dimensional color map.

[0007] In one embodiment, the step of obtaining at least one three-dimensional object model comprises: Using a self-propelled device, photograph environmental information of a target area to acquire map information, and identify the map information to acquire object information; Obtaining a three-dimensional predetermined model corresponding to the object information as the three-dimensional object model from a predetermined three-dimensional model library; and / or In response to a model input command, analyzing the model input command to obtain the three-dimensional object model.

[0008] In one embodiment, the step of acquiring the three-dimensional blank scene comprises: Using the self-propelled device to photograph environmental information of the target area to obtain the map information, and identifying the map information to obtain map boundary information; determining a 3D predetermined scene that matches the map boundary information as the 3D blank scene from a plurality of 3D predetermined scenes; or In response to a scene input command, analyzing the scene input command to obtain the three-dimensional blank scene.

[0009] In one embodiment, the object information includes category information and / or three-dimensional structure information, and acquiring a three-dimensional predetermined model corresponding to the object information as the three-dimensional object model includes: Searching for a three-dimensional predetermined model corresponding to the category information, and setting the searched three-dimensional predetermined model as the three-dimensional object model; and / or The method includes determining a degree of similarity between the three-dimensional structural information and predetermined three-dimensional structural information of each of the three-dimensional predetermined models, and determining the three-dimensional predetermined model with the highest degree of similarity as the three-dimensional object model.

[0010] In one embodiment, the step of placing the three-dimensional object model in a three-dimensional blank scene comprises: acquiring first position and orientation information from the object information, or analyzing a position and orientation setting command to acquire the first position and orientation information; determining a first position and a first orientation corresponding to the first position and orientation information from the 3D blank scene, and placing the 3D object model at the first position, with the 3D object model in the first position and orientation.

[0011] In one embodiment, the step of coloring the three-dimensional object model comprises: determining a target coloring scheme for the three-dimensional object model based on the category information, or analyzing a coloring instruction to obtain the target coloring scheme, wherein the target coloring scheme includes a target color and / or a target texture; coloring the three-dimensional object model based on the target coloring scheme.

[0012] In one embodiment, after obtaining the three-dimensional color map, the method further comprises: and / or, in response to a no-traffic command, setting a no-traffic area or a virtual wall in the three-dimensional color map corresponding to the no-traffic command. The method further includes the step of: in response to a clearance command, setting a clearance area corresponding to the clearance command on the three-dimensional color map.

[0013] In one embodiment, after placing the three-dimensional object model in the three-dimensional blank scene, the method further comprises: acquiring second position and attitude information of the self-propelled device; The method further includes determining a second position and a second orientation from the 3D blank scene corresponding to the second position and orientation information, and placing a 3D device model corresponding to the self-propelled device at the second position, with the 3D device model being in the second position and orientation.

[0014] In one embodiment, after obtaining the three-dimensional color map, the method further comprises: acquiring a motion trajectory of the self-propelled device or an image capturing device mounted on the self-propelled device, and determining a position, orientation, and internal parameters of the image capturing device at each position of the motion trajectory; and / or, processing the 3D color map based on the position, orientation and intrinsic parameters of each of the locations to obtain a 2D picture corresponding to the location. The method further includes the step of modifying the three-dimensional color map in response to a modification command and displaying the modified three-dimensional color map.

[0015] According to another aspect of the present application, there is provided an apparatus for constructing a three-dimensional color map, the apparatus comprising: an acquisition module configured to acquire at least one three-dimensional object model and to acquire a three-dimensional blank scene; a construction module configured to place the three-dimensional object model in the three-dimensional blank scene, color the three-dimensional object model, and obtain the three-dimensional color map.

[0016] In one embodiment, the acquisition module: Using the self-propelled device, photograph environmental information of the target area to obtain map information, and identify the map information to obtain object information; Obtaining a three-dimensional predetermined model corresponding to the object information as the three-dimensional object model from a predetermined three-dimensional model library, and / or In response to a model input command, the system is configured to analyze the model input command to obtain the three-dimensional object model.

[0017] In one embodiment, the acquisition module: The self-propelled device photographs environmental information of the target area to obtain the map information, and identifies the map information to obtain map boundary information; determining a 3D predetermined scene that matches the map boundary information as the 3D blank scene from a plurality of 3D predetermined scenes; or In response to a scene input command, the system is configured to analyze the scene input command to obtain the three-dimensional blank scene.

[0018] In one embodiment, the object information includes category information and / or three-dimensional structure information, and the acquisition module: Searching for a three-dimensional predetermined model corresponding to the category information, and setting the searched three-dimensional predetermined model as the three-dimensional object model; and / or The system is configured to determine the degree of similarity between the three-dimensional structural information and the predetermined three-dimensional structural information of each of the three-dimensional predetermined models, and to determine the three-dimensional predetermined model with the highest degree of similarity as the three-dimensional object model.

[0019] In one embodiment, the construction module comprises: acquiring first position and orientation information from the object information, or analyzing a position and orientation setting command to acquire the first position and orientation information; A first position and a first orientation corresponding to the first position and orientation information are determined from the three-dimensional blank scene, and the three-dimensional object model is placed at the first position and configured to be in the first position and orientation.

[0020] In one embodiment, the construction module comprises: Determine a target coloring scheme for the three-dimensional object model based on the category information, or analyze a coloring instruction to obtain the target coloring scheme, where the target coloring scheme includes a target color and / or a target texture; The three-dimensional object model is configured to be colored based on the target coloring scheme.

[0021] In one embodiment, the construction module further comprises: and / or, in response to a no-traffic command, setting a no-traffic area or a virtual wall in the three-dimensional color map corresponding to the no-traffic command. In response to a clearance command, the device is configured to set a clearance area corresponding to the clearance command in the three-dimensional color map.

[0022] In one embodiment, the construction module further comprises: Acquire second position and attitude information of the self-propelled device; A second position and a second orientation corresponding to the second position and orientation information are determined from the three-dimensional blank scene, and a three-dimensional device model corresponding to the self-propelled device is placed at the second position, and the three-dimensional device model is configured to be in the second position and orientation.

[0023] In one embodiment, the device comprises: acquiring a motion trajectory of the self-propelled device or an image capture device mounted on the self-propelled device, and determining a position, orientation, and internal parameters of the image capture device at each position of the motion trajectory; respectively, processing the 3D color map based on the position, orientation, and intrinsic parameters of each of the positions to obtain a 2D picture corresponding to the position; and displaying the 2D picture; or The system further comprises a post-processing module configured to modify the three-dimensional color map in response to a modification command and display the modified three-dimensional color map.

[0024] According to another aspect of the present application, there is provided a storage medium storing a program or instructions, which, when executed by a processor, realizes the above-described method for constructing a 3D color map. According to another aspect of the present application, there is provided an electronic device including a storage medium and a processor, wherein a computer program is stored in the storage medium, and, when executed by the processor, realizes the above-described method for constructing a 3D color map.

[0025] According to the above technical solution, the present application obtains at least one 3D object module and a 3D blank scene, then places the 3D object model in the 3D blank scene, further simulating the placement of objects in the actual scene. Finally, the target model is colored. In this way, the present embodiment constructs a three-dimensional colored map. Compared with the flat black-and-white map constructed in the exemplary method, the map constructed in this embodiment is more aesthetically pleasing, and each object in the map can be more easily identified, making it easier for users to understand.

[0026] The above description is only a summary of the technical solution of the present application, which can be implemented according to the content of the specification in order to clearly understand the technical means of the present application. In order to make the above and other objectives, features and advantages of the present application clearer and easier to understand, the following provides examples of specific embodiments of the present application. [Brief explanation of the drawings]

[0027] The accompanying drawings described herein are intended to provide a further understanding of the present application and constitute a part of this application, and the illustrative embodiments and description thereof are intended to provide an understanding of the present application and are not intended to unduly limit the present application.

[0028] [Figure 1] 1 is a schematic flowchart of a method for constructing a three-dimensional color map provided by an embodiment of the present application. [Figure 2] 1 is a schematic flowchart of a method for constructing a three-dimensional color map provided by an embodiment of the present application. [Figure 3] 1 is a structural block diagram of a 3D color map construction device provided by an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0029] The present application will be described in detail below in conjunction with the accompanying drawings and examples. It should be noted that the embodiments and features in the embodiments in the present application can be combined with each other unless they are mutually inconsistent.

[0030] This embodiment provides a method for constructing a three-dimensional color map, as shown in FIG. 1, the method includes the following steps: Step 101: obtain at least one three-dimensional object model; and obtain a three-dimensional blank scene; Step 102: Place the 3D object model in the 3D blank scene, and color the 3D object model to obtain the 3D color map.

[0031] An embodiment of the present application provides a method for constructing a 3D color map for use in a self-propelled device such as a cleaning robot. The method uses a 3D object model and a 3D blank scene to simulate the actual object arrangement in an actual 3D scene, constructs a stereoscopic color map, and provides feedback to the user on the constructed 3D color map, allowing the user to intuitively grasp the object arrangement status in the current target area.

[0032] In this manner, the present embodiment constructs a three-dimensional, colored map, and compared to the flat, black-and-white map constructed in the exemplary method, the map constructed in this embodiment is more aesthetically pleasing, and each object in the map is more easily identifiable, making it easier for the user to understand.

[0033] In an embodiment of the present application, in step 101, the step of acquiring at least one three-dimensional object model includes the following steps: step 101-a1, photographing environmental information of a target area using a self-propelled device to acquire map information, identifying the map information to acquire object information, and acquiring a three-dimensional predetermined model corresponding to the object information from a predetermined three-dimensional model library as a three-dimensional object model; and / or Step 101-a2: In response to a model input command, the model input command is analyzed to obtain a three-dimensional object model.

[0034] In this embodiment, a sensing device is mounted on the self-propelled device, and the sensing device can be used to capture map information of the target area. Here, the capturing device may be a color camera, a black-and-white camera, a depth camera, or a radar, or may be a sensor. If the capturing device is a color camera, the map information is a color image. If the capturing device is a black-and-white camera, the map information is a grayscale image. If the capturing device is a depth camera, the map information is a depth image. If the capturing device is a radar, the map information is point cloud information. Furthermore, the map information further includes position and orientation information when the capturing device acquires the map information.

[0035] After the map information is acquired by the image capture device, an identification algorithm is used to acquire object information within the target area based on the map information. It should be understood that the identification algorithm is a set of instructions that takes the map information acquired by the image capture device as input information, processes the input information, and outputs one or more object information of one or more predefined categories within a limited time. The identification algorithm can identify the point cloud map and acquire information such as object size, category, position, and orientation.

[0036] Here, there may be multiple objects or multiple types of objects in the target area, and the object information may include object category, position, orientation, size information, etc. Furthermore, the object information may further include 3D structure information such as a 3D point cloud of furniture, a truncated signed distance function (TSDF), etc.

[0037] Based on the object information, a preset 3D model corresponding to each object is searched for from a preset 3D model library as the 3D object model corresponding to the object. Then, based on the position and orientation information of each object, the 3D object model is placed in a 3D blank scene, and the 3D object model is colored according to a certain rule to obtain a 3D color map.

[0038] Furthermore, it is also possible to receive a model input command input by a user and analyze the command to obtain a three-dimensional object model.

[0039] This embodiment provides two different methods for acquiring a 3D object model, improving the flexibility of acquiring a 3D object model and allowing users to select a suitable method according to their actual needs. Furthermore, when acquiring multiple 3D object models, the two methods can be combined, i.e., some 3D object models are acquired from object information acquired by a self-propelled device, and other 3D object models are acquired based on model data commands.

[0040] In the embodiment of the present application, in one embodiment, in step 101, the step of acquiring a 3D blank scene includes the following steps: Step 101-b1: using a self-propelled device to photograph environmental information of a target area to obtain map information, identify the map information to obtain map boundary information, and determine a 3D predetermined scene that matches the map boundary information as a 3D blank scene from a plurality of 3D predetermined scenes; and / or Step 101-b2: In response to a scene input command, the scene input command is analyzed to obtain a three-dimensional blank scene.

[0041] In this embodiment, the 3D blank scene can be matched with map boundary information, and the map boundary can be obtained from map information captured by the self-propelled device or determined from received data such as a floor plan. Preferably, the map boundary information can include exterior contours, wall contours, door and window contours.

[0042] For example, when a three-dimensional object model corresponding to a table is acquired from map information, where the map boundary information is a boundary corresponding to a kitchen, the three-dimensional object model is placed in the three-dimensional blank scene corresponding to the kitchen.

[0043] Furthermore, it is also possible to receive a scene input command input by a user and analyze the command to obtain a 3D blank scene. For example, if a user inputs a 3D blank scene having only an outer shell, a 3D object model corresponding to a wall and a 3D object model corresponding to a chair are obtained, and these two obtained 3D object models are placed in the 3D blank scene.

[0044] This embodiment provides two different methods for acquiring a 3D blank scene, which improves the flexibility of acquiring a 3D blank scene, and allows users to choose a more suitable method according to their actual needs.

[0045] Furthermore, as a subdivision and extension of the specific embodiment of the above example, in order to fully describe the specific implementation process of this example, another method for constructing a 3D color map is provided, in which the object information includes category information or 3D structure information, and as shown in FIG. 2, the method includes the following steps:

[0046] Step 201: Search for a three-dimensional predetermined model corresponding to the category information, and determine the searched three-dimensional predetermined model as a three-dimensional object model, and / or determine the similarity between the three-dimensional structural information and the predetermined three-dimensional structural information of each three-dimensional predetermined model, and determine the three-dimensional predetermined model with the highest similarity as the three-dimensional object model.

[0047] In the embodiment of the present application, a 3D object model corresponding to the object is selected from a 3D predetermined model library based on the object category or 3D structure information.

[0048] Specifically, a corresponding 3D object model can be searched for in the 3D model library according to the object category information. For example, if the object category information is a table, a 3D model corresponding to the table category is searched for in the 3D model library as the 3D object model.

[0049] Furthermore, the search may be performed according to the three-dimensional structural similarity of the object. Specifically, the three-dimensional structural similarity between each three-dimensional predetermined model and the object is calculated based on the three-dimensional structural information of each three-dimensional predetermined model in the three-dimensional predetermined model library and the three-dimensional structure of the object, and the three-dimensional predetermined model with the closest three-dimensional structure, i.e., the highest similarity, is determined as the three-dimensional object model.

[0050] For example, if the category information of an object is a table and no three-dimensional predetermined model in the table category is searched for in the three-dimensional predetermined model library, the three-dimensional structural similarity between each three-dimensional predetermined model in the model library and the object is calculated, and the three-dimensional predetermined model of a desk is found to be closest to the three-dimensional structure of the object, and therefore the three-dimensional predetermined model of the desk is determined to be the three-dimensional object model.

[0051] The embodiments of the present application provide two methods for determining 3D object models, which can quickly and accurately select a corresponding 3D object model for each object, and then use the 3D object model to label each object on a 3D color map. Furthermore, when determining multiple 3D object models, the two methods can be combined, i.e., some 3D object models can be obtained from the 3D structural similarity of the objects, and other 3D object models can be obtained from the object category information.

[0052] Step 202: Obtain first position and orientation information from object information, or analyze a position and orientation setting command to obtain first position and orientation information; Step 203: A first position and a first orientation corresponding to the first position and orientation information are determined from the three-dimensional blank scene, and the three-dimensional object model is placed at the first position and in the first orientation.

[0053] After determining the 3D object model corresponding to each object, the 3D object model is placed in the 3D blank scene. It should be understood that in an actual usage scene, each object has a corresponding placement position and orientation. Therefore, when constructing a 3D map based on the actual scene, the position and orientation information of each object is mapped onto the 3D map.

[0054] Specifically, after acquiring object information from map information, first position and orientation information is extracted from the object information, a first orientation corresponding to the first position and orientation information and a corresponding first position in the target scene are determined, and then the object is placed at the first position in the first position and orientation.

[0055] For example, for an object called a single bed, first position and orientation information is extracted from the object information, and the first orientation of the single bed is determined to be such that the head of the bed faces east, the foot of the bed faces west, and the first position is by the window.In this case, a three-dimensional object model corresponding to the single bed is placed at the window position in the target scene, with the head of the bed facing east and the foot of the bed facing west.

[0056] Furthermore, each object may be placed at a specified position in a specified orientation in accordance with a user command. Specifically, the position and orientation setting command is analyzed to obtain first position and orientation information, and a 3D object model is placed based on the first position and orientation information in the command. The placement method is similar to that described above, and therefore will not be described in detail here.

[0057] Furthermore, when the three-dimensional object model is placed in the three-dimensional blank scene, the size of the three-dimensional object model may be adjusted based on the size information, which can be obtained from the object information or by analyzing the size setting command.

[0058] In the embodiment of the present application, a 3D object model is positioned based on the object position and orientation information, and the orientation and position of the 3D object model are made to correspond to those of the actual object, so that the resulting 3D color map can more accurately display the item placement status of the target area.

[0059] Step 204: Obtain second position and attitude information of the self-propelled device; Step 205: In the target scene, a second position and a second orientation corresponding to the second position and orientation information are determined, and a three-dimensional device model corresponding to the self-propelled device is placed at the second position in the second orientation.

[0060] In the embodiment of the present application, a three-dimensional device model corresponding to the self-propelled device is also placed in the target scene, and the user can grasp the current location of the self-propelled device based on the three-dimensional color map.

[0061] Specifically, second position and orientation information of the self-propelled device is acquired, and a three-dimensional device model is arranged based on the second position and orientation information, in the same manner as the arrangement method for the three-dimensional object model.

[0062] Here, the second position and attitude information can be acquired by a sensor mounted on the self-propelled device.

[0063] Step 206: Determine a target coloring scheme for the three-dimensional object model based on the category information, or analyze the coloring instruction to obtain a target coloring scheme, where the target coloring scheme includes a target color and / or a target texture; Step 207: color the 3D object model based on the target coloring scheme to obtain a 3D color map.

[0064] In the embodiment of the present application, a color map is obtained by coloring a three-dimensional object model based on a certain rule, i.e., a target coloring scheme.

[0065] Specifically, the target coloring scheme is determined based on the category information in the object information. For example, all 3D object models in the target scene whose category is table can be set to brown, or wood grain texture, or brown wood grain texture.

[0066] Here, the target coloring scheme corresponding to each category may be a preset default scheme or may be a scheme set by the user.

[0067] Furthermore, the target coloring scheme may be determined based on a coloring command input by a user, and the coloring object corresponding to the coloring command may be one or one type of object or several regions. For example, after analyzing the coloring command, if it is determined that the specified coloring object is table Table_A and the target color is brown, the 3D object model corresponding to table Table_A is set to brown.

[0068] The coloring command may further include automatic color change conditions, each of which corresponds to a target coloring scheme. When it is detected that the current condition satisfies the automatic color change condition, the 3D object model is recolored based on the target coloring scheme corresponding to the automatic color change condition to achieve automatic color change. For example, the automatic color change condition may be set to daily, weekly, monthly, quarterly, seasonal, or local holidays in the location of the self-propelled device. When the current time reaches the time corresponding to the automatic color change condition, the target coloring scheme corresponding to the automatic color change condition is executed.

[0069] For example, if the automatic color change condition is a season transition, the target color scheme corresponding to summer is green and the target color scheme corresponding to autumn is yellow. When the season transitions, the object or area corresponding to the color command will change to the color corresponding to the current season.

[0070] Furthermore, a sticker may be attached to the 3D color map in response to a sticker command from a user. Here, the sticker picture may be a picture pre-stored in the self-propelled device or a picture uploaded by the user. The sticker command includes a sticker picture and a sticker position, and the sticker picture is attached to the sticker position based on the command.

[0071] Step 208: In response to the no-entry command, set a no-entry area or virtual wall corresponding to the no-entry command on the three-dimensional color map, and / or in response to the allow-entry command, set a allow-entry area corresponding to the allow-entry command on the three-dimensional color map.

[0072] In the embodiment of the present application, allowed areas and prohibited areas can be set on the three-dimensional color map, where the allowed areas and prohibited areas can be set according to the actual cleaning needs of the user.

[0073] Specifically, the no-entry command may include a no-entry area, and the no-entry area corresponding to the no-entry command may be marked on the 3D color map with a special pattern or color to indicate that the self-propelled device is prohibited from entering the area. The pass-allow command may include a pass-allow area, and the setting method thereof is similar to that of the no-entry area, so details will be omitted here. Furthermore, the no-entry command may include a virtual wall, and the virtual wall may be marked on the 3D color map with a special figure or symbol to indicate that the self-propelled device is prohibited from crossing the virtual wall.

[0074] In an embodiment of the present application, by providing no-passage areas, virtual walls, and permitted passage areas on a 3D color map, a user can intuitively grasp the passable and non-passable areas for a cleaning robot, i.e., the areas that can and cannot be cleaned by the cleaning robot.

[0075] Step 209: Obtaining a motion trajectory of the self-propelled device or the image capture device mounted on the self-propelled device, and determining the position, orientation, and internal parameters of the image capture device at each position of the motion trajectory based on the motion trajectory; In step 210, the 3D color map is processed based on the position, orientation, and intrinsic parameters of each position to obtain a 2D picture corresponding to the position.

[0076] In an embodiment of the present application, the 3D color picture is processed based on the position, orientation, and internal parameters of the camera device to obtain a 2D picture of the camera device at each position on the movement trajectory. Specifically, the position, orientation, and internal parameters of the camera device at each position are determined based on the movement trajectory of the camera device or the self-propelled device. After obtaining the position, orientation, and internal parameters of the camera device, the 3D color map is rendered to generate a 2D picture.

[0077] When displaying a 3D map in 2D, the displayed cross section may be a pre-set cross section (e.g., z=0.1m) or a specific cross section set based on a user command. The rendering method can be selected from rasterization, ray tracing, etc.

[0078] Furthermore, after acquiring two-dimensional pictures corresponding to each position on the motion trajectory, they can be played back on a display device mounted on the self-propelled device.

[0079] Furthermore, after obtaining the 3D color map, the method The method further includes the step of modifying the three-dimensional color map in response to a command and displaying the modified three-dimensional color map.

[0080] In this embodiment, after obtaining the 3D color map, the user modifies the 3D color map according to actual needs, where the modification operation includes one or more of modifying the 3D object model, modifying the 3D blank scene, modifying the first and / or second position and orientation information, and modifying the coloring scheme. For example, a 3D object model corresponding to a sliding door can be modified to a 3D object model corresponding to a folding door, or an east-west facing single bed can be modified to a north-south facing single bed, or the brown wood grain texture of a table can be modified to a white wood grain texture.

[0081] It should be understood that the numbers of each step in the above embodiments do not indicate the order of execution, but rather the order of execution of each process is determined based on its function and inherent logic, and does not in any way limit the implementation process of the embodiments of the present application.

[0082] Furthermore, as a specific implementation of the above-mentioned 3D color map construction method, an embodiment of the present application provides a 3D color map construction device, which, as shown in FIG. 3, includes an acquisition module and a construction module.

[0083] The acquisition module is configured to acquire at least one three-dimensional object model and acquire a three-dimensional blank scene; The construction module is configured to place the three-dimensional object model in the three-dimensional blank scene, color the three-dimensional object model, and obtain a three-dimensional color map.

[0084] In one embodiment, the acquisition module: The self-propelled device photographs environmental information of the target area to obtain map information, identifies the map information to obtain object information, Selecting a three-dimensional predetermined model corresponding to the object information as a three-dimensional object model from a predetermined three-dimensional model library, and / or In response to the model input command, the system is configured to analyze the model input command to obtain a three-dimensional object model.

[0085] In one embodiment, the acquisition module: The self-propelled device photographs the environmental information of the target area to obtain map information, identifies the map information, and obtains map boundary information; determining a 3D predetermined scene that matches the map boundary information as a 3D blank scene from the plurality of 3D predetermined scenes; or In response to a scene input command, the system is configured to analyze the scene input command to obtain a three-dimensional blank scene.

[0086] In one embodiment, the object information includes category information and / or three-dimensional structure information, and the acquisition module: Searching for a three-dimensional predetermined model corresponding to the category information, and determining the searched three-dimensional predetermined model as a three-dimensional object model; and / or The system is configured to determine the degree of similarity between each of the three-dimensional structural information and the predetermined three-dimensional structural information of each three-dimensional predetermined model, and to determine the three-dimensional predetermined model having the highest degree of similarity as the three-dimensional object model.

[0087] In one embodiment, the construction module comprises: Acquire first position and orientation information from the object information, or analyze a position and orientation setting command to acquire the first position and orientation information; The apparatus is configured to determine a first position and a first orientation corresponding to the first position and orientation information in the three-dimensional blank scene, and to place the three-dimensional object model at the first position in the first orientation.

[0088] In one embodiment, the construction module comprises: Determine a target coloring scheme for the three-dimensional object model based on the category information, or analyze the coloring instructions to obtain the target coloring scheme, where the target coloring scheme includes a target color and / or a target texture; The three-dimensional object model is configured to color based on a target coloring scheme.

[0089] In one embodiment, the construction module further comprises: in response to the no-traffic command, setting a no-traffic area or virtual wall corresponding to the no-traffic command on the three-dimensional color map; and / or In response to the clearance command, the device is configured to set a clearance area corresponding to the clearance command on the three-dimensional color map.

[0090] In one embodiment, the construction module further comprises: Acquire second position and attitude information of the self-propelled device; The system is configured to determine a second position and a second orientation in the three-dimensional blank scene corresponding to the second position and orientation information, and place a three-dimensional device model corresponding to the self-propelled device at the second position in the second orientation.

[0091] In one embodiment, the apparatus further comprises a post-processing module, the post-processing module comprising: Acquiring a motion trajectory of the self-propelled device or an image capture device mounted on the self-propelled device, and acquiring the position, orientation, and internal parameters of the image capture device at each position on the motion trajectory; Process the 3D color map based on the position, orientation, and internal parameters of each position, respectively, to obtain a 2D picture corresponding to the position, and display the 2D picture; or The device is configured to modify the three-dimensional color map in response to the modification command and display the modified three-dimensional color map.

[0092] For other corresponding descriptions of each functional module of the 3D color map construction device provided by the embodiment of the present application, please refer to the corresponding descriptions of the above method, and details will be omitted here.

[0093] Based on the above method, an embodiment of the present application further provides a storage medium storing a computer program, which, when executed by a processor, realizes the above method for constructing a three-dimensional color map.

[0094] Based on this understanding, the technical solution of the present application can be realized in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB disk, a portable hard disk, etc.), and includes a number of instructions for causing an electronic device (such as a personal computer, a server, or a network device) to execute the 3D color map construction method described in each embodiment of the present application.

[0095] Based on the method shown in Figures 1 and 2 above and the embodiment of the virtual device shown in Figure 3, in order to achieve the above object, an embodiment of the present application further provides an electronic device, which may be a personal computer, a server, a network device, etc., or a self-propelled device such as a cleaning robot, and the electronic device includes a storage medium and a processor, the storage medium is configured to store a computer program, and the processor is configured to execute the computer program to perform the 3D color map construction method shown in Figures 1 to 3 above.

[0096] In one embodiment, the electronic device may include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include an input unit such as a display and a keyboard, and optionally the user interface may include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (Bluetooth interface, Wi-Fi interface), etc.

[0097] Those skilled in the art will understand that the structure of the electronic device provided in this embodiment does not limit the electronic device, and that the electronic device may include more or fewer components, or may combine some components, or may have different components arranged.

[0098] The storage medium further includes an operating system, a network communication module, etc. The operating system is a program that manages and stores the hardware and software resources of an electronic device and supports the execution of information processing programs and other software and / or programs. The network communication module is configured to enable communication between components within the storage medium and other hardware and software in the physical device.

[0099] From the above description of the embodiments, those skilled in the art will understand that the present application can be realized by combining software with a required general-purpose hardware platform, and can also be realized by hardware.

[0100] Those skilled in the art can understand that the accompanying drawings are merely mode diagrams of preferred embodiments, and the units or flows in the accompanying drawings are not necessarily essential for implementing the present application. Those skilled in the art can understand that the units in the devices of the embodiments may be arranged in the devices of the embodiments according to the descriptions of the embodiments, or may be arranged in one or more devices different from the present embodiment with corresponding changes. The units of the above embodiments may be integrated into one unit or further divided into multiple sub-units.

[0101] The numbers in the above application are for illustrative purposes only and do not indicate the superiority or inferiority of the embodiments. The contents disclosed above are only some specific embodiments of the application, and the application is not limited thereto, and all modifications conceivable by those skilled in the art shall be included in the scope of protection of the application.

Claims

1. A method for constructing a three-dimensional color map, comprising: obtaining at least one three-dimensional object model and obtaining a three-dimensional blank scene; placing the three-dimensional object model in the three-dimensional blank scene; coloring the three-dimensional object model; and obtaining the three-dimensional color map.

2. The step of obtaining at least one three-dimensional object model comprises: The method includes a step of photographing environmental information of a target area using a self-propelled device to obtain map information, and identifying the map information to obtain object information. The step of obtaining at least one three-dimensional object model further comprises: acquiring a three-dimensional predetermined model corresponding to the object information as the three-dimensional object model from a predetermined three-dimensional model library; 2. The method for constructing a three-dimensional color map according to claim 1, further comprising at least one of the steps of: in response to a model input command, analyzing the model input command to obtain the three-dimensional object model.

3. The step of acquiring a three-dimensional blank scene includes: Using the self-propelled device to photograph environmental information of the target area to obtain the map information, and identifying the map information to obtain map boundary information; determining a three-dimensional predetermined scene from a plurality of three-dimensional predetermined scenes that matches the map boundary information as the three-dimensional blank scene; or 3. The method for constructing a three-dimensional color map according to claim 2, further comprising the step of: in response to a scene input command, analyzing said scene input command to obtain said three-dimensional blank scene.

4. The object information includes at least one of category information and three-dimensional structure information, and the step of acquiring a three-dimensional predetermined model corresponding to the object information as the three-dimensional object model includes: A step of searching for a three-dimensional predetermined model corresponding to the category information, and setting the searched three-dimensional predetermined model as the three-dimensional object model; and 3. The method for constructing a three-dimensional color map according to claim 2, further comprising at least one of steps of determining a similarity between the three-dimensional structural information and predetermined three-dimensional structural information of each of the three-dimensional predetermined models, and determining the three-dimensional predetermined model having the highest similarity as the three-dimensional object model.

5. The step of placing the three-dimensional object model in the three-dimensional blank scene includes: acquiring first position and orientation information from the object information or analyzing a position and orientation setting command to acquire the first position and orientation information; 3. The method for constructing a 3D color map according to claim 2, further comprising the steps of: determining a first position and a first orientation corresponding to the first position and orientation information from the 3D blank scene; and placing the 3D object model at the first position, with the 3D object model being in the first orientation.

6. The step of coloring the three-dimensional object model includes: determining a target coloring scheme for the three-dimensional object model based on the category information, or analyzing a coloring instruction to obtain the target coloring scheme, wherein the target coloring scheme includes at least one of a target color and a target texture; and coloring the three-dimensional object model based on the target coloring scheme.

7. After obtaining the three-dimensional color map, the three-dimensional color map construction method includes: In response to a no-traffic command, setting a no-traffic area or a virtual wall corresponding to the no-traffic command on the three-dimensional color map; and 2. The method for constructing a three-dimensional color map according to claim 1, further comprising at least one of the steps of: in response to a clearance command, setting a clearance area corresponding to the clearance command in the three-dimensional color map.

8. After placing the three-dimensional object model in the three-dimensional blank scene, the method for constructing the three-dimensional color map includes: acquiring second position and attitude information of the self-propelled device; 3. The method for constructing a 3D color map according to claim 2, further comprising the steps of: determining a second position and a second orientation corresponding to the second position and orientation information from the 3D blank scene; and placing a 3D device model corresponding to the self-propelled device at the second position, with the 3D device model being in the second orientation.

9. After obtaining the three-dimensional color map, the three-dimensional color map construction method includes: The method further includes a step of acquiring a motion trajectory of the self-propelled device or an image capturing device mounted on the self-propelled device, and determining a position, orientation, and internal parameters of the image capturing device at each position of the motion trajectory; The method for constructing a three-dimensional color map comprises: processing the 3D color map based on the position, orientation, and intrinsic parameters of each of the locations to obtain a 2D picture corresponding to the location; and 9. The method for constructing a three-dimensional color map according to claim 8, further comprising at least one of the steps of: modifying said three-dimensional color map in response to a modification command; and displaying the modified three-dimensional color map.

10. A three-dimensional color map construction device, comprising: an acquisition module configured to acquire at least one three-dimensional object model and to acquire a three-dimensional blank scene; a construction module configured to place the three-dimensional object model in the three-dimensional blank scene, color the three-dimensional object model, and obtain the three-dimensional color map.

11. A storage medium on which a program or instructions are stored, the program or instructions implementing the method for constructing a three-dimensional color map according to any one of claims 1 to 9 when executed by a processor.

12. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the electronic device realizes the method for constructing a three-dimensional color map according to any one of claims 1 to 9.

Images