3D Visualization Method and Device, Robot, Electronic Device, and Storage Medium

Abstract

The present application discloses a three-dimensional visualization method and apparatus, a robot, an electronic device, and a storage medium, belonging to the technical field of robot map processing and perception. Among them, the three-dimensional visualization method includes: obtaining object spatial information and object position information in a two-dimensional map, generating three-dimensional representation information based on the object spatial information and the object position information. Constructing a three-dimensional model of the object based on the three-dimensional representation information, rendering the three-dimensional model to obtain a three-dimensional space corresponding to the two-dimensional map, and performing visual display.

Description

Technical Field

[0001] This application belongs to the technical field of robot map processing and perception, and particularly relates to a three-dimensional visualization method and device, a robot, an electronic device, and a storage medium. Background Art

[0002] In the prior art, in the field of robots, the processing and expression of map information is a very important link. Due to the structural characteristics of commercial lidar, the map representations of current mainstream robots are all simplified to two-dimensional grid maps. In the related art, the map representations of robots all adopt two-dimensional grid maps, and users can only see two-dimensional plane images similar to top views on the client side, resulting in poor user experience. Moreover, there are a large number of noise points in the two-dimensional plane images, which cannot be directly rendered in three dimensions and cannot achieve three-dimensional visualization of the map. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a three-dimensional visualization method and device, a robot, an electronic device, and a storage medium, which can solve the problem that the map representation of the robot adopts a two-dimensional grid map, and users can only see two-dimensional plane images, resulting in poor user experience. The two-dimensional plane images cannot be directly rendered in three dimensions and cannot achieve three-dimensional visualization of the map. Figure 3 Problem

[0004] In a first aspect, the embodiments of this application provide a three-dimensional visualization method for a two-dimensional map, including: obtaining object space information and object position information in the two-dimensional map, and generating three-dimensional representation information based on the object space information and the object position information. Based on the three-dimensional representation information, constructing a three-dimensional model of the object, rendering the three-dimensional model, obtaining a three-dimensional space corresponding to the two-dimensional map, and performing visual display.

[0005] In a second aspect, the embodiments of this application provide a three-dimensional visualization device for a two-dimensional map, including a generation module and a display module. The generation module is used to obtain object space information and object position information in the two-dimensional map, and generate three-dimensional representation information based on the object space information and the object position information; the display module is used to construct a three-dimensional model of the object, render the three-dimensional model, obtain a three-dimensional space corresponding to the two-dimensional map, and perform visual display.

[0006] In a third aspect, the embodiments of this application provide a robot that performs three-dimensional visualization display on a two-dimensional map by using the three-dimensional visualization method for a two-dimensional map as in the first aspect.

[0007] In a fourth aspect, the embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the three-dimensional visualization method for a two-dimensional map as in the first aspect are implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the three-dimensional visualization method of the two-dimensional map as in the first aspect are implemented.

[0009] The present application constructs three-dimensional representation information of a two-dimensional map, and performs three-dimensional modeling and rendering according to the three-dimensional representation information to obtain a three-dimensional space and perform visual display. A more realistic three-dimensional visualization rendering is performed on the two-dimensional map, giving users a more real experience. In this example, there is no restriction on the acquisition method of the two-dimensional map, and any two-dimensional map can be visually displayed through this embodiment, thereby making this embodiment simple and effective, highly customizable, highly versatile, and widely applicable. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 FIG. 1 shows one of the schematic flowcharts of the three-dimensional visualization method provided by an embodiment of the present application;

[0011] Figure 2 FIG. 2 shows another schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0012] Figure 3 FIG. 3 shows a third schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0013] Figure 4 FIG. 4 shows a fourth schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0014] Figure 5 FIG. 5 shows a fifth schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0015] Figure 6 FIG. 6 shows a sixth schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0016] Figure 7 FIG. 7 shows a seventh schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0017] Figure 8 FIG. 8 shows a first structural block diagram of the three-dimensional visualization device provided by an embodiment of the present application;

[0018] Figure 9 FIG. 9 shows a schematic overall solution diagram of the three-dimensional visualization method provided by an embodiment of the present application;

[0019] Figure 10 FIG. 10 shows a schematic diagram of secondary editing of the three-dimensional space provided by an embodiment of the present application;

[0020] Figure 11Shows a schematic diagram of three-dimensional theme editing and switching provided by an embodiment of the present application;

[0021] Figure 12 Shows the eighth schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application;

[0022] Figure 13 Shows a schematic diagram of a two-dimensional grid map provided by an embodiment of the present application;

[0023] Figure 14 Shows a schematic diagram of external building contour information provided by an embodiment of the present application;

[0024] Figure 15 Shows a schematic diagram of internal building contour information provided by an embodiment of the present application;

[0025] Figure 16 Shows a schematic diagram of the frame line of the three-dimensional representation provided by an embodiment of the present application;

[0026] Figure 17 Shows one of the three-dimensional map schematic diagrams of the three-dimensional visualization method provided by an embodiment of the present application;

[0027] Figure 18 Shows the second three-dimensional map schematic diagram of the three-dimensional visualization method provided by an embodiment of the present application;

[0028] Figure 19 Shows a schematic diagram of object position adjustment provided by an embodiment of the present application;

[0029] Figure 20 Shows a schematic diagram of object rotation adjustment provided by an embodiment of the present application;

[0030] Figure 21 Shows a schematic diagram of object stretching adjustment provided by an embodiment of the present application;

[0031] Figure 22 Shows the second structural block diagram of the three-dimensional visualization device provided by an embodiment of the present application;

[0032] Figure 23 Shows the structural block diagram of the electronic device provided by an embodiment of the present application; ]>

[0033] Figure 24 Shows a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.

[0034] Among them, Figures 8 to 24 The corresponding relationship between the reference numerals and the component names in the figure is:

[0035] 100: 3D visualization device; 110: 3D representation generation module; 120: web rendering module; 130: secondary editing module; 140: theme editing and switching module; 200: 2D grid map; 202: structured text data; 204: 3D space rendering; 206: secondary editing of 3D space; 208: editing and switching of 3D themes; 210: wooden box; 300: 3D visualization device 310: generation module; 320: display module; 400: electronic device; 402: processor; 404: memory; 1100: electronic device; 1101: radio frequency unit; 1102: network module; 1103: audio output unit; 1104: input unit; 11041: graphics processor; 11042: microphone; 1105: sensor; 1106: display unit; 11061: display panel; 1107: user input unit; 11071: touch panel; 11072: other input devices; 1108: interface unit; 1109: memory; 1110: processor. Detailed implementation manners

[0036] The technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application belong to the protection scope of the present application.

[0037] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data may be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order different from those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and the number of objects is not limited. For example, the first object may be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means an "or" relationship between the associated objects before and after.

[0038] The following combination of attached Figures 1 to 24 drawings will be used to illustrate in detail the 3D visualization method and device, robot, electronic device, and storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0039] In the embodiments of the present application, a 3D visualization method for a 2D map is provided. Figure 1 FIG. 1 shows one of the flow diagrams of the 3D visualization method provided in the embodiments of the present application. As Figure 1 shown, the 3D visualization method for a 2D map includes:

[0040] Step 102 : Obtain object space information and object position information in the two-dimensional map, and generate three-dimensional representation information based on the object space information and object position information.

[0041] Step 104 : construct a three-dimensional model of the object based on the three-dimensional representation information, render the three-dimensional model, obtain a three-dimensional space corresponding to the two-dimensional map, and perform visual display.

[0042] In related technologies, robot map representations are mostly simplified to two-dimensional grid maps, leaving users with only a two-dimensional image similar to a bird's-eye view on the client. Furthermore, due to issues with the 2D grid map construction algorithm and the accuracy of the LiDAR itself, the 2D image contains a large number of noise points, making direct 3D rendering impossible. Therefore, finding better map visualization to improve the user experience and a universal, efficient map rendering solution have become pressing challenges.

[0043] It is understood that the two-dimensional map in this embodiment refers to a two-dimensional map obtained after preprocessing the original two-dimensional laser map. The two-dimensional map is a two-dimensional map with clear measurements.

[0044] It is understandable that in a two-dimensional map, there are different types of objects. According to the objects that need to be displayed, the map division corresponding to the different types of objects to be displayed is obtained. This embodiment is suitable for visualization of multi-category maps with an unlimited number of categories.

[0045] In this embodiment, the two-dimensional map can be a two-dimensional grid map, the position information of the grid maps with different divisions is the object space information, the orientation information of the grid maps with different divisions is the object position information, the three-dimensional representation information is structured text, and the object space information and object position information are stored in the three-dimensional representation information.

[0046] This embodiment constructs 3D representation information for a 2D map and performs 3D modeling and rendering based on this 3D representation information to obtain a 3D space for visualization. This 3D visualization of the 2D map is more realistic, giving users a more immersive experience. In this example, there are no restrictions on how the 2D map is acquired; any 2D map can be visualized using this embodiment, making this embodiment simple and effective, highly customizable, versatile, and widely applicable.

[0047] In some embodiments of the present application, Figure 2 The second flow chart of the three-dimensional visualization method provided in the embodiment of the present application is shown as follows: Figure 2 As shown, obtaining the spatial information and position information of objects in the two-dimensional map, and generating three-dimensional representation information based on the spatial information and position information of the objects, specifically includes:

[0048] Step 202, obtain the object space information in the two-dimensional map.

[0049] Step 204, obtain the type corresponding to the object.

[0050] Step 206, obtain the first position information of the center point of the object under the absolute scale.

[0051] Step 208, based on the object space information and the type corresponding to the object, obtain the second position information of the object. The position information includes the first position information and the second position information.

[0052] Step 210, perform height alignment on the first position information.

[0053] Step 212, perform horizontal center alignment on the second position information.

[0054] Step 214, generate three-dimensional representation information based on the aligned first position information and second position information.

[0055] In this embodiment, for example, when the two-dimensional map adopts a two-dimensional grid map, the two-dimensional grid map is provided with metric information. After dividing the two-dimensional grid map by category, obtain the types of different divisions, store their numbers in the category field and integrate them into the three-dimensional representation information for storage.

[0056] In this embodiment, obtain the orientation information of the object in the divided two-dimensional grid map, and convert it into rotation information, that is, the first position information of the center point of the object under the absolute scale.

[0057] After determining the object space information and the type corresponding to the object, the second position information of the object can be obtained.

[0058] The first position information needs to be height-aligned to map the two-dimensional center points of different objects to the three-dimensional center points at corresponding different heights. The second position information needs to be horizontally center-aligned. That is, by statistically obtaining the outermost boundary information of the maximum and minimum xy two-dimensional boundary coordinate points of all current points, the coordinates of all points are horizontally center-aligned.

[0059] The first position information after height alignment and the second position information after horizontal center alignment are both stored in the three-dimensional representation information.

[0060] In this embodiment, the three-dimensional representation information is structured text and can be used to store content. For example, the three-dimensional representation information can be combined into a three-dimensional representation file in JSON format and transmitted to the rendering end through the network for subsequent rendering.

[0061] In this embodiment, by obtaining the information in the two-dimensional map, a three-dimensional representation file is generated, and subsequent three-dimensional visualization display is realized through the three-dimensional representation file, instead of directly generating a three-dimensional map from the two-dimensional map, avoiding the influence caused by excessive noise points in the two-dimensional map due to the accuracy of the lidar itself. This can make the map for subsequent three-dimensional visualization display more accurate and the rendering effect better.

[0062] In some embodiments of the present application, obtaining the first position information of the center point of an object under an absolute scale specifically includes:

[0063] Obtaining the rotation angle and position coordinates of the center point of the object under an absolute scale.

[0064] For example, when the two-dimensional map adopts a two-dimensional grid map, according to the orientation information of the grid maps divided differently, the rotation angle and position coordinates of the center point of the object under an absolute scale are calculated, where the first position information includes the rotation angle and position coordinates of the center point of the object under an absolute scale.

[0065] In this embodiment, the two-dimensional map has clear measurement information, and the rotation angle and position coordinates of the center point of the object under an absolute scale can be simply obtained.

[0066] In this embodiment, by obtaining the rotation angle and position coordinate information of the center point of the object under an absolute scale, it is convenient to subsequently establish a three-dimensional model of the object and perform rendering to achieve three-dimensional visualization display.

[0067] In some embodiments of the present application, Figure 3 shows the third schematic flow diagram of the three-dimensional visualization method provided by the embodiments of the present application. As Figure 3 shown, based on the object space information and the type corresponding to the object, obtaining the second position information of the object specifically includes:

[0068] Step 302, based on the type, obtaining the height and thickness of the object.

[0069] Step 304, based on the object space information, obtaining the length of the object.

[0070] For example, when the two-dimensional map adopts a two-dimensional grid map, according to the item types of the grid maps divided differently, the height and thickness of the object are calculated. According to the object space information, the length of the object is calculated, where the second position information includes the height, thickness, and length of the object.

[0071] In this embodiment, according to the object type, the height and thickness of the object are correspondingly determined, and according to the object space information, the length of the object is determined, and the second position information is simply obtained.

[0072] In this embodiment, based on the height, thickness, and length of the object, it is convenient to subsequently establish a three-dimensional model of the object and perform rendering to achieve three-dimensional visualization display.

[0073] In some embodiments of the present application, Figure 4 FIG. 4 shows a schematic flowchart of a three-dimensional visualization method provided by an embodiment of the present application. As Figure 4 shown, before obtaining the type corresponding to the object, it further includes:

[0074] Step 402: Construct the correspondence between the object and the type.

[0075] Step 404: Construct the correspondence between the type and the height and thickness.

[0076] It can be understood that in a two-dimensional grid map, for some objects of different types, the height and thickness are the same, so they can be classified into the same type.

[0077] It can be understood that there are multiple objects to be displayed on the two-dimensional map. First, the correspondence between the object and the type can be established, and then the correspondence between the type and the height and thickness can be established. In this embodiment, by classifying different objects into types, the correspondence between the object and its height and thickness can be simplified, the calculation can be simplified, and the usability of the method can be increased.

[0078] In some embodiments of the present application, the object space information includes the external building outline and / or the internal building outline.

[0079] It can be understood that the two-dimensional map in this embodiment is a preprocessed two-dimensional laser map. The preprocessing process mainly removes noise from the original two-dimensional map through image processing morphology to obtain a two-dimensional map with clear metrics.

[0080] It can be understood that, for example, in this embodiment, the space information may include the external building outline and / or the internal building outline, and the internal building outline may refer to the wall. The object space information is mapped onto a two-dimensional map with absolute scale.

[0081] It should be noted that the object categories applicable to this embodiment include but are not limited to these two display types. Currently, only these two types need to be displayed in common grid maps. This embodiment is not limited by the number of original types and can be adaptively extended according to the number of semantic types of objects included in the map.

[0082] In this embodiment, using the external building outline and / or the internal building outline, which are widely used in two-dimensional grid maps in related technologies, as the object space information can effectively simplify the method steps, while retaining scalability, facilitating future expansion of the object space information.

[0083] In some embodiments of the present application,Figure 5 FIG. 5 shows a schematic flowchart of the three-dimensional visualization method provided by an embodiment of the present application. As Figure 5 shown, based on the three-dimensional representation information, a three-dimensional model of an object is constructed, the three-dimensional model is rendered to obtain a three-dimensional space corresponding to the two-dimensional map, and visual display is performed. Specifically, it includes:

[0084] Step 502: Based on the three-dimensional representation information, construct a polygon network model at the corresponding position and direction.

[0085] Step 504: Use a preset theme style to render the polygon network model to obtain a three-dimensional space corresponding to the two-dimensional map, and perform visual display.

[0086] In this embodiment, the three-dimensional representation information is received, and then the format of the three-dimensional representation information is parsed. The three-dimensional representation information is structured text and can be in JSON format. In this embodiment, a polygon network model, i.e., a Mesh model, is constructed, and modeling and model rendering can be performed on the web side, and finally visual display is performed through the web side.

[0087] It can be understood that, based on the content in the three-dimensional representation information, a Mesh model can be constructed at the corresponding position and direction, and then the Mesh model is rendered using a preset theme style. Among them, the preset theme style can be a main style preset by the user himself, or a default theme style, or the theme style used during the previous rendering.

[0088] In this embodiment, based on the three-dimensional representation information, a three-dimensional model is constructed and rendered, that is, more realistic three-dimensional visualization rendering is performed on the two-dimensional plane grid map, giving users a more real experience.

[0089] In some embodiments of the present application, using a preset theme style to render the polygon network model specifically includes:

[0090] Using a preset theme style, through a rendering engine, render the polygon network model on the web side.

[0091] For the method of image rendering based on WebGL in the related art, the model is parsed and rendered by directly importing an existing model. However, in this embodiment, the two-dimensional visualization map obtained by the robot processing does not have a corresponding three-dimensional model, and due to the problem of too many noise points in the two-dimensional map, the map cannot be directly rendered.

[0092] In the three-dimensional modeling and rendering methods of related technologies, methods that use voxels or utilize depth information to construct a three-dimensional space determine the position of an object in space through the pose information of a camera. However, currently, two-dimensional grid maps lack information on the three-dimensional height, and cannot accurately determine the spatial structure and the position of an object in space. Therefore, three-dimensional rendering of the map cannot be performed.

[0093] In this embodiment, for example, the rendering engine can use WebGL. According to a preset theme style, through WebGL, the Mesh object is rendered to complete the visualization display of the two-dimensional map on the web page.

[0094] In this embodiment, if the user does not have a preset theme style, the display will be in the default style.

[0095] In this embodiment, in order to use WebGL as the rendering engine to render three-dimensional representation information on the web page, it has strong generality and a wide range of applications.

[0096] In some embodiments of the present application, before rendering the polygon network model with a preset theme style, the three-dimensional visualization method of the two-dimensional map further includes:

[0097] Construct at least one theme style.

[0098] In this embodiment, before rendering, a theme style needs to be constructed. The theme style can be edited by the user, or multiple preset theme styles can be provided for the user to select.

[0099] For example, when the user edits the theme style, the user can edit different types of object styles and save the custom theme, including but not limited to color, texture map, thickness, whether to generate shadows, whether to receive shadows, specular reflection, transparency, lighting model, etc.

[0100] For example, when constructing the theme style of the grid map, the user can edit the attribute information corresponding to different category divisions mentioned in the "rendering process of the grid map" by clicking. That is, the user freely combines the desired color scheme and style by selecting and editing the relevant attributes corresponding to the Mesh models generated by different divisions, such as surface roughness, specular reflection, lighting model, geometric information, body color, contour color, emission color, whether to receive shadows, etc. And store the relevant attributes of all division categories in the user-defined theme style representation and three-dimensional representation. In the subsequent model rendering process, the method of this embodiment will preferentially select the corresponding style for rendering. The implementation details are to load and parse the information, and in the process of WebGL rendering, combined with the three-dimensional space representation information, simulate the final rendering effect according to the geometric shapes and lighting models of the corresponding attributes of different divisions.

[0101] This embodiment supports users in constructing a theme style, enhancing the practicality of the method. For users, they can set the theme style according to their habits and needs, simplifying user operations and improving the user experience.

[0102] In some embodiments of the present application, Figure 6 shows the sixth flowchart of the three-dimensional visualization method provided by the embodiments of the present application. As Figure 6 shown, the three-dimensional visualization method of the two-dimensional map further includes:

[0103] Step 602, in response to detecting the selection of the theme style to be switched.

[0104] Step 604, switch the theme style to the theme style to be switched and display it.

[0105] In this embodiment, the user can select an existing theme style to switch the display style.

[0106] This embodiment can use WebGL on the web page to render a three-dimensional model, and supports selecting a preset theme style during rendering. After rendering, the rendered three-dimensional map can be edited again, that is, when the user performs an operation to switch the theme style, in response to detecting that the user performs an operation to select the theme to be switched, the currently displayed theme style is replaced with the theme to be switched selected by the user.

[0107] This embodiment renders a three-dimensional model on WebGL of the web page, and supports the functions of rendering models with different theme styles and subsequent secondary three-dimensional map editing. It performs a more realistic three-dimensional visualization rendering on the planar grid map of the robot, giving the user a more real experience. The setting of switching the theme style simplifies user operations and improves the user experience.

[0108] In some embodiments of the present application, Figure 7 shows the seventh flowchart of the three-dimensional visualization method provided by the embodiments of the present application. As Figure 7 shown, after obtaining the three-dimensional space corresponding to the two-dimensional map and performing visual display, the three-dimensional visualization method of the two-dimensional map further includes:

[0109] Step 702, obtain the object to be added.

[0110] Step 704, adjust the third position information of the object to be added.

[0111] Step 706, based on the third position information, add the object to be added to the three-dimensional space and perform visual display.

[0112] In this embodiment, the user can add a custom object in the already rendered three-dimensional space, and the custom object is the object to be added.

[0113] In this embodiment, the user can select a built-in object as the object to be added and adjust the third position information of the object to be added. For example, the third position can be the position, rotation angle, scaling size, etc. of the object to be added. After determining the third position information, the object to be added is displayed.

[0114] In this embodiment, the information of the object to be added, that is, the information of the newly added three-dimensional object, can be saved into the three-dimensional representation information, that is, a structured text file. When rendering next time, the corresponding three-dimensional representation information edited before can be restored.

[0115] In this embodiment, for the object to be added, the basic information can also be locked and restricted. For example, the height of the y-axis can be locked to keep it at the same height as the map reference plane in the three-dimensional space.

[0116] In this embodiment, adding objects on the three-dimensional visualization map is supported, which simplifies the user operation and improves the user experience. Specific Embodiment 1:

[0118] The main purpose of this embodiment is to realize the three-dimensional visualization rendering of a two-dimensional raster map on the web page, and at the same time support the editability of the generated three-dimensional map and the configurability of the display style. In the related technologies, there is no technical solution for directly performing three-dimensional visualization on the two-dimensional raster map of the robot, and there is only the visualization of the three-dimensional raster map and the construction of the three-dimensional space based on the depth map. And the related technologies also do not provide a post-editing function for the three-dimensional map generated from the two-dimensional raster map, as well as a multi-theme configuration and switching function. This embodiment can directly act on the existing two-dimensional raster map and generate a three-dimensional space with a good user experience.

[0119] Figure 8 One of the structural block diagrams of the three-dimensional visualization device provided by the embodiment of the present application is shown. As Figure 8 shown, the three-dimensional visualization device 100 includes: a three-dimensional representation generation module 110, a web page rendering module 120, a secondary editing module 130, and a theme editing and switching module 140.

[0120] The three-dimensional representation generation module 110 is used to embed the key information of the two-dimensional grid map into the three-dimensional representation as a medium for information interaction. The web rendering module 120 is used to parse the three-dimensional representation generated by the three-dimensional representation generation module 110 and render it using the WebGL engine. The secondary editing module 130 is used to allow users to customize the rotation angle and position of three-dimensional objects in the three-dimensional space rendered by WebGL. The theme editing and switching module 140 provides a function of configurable theme styles and dynamic theme switching.

[0121] The precondition of this embodiment is a preprocessed original two-dimensional laser map. The preprocessing method can adopt the method for extracting the indoor building framework of the two-dimensional solid-state laser map. Its preprocessing process mainly uses a series of algorithms such as image processing morphology to remove noise from the original two-dimensional grid map, obtain the external contour of the building and the internal wall contour, and map them to a two-dimensional map with absolute scale. The above preprocessing method is only an exemplary description of a preprocessing method that meets the preconditions of this embodiment and should not be construed as a limitation of this embodiment. This embodiment is applicable to any visualization of a two-dimensional map with a clear metric as the output result.

[0122] The three-dimensional representation generation module 110 obtains the object space information of two types, namely the external and internal building contours, through the preconditions. It should be noted that the object categories applicable to this embodiment include but are not limited to these two display types. Currently, the common grid maps only have these two types that need to be displayed. This embodiment is not limited by the number of original types and can be adaptively extended according to the number of semantic types of objects included in the map.

[0123] The three-dimensional representation generation module 110 sets corresponding type tags for different types of objects according to the object space information, calculates the rotation angle and position coordinates of the center point of the object under the absolute scale, and sets the corresponding height, thickness of the object according to the corresponding type, and calculates the length of the object according to the object space information.

[0124] Among them, the above position coordinates need to be height-aligned. The three-dimensional representation generation module 110 maps the two-dimensional center points of different objects to the three-dimensional center points at corresponding different heights. The above position information (the corresponding height, thickness, and length of the object) needs to be horizontally center-aligned. The three-dimensional representation generation module 110 horizontally center-aligns the coordinates of all points by statistically calculating the maximum and minimum xy two-dimensional boundary coordinate points (the outermost boundary information) of all current points.

[0125] The three-dimensional representation generation module 110 combines all the above output results into a three-dimensional representation file in JSON format (structured text three-dimensional representation) and transmits it to the rendering end through the network.

[0126] After the web - end rendering module 120 obtains the three - dimensional representation of the structured text generated by the three - dimensional representation generation module 110, it will perform format parsing through the parsing module and construct Mesh objects at corresponding positions and orientations. The web - end rendering module 120 will perform WebGL rendering of corresponding types and styles on the above - mentioned Mesh objects according to the display theme style (if the user does not customize the theme style, it will be displayed according to the default style).

[0127] As Figure 9 shown, the above process in this embodiment is to generate structured text data 202 (three - dimensional representation) according to the two - dimensional grid map 200 and perform three - dimensional space rendering 204.

[0128] As Figure 10 shown, the user can, through the secondary editing module 130, realize the function of adding custom objects in the already rendered three - dimensional space (i.e., secondary editing 206 of the three - dimensional space). The user can select built - in objects, adjust the position, rotation angle, scaling size, etc. of the objects, place the objects in the three - dimensional space, and the information of the newly added three - dimensional objects is also saved in the structured text file (three - dimensional representation file).

[0129] The user can edit the styles of different types of objects through the theme editing and switching module 140 and save the customized theme (including but not limited to color, texture map, thickness, whether to generate shadows, whether to receive shadows, specular reflection, transparency, lighting model, etc.). The user can also select an existing theme style through the theme editing and switching module 140 to switch the display style. As Figure 11 shown, the user can edit and switch the theme style (i.e., editing and switching 208 of the three - dimensional theme).

[0130] This embodiment starts from the processing of basic grid map information, constructs a three - dimensional representation that can be efficiently transmitted, and performs parsing and visual rendering of the three - dimensional representation through the WebGL rendering engine on the web - end. This embodiment also provides a set of three - dimensional map editors that can be secondarily edited, as well as a set of display theme switching devices with multiple configurable styles.

[0131] In this embodiment, a three-dimensional representation is constructed from a two-dimensional grid map, and the three-dimensional representation is rendered into a three-dimensional model on the web browser WebGL. The method supports rendering models in different theme styles and subsequent secondary three-dimensional map editing. It performs a more realistic three-dimensional visualization rendering of the robot's planar grid map, providing a more immersive experience for users. It also plays a very positive role in the secondary verification of the map effect and rapid three-dimensional spatial modeling. At the same time, this embodiment has good versatility. The pre-input conditions of this embodiment are not limited to the two-dimensional grid map generated by lidar. Any two-dimensional map can be visually displayed using the method of this embodiment. The method of this embodiment is simple and effective, highly customizable, and has strong versatility and a wide range of applications. Specific Embodiment 2:

[0133] [[ID=*5]]This embodiment provides a method for generating a three-dimensional representation of a corresponding three-dimensional space using the contour information provided by a two-dimensional grid map, and performing three-dimensional rendering on the representation through a WebGL rendering engine to obtain an interactive three-dimensional map. As Figure 12 shown, the three-dimensional visualization method of a two-dimensional map includes:

[0134] Step 802, rendering of the three-dimensional map.

[0135] In the rendering process of the three-dimensional map, first, the map division of different types of objects to be displayed needs to be obtained according to the two-dimensional map. Figure 13 Taking a grid map as an example, when the type is two categories, the map division includes the inner contour and the outer contour of the building. The outer contour of the building is as Figure 14 shown, and the inner contour of the building is as Figure 15 shown. It should be noted that this embodiment is applicable to the visualization of multi-category maps with an unlimited number of categories. That is, the number and type of example map categories should not be construed as a limitation on this embodiment. For the purpose of illustrating the process and method of this embodiment, only two classification categories are shown. After obtaining the category divisions similar to Figure 14 and Figure 15 shown, the method for generating a corresponding three-dimensional representation according to the metric information of the grid map specifically includes:

[0136] (1) Obtain the position information of the grid map in different divisions and store it in structured text (i.e., the three-dimensional representation);

[0137] (2) Obtain the orientation information of the grid map in different divisions, convert it into rotation information, and store it in structured text (i.e., the three-dimensional representation);

[0138] (3) Obtain the length information of different divisions in the grid map, record it as the corresponding length information, and store it in structured text (i.e., the three-dimensional representation);

[0139] (4) Obtain the types of different divisions, store their numbers in the category field, and integrate them into the structured text (i.e., three-dimensional representation);

[0140] (5) Center the corresponding position information in steps (1) and (2) above according to the maximum and minimum positions of all divisions in the grid map, that is, perform horizontal center alignment of all positions.

[0141] (6) Store all the structured text information.

[0142] (7) Parse the format of the three-dimensional representation of the structured text and construct Mesh objects corresponding to the positions and directions. According to the display theme style, perform WebGL rendering of the above Mesh objects corresponding to the types and styles. The three-dimensional representation wireframe model generated on the computer web page, and the interface diagram of the three-dimensional representation wireframe model generated on the computer web page is as Figure 16 shown. The three-dimensional map obtained by WebGL rendering using the default theme style, and the interface diagram of the three-dimensional map generated on the computer web page is as Figure 17 shown. The three-dimensional map obtained by WebGL rendering using another theme style, and the interface diagram of the three-dimensional map generated on the computer web page is as Figure 18 shown. Figure 17 And Figure 18 When rendering with different theme styles, the colors of the three-dimensional map can be different. For example, Figure 17 it can be gray, Figure 18 it can be blue.

[0143] In the above process, (1) and (2) are used to obtain the position and rotation information of all three-dimensional representations, and (3) and (4) are used to store the information of the Mesh for rendering. (5) is used to perform the centering operation of the entire three-dimensional map because the metrics and positions of the grid map may not be suitable for direct rendering display.

[0144] Step 804, secondary editing and processing of the three-dimensional map.

[0145] In the secondary editing and processing module of the grid map, the user can select the object to be added and edit the position, size, and angle of the object in the three-dimensional map. Assume that the object selected by the user to add is a wooden box 210. The user can adjust the position of the wooden box 210 through the x-axis, y-axis, and z-axis. The interface diagram of adjusting the position of the wooden box 210 through the x-axis, y-axis, and z-axis on the computer web page is as Figure 19 shown. The user can also adjust the rotation angles of the three orientations of the wooden box 210 through the ring. The interface diagram of adjusting the rotation angles of the three orientations of the wooden box 210 through the ring on the computer web page is as Figure 20As shown. The user can also stretch the wooden box 210 in three directions of the x-axis, y-axis, and z-axis. Stretching the wooden box 210 in three directions of the x-axis, y-axis, and z-axis on the computer web page is as Figure 21 shown. In summary, it is possible to perform secondary editing by dragging the x-axis, y-axis, and z-axis or the ring of three colors respectively. Among them, the three axes of the object are the x-axis, y-axis, and z-axis, which can respectively correspond to three different colors. For example, they can be red, blue, and green. The ring can also be set with three colors, corresponding to the rotation of the three axes respectively. In this embodiment, the corresponding editing results will be stored in the three-dimensional representation in the same way, and the corresponding three-dimensional representation information after the previous editing will be restored during the next rendering. And in the actual application of this embodiment, certain locking restrictions can be imposed on the basic information of the added object, that is, it can be preset that the height of the y-axis (green) of the object added by the user is fixed, and locked at the height flush with the map reference plane in the three-dimensional space, etc.

[0146] Step 806, multi-topic style editing and switching of the three-dimensional map.

[0147] During the multi-topic editing and switching of the raster map, the user can edit by clicking on the attribute information corresponding to different category divisions mentioned in the "rendering process of the raster map". That is, the user freely combines the desired color scheme and style by selecting and editing the relevant attributes corresponding to different divisions to generate Mesh, such as surface roughness, reflectivity, lighting model, geometric information, body color, contour color, emission color, whether to receive shadows, and other attributes. And store the relevant attributes of all division categories in the user-defined theme style representation and three-dimensional representation. In the subsequent model rendering process, this embodiment will preferentially select the corresponding style for rendering. The implementation details are to load and parse the information, and during the WebGL rendering process, combine the three-dimensional space representation information to simulate the final rendering effect according to the geometric shape and lighting model of the corresponding attributes of different divisions.

[0148] In the three-dimensional visualization method provided by the embodiments of the present application, the execution subject can be a three-dimensional visualization device. In the embodiments of the present application, taking the three-dimensional visualization device executing the three-dimensional visualization method as an example, the three-dimensional visualization device provided by the embodiments of the present application is described.

[0149] In some embodiments of the present application, a three-dimensional visualization device is provided. Figure 22 The structural block diagram of the three-dimensional visualization device provided by the embodiments of the present application is shown, as Figure 22As shown, the three-dimensional visualization device 300 for a two-dimensional map includes a generation module 310 and a display module 320. The generation module 310 is configured to obtain the object space information and object position information in the two-dimensional map, and generate three-dimensional representation information based on the object space information and the object position information. The display module 320 is configured to construct a three-dimensional model of the object, render the three-dimensional model, obtain the three-dimensional space corresponding to the two-dimensional map, and perform visual display.

[0150] In this embodiment, by constructing the three-dimensional representation information of the two-dimensional map and performing three-dimensional modeling and rendering according to the three-dimensional representation information, a three-dimensional space is obtained and visually displayed. A more realistic three-dimensional visualization rendering is performed on the two-dimensional map, giving users a more real experience. In this example, there is no restriction on the acquisition method for the two-dimensional map, and any two-dimensional map can be visually displayed through this embodiment. Furthermore, this embodiment is simple and effective, highly customizable, has strong versatility, and a wide range of applications.

[0151] The three-dimensional visualization device 300 provided in the embodiment of the present application can implement each process of the above three-dimensional visualization method embodiment and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0152] The three-dimensional visualization device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiment of the present application does not make specific limitations.

[0153] The three-dimensional visualization device in the embodiment of the present application can be a device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems. The embodiment of the present application does not make specific limitations.

[0154] The 3D visualization device provided by the embodiments of the present application can implement each process implemented by the above method embodiments. To avoid repetition, it will not be elaborated here.

[0155] The embodiments of the present application further provide a robot that performs 3D visualization display on a 2D map by using the above 3D visualization method of the 2D map. It can implement each process of the above 3D visualization method embodiments and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0156] Optionally, as Figure 23 shown, the embodiments of the present application further provide an electronic device 400. The electronic device 400 includes a processor 402 and a memory 404. A program or instruction that can run on the processor 402 is stored on the memory 404. When the program or instruction is executed by the processor 402, it implements each step of the above method embodiments and achieves the same technical effects. To avoid repetition, it will not be elaborated here.

[0157] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0158] Figure 24 Schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0159] The electronic device 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, and a processor 1110 and other components.

[0160] Those skilled in the art can understand that the electronic device 1100 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1110 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 24 The electronic device structure shown in does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0161] Among them, the processor 1110 is used to obtain the object space information and object position information in the 2D map, and generate 3D representation information based on the object space information and object position information.

[0162] The processor 1110 is used to construct an object 3D model based on the 3D representation information, render the 3D model, obtain the 3D space corresponding to the 2D map, and perform visual display.

[0163] The processor 110 provided in the embodiment of the present application can implement each process of the above - mentioned three - dimensional visualization method embodiment and achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0164] It should be understood that in the embodiment of the present application, the input unit 1104 may include a Graphics Processing Unit (GPU) 11041 and a microphone 11042. The graphics processor 11041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light - emitting diode, etc. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. The other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0165] The memory 1109 can be used to store software programs and various data. The memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1109 may include a volatile memory or a non-volatile memory, or the memory 1109 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1109 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0166] The processor 1110 may include one or more processing units; optionally, the processor 1110 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1110.

[0167] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above-mentioned embodiment of the three-dimensional visualization method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0168] Among them, the processor is the processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disc, etc.

[0169] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above-mentioned three-dimensional visualization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0170] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0171] The embodiments of the present application provide a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above-mentioned three-dimensional visualization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0172] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the methods and devices in the embodiments of the present application are not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0173] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present application.

[0174] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. A three-dimensional visualization method for a two-dimensional map, characterized in that, Comprising: Obtain the object space information and object position information in the two-dimensional map, and generate three-dimensional representation information based on the object space information and the object position information; Based on the three-dimensional representation information, construct a three-dimensional model of the object, render the three-dimensional model, obtain the three-dimensional space corresponding to the two-dimensional map, and perform visual display; The obtaining of the object space information and object position information in the two-dimensional map and generating the three-dimensional representation information based on the object space information and the object position information specifically includes: Obtain the object space information in the two-dimensional map; Obtain the type corresponding to the object; Obtain the first position information of the center point of the object under the absolute scale; Based on the object space information and the type corresponding to the object, obtain the second position information of the object, where the position information includes the first position information and the second position information; Perform height alignment on the first position information; Perform horizontal center alignment on the second position information; Based on the aligned first position information and second position information, generate the three-dimensional representation information.

2. The three-dimensional visualization method of the two-dimensional map according to claim 1, wherein The obtaining of the first position information of the center point of the object under the absolute scale specifically includes: Obtain the rotation angle and position coordinates of the center point of the object under the absolute scale.

3. The three-dimensional visualization method of the two-dimensional map according to claim 1, wherein The obtaining of the second position information of the object based on the object space information and the type corresponding to the object specifically includes: Based on the type, obtain the height and thickness of the object; Based on the object space information, obtain the object length.

4. The three-dimensional visualization method of the two-dimensional map according to claim 3, wherein Before obtaining the type corresponding to the object, it further includes: Construct the corresponding relationship between the object and the type; Construct the corresponding relationship between the type and the height and thickness.

5. The three-dimensional visualization method of the two-dimensional map according to claim 1, characterized in that, The object space information includes the external building outline and / or the internal building outline.

6. The three-dimensional visualization method of the two-dimensional map according to claim 1, wherein The constructing of the three-dimensional model of the object based on the three-dimensional representation information, rendering the three-dimensional model, obtaining the three-dimensional space corresponding to the two-dimensional map, and performing visual display specifically includes: Based on the three-dimensional representation information, construct a polygon network model at the corresponding position and direction; Use a preset theme style to render the polygon network model, obtain the three-dimensional space corresponding to the two-dimensional map, and perform visual display.

7. The three-dimensional visualization method of the two-dimensional map according to claim 6, characterized in that, The using of the preset theme style to render the polygon network model specifically includes: Use a preset theme style and, through a rendering engine, render the polygon network model on the web page.

8. The three-dimensional visualization method of the two-dimensional map according to claim 6, characterized in that Before using the preset theme style to render the polygon network model, the three-dimensional visualization method of the two-dimensional map further includes: Construct at least one of the theme styles.

9. The three-dimensional visualization method of the two-dimensional map according to claim 6, wherein It further includes: In response to detecting the selection of a theme style to be switched; Switch the theme style to the theme style to be switched and perform display.

10. The three-dimensional visualization method of a two-dimensional map according to any one of claims 1 to 9, characterized in that, After obtaining the three-dimensional space corresponding to the two-dimensional map and performing visual display, the three-dimensional visualization method of the two-dimensional map further includes: Obtain an object to be added; Adjust the third position information of the object to be added; Based on the third position information, add the object to be added to the three-dimensional space and perform visual display.

11. A three-dimensional visualization device for a two-dimensional map, characterized in that, Comprising: A generation module, configured to obtain object spatial information and object position information in a two-dimensional map, and generate three-dimensional representation information based on the object spatial information and the object position information; A display module, configured to construct a three-dimensional model of an object, render the three-dimensional model to obtain a three-dimensional space corresponding to the two-dimensional map, and perform visual display; The generation module is further configured to obtain the object spatial information in the two-dimensional map; Obtain the type corresponding to the object; Obtain first position information of the center point of the object under an absolute scale; Based on the object spatial information and the type corresponding to the object, obtain second position information of the object, where the position information includes the first position information and the second position information; Perform height alignment on the first position information; Perform horizontal center alignment on the second position information; Generate the three-dimensional representation information based on the aligned first position information and the second position information.

12. An electronic device, characterized in that, Comprising: A memory, on which programs or instructions are stored; A processor, configured to implement the steps of the three-dimensional visualization method of the two-dimensional map according to any one of claims 1 to 10 when executing the programs or instructions.

13. A readable storage medium storing a program or instructions thereon, characterized in that, When the programs or instructions are executed by the processor, the steps of the three-dimensional visualization method of the two-dimensional map according to any one of claims 1 to 10 are implemented.

Images