Metaverse space construction system and method, electronic device, and storage medium

Abstract

The application discloses a meta-universe space construction system and method, electronic equipment and a computer readable storage medium. The system comprises a plurality of user nodes and a plurality of data processing nodes, the user nodes construct a first virtual space unit based on the collected visual image and geographical position information of the current real scene; the data processing nodes communicate with one or more user nodes to receive the first virtual space unit data sent by the user nodes, and fit two or more virtual geographical position adjacent / overlapping first virtual space units together to form a second virtual space unit; at least two data processing nodes communicate with each other, and two data processing nodes or any one of them belonging to two virtual geographical position adjacent / overlapping second virtual space units fit the two second virtual space units together. The embodiment disclosed by the application can solve the technical problems of high cost and slow progress in constructing a meta-universe space.

Description

Technical Field

[0001] This invention relates to the field of virtual technology, and in particular to a metaverse space construction system, method, electronic device, and storage medium. Background Technology

[0002] The metaverse, also known as the virtual world, is an open and shared online platform that integrates information technology, communication technology, AR, VR, and other virtual technologies. It is a vast and evolving virtual universe. Although currently limited by the level of technological development, its applications mainly include games, social networking, advertising and marketing, and virtual offices, these applications will expand over time as technology advances, and more and more people will enter the metaverse. The metaverse space itself is the fundamental carrier of this virtual universe, and the quality of its construction directly affects the various scenarios within the metaverse and people's experiences.

[0003] Based on the connection between the metaverse and the real world, they can be roughly divided into three types: the first is a digital world that is completely detached from the real world, such as some online games created by game companies, where people are completely placed in a parallel space that is completely different from reality after entering the game; the second is a fusion space that is a superposition and integration of virtual and real spaces, such as the current use of AR technology to create a digital space that completely overlaps with reality based on the current environment, and to achieve the purpose of augmented reality by adding virtual signs and 3D models in this space; the third is a virtual space that restores and simulates the real space at a 1:1 scale, such as the virtual space in a virtual office scene.

[0004] To construct a metaverse space that replicates reality at a 1:1 scale, spatial information about the real world needs to be collected and modeled. Currently, spatial modeling technologies such as computer 3D vision and spatial laser scanning are becoming increasingly mature, typically with professionals building virtual spaces tailored to specific applications, allowing users to simply use the product. This approach is sufficient for specific products utilizing virtual space for particular applications. However, for the vast metaverse space, relying on professionals and specialized equipment for construction would not only require a significant investment of manpower and resources but also result in slow progress, failing to meet the development needs of the metaverse. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a metaverse space construction system, method, electronic device, and computer-readable storage medium to solve the technical problems of high cost and slow progress caused by professionals and specialized equipment constructing metaverse spaces.

[0006] In a first aspect, embodiments of the present invention provide a metaverse space construction system, including multiple user nodes and multiple data processing nodes, wherein...

[0007] The user node collects visual images and geographic location information of the current real scene, and constructs a first-level virtual space unit based on the collected visual images and geographic location information of the current real scene. The virtual geographic location information of the first-level virtual space unit in the metaverse space corresponds one-to-one with the real geographic location information in the real scene.

[0008] The data processing node communicates with one or more user nodes and is configured to receive first-level virtual space unit data sent by the user nodes, and to fit two or more first-level virtual space units that are adjacent / overlapping in virtual geographic locations together to form a second-level virtual space unit.

[0009] At least two data processing nodes communicate with each other and are configured to exchange the virtual geographic location information of their respective secondary virtual spatial units. When two or more secondary virtual spatial units belonging to different data processing nodes are adjacent / overlapping in virtual geographic location, the two or more data processing nodes or any one of them will fit the two secondary virtual spatial units together.

[0010] Secondly, embodiments of the present invention provide a method for constructing a metaverse space, which is applied to user nodes in the aforementioned system. The method includes the following steps:

[0011] Collect visual images of the current real-world scene and the location information of the user terminal device in the current real-world scene;

[0012] Based on the visual image information, obtain the three-dimensional geometric structure information and texture mapping information of the current real scene;

[0013] Based on the aforementioned three-dimensional geometric structure information, a three-dimensional reconstruction is performed to obtain a three-dimensional geometric model of the real scene;

[0014] The three-dimensional geometric model is rendered based on the texture mapping information to obtain a virtual model of the real scene; and

[0015] Based on the location information and visual image information of the user terminal device in the current real scene, the virtual location information of the virtual model in the metaverse space is determined and calibrated to obtain the first-level virtual space unit of the user's current real scene, wherein the virtual geographical location information of the metaverse space corresponds one-to-one with the real geographical location information of the real scene.

[0016] Thirdly, embodiments of the present invention provide a metaverse space construction device, which is applied in a user node of the aforementioned system, including an image acquisition module, a 3D reconstruction module, a texture mapping module, a rendering module, a positioning module, and a calibration module. The image acquisition module is used to acquire visual image information of the current real-world scene; the 3D reconstruction module is connected to the image acquisition module and configured to generate 3D geometric structure information of the current real-world scene based on the visual image information, and to perform 3D reconstruction based on the 3D geometric structure information to obtain a 3D geometric model of the real-world scene; the texture mapping module is connected to the image acquisition module and configured to generate texture mapping information based on the visual image information. The rendering module is connected to the 3D reconstruction module and the texture mapping module, respectively, and is configured to render the 3D geometric model based on the texture mapping information to obtain a virtual model of the real scene; the positioning module is used to collect the positioning information of the user terminal device in the current real scene; the calibration module is connected to the rendering module and the positioning module, respectively, and determines the virtual position information of the virtual model in the metaverse space based on the positioning information and visual image information of the user terminal device in the current real scene and performs calibration to obtain the first-level virtual space unit of the user's current real scene, wherein the virtual geographical location information of the metaverse space corresponds one-to-one with the real geographical location information of the real scene.

[0017] Fourthly, embodiments of the present invention provide an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method steps executed in the user node or data processing point as described above.

[0018] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method steps performed in the user node or data processing point as described above.

[0019] In a sixth aspect, embodiments of the present invention provide a computer program product, the computer program product including computer program instructions, which, when executed by a processor, implement the method steps performed in the user node or data processing point as described above.

[0020] In a seventh aspect, embodiments of the present invention provide a terminal device, which includes at least a processor, a memory, an image acquisition module, and a location acquisition module, wherein the memory stores computer program instructions, and the processor executes the computer program instructions to implement the method executed in the user node.

[0021] This invention provides a user co-construction and sharing model, in which users build basic first-level virtual space units, thereby overcoming the problem that when professionals use specialized equipment to build the metaverse space, it requires a large investment of manpower and resources and progresses slowly. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below.

[0023] Figure 1 This is a schematic diagram of a metaverse space construction system according to an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of a metaverse space construction system according to another embodiment of the present invention;

[0025] Figure 3 This is a flowchart illustrating the process of constructing a user node in the metaverse space according to an embodiment of the present invention.

[0026] Figure 4 This is a flowchart of a method for a user node to generate a first-level virtual space unit of the user's current real-world scene according to an embodiment of the present invention;

[0027] Figure 5 This is a flowchart of a method for calibrating a virtual model based on geographic location information according to an embodiment of the present invention;

[0028] Figure 6 This is a flowchart of a method for constructing a metaverse space according to an embodiment of the present invention;

[0029] Figure 7 This is a flowchart of a method for constructing a metaverse space according to another embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of a metaverse space construction device according to an embodiment of the present invention;

[0031] Figure 9 This is a schematic diagram of a metaverse space construction device according to another embodiment of the present invention; and

[0032] Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0033] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided to make the principles and spirit of the present invention clearer and more thorough, enabling those skilled in the art to better understand and implement the principles and spirit of the present invention. The exemplary embodiments provided herein are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described herein without inventive effort are within the scope of protection of the present invention.

[0034] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, computer-readable storage medium, or computer program product. Therefore, the present invention can be specifically implemented in at least one of the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0035] According to embodiments of the present invention, the present invention seeks protection for a metaverse space construction system, method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0036] In this document, the elements defined by the phrase "including..." (e.g., components, parts, processes, steps) do not exclude the existence of other elements besides those listed; that is, they may also include other elements not explicitly listed. In this document, any elements and their quantities in the accompanying drawings are for illustrative purposes only and not for limitation, and any names in the drawings are for distinction only and have no limiting meaning.

[0037] The principles and spirit of the present invention will be explained in detail below with reference to several exemplary or representative embodiments.

[0038] Figure 1This is a schematic diagram of a metaverse space construction system according to an embodiment of the present invention. In this embodiment, the metaverse space construction system includes multiple user nodes and multiple data processing nodes. The user nodes collect visual images and geographic location information of the current real-world scene, and construct primary virtual space units based on the collected visual images and geographic location information. The virtual geographic location information of the primary virtual space unit in the metaverse space corresponds one-to-one with the real geographic location information in the real-world scene. The data processing nodes communicate with one or more user nodes to receive primary virtual space unit data sent by the user nodes, and fit two or more primary virtual space units with adjacent / overlapping virtual geographic locations together to form a secondary virtual space unit. At least two data processing nodes communicate with each other, exchanging the virtual geographic location information of their respective secondary virtual space units. When two or more secondary virtual space units belonging to different data processing nodes are geographically adjacent / overlapping, the two data processing nodes, or any one of them, fit the two secondary virtual space units together.

[0039] Figure 1 The user nodes in this example are, for example, personal user terminal devices such as smartphones and tablets, while the data processing nodes are, for example, distributed, decentralized servers or server clusters. In this embodiment, the hardware performance of the user terminal device is sufficient to support various calculations for generating virtual space units. Therefore, the user terminal device collects visual images and geographical locations of the user's real-world scene, generates a first-level virtual space unit, and sends the first-level virtual space unit data to the server or server cluster acting as a data processing node. In another embodiment, such as... Figure 2 As shown, the user node includes user terminal devices and a server. The server is connected to multiple user terminal devices and to one or more data processing nodes. When the hardware performance of a user terminal device is low and cannot complete the corresponding calculations to obtain a virtual space unit, the user terminal device sends the visual image and geographical location information of the user's current real-world scene to the server. The server constructs a first-level virtual space unit corresponding to the user terminal device based on the visual image and geographical location information of the user's current real-world scene collected by the user terminal device, and then sends the first-level virtual space unit data to a data processing node.

[0040] This invention provides a user co-construction and sharing model, in which users build basic first-level virtual space units, thereby overcoming the problem that when professionals use specialized equipment to build the metaverse space, it requires a large investment of manpower and resources and progresses slowly.

[0041] In the following description, unless otherwise specified, user nodes refer to all structures. Figure 1 and Figure 2 The two types of user nodes in the example.

[0042] Figure 3 This is a flowchart illustrating the process of constructing a user node in the metaverse space according to an embodiment of the present invention. The method includes:

[0043] Step S1: Generate a first-level virtual space unit representing the user's current real-world scene.

[0044] Step S2: Obtain the connection parameters of the user node. In one embodiment of this invention, to enable a user node to connect to a data processing node, connection parameters are configured for both the data processing node and the user node, such as a real-world geographical location range, a network IP address range, or a metaverse spatial geographical location range. When a user node needs to connect to a data processing node, its connection parameters need to be determined, such as the user node's geographical location, network IP address, and the spatial geographical location of the generated virtual space unit in the metaverse.

[0045] Step S3: Determine the corresponding data processing node based on the connection parameters. The user node searches for a matching data processing node based on the connection parameters and establishes a connection with it. In this invention, each data processing node is configured with connection parameters, connects to a matching user node based on the connection parameters, and communicates with two or more data processing nodes adjacent to the connection parameter value. Taking the connection parameter as the geographic location range of the metaverse space as an example, each data processing node is responsible for constructing a virtual space of a geographic location range, i.e., a secondary virtual space unit. When the geographic location of the primary virtual space unit constructed by the user node is within the geographic location range of the data processing node, the user node can access the data processing node and send the data of the primary virtual space unit to the data processing node. The data processing node fits together multiple primary virtual space units within its geographic location range, and the fitting together of multiple secondary virtual space units constructed by multiple data processing nodes constitutes the entire metaverse space.

[0046] Step S4: Send the data of the first-level virtual space unit of the user's current real-world scene to the data processing node.

[0047] Figure 4 A flowchart illustrating a method for a user node to generate a first-level virtual space unit representing the user's current real-world scene, according to an embodiment of the present invention. This embodiment includes the following steps:

[0048] Step S11: Collect visual images of the current real-world scene and the location information of the user terminal device in the current real-world scene.

[0049] Step S12: Generate the three-dimensional geometric structure information and texture mapping information of the current real scene based on the visual image information.

[0050] Step S13: Perform three-dimensional reconstruction based on the three-dimensional geometric structure information to obtain a three-dimensional geometric model of the real scene.

[0051] Step S14: Render the three-dimensional geometric model based on the texture mapping information to obtain a virtual model of the real scene.

[0052] Step S15: Based on the user terminal device's location information and visual image information in the current real scene, determine the virtual location information of the virtual model in the metaverse space and calibrate it to obtain the first-level virtual space unit of the user's current real scene.

[0053] In step S11, the present invention utilizes a camera in the user terminal device to capture visual images of the user's current real-world scene. In one embodiment, to obtain comprehensive visual image information of the current real-world scene, the user uses a binocular camera to capture images of the current real-world scene from multiple angles to obtain multiple left and right visual images, thereby forming a set of left and right visual images. In another embodiment, the user uses a binocular camera to capture a panoramic image of the current real-world scene with the current position as the base point.

[0054] After obtaining the visual image, in one embodiment, the three-dimensional geometric structure information of the current real-world scene is obtained in step S12 through the following processing procedure:

[0055] First, the cameras are calibrated based on the acquired image set. The calibration process includes calculating the camera's intrinsic and extrinsic parameters and distortion parameters. This includes determining the pixel coordinate system, image coordinate system, camera coordinate system, and world coordinate system based on the acquired images and lens data, and calculating the overall relationship between the four coordinate systems to obtain the intrinsic parameter matrices of the left and right cameras. The left and right distortion coefficients are calculated based on the camera's lens data, thereby obtaining radial and tangential distortion, which constitute the distortion matrix. Based on the Kruppa equation and Zhang Zhengyou's calibration method, the rotation and translation matrices of one camera relative to another are obtained. These distortion, rotation, and translation matrices are collectively referred to as extrinsic parameter matrices.

[0056] Then, based on the intrinsic and extrinsic parameter matrices, stereo correction is performed on the visual images in the left and right visual image sets, including correcting distortion errors, changing the viewpoint, and horizontal objects, so as to obtain two distortion-free and corrected images located on the same plane.

[0057] Then, stereo matching is performed on the left and right visual image sets to obtain three-dimensional geometric structure information. The stereo matching can employ local matching algorithms, global matching algorithms, or semi-global matching algorithms. The core of these algorithms is to calculate the disparity / depth information of the two images to obtain a disparity map / depth map, and then recover the three-dimensional information of the points based on the disparity information in the disparity map and / or the depth information in the depth map. Since disparity value calculation is usually performed using a small window, it is prone to noise. Therefore, in a better embodiment, image filtering methods such as median filtering, mean filtering, and bilateral filtering can be used to filter the disparity map / depth map to maintain good edge accuracy. After the stereo matching process, the three-dimensional coordinates of points in the real-world scene in the left and right images are determined. These coordinates constitute the three-dimensional geometric structure information of the real-world scene.

[0058] In another embodiment, when the acquired image set consists of real-world scene images acquired from multiple angles, step S12 processes multiple pairs of left and right images to generate multiple three-dimensional geometric structure information of the current real-world scene. At this time, the multiple three-dimensional geometric structure information is further compared and corrected to obtain a corrected three-dimensional geometric structure information. In step S13, three-dimensional reconstruction is performed based on the corrected three-dimensional geometric structure information to obtain a three-dimensional geometric model of the real-world scene.

[0059] In another embodiment, when the acquired image set is real scene images acquired from multiple angles, after step S12, multiple pairs of left and right images are processed to generate multiple three-dimensional geometric structure information of the current real scene. In step S13, based on the multiple three-dimensional geometric structure information, multiple three-dimensional reconstructions are performed to obtain multiple first three-dimensional geometric models of the real scene. Then, the multiple first three-dimensional geometric models are compared and corrected to obtain a corrected three-dimensional geometric model.

[0060] The two embodiments described above improve the accuracy of the final generated three-dimensional geometric model by comparing and correcting the three-dimensional geometric structure information or the three-dimensional geometric model.

[0061] In step S11, when collecting the location information of the user terminal device in the current real-world scenario, in one embodiment, various positioning technologies can be used to obtain the geographical location information of the user node based on the current real-world scenario. For example, in some outdoor scenarios with good GPS signals, such as city streets, GPS information alone can be used to locate the geographical location of the user node, thereby obtaining the geographical location information of the user node, including latitude and longitude coordinates and direction. Alternatively, in scenarios with a large number of mobile terminal base stations and sufficiently strong signals, such as outdoor areas in cities or scenic spots, preliminary geographical location information of the user node can be quickly obtained using mobile terminal base station information, and then corrected using GPS signals to obtain accurate geographical location information of the user node. Alternatively, in indoor or outdoor scenarios with poor GPS signals, such as underground parking lots, geomagnetic information can be collected to obtain the coordinates and direction of the user node. Alternatively, when there is a sufficiently good network signal in the current real-world scenario, such as in shopping malls, restaurants, and hotels, the geographical location information of the user node can be obtained through WiFi information or Bluetooth information. When the user is in motion, the geographical location information can also be determined by IMU inertial information (including acceleration information collected by the accelerometer, angle information collected by the gyroscope, and direction information from the magnetometer) and GPS.

[0062] Figure 5 This is a flowchart of a method for determining the virtual location information of a virtual model in a metaverse space according to an embodiment of the present invention. The method includes the following steps:

[0063] Step S151: Determine the first location of the user terminal device in the current real-world scene based on the location information of the user terminal device in the current real-world scene.

[0064] Step S152: Determine the relative position of the user terminal device in the current real-world scene based on visual image information. For example, based on one or more visual images, the positional relationship between the user terminal device and one or more geographical features in the real-world scene can be determined, such as facing, back to, far from, or near.

[0065] Step S153: Determine the position and location information of the virtual model of the real scene in the real map based on the first position and relative position. Referring to the real map information, obtain the position and location information of the corresponding current real scene in the real map based on the user terminal device's first position in the current real scene and its positional relationship with multiple geographic location features. The location information may include, for example, the geographic location information of the first position, the geographic location information of one or more geographic location features, or of course, all geographic location information in the real scene.

[0066] Step S154: Mark the virtual model using the location information of the virtual model of the real scene on the real map. Since the geographical locations in the metaverse space correspond one-to-one with the geographical locations in the real space, the virtual map in the metaverse space also corresponds one-to-one with the real map. When the real location information is obtained, it can be directly used to mark the virtual model. For example, the geographic coordinates of the real location are added to the data of the points constituting the geometric structure of the virtual model. The data of the marked virtual model includes not only the three-dimensional coordinates of the points but also the virtual geographical location coordinates of the metaverse space. In another embodiment, when the method of expressing the geographical location information of the metaverse space differs from the method of expressing the real geographical location information, the location information of the virtual model of the real scene on the real map is converted into the geographical location information of the metaverse space before marking.

[0067] In another embodiment, after the geographic location information of the metaverse space is calibrated in the virtual model, two points, such as a first virtual point and a second virtual point, are determined on a reference plane in the virtual model. A first relative distance is calculated based on the geographic location information of the two virtual points. Then, a second relative distance is calculated based on the point coordinate data of the first and second virtual points in the geometric structure of the virtual model. The ratio of the second relative distance to the first relative distance yields the scale between the virtual model and the real-world scene. Based on this scale, all length-related data in the geometric structure of the virtual model can be obtained. Examples include the width of streets, the height of buildings, and the length of slide surfaces in amusement parks. All this data lays the data foundation for processing various other activities in the metaverse space.

[0068] Figure 6 This is a flowchart of a metaverse space construction method according to an embodiment of the present invention. In this embodiment, the corresponding schematic diagram of the metaverse space construction system structure is as follows: Figure 1 As shown, in this embodiment, the user nodes are user terminal devices A and B, which are connected to data processing node A, and data processing node A is connected to data processing node B. Of course, there are other user terminal devices connected to data processing node A, multiple user terminal devices connected to data processing node B, and other data processing nodes. For simplicity, [the following is omitted as it is not explicitly stated]. Figure 6 The diagram only shows the processing flow of user terminal devices A and B and data processing nodes A and B during the creation of the metaverse space; the processing flow of other user terminal devices and data processing nodes is similar.

[0069] In this embodiment, the metaverse space construction method includes the following steps:

[0070] Step S11a: User terminal device A generates a first-level virtual space unit a of the user's current real-world scene.

[0071] Step S12a: Determine the data processing node to be accessed, such as determining the data processing node A based on the geographical location of the user terminal device A.

[0072] Step S13a: Send the data of the first-level virtual space unit a to the data processing node A.

[0073] Correspondingly, user terminal device B also sends the data of the first-level virtual space unit b it generated to the data processing node A.

[0074] Correspondingly, in step S21a, the data processing node A receives data of the first-level virtual space unit b generated by the user terminal device B; in step S22a, the data of the first-level virtual space unit a generated by the user terminal device A is received.

[0075] In step S23a, data processing node A monitors whether the locations of the first-level virtual space unit a and the first-level virtual space unit b are adjacent or overlapping.

[0076] When the locations of first-level virtual space unit a and first-level virtual space unit b are adjacent or overlap, in step S24a, data processing node A fits first-level virtual space unit a and first-level virtual space unit b to form second-level virtual space unit a.

[0077] In step S25a, data processing node A and data processing node B exchange the geographical locations of their respective secondary virtual spatial units, and in step S26a, they determine whether the geographical locations of secondary virtual spatial unit a constructed by data processing node A and secondary virtual spatial unit b constructed by data processing node B are adjacent or overlap. If the two have adjacent or overlapping geographical locations, data processing node A will fit them together. Of course, the fitting of the two secondary virtual spatial units can also be done by data processing node B or by both nodes jointly.

[0078] Data processing node B performs the same processing flow as data processing node A, namely, receiving primary virtual space unit data from different user nodes in steps S21b and S22b. In step S23b, multiple primary virtual space units are received to form secondary virtual space unit b. In step S24b, secondary virtual space unit b is fitted with secondary virtual space units from other data processing nodes.

[0079] In this embodiment, the user terminal device collects visual image information and geographical location information of the user's current real scene, and generates a first-level virtual space unit of the user's current real scene based on the visual image information. The generated first-level virtual space unit is labeled with its virtual geographical location information in the metaverse space.

[0080] Figure 7 This is a flowchart of a metaverse space construction method according to an embodiment of the present invention. In this embodiment, the corresponding schematic diagram of the metaverse space construction system structure is as follows: Figure 2 As shown, the user node includes user terminal device A and server A. Server A is connected to multiple user terminal devices. For ease of explanation, this embodiment shows only one user terminal device A. Server A is connected to one or more data processing nodes. In this embodiment, server A can access data processing node A. The processing flow is as follows: Figure 4 As shown:

[0081] For user terminal device A, the processing flow includes:

[0082] Step S1101: User terminal device A acquires visual images.

[0083] In step S1102, user terminal device A evaluates its computing power, for example, by reading the hardware performance index data of user terminal device A. When the computing power of user terminal device A is insufficient to perform relevant calculations to obtain a first-level virtual space unit, in step S1103, the visual image is sent to server A.

[0084] Step S1104: Based on the current scenario, use the appropriate positioning technology to obtain the location information of the user terminal.

[0085] Step S1105: Send the location information of the user terminal to server A.

[0086] For server A, the processing flow includes:

[0087] In step S1111, after receiving the visual image sent by the user terminal device A, server A processes it to obtain the three-dimensional geometric structure information of the corresponding real scene.

[0088] Step S1112: Perform three-dimensional reconstruction based on the three-dimensional geometric structure information to obtain a three-dimensional geometric model.

[0089] Step S1113: Process the visual image to extract texture mapping information.

[0090] Step S1114: Render the three-dimensional geometric model based on the texture mapping information to obtain a virtual model of the real scene.

[0091] Step S1115: Based on the location information of user terminal device A, calibrate the virtual model to obtain a first-level virtual space unit a with location information.

[0092] Step S1116: Query the data processing node based on the connection parameters and establish a communication connection with it.

[0093] Step S1117: Send the data of the first-level virtual space unit a to the data processing node A.

[0094] In this embodiment, when the user terminal device does not have sufficient computing power, the computing work is transferred to the server. This not only makes use of the server's powerful computing capabilities, but also enables the construction of virtual space units with user-provided symbiotic content, thereby accelerating the construction of the metaverse space.

[0095] On the other hand, the present invention also provides a metaverse space construction device, which is applied in user nodes, such as... Figure 8 The diagram shown is a principle block diagram of a metaverse space construction device according to an embodiment of the present invention. The device includes an image acquisition module 1, a 3D reconstruction module 2, a texture mapping module 3, a rendering module 4, a positioning module 5, and a calibration module 6. The image acquisition module 1 is used to acquire visual image information of the current real-world scene. In one embodiment, the image acquisition module 1 acquires visual images of the user's real-world scene through a camera on a user terminal device, such as acquiring multiple visual images from multiple angles or acquiring panoramic images. When there are multiple cameras on the user terminal device, multiple pairs of left and right images are acquired using a binocular camera, thereby forming a left and right visual image set.

[0096] The 3D reconstruction module 2 is connected to the image acquisition module 1. Based on the visual image information, the 3D reconstruction module 2 generates 3D geometric structure information of the current real-world scene, and performs 3D reconstruction based on this information to obtain a 3D geometric model of the real-world scene. This includes camera calibration, image stereo correction, and visual stereo matching to obtain the 3D coordinates of various points constituting physical entities in the real-world scene. These 3D coordinates constitute the 3D geometric structure information of the real-world scene. Then, spatial surfaces are connected based on these coordinates to reconstruct the 3D geometric model of the objects in the real-world scene. Furthermore, the aforementioned process can also include comparison and correction of 3D geometric structure information, or comparison and correction of the 3D geometric model, thereby improving the accuracy of the obtained 3D geometric model.

[0097] The texture mapping module 3 is connected to the image acquisition module 1 and is used to extract texture mapping information from the visual image.

[0098] The rendering module 4 is connected to the 3D reconstruction module 2 and the texture mapping module 3 respectively, and renders the 3D geometric model based on the texture mapping information to obtain a virtual model of the real scene.

[0099] The positioning module 5 is used to collect the positioning information of the user terminal device in the current real scene. Specifically, it determines a suitable positioning technology according to the real scene, and determines the location information of the user terminal based on the information collected by the corresponding location information collection module or sensor in the user terminal settings. The corresponding location information module in the user terminal settings is, for example, a GPS positioning module, a WiFi module, a Bluetooth module, etc., and the sensor is, for example, one or more of a magnetic sensor, an accelerometer, a gyroscope, etc.

[0100] The calibration module 6 is connected to both the rendering module 4 and the positioning module 5. Based on the user terminal device's positioning information and visual image information in the current real-world scene, it determines and calibrates the virtual location information of the virtual model in the metaverse space to obtain the first-level virtual space unit of the user's current real-world scene. The virtual geographical location information in the metaverse space corresponds one-to-one with the real geographical location information in the real-world scene. After calibration, the first-level virtual space unit data of the real-world scene is sent to a data processing node via the user node's communication module.

[0101] When the hardware performance of the user terminal equipment is high enough, it can be composed solely of the user terminal equipment. Figure 8 The user node is formed by the user terminal device and the server when the hardware performance of the user terminal device is insufficient. In this case, the image acquisition module 1 and the positioning module 5 in the metaverse space construction device are located on the user terminal device, while the 3D reconstruction module 2, texture mapping module 3, rendering module 4 and calibration module 6 are located on the server.

[0102] like Figure 9 The diagram shown is a schematic block diagram of a metaverse space construction device according to another embodiment of the present invention. Figure 8 In contrast, the device in this embodiment includes an access module 7, used to acquire the connection parameters of the user node, determine the corresponding data processing node based on the connection parameters, and send the data processing node to the calibration module 6. The calibration module 6 then sends the first-level virtual space unit data to the data processing node. In this embodiment, the metaverse space construction device, based on connection parameters such as the real-world geographical location range, network IP address range, or metaverse space geographical location range, queries the data processing node that matches the user node's connection parameters, thereby determining which data processing node should send the user node's first-level virtual space unit data to, thus ensuring the overall data transmission order of the system.

[0103] This invention also provides an electronic device, including a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the metaverse space construction method of any of the above embodiments. For example, the electronic device is a terminal device serving as a user node, such as a mobile phone, tablet computer, etc., including a processor, a memory, an image acquisition module, and a location acquisition module. The memory stores computer program instructions, and the processor executes the computer program instructions to implement... Figure 1 A method for constructing a metaverse space using user nodes. The electronic device is either a server that serves as part of the user node, or a server that serves as a data processing node.

[0104] Figure 10 A schematic diagram of the hardware structure of an embodiment of the electronic device provided by the present invention is shown.

[0105] like Figure 10 As shown, the electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0106] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0107] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0108] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to one aspect of the invention.

[0109] The processor 601 reads and executes computer program instructions stored in the memory 602 to implement any of the metaverse space construction methods in the above embodiments.

[0110] In one example, the electronic device may also include a communication interface 603 and a bus 610. For example, Figure 8 As shown, the processor 601, memory 602, and communication interface 603 are connected via bus 610 and communicate with each other. The electronic device in this embodiment of the invention can be a server or other computing device, or it can be a cloud server.

[0111] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0112] Bus 610 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0113] Furthermore, in conjunction with the product search method in the above embodiments, this invention can be implemented using a computer storage medium. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the metaverse space construction methods described in the above embodiments.

[0114] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0115] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Machine-readable media can include non-transitory computer-readable storage media, such as electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, and can also include radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0116] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0117] The aspects of the present invention have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0118] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A metaverse space construction system comprising a plurality of user nodes and a plurality of data processing nodes, wherein, the user nodes collect visual images and geographical location information of a current real scene, and construct a first virtual space unit based on the collected visual images and geographical location information of the current real scene, wherein the virtual geographical location information of the first virtual space unit in the metaverse space corresponds to the real geographical location information in the real scene one by one; the data processing nodes communicate with one or more user nodes, configured to receive the first virtual space unit data sent by the user nodes, and fit together two or more first virtual space units with adjacent / overlapping virtual geographical locations to form a second virtual space unit; at least two data processing nodes communicate with each other, configured to interact with each other the virtual geographical location information of the respective second virtual space units; when two or more second virtual space units belonging to different data processing nodes are adjacent / overlapping in virtual geographical location, the two or more data processing nodes or any one of them fit together the two second virtual space units; wherein the data processing nodes are configured with connection parameters, and the data processing nodes are connected with the user nodes meeting the connection parameters according to the connection parameters, and two or more data processing nodes with adjacent connection parameter values communicate with each other.

2. The system of claim 1, wherein the user nodes are user terminal devices.

3. The system of claim 1, wherein the user nodes comprise: a user terminal device configured to collect visual images and geographical location information of a current real scene of a user; and a server connected with a plurality of user terminal devices and one or more data processing nodes, configured to construct a first virtual space unit corresponding to the user terminal device based on the visual images and geographical location information of the current real scene of the user collected by the user terminal device, and send the first virtual space unit data to a data processing node.

4. The system of claim 1, wherein the connection parameters are real geographical location ranges, network IP address ranges, or metaverse space geographical location ranges.

5. A metaverse space construction method applied to the user nodes in the system of any one of claims 1-4, the method comprising: collecting visual images and positioning information of a user terminal device in a current real scene; obtaining three-dimensional geometric structure information and texture mapping information of the current real scene based on the visual image information; performing three-dimensional reconstruction based on the three-dimensional geometric structure information to obtain a three-dimensional geometric model of the real scene; rendering the three-dimensional geometric model based on the texture mapping information to obtain a virtual model of the real scene; and determining the virtual location information of the virtual model in the metaverse space based on the positioning information of the user terminal device in the current real scene and the visual image information, and calibrating to obtain a first virtual space unit of the current real scene where the user terminal device is located, wherein the virtual geographical location information of the metaverse space corresponds to the real geographical location information of the real scene one by one.

6. The method of claim 5, further comprising the steps of: acquiring connection parameters of the user node; determining a corresponding data processing node according to the connection parameters; and sending the first-level virtual space unit data to the data processing node.

7. The method of claim 5, wherein further, based on the binocular camera, visual images of the current real scene are collected from multiple angles; correspondingly, based on the multiple pairs of left and right images acquired by the binocular camera, multiple pieces of three-dimensional geometric structure information of the current real scene are generated respectively; the multiple pieces of three-dimensional geometric structure information are compared and corrected to obtain a corrected three-dimensional geometric structure information; and based on the corrected three-dimensional geometric structure information, three-dimensional reconstruction is performed to obtain a three-dimensional geometric model of the real scene; or based on the multiple pieces of three-dimensional geometric structure information, three-dimensional reconstruction is performed respectively to obtain multiple first three-dimensional geometric models of the real scene; and the multiple first three-dimensional geometric models are compared and corrected to obtain a corrected three-dimensional geometric model.

8. The method of claim 7, wherein when determining the virtual position information of the virtual model in the meta-universe space based on the positioning information and the visual image information of the user terminal device in the current real scene, further comprising: determining a first position of the user terminal device in the current real scene based on the positioning information of the user terminal device in the current real scene; determining a relative position of the user terminal device in the current real scene based on the visual image information; determining the position and position information of the virtual model of the real scene in the real map based on the first position and the relative position; and marking the virtual model with the position information of the virtual model of the real scene in the real map.

9. The method of claim 8, wherein when collecting the positioning information of the user terminal device in the current real scene, one or more of GPS information, geomagnetic information, WiFi information, Bluetooth information, and mobile terminal base station information of the user terminal device are acquired based on the current real scene; correspondingly, the first position of the user terminal device in the current real scene is determined based on one or more of the GPS information, geomagnetic information, WiFi information, Bluetooth information, IMU inertial information, and mobile terminal base station information. When the user node includes a user terminal device and a server, after the user terminal device collects the visual image information of the current real scene, further comprising:

10. The method of claim 5, wherein, evaluating the computing capability of the user terminal device; in response to the computing capability of the user terminal device not supporting the construction of the virtual space, sending the visual image information of the current real scene and the collected positioning information of the user terminal device in the current real scene to the corresponding server; correspondingly, the server performs three-dimensional reconstruction based on the three-dimensional geometric structure information to obtain a three-dimensional geometric model of the real scene; and renders the three-dimensional geometric model based on the texture mapping information to obtain a virtual model of the real scene; and ​ The virtual position information of the virtual model in the meta-universe space is determined based on the positioning information and the visual image information of the user terminal device in the current real scene, and calibration is performed to obtain a first virtual space unit of the current real scene.

11. A meta-universe space construction apparatus applied to a user node in the system of any one of claims 1-4, comprising: an image acquisition module configured to acquire a visual image of a current real scene; a three-dimensional reconstruction module connected to the image acquisition module and configured to generate three-dimensional geometric structure information of the current real scene based on the visual image, and perform three-dimensional reconstruction based on the three-dimensional geometric structure information to obtain a three-dimensional geometric model of the real scene; a texture mapping module connected to the image acquisition module and configured to generate texture mapping information based on the visual image information; a rendering module connected to the three-dimensional reconstruction module and the texture mapping module respectively and configured to render the three-dimensional geometric model based on the texture mapping information to obtain a virtual model of the real scene; a positioning module configured to acquire positioning information of a user terminal device in the current real scene; and a calibration module connected to the rendering module and the positioning module respectively and configured to determine virtual position information of the virtual model in the meta-universe space based on the positioning information and the visual image information of the user terminal device in the current real scene, and perform calibration to obtain a first virtual space unit of the current real scene, wherein the virtual geographic position information of the meta-universe space and the real geographic position information of the real scene are in one-to-one correspondence.

12. The apparatus of claim 11, further comprising an access module configured to obtain connection parameters of the user node and determine a corresponding data processing node according to the connection parameters. a processor and a memory storing computer program instructions; 13. An electronic device comprising: the processor executes the computer program instructions to implement the method of any one of claims 5-10. the computer storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the method of any one of claims 5-10.

14. A computer readable storage medium, wherein, it includes computer program instructions, and the computer program instructions are executed by the processor to implement the method of any one of claims 5-10.

15. A computer program product, characterised in that, the memory stores computer program instructions, and the processor executes the computer program instructions to implement the method of any one of claims 5-10.

16. A terminal device comprising at least a processor, a memory, an image acquisition module and a position acquisition module, wherein, ​

Images