A processing method and system for virtual space fusion

Abstract

The present invention proposes a processing method and system for virtual space fusion. The processing method for virtual space fusion includes: classifying and labeling panoramas, and using a deep learning model to obtain data information of each panorama, wherein the data information includes the depth value, three-dimensional size, and point cloud and Manhattan geometry corresponding to the panorama; using the depth value, three-dimensional size, and point cloud and Manhattan geometry corresponding to the panorama to obtain a virtual space; using the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model. The system includes modules corresponding to the steps of the method.

Description

Technical Field

[0001] The invention proposes a processing method and system for virtual space fusion, belonging to the technical field of virtual space fusion. Background Art

[0002] With the update of equipment, online house viewing has become more and more popular in the real estate field. Solutions for generating house types and models based on three-dimensional reshaping of panoramic images have been widely adopted in the real estate field and used in actual production.

[0003] In the current field of real estate 3D reconstruction, multiple panoramic images of the same scene from different angles and positions are often used for artificial intelligence modeling and post-stitching to generate floor plans. There are two existing methods for processing this data: one is stitching based on Manhattan geometry, and the other is stitching based on monocular depth estimation.

[0004] The advantages and disadvantages of Manhattan geometric stitching are as follows:

[0005] Advantages: simple and intuitive structure, in line with the operating habits of ordinary people, allowing people who have not received professional education to perform simple light processing on the model to obtain a simple and beautiful model

[0006] Disadvantages: The spatial information is fuzzy. In complex scenes, some real spatial information will be lost. When accurate restoration is required, the quality of the personnel processing the model is too high, and the degree of restoration of complex real information is not as good as expected.

[0007] Monocular depth estimation:

[0008] Advantages: precise structure, true restoration of spatial information, accurate data inference, very high degree of restoration of the entire space, greatly reducing labor costs

[0009] Disadvantages: The structure is complex. When the model needs to be lightly processed, it needs to be processed by professionals. It is not flexible enough and has insufficient subsequent expansion capabilities.

[0010] The advantages and disadvantages of both are very obvious, but there is no set of processing methods and solutions in the current market that can integrate management and integrate the two. In the actual production process, it is difficult to integrate the two methods, give full play to their respective strengths, avoid their respective shortcomings and improve actual production efficiency. Summary of the invention

[0011] The present invention provides a processing method and system for virtual space fusion, which is used to solve the technical problems existing in the above-mentioned prior art. The technical solutions adopted are as follows:

[0012] A method for processing virtual space fusion, the method comprising:

[0013] Classifying and labeling the panoramas, and using a deep learning model to obtain data information of each panorama, wherein the data information includes a depth value, a three-dimensional size, and a point cloud and Manhattan geometry structure corresponding to the panorama;

[0014] The virtual space is obtained by using the depth value, the three-dimensional size of the panorama, and the point cloud and Manhattan geometry structure corresponding to the panorama;

[0015] The virtual space is used to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model.

[0016] Furthermore, the panoramas are classified and labeled, and the data information of each panorama is obtained using a deep learning model, including:

[0017] Classify the panoramas, obtain different room locations in each panorama, and output a unique ruler; wherein the different locations include bathrooms, master bedrooms, and secondary bedrooms, etc.; and the unique ruler is data representing the actual length of a unit distance in actual production in any form of input;

[0018] Obtaining a depth value and a three-dimensional size corresponding to each panoramic image using a first deep learning model and a unique ruler;

[0019] Using a second deep learning model to obtain a point cloud of an indoor frame in each panoramic image; wherein the second deep learning model uses a transformer module and an edge enhancement module;

[0020] The point cloud is converted into Manhattan geometry using geometric algorithms.

[0021] Further, the first deep learning model and the unique ruler are used to obtain the depth value and the three-dimensional size corresponding to each panoramic image, including:

[0022] Retrieve the first deep learning model that has completed training and verification;

[0023] Using the first deep learning model that has been trained and verified, perform monocular depth estimation on each panorama to infer the depth value of each panorama;

[0024] Retrieve the scale data information corresponding to the unique scale;

[0025] The three-dimensional dimensions of different rooms in each panoramic image are inferred using a unique ruler; wherein the three-dimensional dimensions include a length dimension, a width dimension, and a height dimension.

[0026] Furthermore, the virtual space is obtained by using the depth value, the three-dimensional size of the panorama, and the point cloud and Manhattan geometry structure corresponding to the panorama, including:

[0027] Extract the depth value, 3D size, point cloud and Manhattan geometry of each panoramic image;

[0028] A virtual space is generated by aggregating the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometry structure corresponding to the panoramic image, wherein the virtual space is used to express the real information of each room.

[0029] Furthermore, using the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model includes:

[0030] Select the first panoramic image data entered into the virtual space as the main space data;

[0031] According to the perspective described by the main space data, the perspectives corresponding to the other panoramas except the main space data are adjusted until the perspectives of all the panoramas are consistent with the viewing perspective of the main space camera data;

[0032] Acquire the relative position relationship between the main space data and other panoramic images other than the main space data according to the main space data of the virtual space, and record it in the space representation data of the virtual space and itself;

[0033] According to the structure described by the virtual space, UV points of points in the geometric structure in the panoramic image are deduced by deducing the equirectangular projection in reverse.

[0034] According to the structure described by the virtual space and the inferred UV points, the texture map and model structure of the three-dimensional model are generated.

[0035] A processing system for virtual space fusion, the processing system for virtual space fusion comprising:

[0036] A data information acquisition module, used to classify and annotate the panoramas, and obtain data information of each panorama using a deep learning model, wherein the data information includes a depth value, a three-dimensional size, and a point cloud and Manhattan geometry structure corresponding to the panorama;

[0037] A virtual space acquisition module, for acquiring a virtual space using a depth value, a three-dimensional size of the panorama, and a point cloud and a Manhattan geometry structure corresponding to the panorama;

[0038] The three-dimensional model construction module is used to use the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model.

[0039] Furthermore, the data information acquisition module includes:

[0040] A classification module is used to classify the panoramas, obtain different room locations in each panorama, and output a unique ruler; wherein the different locations include bathrooms, master bedrooms, and secondary bedrooms, etc.; and the unique ruler is data representing the actual length of a unit distance in actual production in any form of input;

[0041] A first data acquisition module, used to acquire a depth value and a three-dimensional size corresponding to each panoramic image using a first deep learning model and a unique ruler;

[0042] A second data acquisition module, used to acquire a point cloud of an indoor frame in each panoramic image using a second deep learning model; wherein the second deep learning model uses a transformer module and an edge enhancement module;

[0043] The geometry acquisition module is used to convert the point cloud into Manhattan geometry using geometric algorithms.

[0044] Furthermore, the first data acquisition module includes:

[0045] A first deep learning model retrieving module, used to retrieve a first deep learning model that has completed training and verification;

[0046] A depth value acquisition module, used to perform monocular depth estimation on each panoramic image by using the first deep learning model that has been trained and verified, and infer the depth value of each panoramic image;

[0047] A unique scale retrieval module is used to retrieve scale data information corresponding to a unique scale;

[0048] The three-dimensional size acquisition module is used to infer the three-dimensional size of different rooms in each panoramic image using a unique ruler; wherein the three-dimensional size includes a length size, a width size and a height size.

[0049] Furthermore, the virtual space acquisition module includes:

[0050] A data information extraction module is used to extract the depth value, three-dimensional size and point cloud and Manhattan geometry structure corresponding to each panoramic image;

[0051] The virtual space generation module is used to generate a virtual space by utilizing the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometric structure corresponding to the panoramic image, wherein the virtual space is used to express the real information of each room.

[0052] Furthermore, the three-dimensional model building module includes:

[0053] A main space data determination module, used for selecting the first panoramic image data entered into the virtual space as the main space data;

[0054] A viewing angle adjustment module, used to adjust the viewing angles corresponding to other panoramas except the main space data according to the viewing angle described by the main space data until the viewing angles of all panoramas are consistent with the viewing angle direction of the main space camera data;

[0055] A position relationship acquisition module, used for acquiring the relative position relationship between the main space data and other panoramic images except the main space data according to the main space data of the virtual space, and recording it in the space description data of the virtual space and itself;

[0056] A UV point acquisition module is used to infer the UV points of points in the geometric structure in the panoramic image by inversely inferring the equirectangular projection according to the structure described by the virtual space;

[0057] The model and texture generation module is used to generate the texture map and model structure of the three-dimensional model according to the structure described by the virtual space and the inferred UV points.

[0058] Beneficial effects of the present invention:

[0059] In the three-dimensional reconstruction of a house, in order to restore the space as much as possible, multiple panoramic views of the same space at different angles and positions will be taken. In the subsequent model reconstruction, each panoramic view will be analyzed and reconstructed into an independent three-dimensional space model. The whole process will consume a lot of time and manpower costs. A processing method and system for virtual space fusion proposed in the present invention solves this problem. For the same space, a virtual space is agreed upon, and a group of panoramic views describing this space are uniformly mounted in this virtual space. When modeling, features are extracted from different real panoramic views according to the description of the virtual space to synthesize a three-dimensional model. By combining deep learning models and spatial reconstruction technology, high-precision three-dimensional reconstruction of the real world can be achieved. This reconstruction not only retains the shape and size information of the object, but also considers the distance relationship and spatial position relationship of the object. Using texture mapping technology, the texture information in the panorama is mapped to the three-dimensional model, making the three-dimensional model more realistic and delicate visually. This can enhance the user's immersion and interactive experience. The method can process panoramic data of different types and scales, and can perform flexible data processing and model construction as needed. In addition, with the continuous development of deep learning technology, this method is expected to achieve wider application and deeper research in the future. This technical solution has broad application prospects in the fields of virtual reality, augmented reality, digital twins, etc. For example, in the construction of smart cities, this technology can be used to build a three-dimensional digital model of the city, providing strong support for urban planning, traffic management, environmental protection, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1is a flow chart of the method of the present invention;

[0061] Figure 2 The system block diagram of the system of the present invention. DETAILED DESCRIPTION

[0062] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0063] The embodiment of the present invention proposes a processing method for virtual space fusion, such as Figure 1 As shown, the processing method of virtual space fusion includes:

[0064] S1. Classifying and labeling the panoramas, and using a deep learning model to obtain data information of each panorama, wherein the data information includes a depth value, a three-dimensional size, and a point cloud and Manhattan geometry structure corresponding to the panorama;

[0065] S2, obtaining a virtual space using a depth value, a three-dimensional size of the panorama, and a point cloud and a Manhattan geometry structure corresponding to the panorama;

[0066] S3. Using the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model.

[0067] The working principle of the above technical solution is as follows: Phase S1: Classification, annotation and data information acquisition of panoramas

[0068] Classification and labeling: First, the collected panoramas are classified and labeled in detail to facilitate subsequent data processing and analysis.

[0069] Application of deep learning models: Use deep learning models (such as convolutional neural networks (CNNs)) to process panoramas to obtain key data information for each panorama.

[0070] Data information extraction: This data information includes the depth value of the panorama (i.e. the distance relationship of objects in the image), the three-dimensional size (the actual size of the object), and the point cloud (a data set describing the surface shape of the object) and Manhattan geometry (a regular geometric structure used to describe buildings and streets in urban environments) corresponding to the panorama.

[0071] S2 stage: Acquisition of virtual space

[0072] Data integration: Integrate the panorama depth value, 3D size, point cloud, Manhattan geometry and other data obtained in the S1 stage.

[0073] Spatial construction: Using these data, a virtual space is constructed through spatial reconstruction technology (such as 3D reconstruction based on visual geometry or 3D reconstruction based on deep learning). This virtual space is a three-dimensional digital environment that reflects the physical features and spatial relationships of the real world.

[0074] S3 stage: construction of 3D model structure and texture map

[0075] Three-dimensional modeling: Based on the virtual space obtained in the S2 stage, a three-dimensional model structure corresponding to the virtual space is constructed using three-dimensional modeling techniques (such as mesh generation, surface subdivision, etc.).

[0076] Texture Mapping: Map the texture information (such as color, material, etc.) in the panorama to the 3D model to form a realistic texture map. This can further improve the realism and visualization of the 3D model.

[0077] The effect of the above technical solution is: through the combination of deep learning model and spatial reconstruction technology, high-precision three-dimensional reconstruction of the real world can be achieved. This reconstruction not only retains the shape and size information of the object, but also takes into account the distance relationship and spatial position relationship of the object. Using texture mapping technology, the texture information in the panorama is mapped to the three-dimensional model, making the three-dimensional model more realistic and delicate visually. This can enhance the user's immersion and interactive experience. The method can process panoramic data of different types and scales, and can perform flexible data processing and model construction as needed. In addition, with the continuous development of deep learning technology, this method is expected to achieve wider application and deeper research in the future. The technical solution has broad application prospects in the fields of virtual reality, augmented reality, digital twins, etc. For example, in the construction of smart cities, this technology can be used to build a three-dimensional digital model of the city, providing strong support for urban planning, traffic management, environmental protection, etc.

[0078] In summary, the virtual space fusion processing method achieves high-precision three-dimensional reconstruction and improved realism of the real world through the combination of deep learning models and space reconstruction technology, and has broad application prospects and important research value.

[0079] An embodiment of the present invention classifies and annotates panoramic images, and uses a deep learning model to obtain data information of each panoramic image, including:

[0080] S101, classifying the panoramic images, obtaining different room positions in each panoramic image, and outputting a unique ruler; wherein the different positions include a bathroom, a master bedroom, and a secondary bedroom, etc.; and the unique ruler is data representing the real length of a unit distance in actual production input in any form;

[0081] S102, using the first deep learning model and the unique ruler to obtain the depth value and the three-dimensional size corresponding to each panoramic image;

[0082] S103, using a second deep learning model to obtain a point cloud of an indoor frame in each panoramic image; wherein the second deep learning model uses a transformer module and an edge enhancement module;

[0083] S104. Use geometric algorithms to convert point clouds into Manhattan geometry structures.

[0084] The working principle of the above technical solution is as follows: S101: classify the panorama and output a unique scale

[0085] Classification: Using image classification technology, the panoramic image is analyzed through a trained classification model to identify different room types, such as bathroom, master bedroom, second bedroom, etc. The above technical solutions usually rely on deep learning technologies such as convolutional neural networks (CNN) to accurately identify room features in images.

[0086] Unique ruler: A unique ruler is a data that represents the actual length of a unit distance in actual production. In a panoramic image, the ruler can be calibrated by identifying objects of known size in the image (such as doors, windows, etc.). Once the ruler is determined, the actual size can be calculated based on the proportional relationship in the image.

[0087] S102: Obtaining depth value and three-dimensional size using the first deep learning model and the unique ruler

[0088] Depth value acquisition: The panorama is analyzed through a deep learning model (such as a depth estimation network) to predict the depth value of each pixel in the image. These depth values ​​reflect the distance relationship between the object and the camera.

[0089] 3D size calculation: Combining the unique scale and depth value, the 3D size of the object in the panorama can be calculated. The above technical solutions usually involve geometric transformation and projection principles, which can convert the objects in the 2D image into the actual size in 3D space.

[0090] S103: Obtaining point cloud using the second deep learning model

[0091] Point cloud generation: The second deep learning model uses a transformer module and an edge enhancement module to analyze the panorama and generate a point cloud. The transformer module can capture global features in the image, while the edge enhancement module can highlight the edge information in the image, thereby improving the accuracy and completeness of the point cloud.

[0092] Indoor frame recognition: The generated point cloud contains the 3D coordinate information of the objects in the panorama. Through further processing and analysis, the point cloud data of the indoor frame (such as walls, ceilings, etc.) can be identified.

[0093] S104: Using geometric algorithms to convert point clouds into Manhattan geometry

[0094] Application of geometric algorithms: Manhattan geometry is a simplified representation of three-dimensional space in which all walls are assumed to be perpendicular to the three main directions (i.e., X, Y, and Z axes). Point clouds can be converted into Manhattan geometry by processing them using geometric algorithms.

[0095] Structural extraction: By identifying features such as walls, floors, and ceilings in the point cloud, the 3D structural information of the interior can be extracted. This information can be used for subsequent interior layout design, decoration planning, and other applications.

[0096] The effect of the above technical solution is: through the combined application of deep learning models and geometric algorithms, it is possible to accurately identify information such as room types, depth values, and three-dimensional dimensions in the panorama, providing reliable data support for subsequent interior design and planning. This technical solution realizes functions such as automatic classification, depth estimation, and three-dimensional reconstruction of panoramas, greatly improving the degree of automation of interior design and decoration, and reducing the cost and time of manual intervention. Users can quickly obtain indoor three-dimensional structural information through this technical solution, and perform operations such as virtual decoration and layout design. This interactive experience can enhance the user's sense of participation and satisfaction. This technical solution provides strong support for the intelligent development of the interior design and decoration industry. With the continuous advancement of deep learning technology and the continuous expansion of application scenarios, this technical solution is expected to achieve more intelligent and personalized interior design and decoration services in the future.

[0097] In summary, this technical solution realizes functions such as automatic classification, depth estimation, 3D reconstruction and Manhattan geometric structure extraction of panoramas through the combined application of deep learning models and geometric algorithms, bringing significant technical effects and application value to the interior design and decoration industry.

[0098] In one embodiment of the present invention, a first deep learning model and a unique ruler are used to obtain a depth value and a three-dimensional size corresponding to each panoramic image, including:

[0099] S1021, retrieve the first deep learning model that has completed training and verification;

[0100] S1022, using the first deep learning model that has been trained and verified to perform monocular depth estimation on each panoramic image, and infer a depth value of each panoramic image;

[0101] S1023, retrieve the scale data information corresponding to the unique scale;

[0102] S1024. Infer the three-dimensional dimensions of different rooms in each panoramic image using a unique ruler; wherein the three-dimensional dimensions include a length dimension, a width dimension, and a height dimension.

[0103] The working principle of the above technical solution is as follows: S1021: Retrieve the first deep learning model that has been trained and verified

[0104] The above technical solution is the basis for subsequent processing. It is necessary to ensure that the deep learning model has been fully trained and verified and has the ability to perform monocular depth estimation on panoramic images.

[0105] S1022: Using the first deep learning model that has been trained and verified to perform monocular depth estimation on each panoramic image

[0106] The deep learning model analyzes the panorama and uses a multi-layer neural network (usually through a large amount of data and back-propagation algorithms) to automatically learn the feature representation and pattern of the data, and infer the depth value of each pixel in the panorama relative to a reference point (such as the camera position).

[0107] The principle of depth estimation is based on information such as feature points, textures, shadows, etc. in the image, as well as the correlation between depth and image features learned by the deep learning model from a large amount of data.

[0108] S1023: Retrieve the scale data information corresponding to the unique scale

[0109] The above technical solution is to provide a reference standard for converting the depth estimation result into an actual three-dimensional size.

[0110] The ruler data information generally includes information such as the length and unit of the ruler and the position of the ruler in the panoramic image.

[0111] S1024: Using a unique ruler to infer the 3D dimensions of different rooms in each panorama

[0112] After obtaining the depth value of the panorama, the depth value can be converted into an actual three-dimensional size in combination with the ruler data information.

[0113] By analyzing the room structure, walls, ground and other feature points in the panoramic image and combining them with depth information, the three-dimensional dimensions of the room, such as length, width and height, can be inferred.

[0114] The effect of the above technical solution is: this technical solution can use deep learning models and ruler data information to measure the three-dimensional dimensions of the rooms in the panoramic view, and provide accurate data support for subsequent interior design, decoration, renovation, etc. Compared with traditional three-dimensional measurement methods, this technical solution does not require complex physical measurements and calculations. It only needs to use deep learning models and ruler data information to quickly and accurately obtain the three-dimensional dimension information of the panoramic view, greatly improving the measurement efficiency and accuracy. This technical solution is suitable for different types of panoramas and room structures. It only needs to properly train and adjust the deep learning model to adapt to different application scenarios and needs. This technical solution realizes the automatic processing of the entire process from panoramic input to three-dimensional output, without the need for human intervention and manual operation, reducing labor costs and operating difficulties.

[0115] In summary, this technical solution uses deep learning models and ruler data information to realize the three-dimensional dimension measurement of panoramas, which has the advantages of high measurement efficiency, high accuracy, strong adaptability and high degree of automation.

[0116] An embodiment of the present invention uses the depth value, the three-dimensional size of the panoramic image, and the point cloud and Manhattan geometry structure corresponding to the panoramic image to obtain the virtual space, including:

[0117] S201, extracting the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometry structure corresponding to the panoramic image;

[0118] S202, generating a virtual space by aggregating the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometry structure corresponding to the panoramic image, wherein the virtual space is used to express the real information of each room.

[0119] The working principle of the above technical solution is as follows: S201: extracting key information of the panoramic image

[0120] In the above technical solution, the system first extracts key information from the panoramic image, including depth value, three-dimensional size, point cloud data and Manhattan geometry. This information is the basis for building a virtual space.

[0121] The depth value reflects the distance of objects in the image, while the three-dimensional size provides information about the actual size of the object. The point cloud data describes the shape and details of the surface of the object, while the Manhattan geometry simplifies the structures such as walls and ceilings in the three-dimensional space, making them more consistent with the characteristics of actual buildings.

[0122] S202: Aggregation to generate virtual space

[0123] After obtaining the key information of the panoramic image, the system will aggregate this information and generate a virtual space using three-dimensional reconstruction technology.

[0124] The above technical solutions usually involve complex processes such as spatial registration, point cloud fusion, and surface reconstruction. The system will map the point cloud data into three-dimensional space based on the depth value and three-dimensional size information to form a three-dimensional model of the object. At the same time, Manhattan geometry is used to optimize and adjust the structure and layout of the room to make it more in line with the characteristics of the actual building.

[0125] The resulting virtual space is a three-dimensional digital environment that truly reflects the real information of each room, including the room's layout, structure, size, and the position and shape of objects inside.

[0126] The effect of the above technical solution is: the technical solution can realize high-precision three-dimensional reconstruction of real rooms by extracting key information such as depth value, three-dimensional size, point cloud data and Manhattan geometry in panoramic images. This reconstruction not only retains the layout and structural information of the room, but also takes into account the shape and detail features of the object, making the generated virtual space more real and realistic. The technical solution realizes the automated processing process from panoramic images to virtual space without human intervention and manual operation. This greatly improves the processing efficiency and accuracy, and reduces labor costs and time costs. The technical solution is suitable for different types of panoramic images and room structures. Whether it is a simple residential room or a complex commercial space, three-dimensional reconstruction and virtual space generation can be performed through this technical solution. This makes the technical solution have broad application prospects and market demand. The technical solution can also be combined with other technologies, such as virtual reality (VR), augmented reality (AR), etc., to provide users with richer interactive experience and visualization effects. At the same time, users can also customize and modify the generated virtual space according to their needs to meet different application scenarios and needs.

[0127] In summary, this technical solution uses key information such as the depth value, 3D size, point cloud data and Manhattan geometry of panoramic images to achieve high-precision 3D reconstruction of real rooms and generation of virtual spaces. This technical solution has the advantages of high-precision reconstruction, high degree of automation, strong adaptability, scalability and customizability, and provides strong technical support and solutions for interior design, decoration planning, virtual house viewing and other fields.

[0128] In one embodiment of the present invention, the three-dimensional model structure corresponding to the virtual space and the texture map corresponding to the three-dimensional model are constructed by using the virtual space, including:

[0129] S301, selecting the first panoramic image data entered into the virtual space as main space data;

[0130] S302, adjusting the viewing angles corresponding to the other panoramas except the main space data according to the viewing angles described by the main space data until the viewing angles of all the panoramas are consistent with the viewing angles of the main space camera data;

[0131] S303, acquiring a relative position relationship between the main space data and other panoramic images other than the main space data according to the main space data of the virtual space, and recording the relative position relationship in the space representation data of the virtual space and the virtual space itself;

[0132] S304, according to the structure described by the virtual space, inversely inferring the UV points of the points in the geometric structure in the panoramic image by inversely inferring the equirectangular projection;

[0133] S305 , generating a texture map and a model structure of the three-dimensional model according to the structure described by the virtual space and the inferred UV points.

[0134] The working principle of the above technical solution is as follows: S301: Select the main space data

[0135] In the above technical solution, the system will select the first panoramic data from the panoramic data entered in the virtual space as the main space data. The main space data will be used as a reference for constructing the three-dimensional model structure and texture mapping.

[0136] S302: Viewing angle adjustment

[0137] In order to ensure that all panoramas can be seamlessly spliced ​​and integrated when building a 3D model, the system will adjust the perspectives of other panoramas according to the perspective described by the main space data. The above technical solutions usually involve image transformation and registration technology to ensure that the perspectives of all panoramas are consistent with the perspective of the main space data.

[0138] S303: Recording relative position relationship

[0139] After obtaining the relative positional relationships between the main space data and other panoramas, the system will record these positional relationships in the virtual space and its own spatial representation data. These positional relationships are crucial for subsequent 3D model construction and texture map generation.

[0140] S304: Inverse UV points

[0141] Based on the structure described by the virtual space, the system will infer the UV points of the points in the geometric structure in the panorama through methods such as inverse equirectangular projection. UV points are key reference points when mapping texture maps to the surface of a 3D model. They determine the distribution and position of texture maps on the surface of the model.

[0142] S305: Generate texture map and model structure

[0143] After obtaining the UV points, the system will generate the texture map and model structure of the 3D model based on the structure described by the virtual space and the inferred UV points. The texture map contains the color, texture and detail information of the model surface, while the model structure describes the geometric shape and spatial relationship of the model.

[0144] Meanwhile, in the process of generating the texture map and the model structure of the three-dimensional model, the texture map and the model structure generation operation of the three-dimensional model are monitored for abnormalities, including:

[0145] In the process of generating the texture map and model structure of the three-dimensional model, the generation operation parameters are monitored in real time, wherein the generation operation parameters include the distance between UV points, the overlap ratio of UV textures, the area and visual similarity of the completed part, and the rendering frame rate;

[0146] The first evaluation coefficient is obtained by using the distance between the UV points, the overlap ratio of the UV texture, the area of ​​the completed part and the visual similarity; wherein the first evaluation coefficient is obtained by the following formula:

[0147]

[0148] in, E 01 represents the first evaluation coefficient; A Indicates the area of ​​the currently completed part; s Indicates visual similarity; d Represents the average distance between UV points; r Indicates the overlap ratio of UV texture; ε Represents the minimum constant, used to prevent the denominator from being 0; d z Indicates the middle value of the distance between the currently appeared UV points; d max and d min Indicates the maximum and minimum distances between the currently appearing UV points;

[0149] Comparing the first evaluation coefficient with a preset first evaluation coefficient threshold;

[0150] When the first evaluation coefficient is higher than the preset first evaluation coefficient threshold, it is determined that there is no abnormality;

[0151] When the first evaluation coefficient is not higher than a preset first evaluation coefficient threshold, calling the rendering frame rate;

[0152] The rendering frame rate is combined with the first evaluation coefficient to obtain a second evaluation coefficient; wherein the second evaluation coefficient is obtained by the following formula:

[0153]

[0154] in, E 02 represents the second evaluation coefficient; E 01 represents the first evaluation coefficient; f Indicates the current rendering frame rate; f d Indicates the maximum floating ratio of rendering frame rate that has occurred in the current rendering process; ε Represents the minimum constant, used to prevent the denominator from being 0;

[0155] Comparing the first evaluation coefficient with a preset first evaluation coefficient threshold;

[0156] When the first evaluation coefficient is not higher than a preset first evaluation coefficient threshold, an abnormal alarm is issued.

[0157] The effect of the above technical solution is: the technical solution can generate a high-precision three-dimensional model structure by accurately adjusting the viewing angle of the panorama and recording the relative position relationship. This model structure not only retains the layout and structural information of the virtual space, but also takes into account the shape and detail features of the object, making the generated three-dimensional model more real and realistic. By inferring UV points and generating texture maps, the technical solution can achieve seamless mapping of texture maps on the surface of the three-dimensional model. This seamless mapping not only improves the visual effect of the model, but also enables the model to maintain consistent texture effects under different viewing angles and lighting conditions. The technical solution realizes the automated processing process from virtual space data to three-dimensional model structure and texture maps. The operations involved in this process, such as image transformation, registration, inferring UV points, and generating texture maps, can be automatically completed by computer algorithms without human intervention and manual operation. This greatly improves processing efficiency and accuracy, and reduces labor costs and time costs. The technical solution is applicable to different types of virtual space and panorama data. Whether it is a simple indoor space or a complex outdoor scene, three-dimensional model construction and texture map generation can be performed through the technical solution. This makes the technical solution have broad application prospects and market demand.

[0158] At the same time, the corresponding technical solution for abnormal monitoring of the texture mapping and model structure generation operation of the three-dimensional model ensures the real-time monitoring by real-time monitoring of the generation operation parameters, including the distance between UV points, the overlap ratio of UV textures, the area and visual similarity of the completed parts, and the rendering frame rate. This real-time performance is crucial for timely detection and handling of anomalies, and can effectively avoid the accumulation of errors that may occur during the generation process, thereby improving the quality of the final three-dimensional model. These parameters are used to calculate the first evaluation coefficient and the second evaluation coefficient, and multiple dimensions such as area, visual similarity, UV point distribution, and rendering frame rate are comprehensively considered through complex mathematical formulas, making the evaluation more comprehensive and accurate. This accuracy helps to more accurately determine whether there are anomalies in the generation process and reduce the possibility of false alarms and missed reports.

[0159] Moreover, by setting the threshold of the first evaluation coefficient, the scheme can timely trigger a further abnormality detection mechanism when the first evaluation coefficient is lower than the threshold, that is, call the rendering frame rate to calculate the second evaluation coefficient. This mechanism not only improves the sensitivity of abnormality detection, but also can flexibly adjust the judgment criteria according to the specific situation to meet the needs of different scenarios. The introduction of the second evaluation coefficient enables the scheme to further evaluate the stability of the generation process according to the change of the rendering frame rate. By combining the first evaluation coefficient and the rendering frame rate, the scheme can more comprehensively reflect the actual situation of the generation process, so as to make more accurate judgments. This dynamic adjustment capability helps to take timely measures to optimize when an abnormality occurs, such as adjusting rendering parameters, optimizing algorithms, etc., to ensure the smooth progress of the generation process. The scheme can not only detect and alarm in time when an abnormality occurs, but also provide strong support for subsequent abnormality correction through the calculation of the first evaluation coefficient and the second evaluation coefficient. For example, when it is found that the UV point distribution is unreasonable or the rendering frame rate fluctuates too much, the UV layout can be adjusted in time or the rendering algorithm can be optimized to avoid the impact of the abnormality on the quality of the final three-dimensional model. In addition, the scheme fully considers resource utilization and efficiency issues in the abnormality monitoring process. Through real-time monitoring and dynamic adjustment, the solution can minimize unnecessary resource consumption and computing overhead while ensuring the stability of the generation process. This high efficiency makes the solution more competitive and practical in practical applications. Finally, the solution also has good scalability and flexibility. With the continuous development of 3D model generation technology, the solution can be expanded and optimized according to actual needs to adapt to more complex and diverse application scenarios. At the same time, the parameters and thresholds in the solution can also be flexibly adjusted according to specific circumstances to meet the needs and expectations of different users.

[0160] In summary, this technical solution uses virtual space data to construct a three-dimensional model structure and texture mapping, and has the advantages of high-precision three-dimensional models, seamless texture mapping, high degree of automation, and strong adaptability. These advantages make this technical solution have broad application value and prospects in the fields of interior design, decoration planning, virtual house viewing, game development, etc. At the same time, this technical solution has shown significant performance indicators and technical effects in the abnormal monitoring of texture mapping and model structure generation during the three-dimensional model generation process. These effects not only improve the stability and reliability of the generation process, but also provide users with a more efficient, flexible and scalable solution.

[0161] The embodiment of the present invention proposes a processing system for virtual space fusion, such as Figure 2 As shown, the processing system for virtual space fusion includes:

[0162] A data information acquisition module, used to classify and annotate the panoramas, and obtain data information of each panorama using a deep learning model, wherein the data information includes a depth value, a three-dimensional size, and a point cloud and Manhattan geometry structure corresponding to the panorama;

[0163] A virtual space acquisition module, for acquiring a virtual space using a depth value, a three-dimensional size of the panorama, and a point cloud and a Manhattan geometry structure corresponding to the panorama;

[0164] The three-dimensional model construction module is used to use the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model.

[0165] The working principle of the above technical solution is as follows: First, the collected panoramas are classified and labeled in detail to facilitate subsequent data processing and analysis. The panoramas are processed using deep learning models (such as convolutional neural networks (CNNs)) to obtain key data information for each panorama. These data information include the depth value of the panorama (i.e., the distance relationship of objects in the image), the three-dimensional size (the actual size of the object), and the point cloud (a data set describing the surface shape of the object) and Manhattan geometry (a regular geometric structure used to describe buildings and streets in urban environments) corresponding to the panorama.

[0166] Integrate the acquired panorama depth value, 3D size, point cloud, Manhattan geometry and other data. Using these data, construct a virtual space through spatial reconstruction technology (such as 3D reconstruction based on visual geometry or 3D reconstruction based on deep learning). This virtual space is a three-dimensional digital environment that can reflect the physical characteristics and spatial relationships of the real world. Based on the acquired virtual space, use 3D modeling technology (such as mesh generation, surface subdivision, etc.) to construct a 3D model structure corresponding to the virtual space. Map the texture information (such as color, material, etc.) in the panorama to the 3D model to form a realistic texture map. This can further enhance the realism and visualization of the 3D model.

[0167] The effect of the above technical solution is: through the combination of deep learning model and spatial reconstruction technology, high-precision three-dimensional reconstruction of the real world can be achieved. This reconstruction not only retains the shape and size information of the object, but also takes into account the distance relationship and spatial position relationship of the object. Using texture mapping technology, the texture information in the panorama is mapped to the three-dimensional model, making the three-dimensional model more realistic and delicate visually. This can enhance the user's immersion and interactive experience. The method can process panoramic data of different types and scales, and can perform flexible data processing and model construction as needed. In addition, with the continuous development of deep learning technology, this method is expected to achieve wider application and deeper research in the future. The technical solution has broad application prospects in the fields of virtual reality, augmented reality, digital twins, etc. For example, in the construction of smart cities, this technology can be used to build a three-dimensional digital model of the city, providing strong support for urban planning, traffic management, environmental protection, etc.

[0168] In summary, the virtual space fusion processing method achieves high-precision three-dimensional reconstruction and improved realism of the real world through the combination of deep learning models and space reconstruction technology, and has broad application prospects and important research value.

[0169] In one embodiment of the present invention, the data information acquisition module includes:

[0170] A classification module is used to classify the panoramas, obtain different room locations in each panorama, and output a unique ruler; wherein the different locations include bathrooms, master bedrooms, and secondary bedrooms, etc.; and the unique ruler is data representing the actual length of a unit distance in actual production in any form of input;

[0171] A first data acquisition module, used to acquire a depth value and a three-dimensional size corresponding to each panoramic image using a first deep learning model and a unique ruler;

[0172] A second data acquisition module, used to acquire a point cloud of an indoor frame in each panoramic image using a second deep learning model; wherein the second deep learning model uses a transformer module and an edge enhancement module;

[0173] The geometry acquisition module is used to convert the point cloud into Manhattan geometry using geometric algorithms.

[0174] The working principle of the above technical solution is: using image classification technology, the panorama is analyzed through the trained classification model to identify different room types, such as bathrooms, master bedrooms, and secondary bedrooms. The above technical solution usually relies on deep learning technologies such as convolutional neural networks (CNNs) to accurately identify room features in images. The unique ruler is a data representing the actual length of a unit distance in actual production. In the panorama, the ruler can be calibrated by identifying objects of known size in the image (such as doors, windows, etc.). Once the ruler is determined, the actual size can be calculated based on the proportional relationship in the image.

[0175] The panorama is analyzed by a deep learning model (such as a depth estimation network) to predict the depth value of each pixel in the image. These depth values ​​reflect the distance relationship between the object and the camera. Combining the unique ruler and the depth value, the three-dimensional size of the object in the panorama can be calculated. The above technical solutions usually involve geometric transformation and projection principles, which can convert objects in two-dimensional images into actual sizes in three-dimensional space. The second deep learning model uses a transformer module and an edge enhancement module to analyze the panorama and generate a point cloud. The transformer module can capture global features in the image, while the edge enhancement module can highlight the edge information in the image, thereby improving the accuracy and completeness of the point cloud. The generated point cloud contains the three-dimensional coordinate information of the objects in the panorama. Through further processing and analysis, the point cloud data of indoor frames (such as walls, ceilings, etc.) can be identified. Manhattan geometry is a simplified three-dimensional space representation method in which all walls are assumed to be perpendicular to the three main directions (i.e., X, Y, and Z axes). The point cloud can be converted into Manhattan geometry by processing it using a geometric algorithm. By identifying features such as walls, floors and ceilings in the point cloud, the 3D structural information of the room can be extracted. This information can be used for subsequent indoor layout design, decoration planning and other applications.

[0176] The effect of the above technical solution is: through the combined application of deep learning models and geometric algorithms, it is possible to accurately identify information such as room types, depth values, and three-dimensional dimensions in the panorama, providing reliable data support for subsequent interior design and planning. This technical solution realizes functions such as automatic classification, depth estimation, and three-dimensional reconstruction of panoramas, greatly improving the degree of automation of interior design and decoration, and reducing the cost and time of manual intervention. Users can quickly obtain indoor three-dimensional structural information through this technical solution, and perform operations such as virtual decoration and layout design. This interactive experience can enhance the user's sense of participation and satisfaction. This technical solution provides strong support for the intelligent development of the interior design and decoration industry. With the continuous advancement of deep learning technology and the continuous expansion of application scenarios, this technical solution is expected to achieve more intelligent and personalized interior design and decoration services in the future.

[0177] In summary, this technical solution realizes functions such as automatic classification, depth estimation, 3D reconstruction and Manhattan geometric structure extraction of panoramas through the combined application of deep learning models and geometric algorithms, bringing significant technical effects and application value to the interior design and decoration industry.

[0178] In one embodiment of the present invention, the first data acquisition module includes:

[0179] A first deep learning model retrieving module, used to retrieve a first deep learning model that has completed training and verification;

[0180] A depth value acquisition module, used to perform monocular depth estimation on each panoramic image by using the first deep learning model that has been trained and verified, and infer the depth value of each panoramic image;

[0181] A unique scale retrieval module is used to retrieve scale data information corresponding to a unique scale;

[0182] The three-dimensional size acquisition module is used to infer the three-dimensional size of different rooms in each panoramic image using a unique ruler; wherein the three-dimensional size includes a length size, a width size and a height size.

[0183] The working principle of the above technical solution is as follows: The above technical solution is the basis for subsequent processing, and it is necessary to ensure that the deep learning model has been fully trained and verified and has the ability to perform monocular depth estimation on the panorama. The deep learning model analyzes the panorama and uses a multi-layer neural network (usually through a large amount of data and back propagation algorithm) to automatically learn the feature representation and pattern of the data, and infer the depth value of each pixel in the panorama relative to a reference point (such as the camera position). The principle of depth estimation is based on information such as feature points, textures, shadows, etc. in the image, and the correlation between depth and image features learned by the deep learning model from a large amount of data. The above technical solution is to provide a reference standard for converting the results of depth estimation into actual three-dimensional dimensions. The ruler data information usually includes information such as the length, unit, and position of the ruler in the panorama. After obtaining the depth value of the panorama, combined with the ruler data information, the depth value can be converted into an actual three-dimensional size. By analyzing the feature points such as the room structure, wall, and ground in the panorama, combined with the depth information, the three-dimensional dimensions of the room such as length, width, and height can be inferred.

[0184] The effect of the above technical solution is: this technical solution can use deep learning models and ruler data information to measure the three-dimensional dimensions of the rooms in the panoramic view, and provide accurate data support for subsequent interior design, decoration, renovation, etc. Compared with traditional three-dimensional measurement methods, this technical solution does not require complex physical measurements and calculations. It only needs to use deep learning models and ruler data information to quickly and accurately obtain the three-dimensional dimension information of the panoramic view, greatly improving the measurement efficiency and accuracy. This technical solution is suitable for different types of panoramas and room structures. It only needs to properly train and adjust the deep learning model to adapt to different application scenarios and needs. This technical solution realizes the automatic processing of the entire process from panoramic input to three-dimensional output, without the need for human intervention and manual operation, reducing labor costs and operating difficulties.

[0185] In summary, this technical solution uses deep learning models and ruler data information to realize the three-dimensional dimension measurement of panoramas, which has the advantages of high measurement efficiency, high accuracy, strong adaptability and high degree of automation.

[0186] In one embodiment of the present invention, the virtual space acquisition module includes:

[0187] A data information extraction module is used to extract the depth value, three-dimensional size and point cloud and Manhattan geometry structure corresponding to each panoramic image;

[0188] The virtual space generation module is used to generate a virtual space by utilizing the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometric structure corresponding to the panoramic image, wherein the virtual space is used to express the real information of each room.

[0189] The working principle of the above technical solution is as follows: In the above technical solution, the system first extracts key information from the panoramic image, including depth value, three-dimensional size, point cloud data and Manhattan geometry. This information is the basis for building a virtual space. The depth value reflects the distance relationship of objects in the image, and the three-dimensional size provides the actual size information of the object. The point cloud data describes the shape and details of the surface of the object, while the Manhattan geometry simplifies the structures such as walls and ceilings in the three-dimensional space to make it more consistent with the characteristics of actual buildings.

[0190] After obtaining the key information of the panoramic image, the system will aggregate this information and generate a virtual space using 3D reconstruction technology. The above technical solutions usually involve complex processes such as spatial registration, point cloud fusion, and surface reconstruction. The system will map the point cloud data into the 3D space based on the depth value and 3D size information to form a 3D model of the object. At the same time, Manhattan geometry is used to optimize and adjust the structure and layout of the room to make it more in line with the characteristics of the actual building. The final generated virtual space is a 3D digital environment that can truly reflect the real information of each room, including the layout, structure, size of the room, and the position and shape of the internal objects.

[0191] The effect of the above technical solution is: the technical solution can realize high-precision three-dimensional reconstruction of real rooms by extracting key information such as depth value, three-dimensional size, point cloud data and Manhattan geometry in panoramic images. This reconstruction not only retains the layout and structural information of the room, but also takes into account the shape and detail features of the object, making the generated virtual space more real and realistic. The technical solution realizes the automated processing process from panoramic images to virtual space without human intervention and manual operation. This greatly improves the processing efficiency and accuracy, and reduces labor costs and time costs. The technical solution is suitable for different types of panoramic images and room structures. Whether it is a simple residential room or a complex commercial space, three-dimensional reconstruction and virtual space generation can be performed through this technical solution. This makes the technical solution have broad application prospects and market demand. The technical solution can also be combined with other technologies, such as virtual reality (VR), augmented reality (AR), etc., to provide users with richer interactive experience and visualization effects. At the same time, users can also customize and modify the generated virtual space according to their needs to meet different application scenarios and needs.

[0192] In summary, this technical solution uses key information such as the depth value, 3D size, point cloud data and Manhattan geometry of panoramic images to achieve high-precision 3D reconstruction of real rooms and generation of virtual spaces. This technical solution has the advantages of high-precision reconstruction, high degree of automation, strong adaptability, scalability and customizability, and provides strong technical support and solutions for interior design, decoration planning, virtual house viewing and other fields.

[0193] In one embodiment of the present invention, the three-dimensional model building module includes:

[0194] A main space data determination module, used for selecting the first panoramic image data entered into the virtual space as the main space data;

[0195] A viewing angle adjustment module, used to adjust the viewing angles corresponding to other panoramas except the main space data according to the viewing angle described by the main space data until the viewing angles of all panoramas are consistent with the viewing angle direction of the main space camera data;

[0196] A position relationship acquisition module, used for acquiring the relative position relationship between the main space data and other panoramic images except the main space data according to the main space data of the virtual space, and recording it in the space description data of the virtual space and itself;

[0197] A UV point acquisition module is used to infer the UV points of points in the geometric structure in the panoramic image by inversely inferring the equirectangular projection according to the structure described by the virtual space;

[0198] The model and texture generation module is used to generate the texture map and model structure of the three-dimensional model according to the structure described by the virtual space and the inferred UV points.

[0199] The working principle of the above technical solution is as follows: In the above technical solution, the system selects the first panoramic data from the panoramic data entered in the virtual space as the main space data. The main space data will be used as the basis for constructing the three-dimensional model structure and texture mapping. In order to ensure that all panoramas can be seamlessly spliced ​​and merged when constructing the three-dimensional model, the system will adjust the corresponding perspectives of other panoramas according to the perspective described by the main space data. The above technical solution usually involves image transformation and registration technology to ensure that the perspectives of all panoramas are consistent with the perspectives of the main space data. After obtaining the relative positional relationship between the main space data and other panoramas, the system will record these positional relationships in the virtual space and its own spatial representation data. These positional relationships are crucial for subsequent three-dimensional model construction and texture mapping generation. According to the structure described by the virtual space, the system will reversely deduce the UV points of the points in the geometric structure in the panorama by methods such as reverse equirectangular projection. UV points are key reference points when texture mapping is mapped to the surface of the three-dimensional model, and they determine the distribution and position of texture mapping on the surface of the model. After obtaining the UV points, the system will generate the texture map and model structure of the 3D model based on the structure described by the virtual space and the inferred UV points. The texture map contains the color, texture and detail information of the model surface, while the model structure describes the geometric shape and spatial relationship of the model.

[0200] The effect of the above technical solution is: the technical solution can generate a high-precision three-dimensional model structure by accurately adjusting the viewing angle of the panorama and recording the relative position relationship. This model structure not only retains the layout and structural information of the virtual space, but also takes into account the shape and detail features of the object, making the generated three-dimensional model more real and realistic. By inferring UV points and generating texture maps, the technical solution can achieve seamless mapping of texture maps on the surface of the three-dimensional model. This seamless mapping not only improves the visual effect of the model, but also enables the model to maintain consistent texture effects under different viewing angles and lighting conditions. The technical solution realizes the automated processing process from virtual space data to three-dimensional model structure and texture maps. The operations involved in this process, such as image transformation, registration, inferring UV points, and generating texture maps, can be automatically completed by computer algorithms without human intervention and manual operation. This greatly improves processing efficiency and accuracy, and reduces labor costs and time costs. The technical solution is applicable to different types of virtual space and panorama data. Whether it is a simple indoor space or a complex outdoor scene, three-dimensional model construction and texture map generation can be performed through the technical solution. This makes the technical solution have broad application prospects and market demand.

[0201] In summary, this technical solution uses virtual space data to construct a 3D model structure and texture mapping, and has the advantages of high-precision 3D models, seamless texture mapping, high degree of automation, and strong adaptability. These advantages make this technical solution have broad application value and prospects in the fields of interior design, decoration planning, virtual house viewing, game development, etc.

[0202] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for processing virtual space fusion, characterized in that: The processing method of virtual space fusion includes: Classifying and labeling the panoramas, and using a deep learning model to obtain data information of each panorama, wherein the data information includes a depth value, a three-dimensional size, and a point cloud and Manhattan geometry structure corresponding to the panorama; The virtual space is obtained by using the depth value, the three-dimensional size of the panorama, and the point cloud and Manhattan geometry structure corresponding to the panorama; Using the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model; The panorama is classified and labeled, and the data information of each panorama is obtained by using a deep learning model, including: Classify the panoramas, obtain different room positions in each panorama, and output a unique ruler; the unique ruler is data representing the real length of a unit distance in actual production in any form of input; Obtaining a depth value and a three-dimensional size corresponding to each panoramic image using a first deep learning model and a unique ruler; Using a second deep learning model to obtain a point cloud of an indoor frame in each panoramic image; wherein the second deep learning model uses a transformer module and an edge enhancement module; Use geometric algorithms to convert point clouds into Manhattan geometry; Using the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model includes: Select the first panoramic image data entered into the virtual space as the main space data; According to the perspective described by the main space data, the perspectives corresponding to the other panoramas except the main space data are adjusted until the perspectives of all the panoramas are consistent with the viewing perspective of the main space camera data; Acquire the relative position relationship between the main space data and other panoramic images other than the main space data according to the main space data of the virtual space, and record it in the space representation data of the virtual space and itself; According to the structure described by the virtual space, UV points of points in the geometric structure in the panoramic image are deduced by deducing the equirectangular projection in reverse. According to the structure described by the virtual space and the inferred UV points, the texture map and model structure of the three-dimensional model are generated.

2. The method for processing virtual space fusion according to claim 1, characterized in that: The first deep learning model and the unique ruler are used to obtain the depth value and three-dimensional size corresponding to each panorama, including: Retrieve the first deep learning model that has completed training and verification; Using the first deep learning model that has been trained and verified, perform monocular depth estimation on each panorama to infer the depth value of each panorama; Retrieve the scale data information corresponding to the unique scale; The three-dimensional dimensions of different rooms in each panoramic image are inferred using a unique ruler; wherein the three-dimensional dimensions include a length dimension, a width dimension, and a height dimension.

3. The method for processing virtual space fusion according to claim 1, characterized in that: The virtual space is obtained by using the depth value, three-dimensional size of the panorama, and the point cloud and Manhattan geometry structure corresponding to the panorama, including: Extract the depth value, 3D size, point cloud and Manhattan geometry of each panoramic image; A virtual space is generated by aggregating the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometry structure corresponding to the panoramic image, wherein the virtual space is used to express the real information of each room.

4. A processing system for virtual space fusion, characterized in that: The processing system for virtual space fusion includes: A data information acquisition module, used to classify and annotate the panoramas, and obtain data information of each panorama using a deep learning model, wherein the data information includes a depth value, a three-dimensional size, and a point cloud and Manhattan geometry structure corresponding to the panorama; A virtual space acquisition module, for acquiring a virtual space using a depth value, a three-dimensional size of the panorama, and a point cloud and a Manhattan geometry structure corresponding to the panorama; A three-dimensional model construction module, used to use the virtual space to construct a three-dimensional model structure corresponding to the virtual space and a texture map corresponding to the three-dimensional model; The data information acquisition module includes: A classification module is used to classify the panoramas, obtain different room locations in each panorama, and output a unique ruler; the unique ruler is data representing the real length of a unit distance in actual production in any form of input; A first data acquisition module, used to acquire a depth value and a three-dimensional size corresponding to each panoramic image using a first deep learning model and a unique ruler; A second data acquisition module, used to acquire a point cloud of an indoor frame in each panoramic image using a second deep learning model; wherein the second deep learning model uses a transformer module and an edge enhancement module; A geometry acquisition module, used to convert point clouds into Manhattan geometry using geometric algorithms; The three-dimensional model building module includes: A main space data determination module, used for selecting the first panoramic image data entered into the virtual space as the main space data; A viewing angle adjustment module, used to adjust the viewing angles corresponding to other panoramas except the main space data according to the viewing angle described by the main space data until the viewing angles of all panoramas are consistent with the viewing angle direction of the main space camera data; A position relationship acquisition module, used for acquiring the relative position relationship between the main space data and other panoramic images except the main space data according to the main space data of the virtual space, and recording it in the space description data of the virtual space and itself; A UV point acquisition module is used to infer the UV points of points in the geometric structure in the panoramic image by inversely inferring the equirectangular projection according to the structure described by the virtual space; The model and texture generation module is used to generate the texture map and model structure of the three-dimensional model according to the structure described by the virtual space and the inferred UV points.

5. The processing system for virtual space fusion according to claim 4, characterized in that: The first data acquisition module includes: A first deep learning model retrieving module, used to retrieve a first deep learning model that has completed training and verification; A depth value acquisition module, used to perform monocular depth estimation on each panoramic image by using the first deep learning model that has been trained and verified, and infer the depth value of each panoramic image; A unique scale retrieval module is used to retrieve scale data information corresponding to a unique scale; The three-dimensional size acquisition module is used to infer the three-dimensional size of different rooms in each panoramic image using a unique ruler; wherein the three-dimensional size includes a length size, a width size and a height size.

6. The processing system for virtual space fusion according to claim 4, characterized in that: The virtual space acquisition module includes: A data information extraction module is used to extract the depth value, three-dimensional size and point cloud and Manhattan geometry structure corresponding to each panoramic image; The virtual space generation module is used to generate a virtual space by utilizing the depth value and three-dimensional size corresponding to each panoramic image and the point cloud and Manhattan geometric structure corresponding to the panoramic image, wherein the virtual space is used to express the real information of each room.

Images