AR-based decoration result display method and device and electronic equipment

Abstract

The invention discloses an AR-based decoration result display method and device and electronic equipment, and relates to the field of AR. The method comprises the following steps: acquiring a decoration style input by a user; matching a corresponding decorated house and a decorated three-dimensional model corresponding to the decorated house according to the decoration style; controlling AR glasses of the user to scan the undecorated house to obtain an undecorated three-dimensional model of the undecorated house; calculating dimension scale factors of the decorated three-dimensional model and the undecorated three-dimensional model in a preset dimension; and according to the size scale factor, mapping a decoration result of the decorated house to an undecorated three-dimensional model, and displaying the decoration result in AR glasses. By implementing the technical scheme provided by the invention, the virtual decoration effect is combined with the real undecorated house environment, so that the user can intuitively experience the effect after decoration through the AR glasses, and the user experience feeling is improved.

Description

Technical Field

[0001] The present application relates to the AR field, and specifically to an AR-based decoration results display method, device and electronic equipment. Background Art

[0002] With the continuous improvement of people's living standards and the increasing diversification of aesthetic needs, residential decoration has become a topic of great concern to the majority of owners. People not only hope that the renovated house can meet basic functional needs, but also pursue personalized and artistic decoration effects, hoping to create a living space that highlights their taste.

[0003] At present, traditional decoration design mainly relies on plan drawings, renderings and 3D renderings to display and communicate the plan. It is difficult for owners to intuitively and truly feel the decoration effect, resulting in poor user experience.

[0004] Therefore, there is an urgent need for an AR-based decoration results display method, device and electronic equipment. Summary of the invention

[0005] The present application provides an AR-based decoration results display method, device and electronic equipment, which combines the virtual decoration effect with the real undecorated house environment, allowing users to intuitively experience the decoration effect through AR glasses, thereby improving the user experience.

[0006] In a first aspect of the present application, a method for displaying decoration results based on AR is provided, the method comprising: obtaining a decoration style input by a user; matching a corresponding decorated house and a decorated three-dimensional model corresponding to the decorated house according to the decoration style; controlling the user's AR glasses to scan an undecorated house to obtain an undecorated three-dimensional model of the undecorated house; calculating a size ratio factor between the decorated three-dimensional model and the undecorated three-dimensional model in a preset dimension; and mapping the decoration results of the decorated house to the undecorated three-dimensional model according to the size ratio factor, and displaying it in the AR glasses.

[0007] By adopting the above technical solution, by obtaining the decoration style input by the user, matching the corresponding decorated house and the decorated 3D model according to the decoration style, and controlling the user's AR glasses to scan the undecorated house to obtain the undecorated 3D model, and then calculating the size ratio factor of the decorated 3D model and the undecorated 3D model in the preset dimension, and finally mapping the decoration results of the decorated house to the undecorated 3D model according to the size ratio factor and displaying it in the AR glasses, the purpose of previewing the decoration effect in the undecorated house is achieved. This method makes full use of the interactivity and immersion of AR technology, combines the virtual decoration effect with the real undecorated house environment, allows users to experience the effect of decoration in person, and greatly improves the user experience and the intuitiveness and accuracy of the decoration design.

[0008] Optionally, controlling the AR glasses of the user to scan the undecorated house to obtain an undecorated three-dimensional model of the undecorated house specifically includes: controlling the AR glasses to capture multi-angle images of the undecorated house; generating three-dimensional point cloud data of the undecorated house according to the multi-angle images; and constructing the undecorated three-dimensional model of the undecorated house based on the three-dimensional point cloud data.

[0009] By adopting the above technical solution, AR glasses are controlled to collect multi-angle images of undecorated houses, and then three-dimensional point cloud data of the undecorated houses are generated based on the multi-angle images. Finally, an undecorated three-dimensional model of the undecorated house is constructed based on the three-dimensional point cloud data. This method makes full use of the depth sensor and camera integrated in the AR glasses. By collecting multi-angle images of the undecorated house, all-round information of the undecorated house is obtained, and then the multi-angle images are converted into three-dimensional point cloud data and undecorated three-dimensional models, realizing the digital collection and modeling of the undecorated house. At the same time, this method can also capture the structural characteristics of the undecorated house in real time through AR glasses, so that the decoration design can be synchronized with the actual situation of the undecorated house in real time, improving the accuracy and feasibility of the design.

[0010] Optionally, generating the three-dimensional point cloud data of the undecorated house based on the multi-angle images specifically includes: obtaining overlapping areas between the multi-angle images, and extracting feature points between each of the overlapping areas; determining a spatial transformation relationship between two adjacent images based on the feature points; and splicing the multi-angle images based on the spatial transformation relationship to obtain the three-dimensional point cloud data.

[0011] By adopting the above technical solution, the overlapping areas between multi-angle images are first obtained, and the feature points between each overlapping area are extracted. Then, the spatial transformation relationship between two adjacent images is determined based on the feature points. Finally, the multi-angle images are spliced ​​according to the spatial transformation relationship to obtain three-dimensional point cloud data. The technology of image splicing and three-dimensional reconstruction is fully utilized. Through the matching of feature points and the calculation of spatial transformation relationship, the seamless splicing and fusion of multi-angle images are realized, and complete and coherent three-dimensional point cloud data are obtained. By splicing multi-angle images, the panoramic information and detailed features of the undecorated house are obtained, which greatly improves the integrity and accuracy of three-dimensional reconstruction.

[0012] Optionally, mapping the decoration results of the decorated house to the undecorated three-dimensional model according to the size scale factor and displaying the model in the AR glasses specifically includes: acquiring decoration elements in the decorated three-dimensional model, where the decoration elements include ceilings, floors, walls, doors and windows, furniture and lighting; scaling the decoration elements according to the size scale factor to obtain target decoration elements; and mapping the target decoration elements as the decoration results to the undecorated house.

[0013] By adopting the above technical solution, the decoration elements in the decorated three-dimensional model, such as ceiling, floor, wall, door and window, furniture, and lighting, are first obtained, and then the decoration elements are scaled according to the size scale factor to obtain the target decoration elements, and finally the target decoration elements are mapped to the undecorated house as the decoration results. By deconstructing and extracting elements from the existing decoration scheme, the standardization and modularization of the decoration effect are achieved, so that the decoration results can be flexibly mapped and migrated to different house spaces. At the same time, the method also ensures the coordination of the size and proportion of the decoration elements with the undecorated house by scaling the size scale factor. Users can intuitively preview the real decoration effect in the undecorated house, including the material, color, style, and layout of different decoration elements, so as to better evaluate the feasibility and aesthetics of the decoration scheme and make the best decoration choice.

[0014] Optionally, before controlling the AR glasses of the user to scan the undecorated house and obtaining the undecorated three-dimensional model of the undecorated house, the method further includes: obtaining camera parameters of the AR glasses, the camera parameters including focal length, field of view angle, and depth of field; inputting the camera parameters into a preset shooting model to obtain a shooting distance and a shooting angle corresponding to the camera parameters; and marking the shooting distance and the shooting angle on a display interface of the AR glasses to guide the user to take pictures.

[0015] By adopting the above technical solution, the camera parameters of AR glasses, such as focal length, field of view, and depth of field, are first obtained, and then the camera parameters are input into the preset shooting model to obtain the shooting distance and shooting angle. Finally, these shooting information are marked on the display interface of AR glasses to guide users to take correct and standardized shots. This method makes full use of the display and interactive functions of AR glasses, and provides users with clear and accurate shooting instructions through visual shooting guidance. Users do not need to manually set or calculate shooting parameters, but only need to shoot according to the prompts of AR glasses to obtain high-quality multi-angle images, thereby obtaining an accurate and complete three-dimensional model of an undecorated house.

[0016] Optionally, before inputting the camera parameters into a preset shooting model to obtain a shooting distance and a shooting angle corresponding to the camera parameters, the method further includes: obtaining shooting data of the AR glasses in different indoor environments, the shooting data including camera parameters, shooting distance, shooting angle and shooting clarity; screening the shooting data to obtain training data, the training data being shooting data whose shooting clarity is greater than a preset clarity threshold in the shooting data; and constructing a neural network model according to the training data for training to obtain a preset shooting model.

[0017] By adopting the above technical solution, the shooting data of AR glasses in different indoor environments are obtained, including camera parameters, shooting distance, shooting angle and shooting clarity, and then the shooting data is screened, and the shooting data with clarity higher than the preset clarity threshold is selected as training data. Finally, the training data is used to build and train the neural network model to obtain the preset shooting model. Through the learning and training of shooting data, a mapping relationship between shooting parameters and image quality is established, so that according to different camera parameters and shooting environments, a better shooting distance and shooting angle can be adaptively generated, which can intelligently and flexibly adapt to different shooting conditions and requirements, and improve the accuracy and reliability of shooting guidance. At the same time, the method can also continuously improve the performance and generalization ability of the preset shooting model through continuous data collection and model iteration, so that it can be applied to more diverse indoor scenes and AR devices, and provide an intelligent and efficient parameter optimization solution for scanning modeling of undecorated houses.

[0018] Optionally, after mapping the decoration results of the decorated house to the undecorated house according to the size scale factor and displaying them in the AR glasses, the method includes: obtaining price information of each furniture in the decorated three-dimensional model; based on the price information, calculating the sum of the prices of each furniture to obtain a furniture budget amount; obtaining a material budget amount, and obtaining a total budget amount based on the furniture budget amount and the material budget price; and displaying the total budget amount in the AR glasses.

[0019] By adopting the above technical solution, after completing the mapping and display of the decoration effect, you can also obtain the price information of each furniture in the decorated 3D model, and calculate the furniture budget amount. At the same time, combined with the material budget amount, you can get the total budget amount of the entire decoration plan and display it in AR glasses. It can automatically, quickly and accurately calculate the various expenses required for decoration, providing users with an intuitive and transparent budget reference.

[0020] In a second aspect of the present application, a device for displaying decoration results based on AR is provided, and the device includes an acquisition module and a processing module, wherein: the acquisition module is used to acquire the decoration style input by the user; the processing module is used to match the corresponding decorated house and the decorated three-dimensional model corresponding to the decorated house according to the decoration style; the processing module is also used to control the user's AR glasses to scan the undecorated house to obtain the undecorated three-dimensional model of the undecorated house; the processing module is also used to calculate the size ratio factor of the decorated three-dimensional model and the undecorated three-dimensional model in a preset dimension; the processing module is also used to map the decoration results of the decorated house to the undecorated three-dimensional model according to the size ratio factor, and display it in the AR glasses.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the above methods.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions. When the instructions are executed, the method steps shown above are executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By obtaining the decoration style input by the user, the corresponding decorated house and decorated 3D model are matched according to the decoration style, and the user's AR glasses are controlled to scan the undecorated house to obtain the undecorated 3D model, and then the size ratio factor of the decorated 3D model and the undecorated 3D model in the preset dimension is calculated. Finally, according to the size ratio factor, the decoration results of the decorated house are mapped to the undecorated 3D model and displayed in the AR glasses, so as to achieve the purpose of previewing the decoration effect in the undecorated house. This method makes full use of the interactivity and immersion of AR technology, combines the virtual decoration effect with the real undecorated house environment, allows users to experience the effect of decoration in person, and greatly improves the intuitiveness and accuracy of decoration design.

[0024] 2. Collect multi-angle images of the undecorated house by controlling AR glasses, then generate three-dimensional point cloud data of the undecorated house based on the multi-angle images, and finally construct an undecorated three-dimensional model of the undecorated house based on the three-dimensional point cloud data. This method makes full use of the depth sensor and camera integrated in the AR glasses. By collecting multi-angle images of the undecorated house, all-round information of the undecorated house is obtained, and then the multi-angle images are converted into three-dimensional point cloud data and undecorated three-dimensional models, realizing the digital collection and modeling of the undecorated house. At the same time, this method can also capture the structural characteristics of the undecorated house in real time through AR glasses, so that the decoration design can be synchronized with the actual situation of the undecorated house in real time, improving the accuracy and feasibility of the design.

[0025] 3. First, obtain the overlapping areas between multi-angle images and extract the feature points between each overlapping area. Then, determine the spatial transformation relationship between two adjacent images based on the feature points. Finally, stitch the multi-angle images based on the spatial transformation relationship to obtain 3D point cloud data. By making full use of the image stitching and 3D reconstruction technology, the seamless stitching and fusion of multi-angle images are achieved through feature point matching and calculation of spatial transformation relationships, and complete and coherent 3D point cloud data are obtained. By stitching multi-angle images, the panoramic information and detailed features of the undecorated house are obtained, which greatly improves the integrity and accuracy of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of a decoration achievement display method based on AR provided in an embodiment of the present application; Figure 2 It is a module schematic diagram of an AR-based decoration achievement display device provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0027] Explanation of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0028] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0030] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0031] This application provides a decoration results display method based on AR, referring to Figure 1 , Figure 1 1 is a flow chart of a decoration result display method based on AR provided in an embodiment of the present application. The method is applied to a server and includes steps S101 to S105, which are as follows: Step S101: Acquire the decoration style input by the user.

[0032] In step S101, the server establishes a wireless connection with the AR glasses worn by the user. After the connection is established, the server pushes a virtual interface for selecting a decoration style to the AR glasses. The virtual interface uses augmented reality technology to integrate with the real environment in which the user is located. Specifically, through AR glasses, users can see a translucent interface suspended in real space. The translucent interface lists thumbnails and names of various decoration styles, such as modern simplicity, pastoral style, industrial style, and French romance. Each style thumbnail is presented in the form of a three-dimensional model, and the user can interact with it by looking or gestures. When the user focuses his eyes on the thumbnail of a decoration style for more than 2 seconds, or clicks on the thumbnail with his hand, the server will receive the corresponding trigger signal and obtain the decoration style entered by the user.

[0033] Step S102: matching the corresponding decorated house and the decorated three-dimensional model corresponding to the decorated house according to the decoration style.

[0034] In step S102, the server maintains a decoration case library in the database. The decoration case library stores a large amount of information about decorated houses, including the floor plan of the house, decoration style, materials used, furniture configuration and other parameters. Each decorated house corresponds to a decorated three-dimensional model, which is drawn by 3D modeling software and is highly consistent with the size and appearance of the real house. When the server obtains the decoration style selected by the user, it will immediately search in the decoration case library. The server will use the decoration style as a keyword and match it with the decoration style of each decorated house in the decoration case library, thereby determining a batch of decorated houses corresponding to the decoration style selected by the user.

[0035] Step S103: Control the user's AR glasses to scan the undecorated house to obtain an undecorated three-dimensional model of the undecorated house.

[0036] Before step S103, the method also includes: obtaining camera parameters of the AR glasses, the camera parameters including focal length, field of view angle, and depth of field; inputting the camera parameters into a preset shooting model to obtain a shooting distance and shooting angle corresponding to the camera parameters; marking the shooting distance and shooting angle on the display interface of the AR glasses to guide the user to take pictures.

[0037] Specifically, the server first constructs a preset shooting model, and the construction process of the preset shooting model is as follows: obtain the shooting data of AR glasses in different indoor environments, the shooting data includes camera parameters, shooting distance, shooting angle and shooting clarity; filter the shooting data to obtain training data, the training data is the shooting data with shooting clarity greater than the preset clarity threshold; according to the training data, a neural network model is constructed for training to obtain the preset shooting model. Specifically, the server first collects a large amount of shooting data of AR glasses in various indoor scenes. These data include the camera parameters of AR glasses (such as focal length, field of view, depth of field), shooting distance, shooting angle and image clarity obtained by shooting. The shooting data can be data shot and uploaded by historical users using AR glasses. The server analyzes and filters the collected shooting data, and selects data with higher shooting clarity as training data. The server sets a preset clarity threshold, and includes the shooting data with shooting clarity greater than the preset clarity threshold into the training data set. The server uses the filtered training data to construct a neural network model. The input of the neural network model is the camera parameters of the AR glasses, and the output is the corresponding shooting distance and shooting angle. The server inputs the training data into the neural network model and continuously adjusts the weights and biases of the network through the back-propagation algorithm, so that the neural network model can learn the mapping relationship between the camera parameters and the optimal shooting settings. After sufficient training and optimization, the server obtains the preset shooting model. The preset shooting model can predict the optimal shooting distance and shooting angle based on the input AR glasses camera parameters. This preset shooting model can be used to guide users to shoot and improve shooting quality and efficiency.

[0038] For example, suppose the server collects 1,000 sets of AR glasses shooting data in different indoor environments. Each set of data includes the focal length (such as 35mm), field of view (such as 60 degrees), depth of field (such as 1-5 meters), shooting distance (such as 2 meters), shooting angle (such as 30 degrees) and shooting clarity (such as 0.8) of the AR glasses. The server screens these 1,000 sets of data, sets the preset clarity threshold to 0.7, and selects 800 sets of shooting data with a clarity greater than 0.7 as training data. The server builds a neural network model consisting of an input layer, multiple hidden layers and an output layer, and inputs the training data into the neural network model for training. After multiple rounds of iteration and optimization, the prediction accuracy of the preset shooting model continues to improve. Given a new set of AR glasses camera parameters, such as a focal length of 40mm, a field of view of 65 degrees, and a depth of field of 2-6 meters, the preset shooting model can quickly calculate that the shooting distance is 2.5 meters and the shooting angle is 35 degrees.

[0039] With the preset shooting model, when users use AR glasses to shoot undecorated houses, they only need to shoot according to the shooting distance and shooting angle marked on the display interface of the AR glasses to obtain clear and complete images of the house. After receiving the shooting distance and shooting angle sent by the server, the AR glasses mark them on their own display interface. For example, the AR glasses can overlay a virtual reference frame on the display interface, and the size and position of the frame correspond to the optimal shooting distance; at the same time, the AR glasses can also display a virtual reference line on the display interface, and the angle of the line corresponds to the optimal shooting angle. When the user wears the AR glasses, he can adjust the shooting distance and shooting angle between himself and the undecorated house by observing the virtual reference frame and reference line on the display interface. When the user reaches the shooting distance determined by the server and adjusts the camera to the shooting angle determined by the server, the display interface of the AR glasses will give a prompt, such as the reference frame turns green, indicating that the current shooting settings have reached the optimal state.

[0040] For example, a user wants to use AR glasses to shoot an undecorated living room. After the user puts on the AR glasses, the server obtains the camera parameters of the AR glasses: focal length of 35mm, field of view of 60 degrees, depth of field of 1-5 meters. The server inputs these parameters into the preset shooting model, and the preset shooting model calculates that the optimal shooting distance is 2.5 meters and the optimal shooting angle is 30 degrees. The server sends the calculation results to the AR glasses, and the AR glasses display a virtual reference frame with a side length of 2.5 meters and a virtual reference line with an angle of 30 degrees to the ground on the display interface. Based on these visual cues, the user adjusts the distance between himself and the living room until it reaches 2.5 meters; at the same time, the user adjusts the shooting angle of the AR glasses to 30 degrees. When the user meets the optimal shooting settings, the virtual reference frame of the AR glasses turns green, prompting the user to start shooting.

[0041] In step S103, the server controls the AR glasses to collect multi-angle images of the undecorated house; generates three-dimensional point cloud data of the undecorated house according to the multi-angle images; and constructs an undecorated three-dimensional model of the undecorated house based on the three-dimensional point cloud data.

[0042] In a possible implementation, three-dimensional point cloud data of an undecorated house is generated based on multi-angle images, specifically including: obtaining overlapping areas between multi-angle images, and extracting feature points between each overlapping area; determining a spatial transformation relationship between two adjacent images based on the feature points; and splicing the multi-angle images based on the spatial transformation relationship to obtain three-dimensional point cloud data.

[0043] In step S103, when the user is at the correct shooting distance and shooting angle, the server sends a "start acquisition" instruction to the AR glasses. After receiving the instruction, the AR glasses will enter the continuous shooting mode and automatically shoot images of the interior of the house according to the preset time interval (for example, 2 images per second). When the user moves in the room, the AR glasses will record multiple images of different perspectives of the house from different angles and positions. The server first analyzes the received multi-angle images, identifies and extracts the overlapping areas between adjacent images. The overlapping area refers to the part of the house that appears in both images at the same time. By analyzing the overlapping area, the spatial relationship between the images can be determined. The server uses image matching algorithms, such as feature point matching and mutual information matching, to find the overlapping area between each pair of adjacent images. After determining the overlapping area, the server further extracts significant feature points in the overlapping area. Feature points refer to points in the image that have obvious distinguishability and stability, such as corner points and edge points. The server uses feature point detection algorithms, such as SIFT, SURF, and ORB, to detect and extract a set of feature points in each overlapping area. These feature points can accurately represent the structure and content of the overlapping area. Based on the extracted feature points, the server calculates the spatial transformation relationship between adjacent images. The spatial transformation relationship describes the geometric transformation from one image to another, including rotation, translation, and scaling. The server uses feature point matching algorithms, such as RANSAC and ICP, to match the feature points of one image with the feature points of another image and estimate the spatial transformation matrix between them. The spatial transformation matrix can map points in one image to corresponding positions in another image. Based on the calculated spatial transformation relationship, the server stitches multi-angle images and combines them into a complete three-dimensional point cloud data. The server uses image stitching algorithms, such as global optimization stitching and progressive stitching, to align each image to a common coordinate system through spatial transformation relationships and fuse their pixel information. During the stitching process, the server also performs image fusion and color correction processing to eliminate brightness and color differences between images and improve the consistency and realism of the stitching effect.

[0044] When all image frames are processed, the server can obtain sparse 3D point cloud data, where each point corresponds to a feature point in the image. The positions of these points reflect the 3D structure inside the house, but there are still large gaps between the points. In order to obtain a denser and more complete 3D model, the server will further encrypt and interpolate the sparse 3D point cloud data. The server uses the Poisson surface reconstruction method to estimate a continuous surface model based on the sparse 3D point cloud data, fill the gaps between points, and generate a relatively complete 3D mesh model. Finally, the server performs texture mapping on the reconstructed 3D mesh model, maps the pixel information in the original image sequence to the surface of the 3D mesh model, gives the 3D mesh model real color and texture, and then obtains the undecorated 3D model corresponding to the undecorated house.

[0045] Step S104: Calculate the size ratio factor of the decorated 3D model and the undecorated 3D model in a preset dimension.

[0046] In step S104, the server loads the decorated three-dimensional model and the undecorated three-dimensional model, and preprocesses the two models. The preprocessing process includes the unification of the coordinate system of the model and the noise removal operation. The server determines the preset dimensions. The preset dimensions include three dimensions: length, width, and height. Next, the server calculates the dimensions of the decorated three-dimensional model and the undecorated three-dimensional model in the preset dimensions respectively. For each preset dimension, the server extracts the dimension value of the corresponding dimension by analyzing the geometric structure and coordinate information of the three-dimensional model. For example, for the length dimension, the server calculates the difference between the maximum and minimum values ​​of the three-dimensional model in the X-axis direction; for the width dimension, the server calculates the difference between the maximum and minimum values ​​of the three-dimensional model in the Y-axis direction; for the height dimension, the server calculates the difference between the maximum and minimum values ​​of the three-dimensional model in the Z-axis direction. After obtaining the dimensions of the decorated three-dimensional model and the undecorated three-dimensional model in the preset dimensions, the server calculates the size scale factor. The size scale factor refers to the ratio of the size of the decorated three-dimensional model in a preset dimension to the size of the undecorated three-dimensional model in the corresponding dimension. The server calculates the size scale factor for each preset dimension respectively. For example, for the length dimension, the server divides the length of the decorated 3D model by the length of the undecorated 3D model to obtain the size scale factor of the length dimension; for the width dimension and the height dimension, the server performs similar calculations to obtain the corresponding size scale factors. Finally, the server stores the calculated size scale factors for subsequent decoration effect mapping and presentation.

[0047] Step S105: According to the size ratio factor, the decoration results of the decorated house are mapped to the undecorated house and displayed in the AR glasses.

[0048] In step S105, the decoration elements in the decorated three-dimensional model are obtained, and the decoration elements include ceiling, floor, wall, door and window, furniture and lighting; the decoration elements are scaled according to the size ratio factor to obtain the target decoration elements; and the target decoration elements are mapped to the undecorated house as the decoration results.

[0049] Specifically, the server analyzes and deconstructs the decorated 3D model and extracts the decoration elements contained therein. Decoration elements refer to the various components that constitute the decoration effect, such as ceilings, floors, walls, doors and windows, furniture, lighting, etc. The server uses a 3D model segmentation algorithm, such as semantic-based segmentation and geometry-based segmentation, to divide the decorated 3D model into different areas and components, and identify the type of decoration elements to which each area and component belongs. For example, the server can identify the ceiling area as a ceiling element, the ground area as a floor element, the vertical surface area as a wall element, the opening area as a door and window element, the independent object as a furniture element, and the lighting object as a lighting element. The server scales the extracted decoration elements according to the size scale factor. The size scale factor represents the proportional relationship between the decorated house and the undecorated house in terms of length, width, height, etc. The server multiplies the size of each decoration element by the size scale factor of the corresponding dimension to obtain the scaled target decoration element. For example, for a furniture element with a length of 2 meters, a width of 1 meter, and a height of 0.5 meters, if the scale factor of the length dimension is 0.8, the scale factor of the width dimension is 0.9, and the scale factor of the height dimension is 1.2, then the length of the scaled target furniture element is 1.6 meters, the width is 0.9 meters, and the height is 0.6 meters. The server performs the same scaling operation on each decoration element to ensure that their size ratio in the undecorated house matches the actual decoration effect.

[0050] While scaling the decoration elements, the server also makes necessary adjustments and optimizations to the decoration elements according to the preset rules to adapt to the spatial structure and layout of the undecorated house. The preset rules are summarized based on the general principles and experience of decoration design, and the purpose is to ensure that the decoration elements are placed reasonably and naturally in the undecorated house. For example, for the ceiling elements, the server adjusts the vertical position and boundary contour of the ceiling according to the ceiling height and shape of the undecorated house; for the door and window elements, the server adjusts the horizontal position and size of the doors and windows according to the position and size of the wall openings of the undecorated house; for the furniture elements, the server adjusts the plane position and orientation of the furniture according to the floor area and layout of the undecorated house.

[0051] The server uses a spatial mapping algorithm, such as affine transformation, to map the scaled decoration elements to the corresponding positions of the undecorated 3D model. Finally, the server renders the adjusted target decoration elements into the undecorated 3D model of the undecorated house as the decoration results. The server uses a 3D rendering engine, such as Unity 3D and Unreal Engine, to perform material mapping, lighting baking, and shadow rendering on the undecorated 3D model to generate the target model. The server transmits the target model to the AR glasses, which overlay it on the undecorated house in the user's field of view to achieve a real-time preview of the decoration effect. Through AR glasses, users can intuitively see the appearance of the undecorated house after decoration, including changes and matching in various aspects such as ceilings, floors, walls, doors and windows, furniture, and lighting. Users can freely move the perspective and switch decoration styles in AR glasses to fully experience the impact of different decoration styles on the overall effect of the house.

[0052] In a possible implementation, after step S105, the method further includes: obtaining price information of each piece of furniture in the decorated three-dimensional model; based on the price information, calculating the sum of the prices of each piece of furniture to obtain a furniture budget amount; obtaining a material budget amount, and obtaining a total budget amount based on the furniture budget amount and the material budget price; and displaying the total budget amount in the AR glasses.

[0053] Specifically, the server obtains the price information of each piece of furniture in the decorated three-dimensional model. This price information can be pre-entered into the attributes of each furniture element by the designer or modeler when constructing the decorated three-dimensional model. The server needs to calculate the sum of the prices of each piece of furniture to obtain the furniture budget amount in the entire decoration plan. After calculating the furniture budget amount, the server also needs to obtain the material budget amount in the decoration plan. The material budget amount refers to the purchase cost of various raw materials used in the decoration process, such as floor tiles, wall tiles, paint, coatings, wood, cement, sand, etc. The server can obtain the material budget amount through a preset material price database, which contains the required material costs corresponding to various decoration styles. After obtaining the furniture budget amount and the material budget amount, the server adds the two together to obtain the total budget amount (total budget amount) of the entire decoration plan. Finally, the server sends the total budget amount to the AR glasses, and the AR glasses display the budget information in an overlaid manner while displaying the decoration effect. For example, the AR glasses can display a translucent budget information panel in a corner of the field of view, which includes the total budget amount, the furniture budget amount, and the material budget amount.

[0054] Reference Figure 2The present application also provides an AR-based decoration results display device, which is a server, and the server includes an acquisition module 201 and a processing module 202; the acquisition module 201 is used to obtain the decoration style input by the user; the processing module 202 is used to match the corresponding decorated house and the decorated three-dimensional model corresponding to the decorated house according to the decoration style; the processing module 202 is also used to control the user's AR glasses to scan the undecorated house to obtain the undecorated three-dimensional model of the undecorated house; the processing module 202 is also used to calculate the size ratio factor of the decorated three-dimensional model and the undecorated three-dimensional model in a preset dimension; the processing module 202 is also used to map the decoration results of the decorated house to the undecorated three-dimensional model according to the size ratio factor, and display it in the AR glasses.

[0055] In a possible implementation, the processing module 202 controls the user's AR glasses to scan the undecorated house to obtain an undecorated three-dimensional model of the undecorated house, specifically including: the processing module 202 controls the AR glasses to collect multi-angle images of the undecorated house; the processing module 202 generates three-dimensional point cloud data of the undecorated house according to the multi-angle images; the processing module 202 constructs an undecorated three-dimensional model of the undecorated house based on the three-dimensional point cloud data.

[0056] In a possible implementation, the processing module 202 generates three-dimensional point cloud data of an undecorated house based on multi-angle images, specifically including: the acquisition module 201 acquires the overlapping areas between the multi-angle images and extracts feature points between each overlapping area; the processing module 202 determines the spatial transformation relationship between two adjacent images based on the feature points; the processing module 202 splices the multi-angle images based on the spatial transformation relationship to obtain three-dimensional point cloud data.

[0057] In a possible implementation, the processing module 202 maps the decoration results of the decorated house to the undecorated three-dimensional model according to the size scale factor, and displays it in the AR glasses, specifically including: the acquisition module 201 acquires the decoration elements in the decorated three-dimensional model, the decoration elements include ceilings, floors, walls, doors and windows, furniture and lighting; the processing module 202 scales the decoration elements according to the size scale factor to obtain target decoration elements; the processing module 202 maps the target decoration elements as decoration results to the undecorated house.

[0058] In a possible implementation, before the processing module 202 controls the user's AR glasses to scan the undecorated house and obtains the undecorated three-dimensional model of the undecorated house, the method further includes: the acquisition module 201 obtains the camera parameters of the AR glasses, the camera parameters including focal length, field of view angle and depth of field; the processing module 202 inputs the camera parameters into a preset shooting model to obtain the shooting distance and shooting angle corresponding to the camera parameters; the processing module 202 marks the shooting distance and shooting angle on the display interface of the AR glasses to guide the user to take pictures.

[0059] In a possible implementation, the processing module 202 inputs the camera parameters into a preset shooting model to obtain the shooting distance and shooting angle corresponding to the camera parameters. The method also includes: the acquisition module 201 obtains the shooting data of the AR glasses in different indoor environments, and the shooting data includes camera parameters, shooting distance, shooting angle and shooting clarity; the processing module 202 filters the shooting data to obtain training data, and the training data is the shooting data whose shooting clarity is greater than a preset clarity threshold in the shooting data; the processing module 202 constructs a neural network model for training based on the training data to obtain a preset shooting model.

[0060] In a possible implementation, the processing module 202 maps the decoration results of the decorated house to the undecorated house according to the size scale factor, and displays it in the AR glasses. The method includes: the acquisition module 201 acquires the price information of each piece of furniture in the decorated three-dimensional model; the processing module 202 calculates the sum of the prices of each piece of furniture based on the price information to obtain the furniture budget amount; the acquisition module 201 acquires the material budget amount, and obtains the total budget amount according to the furniture budget amount and the material budget price; the processing module 202 displays the total budget amount in the AR glasses.

[0061] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0062] The present application also provides an electronic device. Figure 3 , Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0063] The communication bus 302 is used to realize the connection and communication between these components.

[0064] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0065] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0066] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0067] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a decoration achievement display method based on AR.

[0068] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program stored in the memory 305 for a method of displaying decoration results based on AR. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0069] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments.

[0070] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0072] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0073] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0074] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0075] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0076] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A decoration results display method based on AR, characterized in that: The method comprises: Get the decoration style entered by the user; Matching a corresponding decorated house and a decorated three-dimensional model corresponding to the decorated house according to the decoration style; Controlling the user's AR glasses to scan the undecorated house to obtain an undecorated three-dimensional model of the undecorated house; Calculating a size ratio factor of the decorated three-dimensional model and the undecorated three-dimensional model in a preset dimension; According to the size scale factor, the decoration results of the decorated house are mapped to the undecorated three-dimensional model and displayed in the AR glasses.

2. The method according to claim 1, characterized in that The controlling the AR glasses of the user to scan the undecorated house to obtain the undecorated three-dimensional model of the undecorated house specifically includes: Controlling the AR glasses to collect multi-angle images of the undecorated house; Generating three-dimensional point cloud data of the undecorated house according to the multi-angle images; An undecorated three-dimensional model of the undecorated house is constructed based on the three-dimensional point cloud data.

3. The method according to claim 2, characterized in that Generating the three-dimensional point cloud data of the undecorated house according to the multi-angle image specifically includes: Acquire overlapping areas between the multi-angle images, and extract feature points between the overlapping areas; Determine the spatial transformation relationship between two adjacent images according to the feature points; The multi-angle images are spliced ​​according to the spatial transformation relationship to obtain the three-dimensional point cloud data.

4. The method according to claim 1, characterized in that Mapping the decoration results of the decorated house to the undecorated three-dimensional model according to the size scale factor and displaying it in the AR glasses specifically includes: Acquire decoration elements in the decorated three-dimensional model, wherein the decoration elements include ceilings, floors, walls, doors and windows, furniture, and lighting; Scaling the decoration element according to the size scale factor to obtain a target decoration element; The target decoration element is mapped to the undecorated house as the decoration achievement.

5. The method according to claim 1, characterized in that Before controlling the user's AR glasses to scan the undecorated house to obtain the undecorated three-dimensional model of the undecorated house, the method further includes: Obtaining camera parameters of the AR glasses, wherein the camera parameters include focal length, field of view angle, and depth of field; Input the camera parameters into a preset shooting model to obtain a shooting distance and a shooting angle corresponding to the camera parameters; The shooting distance and the shooting angle are marked on the display interface of the AR glasses to guide the user to take pictures.

6. The method according to claim 5, characterized in that Before inputting the camera parameters into a preset shooting model to obtain a shooting distance and a shooting angle corresponding to the camera parameters, the method further includes: Acquire shooting data of the AR glasses in different indoor environments, where the shooting data includes camera parameters, shooting distance, shooting angle, and shooting clarity; Screening the shooting data to obtain training data, where the training data is shooting data of which shooting clarity is greater than a preset clarity threshold value; Based on the training data, a neural network model is constructed for training to obtain a preset shooting model.

7. The method according to claim 1, characterized in that After mapping the decoration results of the decorated house to the undecorated house according to the size scale factor and displaying them in the AR glasses, the method includes: Obtaining price information of each piece of furniture in the decorated three-dimensional model; Based on the price information, the sum of the prices of the furniture is calculated to obtain a furniture budget amount; Obtaining the material budget amount, and obtaining the total budget amount according to the furniture budget amount and the material budget price; The total budget amount is displayed in the AR glasses.

8. An AR-based decoration achievement display device, characterized in that: The device comprises an acquisition module (201) and a processing module (202), wherein: The acquisition module (201) is used to acquire the decoration style input by the user; The processing module (202) is used to match the corresponding decorated house and the decorated three-dimensional model corresponding to the decorated house according to the decoration style; The processing module (202) is further used to control the user's AR glasses to scan the undecorated house to obtain an undecorated three-dimensional model of the undecorated house; The processing module (202) is further used to calculate a size ratio factor of the decorated three-dimensional model and the undecorated three-dimensional model in a preset dimension; The processing module (202) is further used to map the decoration results of the decorated house to the undecorated three-dimensional model according to the size scale factor, and display it in the AR glasses.

9. An electronic device, characterized in that: The electronic device (300) comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images