Home decoration three-dimensional roaming method and device, equipment, storage medium and program product

Abstract

The invention discloses a home decoration three-dimensional roaming method and device, equipment, a storage medium and a program product, and relates to the technical field of roaming data processing, and the method comprises the steps: constructing a three-dimensional house live-action space according to remote sensing image data and home decoration commodity data of a to-be-decorated house, and carrying out the point cloud layering, and obtaining two-dimensional home decoration facade information; determining viewpoint coordinates and a visual focusing range and an edge blurring range of a visual range by taking a first-person view angle of the user as a reference; loading a corresponding focusing scene image and a corresponding blurring scene image based on the two-dimensional home decoration facade information by taking the viewpoint coordinate as a center; and when roaming mobile data is received, performing roaming filling on the focused scene image and the blurred scene image according to the two-dimensional home decoration facade information, and deleting the scene images outside the visual range. According to the method, roaming is carried out in the visual range of the first-person view angle, so that the loaded data volume can be reduced, the situation of lagging caused by overhigh load of the household equipment is avoided, and the display effect is improved.

Description

Technical Field

[0001] The present application relates to the technical field of roaming data processing, and in particular to a three-dimensional roaming method, device, equipment, storage medium and program product for home decoration. Background Art

[0002] Traditionally, one has to go to building material markets or physical stores to purchase building materials, decorative items, furniture, household appliances and other items. It is difficult to choose items based on the size of one's own house and the matching of furniture. Purchasing items takes time and effort, and it is not easy to choose satisfactory products.

[0003] In order to solve the above problems, some methods have emerged to convert two-dimensional house floor plans into 3D effects for VR (Virtual Reality) roaming interaction. Through the generated panoramic three-dimensional home decoration pictures, customers can roam in three-dimensional space, match their own house space with furniture, realize the overall display of the scene, and enhance the sense of space.

[0004] However, the amount of data generated by using VR to display house spaces and furniture products is very large. Due to the amount of data and the capacity of home devices, when loading all the data, the excessive load can easily cause lag, reducing the display effect of the panoramic three-dimensional picture of the home decoration. Summary of the invention

[0005] The main purpose of this application is to provide a home decoration three-dimensional roaming method, device, equipment, storage medium and program product, aiming to solve the technical problem that due to the amount of data and the capacity of household devices, excessive load can easily cause jamming when loading all data, thereby reducing the display effect of the panoramic three-dimensional picture of the home decoration.

[0006] To achieve the above purpose, the present application proposes a home decoration three-dimensional roaming method, the method comprising: Building a house model based on remote sensing image data of the house to be renovated and home improvement product data to obtain a three-dimensional house real scene space corresponding to the house to be renovated; Performing point cloud stratification on the three-dimensional house real scene space by using an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space; Based on the user's first-person perspective, determine the user's visible range and viewpoint coordinates in the three-dimensional house real scene space, wherein the visible range includes a visual focus range and an edge blur range; Taking the viewpoint coordinates as the center, based on the two-dimensional home decoration facade information, a focused scene image corresponding to the visual focus range and a blurred scene image corresponding to the edge blur range are loaded; When the roaming movement data of the user's first-person perspective is received, the focused scene image and the blurred scene image are roamed and filled according to the roaming movement data and the two-dimensional home decoration facade information, and the scene image outside the visible range is deleted to realize the user's three-dimensional roaming.

[0007] In one embodiment, the step of constructing a house model based on the remote sensing image data of the house to be renovated and the home improvement product data to obtain a three-dimensional house real scene space corresponding to the house to be renovated includes: Performing remote sensing recognition on the interior and exterior of the house to be renovated to obtain remote sensing image data of the house to be renovated; Extracting architectural specification information of the house to be renovated according to the remote sensing image data; Extracting a three-dimensional house model and a three-dimensional home improvement product model from a preset three-dimensional building model library based on the building specification information and the home improvement product data input by the user; A house model is constructed according to the three-dimensional house model and the three-dimensional home improvement product model to obtain a three-dimensional house real scene space corresponding to the house to be decorated.

[0008] In one embodiment, the step of performing point cloud stratification on the three-dimensional house real scene space by using an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space includes: Marking the three-dimensional house real scene space by edge detection algorithm to determine the three-dimensional home decoration area; Performing point cloud extraction on the three-dimensional home decoration area to obtain three-dimensional home decoration point cloud data; Point cloud layering is performed according to the three-dimensional home decoration point cloud data to obtain two-dimensional home decoration facade information of the three-dimensional home decoration area.

[0009] In one embodiment, the step of loading the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range based on the two-dimensional home decoration facade information with the viewpoint coordinates as the center includes: Taking the viewpoint coordinates as the center, determining the edge blur range in the three-dimensional virtual area of ​​the three-dimensional house real scene space; Extracting target facade information corresponding to the three-dimensional virtual area from the two-dimensional home decoration facade information; The target facade information is blurred by using a Gaussian blur algorithm to obtain a blurred scene image; The blurred scene image and the focused scene image corresponding to the visual focus range are loaded into the visible range of the user's first-person perspective.

[0010] In one embodiment, the step of loading the blurred scene image and the focused scene image corresponding to the visual focus range into the visible range of the user's first-person perspective includes: Extracting a similar image region between the blurred scene image and the visual focus range by using a cosine similarity algorithm; Using the nearest neighbor interpolation method to fill the similar image area to obtain a fused image; The blurred scene image, the fused image and the focused scene image are spliced ​​and fused using a wavelet transform fusion algorithm to generate a roaming scene map of the user within a visible range of a first-person perspective.

[0011] In one embodiment, when receiving the roaming movement data of the user's first-person perspective, the step of roaming and filling the focused scene image and the blurred scene image according to the roaming movement data and the two-dimensional home decoration facade information, and deleting the scene image outside the visible range includes: When receiving the roaming movement data of the user's first-person perspective, determining a moving filling area and a moving deleting area in the three-dimensional house real scene space according to the roaming movement data; Taking the moved viewpoint coordinates as the center and based on the moved filling area, returning to the step of determining the edge blur range in the three-dimensional virtual area of ​​the three-dimensional house real scene space; The two-dimensional home decoration facade information corresponding to the mobile deletion area is deleted.

[0012] In addition, to achieve the above purpose, the present application also proposes a home decoration three-dimensional roaming device, the device comprising: A real-scene space module is used to construct a house model based on remote sensing image data of the house to be renovated and home improvement product data, and obtain a three-dimensional house real-scene space corresponding to the house to be renovated; A point cloud stratification module is used to perform point cloud stratification on the three-dimensional house real scene space through an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space; A visual processing module, used to determine the visual range and viewpoint coordinates of the user in the three-dimensional house real scene space based on the user's first-person perspective, wherein the visual range includes a visual focus range and an edge blur range; An image loading module, used to load the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range based on the two-dimensional home decoration facade information, with the viewpoint coordinates as the center; The perspective roaming module is used to roam and fill the focused scene image and the blurred scene image according to the roaming movement data and the two-dimensional home decoration facade information when receiving the roaming movement data of the user's first-person perspective, and delete the scene image outside the visible range to realize the user's three-dimensional roaming.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a home decoration three-dimensional roaming device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the home decoration three-dimensional roaming method as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the home improvement three-dimensional roaming method described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the home improvement three-dimensional roaming method as described above.

[0016] One or more technical solutions proposed in the present application have at least the following technical effects: the present application first constructs a house model according to the remote sensing image data and home decoration product data of the house to be decorated, and obtains the three-dimensional house real scene space corresponding to the house to be decorated; then, the three-dimensional house real scene space is layered by point cloud through an edge detection algorithm to obtain the two-dimensional home decoration facade information of the three-dimensional house real scene space; then, based on the user's first-person perspective, the user's visible range and viewpoint coordinates in the three-dimensional house real scene space are determined, and the visible range includes a visual focus range and an edge blur range; then, with the viewpoint coordinates as the center, the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range are loaded based on the two-dimensional home decoration facade information; finally, when the roaming movement data of the user's first-person perspective is received, the focused scene image and the blurred scene image are roamed and filled according to the roaming movement data and the two-dimensional home decoration facade information, and the scene image outside the visible range is deleted to realize the three-dimensional roaming of the user. Since this application uses the visible range and viewpoint coordinates of the first-person perspective as a reference to load focused scene images and blurred scene images, when the user roams, roaming fills in and deletes scene images outside the visible range. This can reduce the amount of loaded data, avoid freezes caused by excessive load on home devices, and thus improve the display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart diagram of the first embodiment of the home improvement three-dimensional roaming method of the present application.

[0020] Figure 2 A flow chart of the second embodiment of the home improvement three-dimensional roaming method of the present application.

[0021] Figure 3 A flowchart diagram of the third embodiment of the home improvement three-dimensional roaming method of the present application.

[0022] Figure 4 This is a schematic diagram of the module structure of the home decoration three-dimensional roaming device according to an embodiment of the present application.

[0023] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the home improvement three-dimensional roaming method in the embodiment of the present application.

[0024] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0025] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0026] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0027] It should be noted that the execution subject of this embodiment can be an electronic device with model building, point cloud layering, image loading and 3D roaming functions, such as an all-in-one VR device, VR glasses, etc., or an electronic device capable of realizing the above functions, a home decoration 3D roaming device (such as a VR head-mounted display device) that executes the home decoration 3D roaming method of this application, etc., and this embodiment does not limit this. The following takes a home decoration 3D roaming device as an example to illustrate this embodiment and the following embodiments.

[0028] Based on this, the present application embodiment provides a home improvement three-dimensional roaming method, referring to Figure 1 , Figure 1A flowchart diagram of the first embodiment of the home improvement three-dimensional roaming method of the present application.

[0029] In this embodiment, the home improvement three-dimensional roaming method includes steps S10 to S50: Step S10: constructing a house model according to the remote sensing image data of the house to be renovated and the home improvement product data, and obtaining a three-dimensional house real scene space corresponding to the house to be renovated.

[0030] It should be noted that the house to be decorated in this embodiment refers to a house that has completed the main construction and is waiting for home decoration. The house usually completes the basic structural construction, including wall treatment, floor paving and other projects, but a series of decoration work such as room layout and furniture arrangement is still in the assembly state.

[0031] It should be noted that the remote sensing image data is image information data about the internal structure and external main structure of the house to be renovated obtained through remote sensing technology.

[0032] Among them, the home decoration 3D roaming equipment can be used to shoot and collect data of the house to be decorated, such as the shape, structure, floor area and room layout of the house, providing a reference for the planning of house decoration.

[0033] It should be noted that home improvement product data is a collection of information on various products related to home decoration.

[0034] Exemplarily, home furnishing product data includes data on product type (e.g., dining table, coffee table, refrigerator, microwave oven, etc.), brand, specification, model, material, color, price, functional features, applicable scenarios, etc.

[0035] Among them, the home improvement product data may also include information such as the spatial position and layout of various products in the house to be decorated, so as to present the actual scene of the home improvement product data in the house to be decorated.

[0036] It should be noted that the three-dimensional house real-life space is constructed based on remote sensing image data and home improvement product data, including the interior space information, layout, size, walls, ceiling, floor, furniture placement and various details of the house, and presents this information in the form of a three-dimensional model in the real-life space.

[0037] Through the three-dimensional real-life space of a house, users can immersively complete the selection and purchase of building materials, decorative items, furniture, home appliances and other items.

[0038] Specifically, the process of building a house model may include: extracting the structural data of the house to be renovated from the remote sensing image data to preliminarily build a house model of the same scale. Then extract the image of the interior of the house to be renovated from the remote sensing image data, and map it to the internal space of the house model, so that the interior of the model is consistent with the house to be renovated. Then the user inputs the home improvement product data, including the type, brand, specifications, spatial position and layout of the product in the house to be renovated, and a proportional three-dimensional product model is established based on the information entered by the user, and it is rendered into a three-dimensional physical model consistent with the real object. Then each three-dimensional physical model is placed in the internal space of the house model, and finally a three-dimensional house real scene space with a completed home decoration is presented.

[0039] In this embodiment, through the three-dimensional house real-life space, the user can decorate the house to be decorated according to the home improvement product data to present the final effect.

[0040] In a feasible implementation manner, step S10 of this embodiment may include the steps of: performing remote sensing identification on the interior and exterior of the house to be renovated to obtain remote sensing image data of the house to be renovated; extracting architectural specification information of the house to be renovated based on the remote sensing image data; extracting a three-dimensional house model and a three-dimensional home improvement product model from a preset three-dimensional architectural model library based on the architectural specification information and the home improvement product data input by the user; constructing a house model based on the three-dimensional house model and the three-dimensional home improvement product model to obtain a three-dimensional house real scene space corresponding to the house to be renovated.

[0041] It should be noted that building specification information is information that describes various specific parameters and characteristics of a house in terms of construction.

[0042] Exemplarily, the building specification information may include the floor area of ​​the house to be renovated, the house to be renovated, the apartment layout (such as the number of bedrooms, living rooms, and bathrooms), the span and depth dimensions, the wall thickness, the specifications of doors and windows, the building structure type (such as brick-concrete structure, frame structure, etc.), the roof form, etc.

[0043] It should be noted that the 3D architectural model library is a resource library that centrally stores and manages a large number of 3D architectural models, including architectural models of various types, styles, and scales presented in a 3D digital form. Similarly, a similar model library of the 3D architectural model library may also include home furnishing product models of various types and specifications presented in a 3D digital form.

[0044] Through the model library, various architectural models or home improvement product models can be reused, called and viewed, saving the time and effort of creating models from scratch.

[0045] It should be noted that the three-dimensional house model is a model in the model library that presents the spatial structure, appearance, etc. of the house to be renovated in three dimensions in a digital way.

[0046] The three-dimensional home decoration product model is a product model in the model library that presents various products used for house decoration (such as furniture, lamps, decorations, etc.) in three dimensions.

[0047] In this embodiment, through the existing three-dimensional building model library and other similar model libraries, three-dimensional house models and three-dimensional home furnishing product models can be quickly extracted to complete the construction of the three-dimensional house real scene space.

[0048] Step S20: performing point cloud stratification on the three-dimensional house real scene space through an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space.

[0049] It should be noted that edge detection algorithms are used to detect locations where features such as brightness, color, and differences between pixels in an image change significantly. Through edge detection algorithms, edge information of the boundaries of different objects or areas in the three-dimensional house real-life space can be extracted to facilitate point cloud stratification and extract two-dimensional home decoration facade information.

[0050] It should be noted that the two-dimensional home decoration facade information is the information of the three-dimensional house real space displayed in a two-dimensional plane. It can be understood that when the perspective does not change, the scene of the three-dimensional house real space that can be observed from the current perspective is converted into picture information (similar to a camera picture), but unlike the camera picture, the two-dimensional home decoration facade information includes not only the image of the scene, but also the spatial information of the image in the three-dimensional house real space. In this way, while ensuring the visual realism of the three-dimensional house real space, the amount of VR data loaded can be further reduced.

[0051] Specifically, the process of point cloud stratification may be implemented by an edge detection algorithm or a watershed algorithm, which is not limited in this embodiment.

[0052] In this embodiment, by using two-dimensional home decoration facade information, the amount of VR data loading can be further reduced while ensuring the visual realism of the three-dimensional house real space.

[0053] In a feasible implementation, step S20 of this embodiment may include the steps of: marking the three-dimensional house real-scene space through an edge detection algorithm to determine the three-dimensional home decoration area; performing point cloud extraction on the three-dimensional home decoration area to obtain three-dimensional home decoration point cloud data; performing point cloud stratification according to the three-dimensional home decoration point cloud data to obtain two-dimensional home decoration facade information of the three-dimensional home decoration area.

[0054] It should be noted that the three-dimensional home improvement area is the area in the three-dimensional house real-life space where various home improvement products are located.

[0055] Among them, 3D home decoration point cloud data is point cloud data used to represent the space where the 3D home decoration area is located and the geometric shape and structure of home decoration products. 3D home decoration point cloud data can be composed of a large number of 3D coordinate points, accurately recording the position, shape and characteristics of home decoration products in the 3D home decoration area.

[0056] In this implementation, it is considered that when a user roams in a three-dimensional house real scene space, the human eye can only see a part of the scene in the space (for example, when the human eye is facing a cup, it can only see the front and part of the side of the cup, but the actual bottom, inside and back cannot be seen). Therefore, after obtaining the three-dimensional home decoration point cloud data, only the scene that the human eye can see can be extracted as the two-dimensional home decoration facade information, so as to realize the two-dimensionalization of the scene data of the three-dimensional house real scene space while ensuring the three-dimensional spatialization, so as to reduce the loading of VR data.

[0057] Step S30: Based on the user's first-person perspective, determine the user's visible range and viewpoint coordinates in the three-dimensional house real scene space, where the visible range includes a visual focus range and an edge blur range.

[0058] It should be noted that the first-person perspective is the perspective from which the user observes the three-dimensional real-life space of the house.

[0059] It should be noted that the visible range is the field of view that the user can see in the first-person perspective, which can be determined by the field of view angle, depth range, near clipping plane and far clipping plane.

[0060] The viewpoint coordinates are the coordinate points used to define the user's observation position in the three-dimensional house real scene space.

[0061] When the user uses the home improvement 3D roaming device, the viewpoint coordinates may be generated randomly or from the doorway of the house to be decorated, which is not limited in this embodiment.

[0062] It should be noted that the visual focus range is the area that the user can see clearly from a first-person perspective. Objects within this range are usually clear, while objects outside this range may become blurred.

[0063] Among them, the visual focus range is not only determined according to the viewpoint coordinates, but also can be further determined according to the movement of the user's eyeballs to improve the experience.

[0064] The edge blur range refers to the area outside the visual focus range where the eyes cannot maintain clear focus on the object, resulting in blurred vision.

[0065] Step S40: With the viewpoint coordinates as the center, based on the two-dimensional home decoration facade information, the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range are loaded.

[0066] It should be noted that the focused scene image is an image that clearly displays the two-dimensional home decoration facade information of the three-dimensional house real space within the visual focus range. At this time, when the user focuses on this area, the scene of the three-dimensional house real space can be clearly presented.

[0067] It should be noted that the blurred scene image is an image in which the two-dimensional home decoration facade information of the three-dimensional house real scene space is blurred to a certain extent (for example, some pixels are reduced to reduce clarity and details) within the edge blur range. At this time, since the user is not focusing on this, even if the scene image is blurred, it will not reduce the user's experience.

[0068] Specifically, in the process of loading the two-dimensional home decoration facade information, it is only necessary to blur the two-dimensional home decoration facade information within the edge blur range through a fuzzy algorithm (such as a mean fuzzy algorithm, a Gaussian fuzzy algorithm, etc.), and there is no need to process the two-dimensional home decoration facade information within the visual focus range.

[0069] In this embodiment, since VR interaction is performed from the user's first-person perspective, in the user's actual observation, the user's perspective is generally focused on a range, and other areas are blurred because the eyes cannot maintain clear focus on objects. Therefore, the two-dimensional home decoration facade information in the edge blur range can be blurred to further reduce the amount of VR data loading.

[0070] In a feasible implementation, step S40 of this embodiment may include the steps of: determining, with the viewpoint coordinates as the center, a three-dimensional blurred area of ​​the edge blur range in the three-dimensional house real scene space; extracting the target facade information corresponding to the three-dimensional blurred area from the two-dimensional home decoration facade information; blurring the target facade information through a Gaussian blur algorithm to obtain a blurred scene image; and loading the blurred scene image and the focused scene image corresponding to the visual focus range into the visible range of the user's first-person perspective.

[0071] It should be noted that the three-dimensional blurred area is an area in the three-dimensional house real scene space that is within the edge blur range and needs to be visually blurred.

[0072] The target facade information is information about the three-dimensional house real scene space displayed in a two-dimensional plane in the three-dimensional virtual area.

[0073] It should be noted that the Gaussian blur algorithm is an algorithm that uses a Gaussian function (a bell-shaped curve function) to calculate the weighted average value around each pixel to blur the image. Through the Gaussian blur algorithm, the details in the image generated by the target facade information can be reduced to achieve a blurred visual effect.

[0074] In this embodiment, the viewpoint coordinates can be determined first, and the visual focus range and edge blur range can be calculated with the given viewpoint coordinates as the center. Then, the edge blur range is determined in the three-dimensional virtual area of ​​the three-dimensional house real scene space with the viewpoint coordinates as the center. Then, the two-dimensional home decoration facade information in the three-dimensional virtual area is read as the target facade information for blurring, such as blurring the image within the range through a Gaussian blur algorithm to achieve a blurring effect. Finally, the extracted focused scene image and the processed blurred scene image are loaded and integrated according to the corresponding positions and levels to ensure that the expected effect in the user's first-person perspective can be correctly presented when displayed.

[0075] Step S50: When receiving the roaming movement data of the user's first-person perspective, the focused scene image and the blurred scene image are roamed and filled according to the roaming movement data and the two-dimensional home decoration facade information, and the scene image outside the visible range is deleted to realize the user's three-dimensional roaming.

[0076] It should be noted that the roaming movement data is the movement data generated by the user roaming in the three-dimensional house real space. The roaming movement data includes not only the spatial movement of the viewpoint coordinates, but also the rotation or orientation of the user's eyeball to ensure the accuracy of the visual focus range and edge blur range.

[0077] In this embodiment, as the user moves in the 3D house real scene space, the scene in the space that the human eye can see is constantly changing. Therefore, the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range can be reloaded within the new viewing angle range, and the original scene image of the area that has moved out of the visible range can be deleted. In this way, the load on the home decoration 3D roaming device can be reduced when the user cannot observe it.

[0078] In the technical solution provided in this embodiment, the home decoration three-dimensional roaming device can first extract the structural data of the house to be decorated in the remote sensing image data to preliminarily construct a house model of the same scale. Then extract the image picture of the interior of the house to be decorated in the remote sensing image data, and map it to the internal space of the house model, so that the interior of the model is consistent with the house to be decorated. Then the user inputs the home decoration product data, including the type, brand, specification, spatial position, layout and other information of the product in the house to be decorated, and establishes a proportional three-dimensional product model according to the information input by the user, and renders it into a three-dimensional physical model consistent with the real object. Then each three-dimensional physical model is placed in the internal space of the house model, and finally presents the three-dimensional house real scene space of the completed home decoration. Next, the three-dimensional house real scene space is layered by the edge detection algorithm. After obtaining the three-dimensional home decoration point cloud data, only the scene that can be seen by the human eye can be extracted as the two-dimensional home decoration facade information. Then, based on the user's first-person perspective, the user's visible range and viewpoint coordinates in the three-dimensional house real scene space are determined, and the visible range includes the visual focus range and the edge blur range. Then, in the process of loading the two-dimensional home decoration facade information, it is only necessary to blur the two-dimensional home decoration facade information in the edge blur range through a fuzzy algorithm (such as a mean fuzzy algorithm, a Gaussian fuzzy algorithm, etc.), and there is no need to process the two-dimensional home decoration facade information in the visual focus range. Finally, when the user moves in the three-dimensional house real scene space, considering that the scene in the space that the human eye can see is constantly changing. Therefore, the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range can be reloaded within the new viewing angle range, and the original scene image of the area moved out of the visible range can be deleted. In this way, the load on the home decoration three-dimensional roaming device can be reduced when the user cannot observe it. Since this embodiment uses the visible range and viewpoint coordinates of the first-person perspective as the reference, the focused scene image and the blurred scene image are loaded. When the user roams, the roaming fill and the scene image outside the visible range are deleted, which can reduce the amount of loaded data, avoid the situation of jamming caused by excessive load of home devices, and thus improve the display effect.

[0079] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 1 and Figure 2 , Figure 2 A flow chart of the second embodiment of the home improvement three-dimensional roaming method of the present application.

[0080] The step of loading the blurred scene image and the focused scene image corresponding to the visual focus range into the visible range of the user's first-person perspective in this example further includes steps S401 to S403: Step S401: extracting a similar image region between the blurred scene image and the visual focus range by using a cosine similarity algorithm.

[0081] It should be noted that the cosine similarity algorithm is a measurement method used to measure the similarity between two objects. The cosine similarity algorithm can be used to determine the similar areas (i.e., similar image areas) adjacent to the boundaries of the blurred scene image and the visual focus range.

[0082] Step S402: using the nearest neighbor interpolation method to fill the similar image area to obtain a fused image.

[0083] It should be noted that the nearest neighbor interpolation method is an interpolation technology used for image resampling. Through the nearest neighbor interpolation method, the blurred scene image in the similar image area can be further filled with pixels to obtain a fused image. This allows the subsequent splicing between the focused scene image and the blurred scene image to be transitioned through the fused image without appearing abrupt.

[0084] Step S403: using a wavelet transform fusion algorithm to splice and fuse the blurred scene image, the fused image and the focused scene image, to generate a roaming scene map of the user within the visible range of the first-person perspective.

[0085] It should be noted that the wavelet transform fusion algorithm is an algorithm that decomposes the image into different scales and directions, then fuses the decomposed coefficients, and then obtains the fused image through inverse wavelet transform.

[0086] Through the wavelet transform fusion algorithm, the stitching seams and ghosting problems in the stitching and fusion process of blurred scene images, fused images and focused scene images can be eliminated, making the image stitching and fusion smoother and achieving better fusion effects.

[0087] It should be noted that the roaming scene graph is a scene graph of the three-dimensional house real space presented in the user's first-person perspective after the above-mentioned splicing and fusion are completed.

[0088] In the technical solution provided in this embodiment, a series of image processing is performed through the above-mentioned cosine similarity algorithm, nearest neighbor interpolation method and wavelet transform fusion algorithm, which can make the splicing and fusion of the focused scene image and the blurred scene image smoother and achieve a better fusion effect.

[0089] Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 1 and Figure 3 , Figure 3 A flowchart diagram of the third embodiment of the home improvement three-dimensional roaming method of the present application.

[0090] In this example, step S50 includes steps S51 to S53: Step S51: upon receiving the roaming movement data of the user's first-person perspective, determining a movement filling area and a movement deleting area in the three-dimensional house real scene space according to the roaming movement data.

[0091] It should be noted that the moving fill area is the area of ​​the three-dimensional house real scene space that appears in the perspective when the user's first-person perspective moves. The moving deletion area is the area of ​​the three-dimensional house real scene space that is removed from the original perspective when the user's first-person perspective moves.

[0092] Step S52: Taking the moved viewpoint coordinates as the center and based on the moved filling area, returning to the step of determining the three-dimensional virtual area of ​​the edge blur range in the three-dimensional house real scene space.

[0093] Step S53: deleting the two-dimensional home decoration facade information corresponding to the move deletion area.

[0094] In this embodiment, when the user's first-person perspective moves, a new moving filling area appears, so the home decoration 3D roaming device needs to re-determine the visual focus range and edge blur range of the moving filling area. Then, according to the re-determined visual focus range and edge blur range of the moving filling area, the above-mentioned process of loading the 2D home decoration facade information is performed. For the moving deletion area, the corresponding 2D home decoration facade information can be deleted.

[0095] In the technical solution provided in this embodiment, by re-determining the moving filling area and the moving deleting area, the scene loading of the 3D house real scene space in the first-person perspective can be continuously changed when the user's first-person perspective moves. In this way, the load on the home decoration 3D roaming device can be reduced when the user cannot observe.

[0096] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the home improvement three-dimensional roaming method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0097] This application also provides a home decoration three-dimensional roaming device, please refer to Figure 4 , Figure 4 This is a schematic diagram of the module structure of the home decoration 3D roaming device according to the embodiment of the present application; the home decoration 3D roaming device includes: The real-scene space module 401 is used to construct a house model according to the remote sensing image data of the house to be renovated and the home improvement product data, and obtain a three-dimensional house real-scene space corresponding to the house to be renovated; The point cloud stratification module 402 is used to perform point cloud stratification on the three-dimensional house real scene space by using an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space; A visual processing module 403 is used to determine the visual range and viewpoint coordinates of the user in the three-dimensional house real scene space based on the user's first-person perspective, wherein the visual range includes a visual focus range and an edge blur range; An image loading module 404 is used to load the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range based on the two-dimensional home decoration facade information, with the viewpoint coordinates as the center; The perspective roaming module 405 is used to roam and fill the focused scene image and the blurred scene image according to the roaming movement data and the two-dimensional home decoration facade information when receiving the roaming movement data of the user's first-person perspective, and delete the scene image outside the visible range to realize the user's three-dimensional roaming.

[0098] In this embodiment, the real-scene space module 401 is also used to perform remote sensing identification on the interior and exterior of the house to be renovated to obtain remote sensing image data of the house to be renovated; extract architectural specification information of the house to be renovated based on the remote sensing image data; extract a three-dimensional house model and a three-dimensional home improvement product model from a preset three-dimensional architectural model library based on the architectural specification information and the home improvement product data input by the user; construct a house model based on the three-dimensional house model and the three-dimensional home improvement product model to obtain a three-dimensional house real-scene space corresponding to the house to be renovated.

[0099] Furthermore, the point cloud stratification module 402 is also used to mark the three-dimensional house real-scene space through an edge detection algorithm to determine the three-dimensional home decoration area; perform point cloud extraction on the three-dimensional home decoration area to obtain three-dimensional home decoration point cloud data; perform point cloud stratification based on the three-dimensional home decoration point cloud data to obtain two-dimensional home decoration facade information of the three-dimensional home decoration area.

[0100] Furthermore, the image loading module 404 is also used to determine the three-dimensional blurred area of ​​the edge blur range in the three-dimensional house real scene space with the viewpoint coordinates as the center; extract the target facade information corresponding to the three-dimensional blurred area from the two-dimensional home decoration facade information; blur the target facade information through a Gaussian blur algorithm to obtain a blurred scene image; and load the blurred scene image and the focused scene image corresponding to the visual focus range into the visible range of the user's first-person perspective.

[0101] Furthermore, the image loading module 404 is also used to extract similar image areas between the blurred scene image and the visual focus range through a cosine similarity algorithm; perform image filling on the similar image areas using a nearest neighbor interpolation method to obtain a fused image; and perform splicing and fusion of the blurred scene image, the fused image and the focused scene image using a wavelet transform fusion algorithm to generate a roaming scene map of the user within the visible range of the first-person perspective.

[0102] Furthermore, the perspective roaming module 405 is also used to determine the moving filling area and the moving deletion area in the three-dimensional house real scene space according to the roaming movement data when receiving the roaming movement data of the user's first-person perspective; with the viewpoint coordinates after the movement as the center, based on the moving filling area, return to execute the operation of determining the three-dimensional virtual area of ​​the edge blur range in the three-dimensional house real scene space; and delete the two-dimensional home decoration facade information corresponding to the moving deletion area.

[0103] Other embodiments or specific implementations of the home decoration three-dimensional roaming device of the present application can refer to the above-mentioned method embodiments, which will not be repeated here.

[0104] The home decoration 3D roaming device provided by the present application adopts the home decoration 3D roaming method in the above embodiment, which can solve the technical problem that the existing method of using VR to display house space and furniture products, due to the amount of data and the capacity of household equipment, when loading all data, excessive load is easy to cause jamming, reducing the display effect of the home decoration panoramic 3D picture. Compared with the prior art, the beneficial effects of the home decoration 3D roaming device provided by the present application are the same as the beneficial effects of the home decoration 3D roaming method provided by the above embodiment, and the other technical features of the home decoration 3D roaming device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0105] The present application provides a home decoration three-dimensional roaming device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the home decoration three-dimensional roaming method in the above-mentioned embodiment one.

[0106] Reference below Figure 5 , Figure 5The device structure diagram of the hardware operating environment involved in the home decoration 3D roaming method in the embodiment of the present application shows a schematic diagram of the structure of the home decoration 3D roaming device suitable for implementing the embodiment of the present application. The home decoration 3D roaming device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The home improvement three-dimensional roaming device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0107] like Figure 5 As shown, the home improvement three-dimensional roaming device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the home improvement three-dimensional roaming device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the home improvement 3D roaming device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a home improvement 3D roaming device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0108] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0109] The home decoration 3D roaming device provided by the present application adopts the home decoration 3D roaming method in the above embodiment, which can solve the technical problem that the existing method of using VR to display house space and furniture products, due to the amount of data and the capacity of home equipment, when loading all data, excessive load is easy to cause jamming, reducing the display effect of the home decoration panoramic 3D picture. Compared with the prior art, the beneficial effects of the home decoration 3D roaming device provided by the present application are the same as the beneficial effects of the home decoration 3D roaming method provided by the above embodiment, and the other technical features of the home decoration 3D roaming device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0110] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0111] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0112] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the home improvement three-dimensional roaming method in the above-mentioned embodiment.

[0113] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0114] The computer-readable storage medium may be included in the home decoration 3D roaming device; or may exist independently without being assembled into the home decoration 3D roaming device.

[0115] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the home decoration three-dimensional roaming device, the home decoration three-dimensional roaming device: constructs a house model according to the remote sensing image data and home decoration product data of the house to be decorated, and obtains the three-dimensional house real scene space corresponding to the house to be decorated; performs point cloud stratification on the three-dimensional house real scene space through an edge detection algorithm to obtain the two-dimensional home decoration facade information of the three-dimensional house real scene space; determines the user's visible range and viewpoint coordinates in the three-dimensional house real scene space based on the user's first-person perspective, and the visible range includes a visual focus range and an edge blur range; takes the viewpoint coordinates as the center, and loads the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range based on the two-dimensional home decoration facade information; when receiving the roaming movement data of the user's first-person perspective, roams and fills the focused scene image and the blurred scene image according to the roaming movement data and the two-dimensional home decoration facade information, and deletes the scene image outside the visible range to realize the three-dimensional roaming of the user.

[0116] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0119] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned home improvement three-dimensional roaming method, and can solve the technical problem that the existing method of using VR to display house space and furniture products, due to the amount of data and the capacity of household devices, when loading all data, excessive load is easy to cause jamming, reducing the display effect of the home improvement panoramic three-dimensional picture. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the home improvement three-dimensional roaming method provided by the above-mentioned embodiment, and will not be repeated here.

[0120] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned home improvement three-dimensional roaming method when executed by a processor.

[0121] The computer program product provided by the present application can solve the technical problem that the existing method of using VR to display house space and furniture products, due to the amount of data and the capacity of household devices, when loading all data, excessive load is easy to cause jamming, reducing the display effect of the panoramic three-dimensional picture of home decoration. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the three-dimensional roaming method for home decoration provided by the above embodiment, which will not be repeated here.

[0122] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A three-dimensional roaming method for home decoration, characterized in that: The method includes: Building a house model based on remote sensing image data of the house to be renovated and home improvement product data to obtain a three-dimensional house real scene space corresponding to the house to be renovated; Performing point cloud stratification on the three-dimensional house real scene space by using an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space; Based on the user's first-person perspective, determine the user's visible range and viewpoint coordinates in the three-dimensional house real scene space, wherein the visible range includes a visual focus range and an edge blur range; Taking the viewpoint coordinates as the center, based on the two-dimensional home decoration facade information, a focused scene image corresponding to the visual focus range and a blurred scene image corresponding to the edge blur range are loaded; When the roaming movement data of the user's first-person perspective is received, the focused scene image and the blurred scene image are roamed and filled according to the roaming movement data and the two-dimensional home decoration facade information, and the scene image outside the visible range is deleted to realize the user's three-dimensional roaming.

2. The method according to claim 1, characterized in that The step of constructing a house model according to the remote sensing image data of the house to be renovated and the home improvement product data to obtain a three-dimensional house real scene space corresponding to the house to be renovated comprises: Performing remote sensing recognition on the interior and exterior of the house to be renovated to obtain remote sensing image data of the house to be renovated; Extracting architectural specification information of the house to be renovated according to the remote sensing image data; Extracting a three-dimensional house model and a three-dimensional home improvement product model from a preset three-dimensional building model library based on the building specification information and the home improvement product data input by the user; A house model is constructed according to the three-dimensional house model and the three-dimensional home improvement product model to obtain a three-dimensional house real scene space corresponding to the house to be decorated.

3. The method according to claim 1, characterized in that The step of performing point cloud stratification on the three-dimensional house real scene space by using an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space includes: Marking the three-dimensional house real scene space by edge detection algorithm to determine the three-dimensional home decoration area; Performing point cloud extraction on the three-dimensional home decoration area to obtain three-dimensional home decoration point cloud data; Point cloud layering is performed according to the three-dimensional home decoration point cloud data to obtain two-dimensional home decoration facade information of the three-dimensional home decoration area.

4. The method according to any one of claims 1 to 3, characterized in that The step of loading the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range based on the two-dimensional home decoration facade information with the viewpoint coordinates as the center includes: Taking the viewpoint coordinates as the center, determining the edge blur range in the three-dimensional virtual area of ​​the three-dimensional house real scene space; Extracting target facade information corresponding to the three-dimensional virtual area from the two-dimensional home decoration facade information; The target facade information is blurred by using a Gaussian blur algorithm to obtain a blurred scene image; The blurred scene image and the focused scene image corresponding to the visual focus range are loaded into the visible range of the user's first-person perspective.

5. The method according to claim 4, characterized in that The step of loading the blurred scene image and the focused scene image corresponding to the visual focus range into the visible range of the user's first-person perspective includes: Extracting a similar image region between the blurred scene image and the visual focus range by using a cosine similarity algorithm; Using the nearest neighbor interpolation method to fill the similar image area to obtain a fused image; The blurred scene image, the fused image and the focused scene image are spliced ​​and fused using a wavelet transform fusion algorithm to generate a roaming scene map of the user within a visible range of a first-person perspective.

6. The method according to claim 4, characterized in that The step of performing roaming filling on the focused scene image and the blurred scene image according to the roaming movement data and the two-dimensional home decoration facade information when receiving the roaming movement data of the user's first-person perspective, and deleting the scene image outside the visible range includes: When receiving the roaming movement data of the user's first-person perspective, determining a moving filling area and a moving deleting area in the three-dimensional house real scene space according to the roaming movement data; Taking the moved viewpoint coordinates as the center and based on the moved filling area, returning to the step of determining the edge blur range in the three-dimensional virtual area of ​​the three-dimensional house real scene space; The two-dimensional home decoration facade information corresponding to the mobile deletion area is deleted.

7. A three-dimensional roaming device for home decoration, characterized in that: The device comprises: A real-scene space module is used to construct a house model based on remote sensing image data of the house to be renovated and home improvement product data, and obtain a three-dimensional house real-scene space corresponding to the house to be renovated; A point cloud stratification module is used to perform point cloud stratification on the three-dimensional house real scene space through an edge detection algorithm to obtain two-dimensional home decoration facade information of the three-dimensional house real scene space; A visual processing module, used to determine the visual range and viewpoint coordinates of the user in the three-dimensional house real scene space based on the user's first-person perspective, wherein the visual range includes a visual focus range and an edge blur range; An image loading module, used to load the focused scene image corresponding to the visual focus range and the blurred scene image corresponding to the edge blur range based on the two-dimensional home decoration facade information, with the viewpoint coordinates as the center; The perspective roaming module is used to roam and fill the focused scene image and the blurred scene image according to the roaming movement data and the two-dimensional home decoration facade information when receiving the roaming movement data of the user's first-person perspective, and delete the scene image outside the visible range to realize the user's three-dimensional roaming.

8. A three-dimensional roaming device for home decoration, characterized in that: The device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the home improvement three-dimensional roaming method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the home improvement three-dimensional roaming method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the home improvement three-dimensional roaming method according to any one of claims 1 to 6 are implemented.

Images