AR scene furniture identification and dynamic removal system based on artificial intelligence

Abstract

The invention belongs to the technical field of augmented reality, and discloses an AR scenario furniture identification and dynamic removal system based on artificial intelligence, and the system comprises the steps: carrying out the multi-view image registration, furniture entity segmentation and shielding completion analysis based on the obtained hotel scene AR image sequence data, and forming a complete furniture contour feature; the style similarity of the furniture multi-dimensional feature vectors is analyzed, and a furniture style recognition result is obtained; the method comprises the following steps: acquiring AR scanning data of a home environment, performing furniture removal priority ranking through space conflict analysis and style coordination evaluation, generating a dynamic removal strategy, establishing furniture three-dimensional model data based on the dynamic removal strategy, and performing material texture mapping and shadow fusion to obtain a realistic rendering effect; an interactive optimization adjustment scheme is generated, and scene parameter real-time adjustment and optimization and visual effect improvement are realized through user feedback; the reality sense of virtual home display and the user experience effect are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality technology, more particularly, the present application relates to an AR scene-based furniture recognition and dynamic removal system based on artificial intelligence. BACKGROUND

[0002] With the deepening of augmented reality technology and the rapid development of smart home industry, AR scene-based furniture recognition and replacement system has become a key support to improve indoor design efficiency and user experience. Current AR furniture recognition and replacement systems mainly achieve virtual display and replacement effect of furniture through image recognition, three-dimensional reconstruction, virtual rendering and other technologies; however, these systems still have obvious deficiencies in accurate recognition of furniture boundaries and occlusion completion in complex lighting environments.

[0003] Existing AR furniture systems usually ignore the coupling relationship between furniture surface material reflection characteristics and environmental lighting changes, which can cause the furniture edge features to be blurred or distorted in the image; the occluded area of the furniture will produce incomplete and discontinuous contour features, and the dynamic change of the environmental lighting will interfere with these feature signals. Due to the lack of real-time decoupling analysis of the coupling effect of furniture geometry and lighting, it is difficult to accurately capture the complete three-dimensional structure of the furniture in actual AR application process, resulting in distorted virtual replacement effect, and thus unable to effectively implement immersive experience strategy; especially in complex lighting environments such as hotels and exhibition halls, the composite interference of multi-light source cross irradiation and shadow projection will further reduce the recognizability of furniture features. The existing multi-view fusion system fails to consider the distortion and non-uniformity of image data under different viewing angles, and the simple image stitching method often amplifies rather than suppresses the interference of viewing angle difference, resulting in poor three-dimensional reconstruction effect. The limitations of this processing method make the system's recognition accuracy of furniture style significantly decrease in complex scene conditions, with low average fusion reality sensitivity, which seriously affects the immersion and satisfaction of user visual experience; in addition, different material furniture has significant differences in visual characteristics of light reflection, and the current AR rendering platform lacks adaptive analysis capability for light adaptation characteristics of different material furniture, resulting in insufficient universality of rendering effect algorithm, which cannot cope with the visual feature variation caused by material diversification, and thus affects the reality of scene fusion and the robustness of rendering model.

[0004] In view of this, the present application proposes an AR scene-based furniture recognition and dynamic removal system based on artificial intelligence to solve the above problems. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purpose, the present application provides the following technical solutions: The AR scene-based furniture recognition and dynamic removal system based on artificial intelligence comprises: The furniture recognition module is used for acquiring hotel scene AR image sequence data, furniture style sample library data and illumination environment parameter data; multi-view image registration and furniture entity segmentation are performed according to the collected AR image data sequence, and furniture candidate region data is obtained; occlusion completion analysis is performed on the furniture candidate region data, and a furniture multi-dimensional feature vector is constructed in combination with the illumination environment parameter data; style similarity calculation is performed based on the furniture multi-dimensional feature vector and the furniture style sample library data, and a furniture style recognition result is obtained; The furniture removal module is used for performing space depth reconstruction and existing furniture layout recognition according to the pre-acquired home environment AR scanning data, and obtaining a home furniture distribution map; space occupation conflict analysis and style coordination evaluation are respectively performed based on the home furniture distribution map and the furniture style recognition result, a style conflict degree index is obtained, and a dynamic removal strategy is generated based on the style conflict degree index; The furniture fusion module performs virtual removal processing on the home furniture distribution map based on the dynamic removal strategy, and obtains empty space region data; geometric morphological parameterization reconstruction is performed, and furniture three-dimensional model data is obtained; material texture mapping and target position illumination characteristic analysis are respectively performed based on the furniture three-dimensional model data; corresponding texture furniture model and environment illumination parameters are obtained; color migration processing is performed on the texture furniture model based on the environment illumination parameters, and the texture furniture model is placed in the empty space region data to obtain a fusion scene rendering result; The fusion optimization module performs reality consistency detection based on the fusion scene rendering result, and performs visual abnormal point identification based on the reality consistency detection, and obtains abnormal region marking data; interactive parameter adjustment is performed based on the abnormal region marking data, and rendering parameter optimization is performed on the illumination-adapted furniture model according to the adjustment result, and optimized scene data is obtained.

[0006] Further, the acquisition process of the furniture style recognition result comprises: A plurality of angle images of a hotel scene are collected by an AR device to obtain a set of original image frames; the set of original image frames is time-stamped and calibrated to obtain a synchronized image sequence; Device pose data of each image frame in the synchronized image sequence is acquired; camera motion trajectory estimation is performed based on the device pose data, and camera trajectory parameters are obtained; feature point matching is performed on the synchronized image sequence based on the camera trajectory parameters, and a set of matched feature points is obtained; Triangulation calculation is performed on the set of matched feature points to obtain a sparse space point cloud; dense reconstruction is performed on the sparse space point cloud to obtain scene space point cloud data; Plane extraction analysis is performed on the scene space point cloud data to obtain scene structure plane data; spatial semantic segmentation is performed according to the scene structure plane data to obtain object region labels, including ground, wall and object region labels; The object region label is subjected to connectivity cluster analysis to obtain an independent object cluster set; geometric shape regularity determination is performed on the independent object cluster set to obtain furniture candidate region data; The furniture candidate region data is subjected to occlusion edge detection to obtain an occlusion boundary contour; and complementary information is extracted from the multi-view image according to the occlusion boundary contour to obtain occlusion region completion data; edge smoothing connection is performed based on the occlusion region completion data to obtain complete furniture contour data; Light environment parameter data at the image acquisition time is obtained; the furniture candidate region data is subjected to light normalization processing to obtain a light invariance feature map; and geometric shape features, texture distribution features and color statistical features are extracted from the light invariance feature map to obtain a furniture multi-dimensional feature vector; Style label clustering is performed on the furniture style sample library data to obtain a style category prototype feature library; Mahalanobis distance measurement is performed on the furniture multi-dimensional feature vector and the style category prototype feature library to obtain a style similarity score; and the style category to which the furniture belongs is determined according to the style similarity score to obtain a furniture style recognition result.

[0007] Further, the occlusion region completion data comprises: Gradient direction consistency analysis is performed on the furniture candidate region data to obtain an edge continuity breaking point; the occlusion occurrence position is determined according to the edge continuity breaking point to obtain occlusion region positioning data; Boundary contour tracking is performed on the occlusion region positioning data to obtain an occlusion boundary contour; and view angle traversal search is performed on the multi-view image according to the occlusion boundary contour to obtain a candidate completion view angle set; The view angle with the highest visibility of the occlusion region is selected from the candidate completion view angle set to obtain an optimal completion view angle; point cloud data of the corresponding region in the optimal completion view angle is extracted to obtain occlusion region completion point cloud; and the occlusion region completion point cloud is mapped back to the original view angle coordinate system to obtain the occlusion region completion data.

[0008] Further, the acquisition process of the dynamic removal strategy comprises: The home environment is scanned and collected by an AR device to obtain home environment image data; binocular stereo matching is performed on the home environment image data to obtain depth image data; Voxel space modeling is performed according to the depth image data to obtain a home space voxel model; grid optimization is performed on the home space voxel model to obtain a home space three-dimensional model; Object instance segmentation is performed on the home space three-dimensional model to obtain an existing furniture instance set; spatial position extraction is performed on the existing furniture instance set to obtain a home furniture distribution map; The three-dimensional bounding box size of the target furniture is calculated according to the complete furniture contour data to obtain target furniture space occupation parameters; The target furniture space occupation parameter is virtually placed and traversed in the home furniture distribution map to obtain a placement candidate position set; and collision detection calculation is performed on each position in the placement candidate position set to obtain a space conflict matrix; A style attribute vector of the target furniture is extracted according to the furniture style recognition result; style attribute vectors of existing furniture instances are extracted; cosine similarity of the target furniture style attribute vector and the existing furniture style attribute vectors is calculated to obtain a style coordination degree value; Reverse scoring is performed based on the style coordination degree value to obtain a style conflict degree index; the space conflict matrix and the style conflict degree index are comprehensively sorted to obtain a furniture removal priority sequence; and a dynamic removal strategy is generated based on the furniture removal priority sequence.

[0009] Further, the acquisition process of the scene rendering result includes: The highest priority furniture object is selected from the home furniture distribution map according to the dynamic removal strategy to obtain a furniture to be removed identifier; a virtual emptying process is performed on a space region corresponding to the furniture to be removed identifier to obtain empty space region data; Skeleton extraction is performed on the complete furniture contour data to obtain a furniture skeleton structure diagram; parameterized shape fitting is performed based on the furniture skeleton structure diagram to obtain a furniture geometric parameter set; A furniture three-dimensional mesh model is constructed according to the furniture geometric parameter set to obtain furniture three-dimensional model data; A furniture surface texture image is extracted from the AR image sequence data to obtain original texture data; perspective distortion correction is performed on the original texture data to obtain a corrected texture image; The corrected texture image is mapped to a UV coordinate space of the furniture three-dimensional model data to obtain a textured furniture model; Local light source position estimation is performed on the empty space region data in the home space three-dimensional model to obtain a main light source direction vector; ambient light intensity distribution is calculated according to the main light source direction vector to obtain an ambient light parameter; Material reflection properties of the textured furniture model are extracted to obtain a material reflection coefficient; light transmission calculation is performed based on the ambient light parameter and the material reflection coefficient to obtain a target light rendering parameter; Color space conversion is performed on the textured furniture model according to the target light rendering parameter to obtain a color adjustment matrix; color migration is performed on the furniture model texture based on the color adjustment matrix to obtain a light-adapted furniture model; The light-adapted furniture model is placed in the empty space region data; a shadow projection region is calculated according to the main light source direction vector and the furniture geometric parameter set to obtain shadow mask data; soft shadow blur processing is performed on the shadow mask data to obtain a soft shadow image; Alpha blend the soft shadow image with the home space three-dimensional model to obtain scene data with shadow; perform depth sorting rendering on the light-adapted furniture model and the scene data with shadow to obtain a fusion scene rendering result.

[0010] Further, the obtaining process of the furniture geometry parameter set comprises: Perform a morphological thinning operation on the complete furniture contour data to obtain a single-pixel width skeleton line; perform branch point and endpoint detection on the single-pixel width skeleton line to obtain a skeleton key node set; and construct a skeleton topology connection relationship according to the skeleton key node set to obtain a furniture skeleton structure diagram; Perform structure type matching on the furniture skeleton structure diagram to obtain a furniture category label; and select a corresponding parameterized shape template according to the furniture category label to obtain a basic shape template; Perform least squares fitting on the control parameters of the basic shape template and the complete furniture contour data to obtain optimal shape parameter values; and generate a furniture geometry parameter set based on the combination thereof.

[0011] Further, the obtaining process of the target light rendering parameter comprises: Perform type identification on the surface material of the textured furniture model to obtain a material type label, and query a corresponding bidirectional reflectance distribution function from a preset material attribute database according to the material type label to obtain a material BRDF model; calculate the diffuse reflection coefficient and the specular reflection coefficient based on the material BRDF model to obtain the material reflection coefficient; Perform light intensity calculation on the basis of the main light source direction vector and intensity value in the ambient light parameter, in combination with the material reflection coefficient, to obtain the light intensity value of each vertex; Perform normalization processing on the light intensity value of each vertex to obtain a normalized light intensity distribution; and generate the target light rendering parameter according to the normalized light intensity distribution.

[0012] Further, the obtaining process of the optimized scene data comprises: Perform image quality evaluation on the fusion scene rendering result to obtain an image sharpness index; and perform light consistency detection on the fusion scene rendering result to obtain a light consistency score; Perform edge fusion degree analysis on the fusion scene rendering result to obtain edge transition smoothness; and perform weighted comprehensive scoring based on the image sharpness index, the light consistency score, and the edge transition smoothness to obtain a scene reality score; Set an abnormality detection threshold value according to the scene reality score; and perform positioning marking on a region with a scene reality score lower than the abnormality detection threshold value to obtain abnormal region marking data; Present the fusion scene rendering result and the abnormal region marking data to the user; and collect touch interaction operation data of the user to obtain an interaction operation sequence; The operation type recognition is performed on the interaction operation sequence, so as to obtain an adjustment operation type; and the user adjustment intention is analyzed according to the adjustment operation type, so as to obtain an adjustment direction instruction; The position, rotation angle and zoom ratio of the light-adapted furniture model are corrected based on the adjustment direction instruction, so as to obtain geometric adjustment parameters; and the color, brightness and saturation of the light-adapted furniture model are corrected, so as to obtain color adjustment parameters; The geometric adjustment parameters and the color adjustment parameters are integrated, so as to obtain user tuning parameters; the user tuning parameters are applied to regenerate the fusion scene rendering result, so as to obtain optimized scene data and display the optimized scene data to the user.

[0013] Further, the obtaining process of the scene reality score includes: The Laplacian gradient calculation is performed on the fusion scene rendering result, so as to obtain an edge gradient intensity map; and the statistical variance calculation is performed on the edge gradient intensity map, so as to obtain an image definition index; The light distribution histogram of the furniture region and the surrounding environment region in the fusion scene rendering result is extracted, so as to obtain a region light histogram set; the histogram correlation calculation is performed on the region light histogram set, so as to obtain a light consistency score; The color difference analysis is performed on the joint edge of the furniture and the environment in the fusion scene rendering result, so as to obtain an edge color jump amplitude; the gradient continuity of the edge transition region is calculated according to the edge color jump amplitude, so as to obtain an edge transition smoothness; The image definition index, the light consistency score and the edge transition smoothness are weighted and summed, so as to obtain the scene reality score.

[0014] Further, the preset material attribute database stores material entries of hundreds of common materials, including material name, BRDF model type, model parameter, reflectivity spectrum and roughness range.

[0015] The technical effects and advantages of the AR scene-based furniture recognition and dynamic removal system based on artificial intelligence are as follows: This invention utilizes multi-view image registration and spatial point cloud reconstruction to form a complete furniture outline feature and style recognition mapping network. This network comprehensively captures the geometric features of furniture from different angles, effectively identifies occluded areas, and accurately completes the data, significantly improving the completeness and accuracy of furniture recognition. By constructing a geometrically parametric reconstruction model that integrates the advantages of material texture mapping and lighting adaptation mechanisms, it accurately reproduces the three-dimensional form and visual texture of furniture, providing a realistic virtual replacement effect and effectively avoiding visual inconsistencies and fusion distortion problems caused by traditional AR rendering methods. By establishing a style coordination evaluation index system and combining it with spatial conflict analysis for multi-dimensional evaluation, it objectively assesses the harmony of furniture combinations, addressing complex design needs in different interior environments and improving the applicability and aesthetics of virtual furniture replacement. Interactive optimization technology enables real-time scene adjustment and user feedback response. When anomalies occur in the fusion effect, it can promptly optimize and provide precise parameter correction suggestions, transforming furniture display from a traditional static preset to a dynamic interactive experience. This significantly improves user satisfaction and enhances the immersion and practicality of AR applications. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the AI-based AR-based scene-based furniture recognition and dynamic removal system of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 like Figure 1 As shown, the AI-based AR-based scene-based furniture recognition and dynamic removal system provided by this invention includes: The furniture recognition module is used to acquire AR image sequence data of hotel scenes, furniture style sample library data, and lighting environment parameter data; it performs multi-view image registration and furniture entity segmentation based on the acquired AR image data sequence to obtain furniture candidate region data; it performs occlusion completion analysis on the furniture candidate region data and constructs a multi-dimensional feature vector of furniture by combining it with the lighting environment parameter data; it calculates style similarity based on the multi-dimensional feature vector of furniture and furniture style sample library data to obtain the furniture style recognition result. The furniture removal module is used for spatial depth reconstruction and existing furniture layout identification according to pre-acquired home environment AR scanning data, to obtain a home furniture distribution map; spatial occupancy conflict analysis and style coordination evaluation are respectively performed based on the home furniture distribution map and furniture style identification results, to obtain a style conflict degree index, and a dynamic removal strategy is generated based on the style conflict degree index; The furniture fusion module performs virtual removal processing on the home furniture distribution map based on the dynamic removal strategy, to obtain empty space region data; and performs geometric morphological parameterization reconstruction, to obtain furniture three-dimensional model data; material texture mapping and target position light characteristic analysis are respectively performed based on the furniture three-dimensional model data; corresponding texture-equipped furniture models and environment light parameters are obtained; color migration processing is performed on the texture-equipped furniture models based on the environment light parameters, and the texture-equipped furniture models are placed into the empty space region data, to obtain fusion scene rendering results; The fusion optimization module performs reality consistency detection based on the fusion scene rendering results, and performs visual abnormal point identification based on the reality consistency detection, to obtain abnormal region marking data; interactive parameter adjustment is performed based on the abnormal region marking data, and rendering parameter optimization is performed on the light-adapted furniture models according to the adjustment results, to obtain optimized scene data.

[0019] In this embodiment, the process of obtaining the furniture style identification result includes: A plurality of angle image sequences of the hotel scene are collected by the AR device to obtain a set of original image frames; the set of original image frames is time-stamped and calibrated to obtain a synchronized image sequence; Specifically, the user performs a surround shooting around the target furniture by holding an AR device (such as a smart terminal equipped with a depth camera or professional AR glasses). During the shooting process, the AR device continuously collects RGB images and depth images at a rate of 15 to 30 frames per second, and records the six-degree-of-freedom pose data of the device corresponding to each frame of image, including three-dimensional space position coordinates and three-axis rotation angles, to obtain an element image frame set, which covers the front, back, left, right and top view angles of the furniture.

[0020] When the set of original image frames is time-stamped and calibrated, the timestamp of each image is first extracted, and abnormal jump points in the timestamp sequence are detected; for the case that the timestamp is not continuous due to delay or frame loss, the timestamp is corrected by using a linear interpolation method; at the same time, the RGB images and the depth images are frame-aligned to obtain a spatio-temporally synchronized image sequence.

[0021] The device pose data of each image frame in the synchronized image sequence is obtained; and camera motion trajectory estimation is performed based on the device pose data, to obtain camera trajectory parameters; feature point matching is performed on the synchronized image sequence based on the camera trajectory parameters, to obtain a set of matched feature points; Specifically, the pose estimation result provided by a visual-inertial odometry (VIO) terminal built in the AR device is used preferentially, including a plurality of discrete pose points; and the discrete pose points are fitted into a continuous spatial curve to obtain camera motion trajectory parameters, including total length of the trajectory, motion speed variation curve and view coverage range and the like information; a scale invariant feature transform (SIFT) algorithm is used to extract image keys and perform feature point matching; for adjacent frame images, a fast nearest neighbor search algorithm is used for feature point matching, and a ratio test is used in the matching process to filter out fuzzy matching and retain feature point pairs with a matching confidence higher than 0.8; for non-adjacent frame images, especially for image pairs with a large view difference, a bag-of-words model is used to accelerate the matching process. Through these matching strategies, a matching feature point set covering the entire image sequence is obtained; wherein the bag-of-words model is an existing model, and the present application does not elaborate on the process.

[0022] Triangulation calculation is performed on the matching feature point set to obtain a sparse spatial point cloud; dense reconstruction is performed on the sparse spatial point cloud to obtain scene spatial point cloud data; Specifically, the three-dimensional spatial coordinates of the feature points are calculated through the triangulation principle by using the known camera pose and the pixel coordinates of the feature points in different images; the direct linear transformation (DLT) method is used for triangulation, and for the feature points observed in multiple views, the least squares method is used to optimize the estimation of their three-dimensional coordinates to improve the reconstruction accuracy; through triangulation, the two-dimensional image features are converted into three-dimensional space points to obtain a sparse spatial point cloud, and when dense reconstruction is performed on the sparse spatial point cloud, the depth of each pixel in the image is estimated based on the known camera pose; the plane scanning strategy is used for depth estimation, a plurality of candidate depth planes are uniformly sampled in the depth range, and the luminosity consistency of each pixel under different depth assumptions with adjacent views is calculated; the normalized cross correlation (NCC) is used to measure the luminosity consistency, and the depth with the highest consistency is selected as the depth estimation value of the pixel; by performing depth estimation on all pixels and converting them into three-dimensional coordinates, dense scene spatial point cloud data is obtained.

[0023] Plane extraction analysis is performed on the scene spatial point cloud data to obtain scene structure plane data; spatial semantic segmentation is performed according to the scene structure plane data to obtain object region labels, including ground, wall and object region labels; Specifically, three points in the point cloud are randomly sampled to construct a candidate plane, and the number of points supporting the plane is calculated. After multiple iterations, the plane with the most supporting points is selected as the extraction result to obtain the scene structure plane data. The main extracted planes include the ground, wall, ceiling, and plane surface of large furniture. In the plane extraction parameter setting, a distance threshold is usually set, representing the maximum allowed distance of points to the plane. Then, the semantic inference is performed using the geometric properties and spatial position relationship of the plane. For example, the ground plane is usually the largest horizontal plane and is located at the bottom of the scene; the wall is a vertical plane and is orthogonal to the ground; and the object region is the part of the point cloud that does not belong to the ground and wall. Each point in the point cloud is assigned a semantic label to obtain the segmentation result of the ground, wall, and object region.

[0024] The object region label is subjected to connectivity clustering analysis to obtain an independent object cluster set. The independent object cluster set is subjected to geometric shape regularity determination to obtain furniture candidate region data. Specifically, adjacent object points with a spatial distance less than a set spatial distance threshold are grouped into the same cluster, and the clustering is expanded by region growing. The clustering process can separate independent object instances to obtain an independent object cluster set, with each cluster corresponding to a candidate object in the scene. Then, the geometric features of each object cluster are calculated, including the main direction axis, length-width-height ratio, volume, surface flatness, etc. By setting a geometric regularity threshold, object clusters that meet the furniture characteristics are selected to obtain furniture candidate region data.

[0025] The furniture candidate region data is subjected to occlusion edge detection to obtain an occlusion boundary contour. Based on the occlusion boundary contour, complementary information is extracted from multiple view images to obtain occlusion region completion data. Based on the occlusion region completion data, edge smoothing connection is performed to obtain complete furniture contour data. Specifically, joint analysis of normal vector gradient and point density gradient is used to identify the location of occlusion. Occlusion edges appear as incomplete contour lines with obvious breaks or missing parts. By edge tracking, the occlusion boundary contour is extracted along the point cloud boundary, and the region range that needs to be completed is marked. The visibility of the occlusion region is analyzed from different angles. Since multiple-angle surround shooting is used, the occluded region in one view may be visible in other views. By traversing the search through different angles, the best complementary view that can observe the occlusion region is found. From the point cloud data of these complementary views, the three-dimensional points corresponding to the occlusion region are extracted to obtain the occlusion region completion point cloud. The completion point cloud is transformed to the original view coordinate system and filled into the occlusion region. The boundary of the completed region is smoothed, and through surface fitting and normal vector smoothing, complete furniture contour data with good geometric continuity is obtained.

[0026] Obtaining the light environment parameter data at the image acquisition time; performing light normalization processing on the furniture candidate region data to obtain a light invariance feature map; and extracting geometric shape features, texture distribution features and color statistical features from the light invariance feature map to obtain a furniture multi-dimensional feature vector; Specifically, the light environment parameters are extracted from the original frame image sequence, including the main light source direction, the light intensity, the color temperature and the ambient light distribution, etc. The main light source direction is inferred by analyzing the light and shade distribution and the shadow direction on the surface of the furniture, and the light source position is estimated from multi-view images by using photometric stereo vision method. The light intensity is calculated by analyzing the average brightness and exposure parameters of the image. The color temperature is estimated by analyzing the color bias (warm or cold) of the image, which is usually represented by the color temperature value (unit: Kelvin K). The ambient light distribution is modeled by a spherical harmonic function to capture the contribution of ambient light from different directions.

[0027] Further, the Retinex theory is used to decompose the image into a reflection component and a light component, and the influence of the light component is estimated and removed by using a multi-scale Retinex algorithm to obtain a light invariance feature map. Geometric shape features (including the length-width-height ratio, symmetry, edge straightness, curvature distribution, etc. of the furniture), texture distribution features (including the contrast, correlation, energy and entropy, etc. of the texture statistics) and color statistical features (including the color histogram, dominant color, color richness, etc.) are extracted by using multi-level features. The multi-dimensional features are organized into a feature vector form to obtain a furniture multi-dimensional feature vector.

[0028] The furniture style sample library data is clustered by style labels to obtain a style category prototype feature library. The furniture multi-dimensional feature vector is measured by Mahalanobis distance with the preset style category prototype feature library to obtain a style similarity score. The style category to which the furniture belongs is determined according to the style similarity score to obtain a furniture style recognition result.

[0029] Specifically, the style category prototype feature library contains thousands to tens of thousands of pre-labeled furniture samples, each of which is labeled with a style category (such as modern minimalist, Nordic style, Chinese classical, European luxury, etc.) and a corresponding feature vector. For each style category, the average value of the features of all samples in the category is calculated to obtain the prototype feature vector of the style. The prototype feature vectors of all style categories constitute the style category prototype feature library, which serves as a reference benchmark for style recognition. The Mahalanobis distance between the furniture to be identified and all style categories is calculated, and the Mahalanobis distance is converted into a style similarity score. According to the style similarity score, the style category with the highest score is selected as the recognition result to obtain the furniture style recognition result.

[0030] In this embodiment, the occlusion region completion data includes: Gradient direction consistency analysis is performed on the furniture candidate region data to obtain edge continuity breaking points; a shielding occurrence position is determined according to the edge continuity breaking points to obtain shielding region positioning data; Specifically, first, for each point on the boundary, the normal vector of the point is estimated by analyzing the distribution of the local neighborhood points, and the included angle between the normal vectors of adjacent points on the boundary is calculated; when the included angle between the normal vectors of adjacent points is less than a set angle threshold, it is considered that the edge continuity is good; when the included angle suddenly increases and is not less than the angle threshold, it is marked as an edge continuity breaking point; at the same time, the point cloud density distribution is analyzed, and the local point density of the boundary point is calculated to identify the positions of sharp changes in density, which together with the normal vector breaking points constitute the edge continuity breaking point set; The identified breaking points are subjected to clustering analysis, and the breaking points with close spatial distances are classified into the same group to obtain a plurality of groups of breaking point clusters, and each group corresponds to an independent shielding region; for each breaking point cluster, the spatial distribution characteristics and normal vector variation mode are analyzed, and noise and pseudo-breaking points are filtered out based on the same to confirm the real shielding occurrence position; for each confirmed shielding occurrence position, the spatial coordinate range, boundary shape and shielding depth estimation value are extracted to obtain the shielding region positioning data.

[0031] The boundary contour tracking is performed on the shielding region positioning data to obtain a shielding boundary contour; and the multi-view image is searched according to the shielding boundary contour to obtain a candidate complete view set. Specifically, the contour tracking is performed along the point cloud boundary from the starting breaking point of the shielding region; the boundary point sequence obtained by the tracking constitutes the shielding boundary contour, and the contour is stored in the form of an ordered point list, each point containing three-dimensional coordinate and normal vector information; the contour tracking adopts an edge walking method, that is, the next boundary point is searched in the neighborhood of the current boundary point, and the point with the closest distance and continuous normal vector change is selected as the next tracking point; and for a closed shielding boundary, the tracking process is terminated when returning to the starting point; for an open boundary, the tracking is terminated when reaching the boundary endpoint or encountering another shielding region; Project the three-dimensional coordinates of the occlusion boundary contour onto the image plane of each candidate view; and for each view, check whether the projected position of the occlusion region in the view image falls within the image range. If the projected position exceeds the image boundary, it indicates that the view cannot observe the occlusion region, and the view is excluded. If the projection is within the image, further check the visibility of the occlusion region. The visibility check is performed by calculating the expected depth value of the sampling points in the occlusion region under the view, and comparing it with the depth value of the corresponding pixels in the actual depth map. If the expected depth is not less than the actual depth, it indicates that the view is visible. If the expected depth is greater than the actual depth, it indicates that the region is still occluded. By traversing all views and performing visibility checks, a set of candidate completion views is obtained.

[0032] From the set of candidate completion views, the view with the highest occlusion region visibility is selected to obtain the optimal completion view. The point cloud data corresponding to the region in the optimal completion view is extracted to obtain the occlusion region completion point cloud. The occlusion region completion point cloud is mapped back to the original view coordinate system to obtain the occlusion region completion data.

[0033] Specifically, when selecting the view with the highest occlusion region visibility from the set of candidate completion views, the observation quality score of each candidate view on the occlusion region is calculated. The observation quality score considers multiple factors, including the visible area ratio, the view angle, the image resolution, and the lighting quality. The brightness distribution of the occlusion region in the view image is analyzed. When the lighting is uniform, there is no overexposure or underexposure, the score is high. When the lighting is uneven or there is strong shadow, the score is low. The four factors are normalized and then weighted to calculate the total score. The score of each view in the set of candidate completion views is obtained, and the view with the highest score is selected as the optimal completion view.

[0034] The projection position of the occlusion region in the optimal completion view image is located. Through back projection, the three-dimensional point cloud of the occlusion region under the optimal completion view is obtained. The extracted point cloud is checked for quality, and abnormal depth values (such as zero or infinity), excessive noise, or isolated outliers are filtered out. After filtering, the occlusion region completion point cloud is obtained.

[0035] The occlusion region completion point cloud is mapped to the same coordinate system as the original furniture point cloud. After the mapping is completed, the completion point cloud is fused with the original point cloud, and the missing three-dimensional points in the occlusion region are filled to obtain complete and continuous occlusion region completion data.

[0036] In this embodiment, the acquisition process of the dynamic removal strategy includes: The home environment is scanned and collected by an AR device to obtain home environment image data. The home environment image data is subjected to binocular stereo matching to obtain depth image data. Specifically, the user scans the home room in all directions through the AR device equipped with a structured light depth sensor, and obtains depth image data corresponding to each frame of RGB image.

[0037] According to the depth image data, voxelization space modeling is performed to obtain a home space voxel model; and grid optimization is performed on the home space voxel model to obtain a home space three-dimensional model. Specifically, for each depth image, the depth value of a pixel point is converted into a three-dimensional coordinate according to the camera pose, and the coordinate is mapped to a corresponding voxel position; a truncated signed distance function (TSDF) is used for voxel fusion, and the TSDF value represents the signed distance from the voxel center to the nearest surface; and by fusing the TSDF values of multiple frames of depth images, a stable space representation is obtained; after fusion, a voxel model of the home space is formed; then, each voxel in the voxel model is traversed, and according to the change of the TSDF value of the voxel vertex, a triangular facet passing through the voxel is generated; and the number of extracted triangular facets is reduced through an iterative edge collapse operation, which greatly reduces the model complexity while ensuring the visual quality, and a lightweight home space three-dimensional model is obtained.

[0038] The home space three-dimensional model is subjected to object instance segmentation to obtain an existing furniture instance set; and the spatial position of the existing furniture instance set is extracted to obtain a home furniture distribution map. Specifically, first, the three-dimensional model is subjected to plane extraction and surface normal vector analysis to identify the main structure planes (floor, wall, ceiling), then the regions other than the structure planes are regarded as object candidates, and connectivity analysis and geometric clustering are performed on the object candidates, and the facets that are continuous in space and similar in geometric features are classified into the same object instance; geometric features such as size, shape, position, etc. are extracted for each object instance, and object category inference is performed in combination with semantic rules (such as a cuboid with a height suitable for being located on the ground may be a table or a cabinet) to obtain an existing furniture instance set, which contains information such as three-dimensional grid, spatial position, and category label of the furniture; and the spatial position parameters of each furniture instance are synchronously obtained, including the center coordinates of the furniture (obtained by calculating the geometric center of the instance bounding box), the floor area (the polygon area projected onto the ground), the height range, the orientation angle, etc.; these spatial position parameters are organized in the form of a two-dimensional plan view to form a home furniture distribution map.

[0039] The three-dimensional bounding box size of the target furniture is calculated according to the complete furniture contour data to obtain the spatial occupation parameters of the target furniture. Specifically, the principal component analysis (PCA) method is used to determine the main direction axis of the furniture, including three orthogonal main directions corresponding to the length, width and height directions of the furniture; the maximum extension range of the point cloud is calculated along the three main directions to obtain an aligned three-dimensional bounding box, which is defined by a center position, three axial dimensions (length L, width W, height H) and rotation angles. These parameters constitute the target furniture space occupation parameters.

[0040] The target furniture space occupation parameters are virtually placed and traversed in the home furniture distribution map to obtain a set of placement candidate positions; and collision detection calculation is performed on each position in the set of placement candidate positions to obtain a space conflict matrix. Specifically, a placement candidate position grid is set in the home space, and the grid interval is usually 20 to 50 centimeters; for each grid position, the bounding box of the target furniture is virtually placed at the position, and for each placement posture, it is checked whether the basic constraint conditions are met, such as the furniture bottom must be located on the ground, cannot be embedded in the wall, and necessary passage space must be reserved; the positions and postures that meet the constraint conditions are added to the candidate set to form a set of placement candidate positions, which contains all feasible furniture placement schemes; For a candidate position, intersection detection is performed between the bounding box of the target furniture and the bounding boxes of all existing furniture in the home environment, and the separation axis theorem (SAT) is used for fast collision judgment. If the projections of the two bounding boxes on any separation axis do not overlap, it is determined that there is no collision; otherwise, it is determined that a collision occurs; the collision relationship is recorded and organized in the form of a matrix to obtain a space conflict matrix. The matrix elements represent whether the candidate position collides with the existing furniture. (1 represents conflict, and 0 represents no conflict).

[0041] According to the furniture style recognition result, the style attribute vector of the target furniture is extracted; the style attribute vectors of the existing furniture instance set are extracted; the cosine similarity of the target furniture style attribute vector and the existing furniture style attribute vector is calculated to obtain a style coordination degree value; Specifically, the identified furniture style recognition result is converted into a numerical style attribute representation; the style attribute vector contains multiple dimensions, such as design era, style genre, color keynote and material type; each dimension is represented by one-hot encoding or continuous numerical value, which is combined into a high-dimensional vector; for each piece of furniture in the existing furniture instance set, its style attribute vector is also extracted, and the same feature dimensions and encoding method as the target furniture are used; the style similarity of the target furniture and all existing furniture is calculated to obtain a set of style coordination degree values.

[0042] Based on the style coordination degree value, a reverse score is obtained to obtain a style conflict degree index; according to the space conflict matrix and the style conflict degree index, a comprehensive ranking is obtained to obtain a furniture removal priority sequence; and a dynamic removal strategy is generated based on the furniture removal priority sequence; Specifically, the style coordination degree value is converted into a style conflict degree, which is weighted and summed with the space conflict degree to calculate the priority scores of all existing furniture, and the furniture removal priority sequence is obtained by sorting the scores from high to low. When generating a dynamic removal strategy based on the sequence, the dynamic removal strategy includes removal order, number of furniture removed each time, space state update rule after removal, etc.

[0043] In this embodiment, the acquisition process of the fusion scene rendering result includes: According to the dynamic removal strategy, the furniture object with the highest priority is selected from the home furniture distribution map to obtain a furniture identifier to be removed; the space region corresponding to the furniture identifier to be removed is virtually emptied to obtain empty space region data; Specifically, the furniture removal priority sequence is queried, the first ranked furniture instance is extracted as the primary removal object, and the instance identifier, spatial position and occupied range of the furniture are recorded to obtain the furniture identifier to be removed; further, the three-dimensional region occupied by the furniture is located in the home space three-dimensional model, and the furniture grid in the region is deleted from the scene while retaining the background structure such as the ground and the wall; the emptied region forms an available empty space, the home furniture distribution map is updated, and the region is marked as a placeable state to obtain empty space region data.

[0044] The complete furniture contour data is skeletonized to obtain a furniture skeleton structure diagram; and a parameterized shape fitting is performed based on the furniture skeleton structure diagram to obtain a furniture geometric parameter set; Specifically, first, the internal distance field of the furniture point cloud is calculated, which represents the shortest distance from each point to the furniture surface. The local maximum point of the distance field corresponds to the skeleton point, i.e., the internal point with the farthest distance to the surface. By extracting the ridge line of the distance field, the central axis skeleton of the furniture is obtained, and the skeleton is simplified and smoothed to remove noise branches to obtain the furniture skeleton structure diagram. According to the topological type (such as tree, ring, and net) and the number of branches of the furniture skeleton structure diagram, the category of the furniture (such as chair, table, and cabinet) is determined, and a parameterized shape template corresponding to the category is selected. The template is composed of several basic geometric bodies (such as cuboid, cylinder, and sphere), and each geometric body is controlled by parameters (such as center position, size, and rotation angle). By adjusting the template parameters through an optimization algorithm, the template shape is best matched with the furniture point cloud to obtain the furniture geometric parameter set, which includes the type, size, position, rotation, and other parameters of each geometric body.

[0045] A furniture three-dimensional grid model is constructed according to the furniture geometric parameter set to obtain furniture three-dimensional model data. Specifically, a standard mesh (such as a hexahedral mesh of a cuboid or a cylindrical mesh of a cylinder) is generated for each basic geometric body according to the furniture geometric parameter set, and then the mesh is scaled, rotated and translated according to the parameters and placed at a specified position; the meshes of all geometric bodies are combined by Boolean operation to form a unified three-dimensional mesh model, and subdivision and smoothing processing is performed to increase the patch density and improve the surface quality, thereby obtaining the furniture three-dimensional model data.

[0046] A furniture surface texture image is extracted from the AR image sequence data to obtain original texture data; and the original texture data is corrected for perspective distortion to obtain a corrected texture image. Specifically, an original image frame with the best furniture visibility, uniform illumination and a directly opposite view angle is selected as a texture source; a region corresponding to a surface is framed in the image, and pixel data is extracted to obtain original texture data; then, a homography matrix of the texture region is calculated using three-dimensional geometric information of the furniture surface and camera poses, and the homography transformation can map the perspective projection texture back to the orthographic projection to eliminate the distortion caused by the view angle, thereby obtaining the corrected texture image.

[0047] The corrected texture image is mapped to the UV coordinate space of the furniture three-dimensional model data to obtain a textured furniture model. Specifically, first, two-dimensional UV coordinates are assigned to each vertex of the three-dimensional model, and the UV coordinates define the corresponding position of the vertex on the texture image; UV unfolding is used to parameterize and map the three-dimensional surface to a two-dimensional plane, and then the corrected texture image is used as a texture map and attached to the model surface according to the UV coordinate relationship, thereby obtaining the textured furniture model.

[0048] Local light source position estimation is performed on the empty space region data in the home space three-dimensional model to obtain a main light source direction vector; and the ambient light intensity distribution is calculated according to the main light source direction vector to obtain ambient light parameters. Specifically, by detecting highlight points and shadow boundaries in the scene, a light direction voting algorithm is used to infer the source direction of the main light source to obtain a main light source direction vector, which represents a unit direction vector from the light source to the scene; when calculating the ambient light intensity distribution according to the main light source direction vector, the brightness information of the known surfaces in the scene is combined, an inverse light estimation method is used to solve the intensity parameters and color parameters of the ambient light, and the ambient light parameters, including the light intensity value, the light color and the ambient light contribution ratio, are obtained.

[0049] The material reflection properties of the textured furniture model are extracted to obtain a material reflection coefficient; and light transmission calculation is performed based on the ambient light parameters and the material reflection coefficient to obtain target light rendering parameters. Specifically, according to the material type of the furniture (such as wood, metal, cloth, etc.), the corresponding reflection characteristic parameters are queried from the preset material attribute database, including the diffuse reflection coefficient, the specular reflection coefficient, the roughness, and the metal degree, etc. These parameters describe the response characteristics of the material to light, and the material reflection coefficient is obtained. Then, by using the physically-based rendering (PBR) model, the reflection and scattering process of light on the material surface is simulated, and the color and brightness that each point on the furniture surface should present under the ambient light condition is calculated, and the target light rendering parameter is obtained.

[0050] According to the target light rendering parameter, color space conversion is performed on the textured furniture model to obtain a color adjustment matrix. Based on the color adjustment matrix, color migration is performed on the texture of the furniture model to obtain a light-adapted furniture model. Specifically, the RGB color of the texture is converted to the HSV (hue, saturation, value) color space. In the HSV space, according to the target light rendering parameter, the value of the brightness of each pixel is calculated and compared with the original brightness value to obtain a brightness adjustment factor, and a color adjustment matrix is constructed, which contains adjustment coefficients for the H, S, and V channels. Then, the color adjustment matrix is applied to the textured furniture model for color migration. By converting the texture image to the HSV space and applying the adjustment coefficients to modify the hue, saturation, and brightness values, and then converting back to the RGB space, the light-adapted furniture model is obtained.

[0051] When the light-adapted furniture model is placed in the empty space region data, according to the user-specified position or the system-recommended optimal position, the three-dimensional coordinates and rotation angle of the furniture in the home space are determined, and the scene data is updated.

[0052] When calculating the shadow projection area according to the main light source direction vector and the furniture geometry parameter set, a ray casting algorithm is used to emit a ray from the geometric boundary of the furniture in the opposite direction of the main light source direction vector. The intersection points of the ray and the ground or wall form the projection area of the shadow, and the shadow mask data is obtained.

[0053] When performing soft shadow blur processing on the shadow mask data, a Gaussian blur filter is applied, and the blur radius is adjusted according to the size and distance parameters of the light source to simulate the soft transition effect of the shadow in the real world, and the soft shadow image is obtained.

[0054] When performing Alpha blending on the soft shadow image and the home space three-dimensional model, the transparency parameter of the shadow is set, usually between 0.3 and 0.6. The shadow image is superimposed on the corresponding position of the scene, and the pixel color after blending is calculated using the Alpha blending formula, and the shadow scene data is obtained.

[0055] When the light-adapted furniture model is rendered in depth sorting with shadowed scene data, each geometric object is sorted according to the distance from the camera, and objects far away are rendered first, and objects close are rendered later, the occlusion relationship is correctly handled, and finally a complete scene image containing furniture and shadows is generated, and a fused scene rendering result is obtained.

[0056] It needs to be further explained that, in the specific implementation process, the acquisition process of the furniture geometric parameter set includes: The complete furniture contour data is subjected to a morphological thinning operation to obtain a single-pixel width skeleton line; the single-pixel width skeleton line is subjected to branch point and end point detection to obtain a skeleton key node set; and a skeleton topological connection relationship is constructed according to the skeleton key node set to obtain a furniture skeleton structure diagram; Specifically, first, the three-dimensional furniture point cloud data is projected onto a main plane to obtain a two-dimensional contour image; after the point cloud is projected onto the main plane, rasterization processing is performed to discretize the continuous spatial coordinates into pixel coordinates to obtain a binary image, wherein the grid resolution is usually set to correspond to 2 to 5 pixels per centimeter, and the furniture contour area is marked as foreground (pixel value is 1), and the background area is marked as background (pixel value is 0).

[0057] Morphological thinning is performed on the binary image to peel off the contour boundary pixels layer by layer while maintaining the connectivity and topological structure of the contour unchanged; and the thinning process follows a specific deletion rule, and only boundary points that meet the following conditions are deleted each time: the deletion of the point will not cause the contour to break, and will not change the number of connected components of the contour; the iteration continues until no pixel can be deleted, and a single-pixel width skeleton line is obtained. For each pixel point on the single-pixel width skeleton line, the number of skeleton pixels in its eight-neighborhood is counted, when there is only one skeleton pixel in the neighborhood, the pixel point is marked as an end point, indicating the end of the skeleton; when there are three or more skeleton pixels in the neighborhood, the pixel point is marked as a branch point, indicating the intersection of the skeleton; when there are two skeleton pixels in the neighborhood, the point is a normal connection point; by traversing, the coordinates of all end points and branch points are extracted to form a skeleton key node set.

[0058] Further, based on the skeleton key node set, a plurality of branches are constructed, and the starting node, the ending node, the branch length and the direction angle are recorded; all branches and their connection relationships are organized into a graph structure, the nodes represent the key points, the edges represent the skeleton branches, and the attributes of the edges include the length and direction information, and a furniture skeleton structure diagram is obtained.

[0059] The furniture skeleton structure diagram is subjected to structure type matching to obtain a furniture category label; and a corresponding parameterized shape template is selected according to the furniture category label to obtain a basic shape template; Specifically, first, a furniture category template library is established, containing standard skeleton topological structures of common furniture types, such as a chair usually having four leg branches connected to a seat surface node, a table having four or more support legs connected to a table top center, and a sofa having armrest and backrest feature branch structures; The topological features of the furniture skeleton structure graph to be recognized are extracted, including the number of nodes, the number of branches, the branch length ratio, the connection mode, and the like, similarity calculation is performed on the features of each category in the furniture category template library, and the template category with the highest similarity is selected as the matching result to obtain a furniture category label; a corresponding standard geometric template is selected from a preset parameterized template library according to the furniture category label; wherein the parameterized shape template defines the basic geometric structure and adjustable control parameters of the furniture; taking a chair as an example, the basic shape template includes a seat surface component, a backrest component, and a leg component, and the control parameters include seat surface length, seat surface width, seat surface height, backrest height, backrest inclination angle, leg diameter, and the like. Taking a table as an example, the template includes a table top and a table leg component, and the control parameters include table top length, table top width, table top thickness, table top height, table leg number, table leg position distribution, and the like. Each parameter has a value range constraint to ensure that the generated model conforms to the physical characteristics of the actual furniture, and a basic shape template is obtained.

[0060] Specifically, when performing least squares fitting on the control parameters of the basic shape template and the complete furniture contour data, an optimization objective function is established to minimize the error between the shape generated by the parameterized template and the actual furniture contour; the optimization objective function is optimized and solved, the control parameter values are iteratively adjusted, the objective function value is gradually reduced, and when the objective function converges or reaches the maximum number of iterations, the optimization is stopped, and the optimal shape parameter value is obtained. Based on the optimal shape parameter value combination, a furniture geometric parameter set is generated, all control parameters and their optimized values are organized into a structured data set; the furniture geometric parameter set is stored in the form of key-value pairs, with the key being the parameter name and the value being the parameter value and unit.

[0061] In this embodiment, the process of obtaining the target light rendering parameter includes: The surface material of the textured furniture model is identified to obtain a material type label. Specifically, first, the texture features and geometric features of each component of the furniture model are extracted, and the extracted texture features and geometric features are input into a pre-trained material classification model, which adopts a convolutional neural network architecture and is trained on a large-scale material data set; the material classification model outputs the probability distribution of each material category, and the category with the highest probability is selected as the recognition result to obtain the material type label, including common labels such as wood, plywood, metal, leather, cloth, glass, and plastic.

[0062] According to the material type label, a corresponding bidirectional reflectance distribution function (BRDF) model is queried from a material attribute database to obtain the material BRDF model; Specifically, first, an existing material attribute database is accessed, which stores material entries of hundreds of common materials, including material name, BRDF model type, model parameter, reflectivity spectrum, roughness range, and other detailed information. According to the material type label, the corresponding material entry is located, the BRDF model description of the material is read, and the material BRDF model is obtained. The BRDF model defines the energy distribution relationship between the incident light and the reflected light.

[0063] Based on the material BRDF model, diffuse reflection coefficients and specular reflection coefficients are calculated to obtain material reflection coefficients; Specifically, according to the mathematical expression of the BRDF model, reflection parameters including diffuse reflection coefficients and specular reflection coefficients are extracted. The reflection parameters are converted into vector form to obtain material reflection coefficients, including diffuse reflection coefficient vectors and specular reflection coefficient vectors. The diffuse reflection coefficient represents the diffuse scattering ability of the material surface to incident light, and its value depends on the inherent color and surface roughness of the material. The specular reflection coefficient represents the highlight reflection intensity of the material surface, and is related to the smoothness and refractive index of the material.

[0064] According to the main light source direction vector and intensity value in the environmental light parameter, and in combination with the material reflection coefficient, the illumination intensity is calculated to obtain the illumination intensity value of each vertex. Specifically, according to the main light source direction vector and intensity value in the ambient light parameter, and in combination with the material reflection coefficient, a physical-based light calculation model is used for light intensity calculation. For each vertex of the furniture model surface, first, the three-dimensional position coordinates and normal vector of the vertex are obtained. The main light source direction vector represents a unit vector from the light source to the scene, and the ambient light intensity value represents the radiation intensity of the light source, usually in lux or lumens. When calculating the diffuse reflection light contribution at the vertex, the Lambert cosine law is used, and the light intensity is proportional to the cosine of the angle between the light source direction and the surface normal vector. The specific calculation formula is: the diffuse reflection light intensity is equal to the ambient light intensity multiplied by the diffuse reflection coefficient and then multiplied by the maximum value of the dot product of the normal vector and the light source direction vector (compared with zero). This ensures that the surface facing away from the light source will not receive a negative light intensity. When calculating the specular reflection light contribution, the viewing direction, i.e., the vector from the vertex to the virtual camera, needs to be considered. The half-angle vector method is used to calculate the half-angle vector of the light source direction and the viewing direction, and then the angle between the half-angle vector and the surface normal vector is calculated. The specular reflection light intensity is equal to the ambient light intensity multiplied by the specular reflection coefficient and then multiplied by the power of the dot product of the half-angle vector and the normal vector, and the power index represents the glossiness of the material, and the larger the value, the more concentrated the highlight. The diffuse reflection light intensity and the specular reflection light intensity are added together, and the base contribution of the ambient light is added to obtain the light intensity value of each vertex. For a furniture model containing thousands to tens of thousands of vertices, the above calculation needs to be performed for each vertex respectively, and finally the vertex light intensity numerical value is obtained.

[0065] The light intensity value of each vertex is normalized to obtain a normalized light intensity distribution; and target light rendering parameters are generated according to the normalized light intensity distribution; Specifically, when normalizing the light intensity value of each vertex, first, the maximum and minimum values of the light intensity of all vertices are counted to determine the dynamic range of the light intensity. The maximum and minimum normalization method is used to map the light intensity value to the standard interval of zero to one. The normalization formula is: the normalized light intensity is equal to the current vertex light intensity minus the minimum light intensity, and then divided by the difference between the maximum light intensity and the minimum light intensity. The normalization process eliminates the dimensional influence of the absolute light intensity, so that the light distribution can be compared and migrated between different environments and scenes. For the case where there are extreme bright spots or dark spots, the quantile normalization method can be used, using the 500th percentile and the 95th percentile as the upper and lower bounds of normalization to avoid the influence of abnormal values on the normalization result. The normalized light intensity retains the relative distribution characteristics of the light, and a normalized light intensity distribution is obtained, which is stored in the form of an array with vertex index as key and normalized intensity value as value.

[0066] In this embodiment, the acquisition process of the optimized scene data includes: An image quality evaluation is performed on the fusion scene rendering result to obtain an image definition index; a lighting consistency detection is performed on the fusion scene rendering result to obtain a lighting consistency score; an edge fusion degree analysis is performed on the fusion scene rendering result to obtain an edge transition smoothness; Specifically, a Laplacian operator is used to calculate a second derivative of the image to extract edges and detail information of the image; a variance of the Laplacian response value is calculated, and the greater the variance value, the richer the high-frequency details contained in the image, and the higher the definition, so as to obtain the image definition index; At the same time, the pixel brightness distribution of the furniture region and the surrounding environment region is extracted respectively, the brightness histogram of each is calculated, and the similarity of the two histograms is evaluated by using a histogram correlation measurement method such as Pearson correlation coefficient or Bhattacharyya distance, and the higher the similarity, the better the lighting consistency, so as to obtain the lighting consistency score; Further, the edge of the junction of the furniture and the environment is detected, the edge pixels are located by using edge detection, and then the color gradient change on both sides of the edge is calculated; if the color change is smooth and continuous, it indicates that the fusion is natural, and the edge transition is smooth; if there is a mutation, it indicates that the fusion is harsh; by counting the gradient continuity index of the edge region, the edge transition smoothness is obtained.

[0067] Based on the image definition index, the lighting consistency score and the edge transition smoothness, a weighted comprehensive score is obtained, and a scene reality score is obtained; An abnormality detection threshold is set according to the scene reality score; a region whose scene reality score is lower than the abnormality detection threshold is located and marked to obtain abnormal region marking data; the user is presented with the fusion scene rendering result and the abnormal region marking data; Specifically, when the region whose scene reality score is lower than the abnormality detection threshold is located and marked, the scene is divided into a plurality of sub-regions, the reality score of each sub-region is calculated, the sub-region whose score is lower than the threshold is marked, and the abnormal region marking data is obtained by using a high-light frame or a semi-transparent mask to identify on the image; and the rendering scene and the abnormal region annotation are displayed on the display screen of the AR device at the same time, prompting the user to pay attention to the part that needs to be improved.

[0068] Touch interaction operation data of the user is collected to obtain an interaction operation sequence; an operation type recognition is performed on the interaction operation sequence to obtain an adjustment operation type; an adjustment direction instruction is obtained by analyzing the user adjustment intention according to the adjustment operation type; Based on the adjustment direction instruction, the position, rotation angle and scaling ratio of the light-adapted furniture model are corrected to obtain geometric adjustment parameters; the color, brightness and saturation of the light-adapted furniture model are corrected to obtain color adjustment parameters; The geometry adjustment parameter and the color adjustment parameter are integrated to obtain user tuning parameters; the user tuning parameters are applied to regenerate the fusion scene rendering result, to obtain the optimized scene data and display the same to the user; Specifically, when collecting the touch interaction operation data of the user, the touch point coordinates of the user on the screen, the touch time, the gesture type (such as clicking, dragging, zooming, rotating, etc.) are recorded to form a time sequence of the interaction operation sequence. When performing operation type identification on the interaction operation sequence, the user operation is classified into operation types such as position adjustment, angle rotation, size scaling, brightness adjustment, color correction, etc. according to the gesture features and the operation mode, to obtain the adjustment operation type. When analyzing the user adjustment intention according to the adjustment operation type, the direction and amplitude of the operation are analyzed, for example, the drag operation corresponds to the position movement intention, the rotation gesture corresponds to the angle adjustment intention, and the double-finger scaling corresponds to the size adjustment intention, to form specific adjustment direction instructions such as moving 10 centimeters to the left and rotating 15 degrees clockwise.

[0069] When the position, rotation angle and scaling ratio of the light-adapted furniture model are parameter-corrected based on the adjustment direction instruction, the operation amount of the user is converted into the increment of the model parameter, the transformation matrix of the furniture model is updated, and the geometry adjustment parameter is obtained. When the color, brightness and saturation of the light-adapted furniture model are parameter-corrected, the color adjustment increment value is calculated according to the user's adjustment operation, the color parameter of the texture is updated, and the color adjustment parameter is obtained. When the geometry adjustment parameter and the color adjustment parameter are integrated, they are organized into a unified parameter structure containing the parameter values of all adjustment items to obtain the user tuning parameters. When the user tuning parameters are applied to regenerate the fusion scene rendering result, the updated furniture model parameters are used to re-execute the rendering process, including light calculation, shadow generation and scene synthesis, to generate a new scene image, obtain the optimized scene data, and display the same to the user in real time, forming an interactive optimization closed loop.

[0070] In this embodiment, the acquisition process of the scene reality score includes: The Laplacian gradient calculation is performed on the fusion scene rendering result to obtain an edge gradient intensity map. The Laplacian operator is a second-order differential operator that can effectively detect edges and details in an image.

[0071] Specifically, first, the fusion scene rendering result is converted into a gray image to eliminate the interference of color information on the definition evaluation; then a Laplace convolution kernel is applied to the convolution operation of the gray image, and the commonly used Laplace kernel is a 3x3 matrix, with 8 in the center element and -1 in the surrounding eight elements; the convolution operation calculates the second derivative of each pixel and its neighborhood pixels, and the absolute value of the derivative value represents the edge strength of the position; finally, the absolute value of the convolution result is taken and normalized to form an edge gradient intensity map; in the edge gradient intensity map, the high-intensity area corresponds to the edge and detail part of the image, and the low-intensity area corresponds to the flat area.

[0072] The edge gradient intensity map is subjected to statistical variance calculation to obtain an image definition index; the variance can reflect the dispersion degree of the pixel gray value in the image, and the clear image contains rich high-frequency details, so the variance of the edge gradient intensity map is large; the fuzzy image lacks detail information, so the variance of the edge gradient intensity map is small.

[0073] Specifically, first, the mean value of all pixel values of the edge gradient intensity map is calculated; then the square of the difference between each pixel value and the mean value is calculated, and the sum of all pixels is calculated; finally, the variance value is obtained by dividing the total number of pixels, which is the image definition index.

[0074] The light distribution histogram of the furniture region and the surrounding environment region in the fusion scene rendering result is extracted to obtain a set of regional light histograms. First, based on the spatial position information of the light-adapted furniture model generated in the early stage, the furniture region is segmented in the fusion scene rendering result, and a pixel-level semantic segmentation method is used to mark the pixels belonging to the furniture as foreground and the remaining pixels as background; then a certain distance (usually 10% to 20% of the width of the furniture bounding box) is extended outward from the boundary of the furniture region to demarcate the surrounding environment region, which should have direct visual contact and light interaction with the furniture; the light distribution features of the furniture region and the surrounding environment region are extracted respectively, the RGB image of each region is converted into HSV color space or Lab color space, the lightness channel (V channel of HSV or L channel of Lab) is extracted, which reflects the distribution of light intensity; the gray histogram of the lightness channel is counted, and the horizontal axis of the histogram represents the lightness level (usually divided into 256 levels), and the vertical axis represents the number of pixels corresponding to the lightness level; the furniture region histogram and the environment region histogram are organized into a set of regional light histograms. The light distribution histogram can comprehensively characterize the brightness statistical characteristics of the region and provide a quantitative basis for light consistency evaluation.

[0075] The histogram correlation of the regional light histograms is calculated to obtain a light consistency score. The histogram correlation measures the similarity of two distributions, and common methods include the Pearson correlation coefficient, Bhattacharyya distance and histogram intersection method; the embodiment preferentially adopts a weighted combination of the Pearson correlation coefficient and the Bhattacharyya distance, and the calculation formula is: light consistency score = a x Pearson correlation coefficient + (1-a) x (1-Bhattacharyya distance), wherein a is a weight coefficient, and is usually set to 0.6; the finally obtained light consistency score ranges from 0 to 1.

[0076] Specifically, the color difference of the junction edge of the furniture and the environment in the fusion scene rendering result is analyzed to obtain an edge color jump amplitude; specifically, first, the edge detection is used to locate the junction boundary of the furniture and the environment, and the Canny edge detector or the boundary extraction method based on depth information can be used; the Canny edge detector can accurately locate the edge position by calculating the image gradient and performing non-maximum suppression and double-threshold detection; based on the depth information, the depth map of the furniture three-dimensional model is used to detect the boundary through the depth mutation, and the combination of the two methods can improve the accuracy of boundary positioning; then, along the detected boundary line, the sampling bands are set on both sides of the boundary, the inner sampling band is located in the furniture area and has a width of 5 to 10 pixels, and the outer sampling band is located in the environment area and has the same width; in the RGB color space or the Lab color space, the average color values of the inner sampling band and the outer sampling band are calculated respectively; for each sampling point on the boundary, the color distance of the colors on both sides is calculated by using the Euclidean distance formula or the corresponding formula in the Lab space; the color distances of all sampling points are counted to calculate the maximum value, the mean value and the standard deviation, and these statistics constitute the edge color jump amplitude. The edge color jump amplitude reflects the harshness of the fusion boundary, and the greater the amplitude, the more unnatural the edge.

[0077] The gradient continuity of the edge transition region is calculated according to the edge color jump amplitude to obtain the edge transition smoothness. Specifically, first, a sampling profile is established in the boundary normal direction, the profile extends from the inside of the furniture to the outside of the environment, across the boundary, and the length is usually 20 to 30 pixels; the brightness value or color value of the pixels along the profile is extracted to form a one-dimensional signal sequence; the first-order difference of the sequence, that is, the difference between adjacent pixel values, is calculated, and the difference value represents the local gradient; an ideal smooth transition should present a gradual change characteristic, and the gradient value should be continuous and change slowly; if there is a gradient mutation at the boundary position, that is, the absolute value of the difference value is much larger than the surrounding value, it indicates that the transition is too harsh; the continuity of the gradient is quantified by calculating the coefficient of variation (standard deviation divided by mean) of the gradient sequence, and the smaller the coefficient of variation, the more continuous the gradient; the average of the coefficients of variation of all boundary profiles is calculated, and inverse normalization processing is performed, that is, edge transition smoothness = 1 / (1+average coefficient of variation), which ensures that the smoothness value is between 0 and 1, and the larger the value, the smoother the transition. The edge transition smoothness can sensitively capture the visual defects of the fusion boundary, and is crucial for improving the immersion of the AR scene.

[0078] The obtained image sharpness index, lighting consistency score and edge transition smoothness are weighted and summed to obtain a scene reality score.

[0079] Specifically, the obtained image sharpness index, lighting consistency score and edge transition smoothness are weighted and summed to obtain a scene reality score. The weighting formula is: scene reality score = w1 x image sharpness index + w2 x lighting consistency score + w3 x edge transition smoothness, wherein w1, w2 and w3 are weight coefficients, and w1 + w2 + w3 = 1; the setting of the weight coefficients needs to be adjusted according to the application scenario and user perception characteristics; for example, for applications that focus on visual immersion, the weight of the edge transition smoothness should be higher.

[0080] Through the description of the above detailed embodiments, the AR scene-based furniture recognition and dynamic removal system based on artificial intelligence provided by the present application can accurately perceive the three-dimensional structure of the indoor environment, intelligently recognize furniture objects and their physical characteristics, generate personalized layout schemes based on user needs, and realize seamless removal and virtual relocation of furniture in the AR environment, providing users with intuitive and realistic furniture layout preview experience, effectively improving the space planning efficiency and customer experience of the hotel environment.

[0081] The above merely describes preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, the technical solutions recorded in the foregoing embodiments can be modified or some technical features thereof can be replaced equivalently by those skilled in the art, without departing from the spirit and principle of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of the present application.

[0082] The formula in the specification is a dimensionless value, and the formula is obtained by collecting a large amount of data to simulate a recent real situation. The preset parameters and threshold values in the formula are set by those skilled in the art according to actual conditions.

[0083] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. An AI-based AR-based scene-based furniture recognition and dynamic removal system, characterized in that, include: The furniture recognition module is used to acquire AR image sequence data of hotel scenes, furniture style sample library data, and lighting environment parameter data; Based on the collected AR image data sequence, multi-view image registration and furniture entity segmentation are performed to obtain furniture candidate region data; occlusion completion analysis is performed on the furniture candidate region data, and a multi-dimensional feature vector of furniture is constructed by combining it with lighting environment parameter data; Furniture style recognition results are obtained by calculating style similarity based on furniture multidimensional feature vectors and furniture style sample database data; The furniture removal module is used to perform spatial depth reconstruction and existing furniture layout identification based on pre-acquired AR scanning data of the home environment to obtain a home furniture distribution map; based on the home furniture distribution map and furniture style identification results, it performs space occupancy conflict analysis and style coordination assessment to obtain a style conflict degree index, and generates a dynamic removal strategy based on it; The furniture integration module uses a dynamic removal strategy to virtually remove furniture from the home furniture distribution map, thus obtaining data on vacant space areas. The geometric shape is parametrically reconstructed to obtain the 3D model data of the furniture; material texture mapping and target position lighting characteristics analysis are performed based on the 3D model data of the furniture; the corresponding textured furniture model and ambient lighting parameters are obtained; the textured furniture model is color-transferred based on the ambient lighting parameters and placed into the empty space area data to obtain the fused scene rendering result; The fusion optimization module performs realism consistency detection based on the fusion scene rendering results, and identifies visual anomalies based on them to obtain anomaly region marking data. Interactive parameter adjustments are performed based on anomaly region marker data, and rendering parameters are optimized for the lighting-adapted furniture model based on the adjustment results to obtain optimized scene data.

2. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 1, characterized in that, The process of obtaining furniture style recognition results includes: A multi-angle image sequence of the hotel scene is acquired using AR devices to obtain an original image frame set; the original image frame set is then time-stamped and synchronized to obtain a synchronized image sequence. The device pose data of each image frame in the synchronized image sequence is acquired; the camera motion trajectory is estimated based on the pose data to obtain the camera trajectory parameters; and the feature points of the synchronized image sequence are matched based on the camera trajectory parameters to obtain the matching feature point set. Triangulation calculations are performed on the matching feature point set to obtain a sparse spatial point cloud; the sparse spatial point cloud is then densified and reconstructed to obtain scene spatial point cloud data. Planar extraction and analysis are performed on the scene spatial point cloud data to obtain scene structure plane data; spatial semantic segmentation is performed based on the scene structure plane data to obtain object region labels, including ground, wall and object region labels; Connectivity clustering analysis is performed on the object region labels to obtain a set of independent object clusters; geometric regularity is determined on the set of independent object clusters to obtain furniture candidate region data; Occlusion edge detection is performed on the furniture candidate region data to obtain the occlusion boundary contour; complementary information is extracted from multi-view images based on the occlusion boundary contour to obtain occlusion region completion data; edge smoothing connection is performed based on the occlusion region completion data to obtain complete furniture contour data; Acquire the illumination environment parameter data at the time of image acquisition; perform illumination normalization processing on the furniture candidate region data to obtain the illumination invariant feature map; and extract geometric morphology features, texture distribution features and color statistical features from the illumination invariant feature map to obtain the furniture multidimensional feature vector; Style tag clustering is performed on the furniture style sample database to obtain a style category prototype feature library; Mahalanobis distance is used to measure the multidimensional feature vectors of the furniture and the style category prototype feature library to obtain a style similarity score; the style category to which the furniture belongs is determined based on the style similarity score to obtain the furniture style recognition result.

3. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 2, characterized in that, Complete the data for occluded areas, including: Gradient direction consistency analysis is performed on the furniture candidate region data to obtain edge continuity breakpoints; the location of occlusion is determined based on the edge continuity breakpoints to obtain occlusion area location data. Boundary contour tracking is performed on the occluded area localization data to obtain the occlusion boundary contour; based on the occlusion boundary contour, a view traversal search is performed in the multi-view image to obtain a set of candidate completion viewpoints. The optimal completion view is obtained by selecting the view with the highest visibility of the occluded area from the candidate completion view set; the point cloud data of the corresponding area in the optimal completion view is extracted to obtain the occluded area completion point cloud; the occluded area completion point cloud is mapped back to the original view coordinate system to obtain the occluded area completion data.

4. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 2, characterized in that, The process of obtaining the dynamic removal strategy includes: The home environment is scanned and collected using AR devices to obtain home environment image data; the home environment image data is then subjected to binocular stereo matching to obtain depth image data. Voxelized spatial modeling is performed based on depth image data to obtain a voxel model of the home space; mesh optimization is performed on the voxel model of the home space to obtain a three-dimensional model of the home space. The 3D model of the home space is segmented into object instances to obtain a set of existing furniture instances; the spatial location of the existing furniture instances is extracted to obtain a home furniture distribution map. Calculate the three-dimensional bounding box dimensions of the target furniture based on the complete furniture outline data to obtain the space occupancy parameters of the target furniture; The space occupancy parameters of the target furniture are virtually traversed in the furniture distribution map to obtain a set of candidate placement locations; and collision detection is performed on each location in the set of candidate placement locations to obtain a spatial conflict matrix. Extract the style attribute vector of the target furniture based on the furniture style recognition results; extract the style attribute vector of each of the existing furniture instances; calculate the cosine similarity between the style attribute vector of the target furniture and the style attribute vector of the existing furniture to obtain the style harmony value; A style conflict index is obtained by performing inverse scoring based on the style harmony value; a furniture removal priority sequence is obtained by comprehensively ranking the spatial conflict matrix and the style conflict index; and a dynamic removal strategy is generated based on the furniture removal priority sequence.

5. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 4, characterized in that, The process of obtaining the fused scene rendering results includes: Based on the dynamic removal strategy, the furniture objects with the highest priority are selected from the furniture distribution map to obtain the furniture to be removed icons; the space area corresponding to the furniture to be removed icons is virtually cleared to obtain the empty space area data; Skeleton extraction is performed on the complete furniture outline data to obtain the furniture skeleton structure diagram; parametric shape fitting is performed based on the furniture skeleton structure diagram to obtain the furniture geometric parameter set; A 3D mesh model of the furniture is constructed based on the set of furniture geometric parameters, and the 3D model data of the furniture is obtained. Furniture surface texture images are extracted from AR image sequence data to obtain raw texture data; viewpoint distortion correction is performed on the raw texture data to obtain corrected texture images; The corrected texture image is mapped to the UV coordinate space of the furniture 3D model data to obtain a textured furniture model; Local light source positions are estimated from the vacant space area data in the 3D model of the home space to obtain the main light source direction vector; the ambient light intensity distribution is calculated based on the main light source direction vector to obtain the ambient light parameters. Extract the material reflection properties of the textured furniture model to obtain the material reflection coefficient; perform light transmission calculation based on ambient lighting parameters and material reflection coefficient to obtain the target lighting rendering parameters; The color space of the textured furniture model is converted according to the target lighting rendering parameters to obtain a color adjustment matrix; the color of the furniture model texture is then transferred based on the color adjustment matrix to obtain a lighting-adapted furniture model. The lighting-adapted furniture model is placed in the empty space area data; the shadow projection area is calculated based on the main light source direction vector and the furniture geometric parameter set to obtain shadow mask data; the shadow mask data is processed with soft shadow blurring to obtain a soft shadow image; The soft shadow image is alpha-blended with the 3D model of the home space to obtain shadowed scene data; the lighting-adapted furniture model is then depth-sorted and rendered with the shadowed scene data to obtain the blended scene rendering result.

6. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 5, characterized in that, The process of obtaining the furniture geometry parameter set includes: Morphological refinement is performed on the complete furniture outline data to obtain a single-pixel-width skeleton line; branch points and endpoints of the single-pixel-width skeleton line are detected to obtain a set of key skeleton nodes; and the skeleton topological connection relationship is constructed based on the set of key skeleton nodes to obtain the furniture skeleton structure diagram. The furniture skeleton structure diagram is matched with the structure type to obtain the furniture category label; and the corresponding parametric shape template is selected according to the furniture category label to obtain the basic shape template. The control parameters of the basic shape template are fitted with the complete furniture outline data using least squares to obtain the optimal shape parameter values; and a set of furniture geometric parameters is generated based on their combination.

7. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 5, characterized in that, The process of obtaining target lighting rendering parameters includes: The surface material of the textured furniture model is identified to obtain a material type label. Based on the material type label, the corresponding bidirectional reflectance distribution function is queried from the preset material attribute database to obtain the material BRDF model. The diffuse reflectance coefficient and specular reflectance coefficient are calculated based on the material BRDF model to obtain the material reflectance coefficient. Based on the main light source direction vector and intensity value in the ambient lighting parameters, and combined with the material reflection coefficient, the lighting intensity is calculated to obtain the lighting intensity value of each vertex. The illumination intensity value of each vertex is normalized to obtain a normalized illumination intensity distribution; the target illumination rendering parameters are generated based on the normalized illumination intensity distribution.

8. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 5, characterized in that, The optimized process for acquiring scene data includes: Image quality is evaluated on the rendered results of the fused scene to obtain an image sharpness index; illumination consistency is detected on the rendered results of the fused scene to obtain an illumination consistency score. Edge blending degree analysis is performed on the rendering results of the fused scene to obtain the edge transition smoothness; a weighted comprehensive score is performed based on image sharpness index, illumination consistency score and edge transition smoothness to obtain the scene realism score; Anomaly detection thresholds are set based on scene realism scores; areas with scene realism scores below the anomaly detection thresholds are located and marked to obtain anomaly area marking data. Present the user with the blended scene rendering results and abnormal area marker data; collect the user's touch interaction data to obtain the interaction sequence; The operation type is identified from the interactive operation sequence to obtain the adjustment operation type; the user's adjustment intention is parsed based on the adjustment operation type to obtain the adjustment direction instruction; Based on the direction adjustment command, the position, rotation angle, and scaling ratio of the lighting-adapted furniture model are corrected to obtain geometric adjustment parameters; the color, brightness, and saturation of the lighting-adapted furniture model are corrected to obtain color adjustment parameters. The geometric adjustment parameters and color adjustment parameters are integrated to obtain the user-optimized parameters; the user-optimized parameters are then applied to regenerate the blended scene rendering result, resulting in optimized scene data which is then displayed to the user.

9. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 8, characterized in that, The process of obtaining the scene realism score includes: The Laplacian gradient of the rendered fused scene is calculated to obtain the edge gradient intensity map; and the statistical variance of the edge gradient intensity map is calculated to obtain the image sharpness index. Extract the illumination distribution histograms of the furniture area and the surrounding environment area from the fused scene rendering results to obtain a set of regional illumination histograms; calculate the histogram correlation of the regional illumination histogram set to obtain the illumination consistency score; Color difference analysis is performed on the edges where furniture and environment meet in the rendered scene to obtain the edge color jump amplitude; the gradient continuity of the edge transition area is calculated based on the edge color jump amplitude to obtain the edge transition smoothness. The obtained image sharpness index, illumination consistency score, and edge transition smoothness are weighted and summed to obtain the scene realism score.

10. The AI-based AR-based scene-based furniture recognition and dynamic removal system according to claim 7, characterized in that, The preset material property database stores material entries for hundreds of common materials, including material name, BRDF model type, model parameters, reflectivity spectrum, and roughness range.

Images