A training method and a defect identification method of an interior design defect identification model

Abstract

This disclosure provides a training method and a defect recognition method for an interior design defect recognition model, relating to the fields of artificial intelligence and interior design assistance technology. It can identify problems in interior design schemes in advance, reduce rework costs, and thus improve the subsequent rationality of the interior design scheme. The method includes: acquiring training data, which includes video data of the interior design scheme, the corresponding design rules, and user operation instructions. Based on the video data and operation instructions, an interior design defect recognition model is trained. The interior design defect recognition model includes a visual encoder, a scene generation network, an interaction response network, and a defect recognition network.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and interior design assistance technology, and in particular to a training method and a defect identification method for an interior design defect identification model. Background Technology

[0002] With the steady development of the real estate industry and the continuous improvement of people's quality of life, interior design has become a core element in the creation of living and working spaces, directly affecting the comfort, functionality, and aesthetics of the space. As a crucial step connecting space planning and construction implementation, the scientific nature of the interior design plan directly determines the smoothness of subsequent construction.

[0003] Currently, the interior design industry mainly adopts a combination of traditional design methods and digital-assisted design to carry out its work.

[0004] Although related technologies have improved the efficiency and visualization of interior design to some extent, many problems still exist, making it difficult to identify problems in interior design schemes and control rework costs. Summary of the Invention

[0005] This disclosure provides a training method and a defect identification method for an interior design defect identification model, which can identify problems in interior design schemes in advance, reduce rework costs, and thus improve the rationality of subsequent interior design schemes.

[0006] Firstly, this disclosure provides a training method for an interior design defect identification model, the method comprising: Acquire training data, including video data of interior design schemes, corresponding design rules, and user operation instructions. Based on the video data and operation instructions, train an interior design defect recognition model, which includes a visual encoder, a scene generation network, an interaction response network, and a defect recognition network.

[0007] The visual encoder is used to encode video data to obtain the visual features of the interior design scheme; the scene generation network is used to obtain the simulated scene and scene feature data corresponding to the interior design scheme based on the visual features; the interactive response network is used to obtain the scene state data corresponding to the simulated scene under the user's selected perspective based on the operation instructions; and the defect identification network is used to identify defects in the scene feature data and scene state data based on the design rules to obtain the defect identification results of the interior design scheme.

[0008] The solution provided in this disclosure trains the interior design defect identification model using training data, enabling it to obtain the simulated scene and scene feature data of the interior design scheme based on video frame data. Based on the user's operation commands on the model scene, it obtains the scene state data corresponding to the simulated scene from the user's selected perspective. By using design rules to identify defects in the scene feature data and scene state data, the defect identification results of the interior design scheme are obtained. This allows for the early detection of problems and defects in the interior design scheme, effectively improving the subsequent rationality and practicality of the interior design scheme.

[0009] Secondly, this disclosure provides a method for identifying interior design defects, the method comprising: The system acquires video data of the interior design scheme input by the user, the design rules corresponding to the interior design scheme, and the user's operation instructions; it inputs the video data, design rules, and operation instructions into the interior design defect identification model to obtain the defect identification results output by the interior design defect identification model; and it generates an optimization suggestion report corresponding to the interior design scheme based on the defect identification results.

[0010] Thirdly, this disclosure provides an electronic device including a display, a processor, a communication interface, and a memory. The memory stores at least one computer program, which is loaded and executed by the processor to implement the training method for the aforementioned interior design defect identification model, or to implement the aforementioned interior design defect identification method.

[0011] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in this disclosure, the accompanying drawings used in some embodiments of this disclosure will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 A schematic diagram of the architecture of a world model provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure; Figure 3 A schematic diagram of the architecture of a training system for an interior design defect identification model provided in this embodiment of the disclosure; Figure 4 A flowchart illustrating a training method for an interior design defect identification model provided in this embodiment of the disclosure; Figure 5 A schematic diagram of the architecture of an interior design defect identification model provided in this embodiment of the disclosure; Figure 6 A flowchart illustrating an interior design defect identification method provided in this embodiment of the disclosure; Figure 7 This is a schematic diagram of the structure of a training device for an interior design defect identification model provided in an embodiment of this disclosure. Detailed Implementation

[0014] The training method and defect identification method of the interior design defect identification model provided in this application embodiment are described in detail below with reference to the accompanying drawings.

[0015] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0016] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0017] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0018] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0019] Based on the aforementioned background technology, the existing world model based on self-distillation with no labels (DINO) is a latent space prediction model based on the second-generation self-distillation with no labels (DINOv2) visual encoder. Its core advantage lies in learning the temporal dynamics of the environment using large-scale video data, supporting feature prediction of future frames and trajectory planning under action conditions, and has been applied in fields such as driving and robot control.

[0020] The core architecture of DINO-world consists of a frozen DINOv2 visual encoder and a Transformer-based predictor. Pre-trained on a large-scale unlabeled video dataset, it offers the following advantages: (1) Feature prediction capability: The algorithm of this world model is based on the patch features of predicting future frames in the latent space, avoiding redundant calculations of pixel-level modeling, and the training efficiency is better than that of traditional pixel-level generation models. (2) Flexible Adaptability: The world model supports variable frame rate, context length and resolution, and adapts to agent control through fine-tuning of action blocks. The action blocks inject action signals in the form of q+MLP(LN([q,a])), which supports flexible expansion of motion interaction from an architectural perspective.

[0021] (3) Advantages of native motion modeling: By capturing spatiotemporal relationships through 3D Rotary Position Embedding (RoPE), the motion trajectory of objects can be accurately modeled. Action blocks can be efficiently associated with the "observation-action-motion response" mapping, adapting to the motion simulation needs of interactive scenarios.

[0022] This world model provides a completely new technical approach to identifying defects in interior design schemes.

[0023] Figure 1 This is a schematic diagram of the architecture of a world model provided in an embodiment of this disclosure.

[0024] like Figure 1 The world model architecture consists of a frozen DINOv2 visual encoder 101, a Transformer-based predictor 102, an MLP (Multilayer Perceptron) 103, and a decoder 104. The Transformer-based predictor 102 includes RoPE and a Causal Mask (block triangular causal mask).

[0025] The core process of this world model is as follows: First, the input video frames are converted into compact latent space feature maps using a frozen DINOv2 visual encoder 101. Then, the latent space feature maps are fed into a Transformer-based predictor 102, which uses RoPE spatiotemporal coding and Causal Mask for temporal modeling, predicting latent features for specific future spatiotemporal coordinates in parallel. Next, the input action commands are encoded using an MLP 103 and interact with the latent features output by the Transformer-based predictor 102 to obtain action-condition-guided latent features. Finally, depending on the task requirements, a decoder 104 selectively maps the action-condition-guided latent features to specific task outputs. For example, the task could be a semantic segmentation map or a depth map.

[0026] While this world model has significant advantages in areas such as physical dynamics prediction and intelligent agent trajectory planning, the simulation scenarios for interior design solutions have unique industry requirements and technical demands. Existing models are not designed to adapt to this scenario, resulting in many inherent shortcomings when directly applied, as follows: (1) Lack of multimodal data processing capabilities. Interior design scenarios rely on multimodal data such as drawings, natural language, design draft images, and voice as input. However, the existing world model only supports visual frame input and cannot parse non-visual modal design information, resulting in the design requirements and model input not being effectively connected. (2) Insufficient ability to build customized scenes: Interior design needs to generate exclusive spaces that conform to layout specifications and size constraints based on scene type and functional requirements. However, the core function of the existing model is to predict the characteristics of future potential spaces based on historical video frames, and it lacks the ability to analyze design intentions and generate customized spatial structures and object layouts. (3) Insufficient real-time interactive adaptability: The scene simulation of the interior design scheme needs to support users to perform high-frequency interactive operations such as scene roaming and perspective switching. The model is required to have low latency response capability. However, the existing model adopts an autoregressive prediction architecture, which has a slow inference speed and does not have a dedicated adaptation mechanism for perspective switching and path movement, making it difficult to meet the real-time interactive requirements. (4) Lack of professional spatial knowledge: Interior design involves professional knowledge such as ergonomic standards, spatial layout specifications, and scene-specific requirements. However, the pre-training data of existing models are general videos, which cannot incorporate professional knowledge, resulting in the generated scenes not matching industry standards and thus failing to identify design defects.

[0027] To address the aforementioned issues, this disclosure provides a training method for an interior design defect identification model. This model includes a visual encoder, a simulated scene generation network, and a defect identification network. By training the model with training data, it can obtain simulated scenes and scene feature data of the interior design scheme based on video frame data. Furthermore, based on user commands to the model scene, it obtains scene state data corresponding to the simulated scene from the user's selected perspective. Finally, it identifies defects in the scene feature data and scene state data using design rules, yielding the defect identification results for the interior design scheme. This allows for the early detection of problems and defects in the interior design scheme, effectively improving its subsequent rationality and practicality.

[0028] The training method for the interior design defect identification model provided in this disclosure can be applied to a training device for the interior design defect identification model. The training device can be an electronic device such as a server, server cluster, high-performance computing platform, or dedicated simulation server. This disclosure does not impose any restrictions on the specific form of the training device for the interior design defect identification model.

[0029] Taking servers as an example of electronic devices, Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0030] like Figure 2 As shown, the electronic device 200 includes: a display screen 201, a processor 202, a memory 203, a communication interface 204, and a bus 205. The processor 202, the memory 203, and the communication interface 204 can be connected via the bus 205.

[0031] The display screen 201 is the output and feedback window of the electronic device 200. It can be used to present key information to users or designers, and can also collect feedback data for training methods of the interior design defect identification model and for the whole process optimization of the interior design defect identification method.

[0032] The processor 202 is the control center of the electronic device 200. It can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0033] As an example, processor 202 may include one or more CPUs, for example Figure 2 CPU 0 and CPU 1 are shown in the diagram.

[0034] The memory 203 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0035] In one possible implementation, the memory 203 may exist independently of the processor 202. The memory 203 can be connected to the processor 202 via a bus 205 and is used to store data, instructions, or program code. When the processor 202 calls and executes the instructions or program code stored in the memory 203, it can implement the communication method provided in the embodiments of this application.

[0036] In another possible implementation, the memory 203 can also be integrated with the processor 202.

[0037] The communication interface 204 is used for the electronic device 200 to connect with other devices via a communication network, which may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 204 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0038] Bus 205 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0039] It should be pointed out that, Figure 2 The structure shown does not constitute a limitation on the electronic device 200, except Figure 2In addition to the components shown, the electronic device 200 may include more or fewer components than those shown, or combine certain components, or have different component arrangements.

[0040] In some embodiments, the training device for the interior design defect identification model belongs to the training system for the interior design defect identification model. Figure 3 This is a schematic diagram of the architecture of a training system for an interior design defect identification model provided in an embodiment of this disclosure. Figure 3 As shown, the training system for the interior design defect identification model may include an interior design data receiving device 301, an interior design defect identification model training device 302, and an interior design defect identification model application device 303. The interior design data receiving device 301, the interior design defect identification model training device 302, and the interior design defect identification model application device 303 can communicate with each other.

[0041] The interior design data receiving device 301 may include a multimodal data receiving unit and a preprocessing unit. The multimodal data receiving unit can receive user-input data related to the interior design scheme, such as videos of the interior design scheme, design drafts, CAD drawings, and design rule texts. The multimodal data receiving unit can also receive user operation commands. The preprocessing unit is used to transform the received data related to different modalities of the interior design scheme into a unified tensor format acceptable to the model through operations such as scaling, cropping, filling, and resampling.

[0042] The training device 302 of the interior design defect identification model acquires the preprocessed multimodal data and operation instructions from the interior design data receiving device 301, and performs phased or end-to-end joint training on the interior design defect identification model. Finally, a trained interior design defect identification model is obtained, with the input being multimodal data and operation instructions related to the interior design scheme and the output being the defect results of the interior design scheme.

[0043] In some embodiments, the training device 302 for the interior design defect identification model integrates and deploys a video encoder, a simulated scene generation network, and a defect identification network.

[0044] In other embodiments, the training device 302 for the interior design defect identification model is also integrated with a multimodal encoder, such as a video encoder, a text encoder, a speech encoder, a document encoder, etc.

[0045] In some embodiments, the training device 302 for the interior design defect recognition model integrates a multimodal data receiving unit, a preprocessing unit, a multimodal encoder, a simulated scene generation network, and a defect recognition network, allowing the entire process from data acquisition and preprocessing to model training and optimization to be completed directly on a single device. This disclosure also provides an interior design defect recognition method that, based on user-inputted video data of the interior design scheme, design rules, and operation instructions, obtains defect recognition results for the interior design scheme through a trained interior design defect recognition model.

[0046] Understandably, this method for identifying interior design defects can be applied to... Figure 3 The application device 303 for identifying interior design defects shown can also be applied to the training device 302 for identifying interior design defects, and this embodiment does not impose any specific limitations on this.

[0047] The training method for the interior design defect identification model provided in this application is described below with reference to specific embodiments and accompanying drawings.

[0048] Figure 4 This is a flowchart illustrating the training method for an interior design defect identification model provided in this disclosure. Figure 4 As shown, the specific steps of the training method for the interior design defect identification model include S401 and S402.

[0049] S401. Obtain training data.

[0050] The training data includes video data of interior design schemes, design rules corresponding to the interior design schemes, and user operation instructions.

[0051] Video data refers to a series of static images arranged in chronological order that make up a video; that is, each frame of video is a static image.

[0052] The design rules corresponding to an interior design scheme refer to the standards and constraints used to guide the interior design scheme, which can clearly define the types of indicators and the corresponding requirements and standards.

[0053] The types of indicators include spatial geometry indicators, physical environment indicators, material and process indicators, equipment and electrical indicators, and aesthetic and style indicators. For example, the width of the passageway should not be less than 1.8 meters, and the room lighting should be soft and warm, not too glaring.

[0054] Operation commands refer to action signals that drive the model to predict how the simulated scene corresponding to the interior design scheme will change.

[0055] In one example, the operation command can be triggered by an input device. For example, the input device could be a game controller, mouse, keyboard, etc.

[0056] In another example, operation instructions can also be triggered by text input, and the specific operation instructions can be parsed out using natural language understanding technology.

[0057] In another example, operation instructions can also be triggered by voice. The user's voice is received through a microphone, converted into text using speech recognition technology, and then parsed into specific operation instructions using natural language understanding technology.

[0058] S402. Based on video data and operation instructions, train an interior design defect recognition model. The interior design defect recognition model includes a visual encoder, a scene generation network, an interactive response network, and a defect recognition network.

[0059] The visual encoder is used to encode video data to obtain the visual features of the interior design scheme.

[0060] In some embodiments, the scene generation network is used to obtain simulated scenes and scene feature data corresponding to the interior design scheme based on visual features.

[0061] A simulated scene refers to the transformation of the visual features of an interior design scheme into a three-dimensional (3D) virtual scene that can be directly observed, used for visualizing, experiencing, or verifying the interior design scheme.

[0062] Scene feature data, also known as scene key parameters, is used to describe, represent, or quantify the information of various element attributes and their spatial relationships in the simulated scene corresponding to the interior design scheme. In other words, scene feature data is the conversion of the visual content (i.e. the image that the user can see) of the simulated scene corresponding to the interior design scheme into data (such as numbers, vectors, or matrices) that the scene generation network can process.

[0063] In some embodiments, the interactive response network is used to obtain scene state data corresponding to the simulated scene from the user's selected perspective, based on operation instructions.

[0064] Scene state data refers to the collection of information about all objects or the environment in the simulated scene from the user's selected perspective.

[0065] For example, scene state data can include the position information of objects from the user's selected viewpoint, object orientation information, object material information, illumination intensity information, and illumination direction information.

[0066] In some embodiments, the defect identification network is used to identify defects in scene feature data and scene state data based on design rules, thereby obtaining defect identification results for the interior design scheme.

[0067] In this embodiment, the interior design defect identification model is trained using training data. This allows the model to obtain the simulated scene and scene feature data of the interior design scheme based on video frame data. Based on the user's operation commands on the model scene, the model obtains the scene state data corresponding to the simulated scene from the user's selected perspective. By using design rules to identify defects in the scene feature data and scene state data, the model obtains the defect identification results of the interior design scheme. This allows for the early detection of problems and defects in the interior design scheme, effectively improving the subsequent rationality and practicality of the interior design scheme.

[0068] The following embodiments provide a detailed description of the visual encoder, scene generation network, interactive response network and defect recognition network in the interior design defect identification model, including (1)-(4).

[0069] (1) A visual encoder is used to encode video data related to interior design schemes into visual features that the model can understand and compute.

[0070] In some embodiments, the visual encoder can be a frozen DINOv2 visual encoder used to convert image or video data into high-dimensional semantic feature vectors, i.e., visual features in this disclosure. This encoder acts like a "universal visual translator," extracting deep information such as shape, texture, and objects from image or video data without training, providing a stable and compact visual representation for subsequent models. At the same time, due to parameter freezing, it can significantly save computational resources.

[0071] (2) The scene generation network is used to obtain the simulated scene and scene feature data corresponding to the interior design scheme based on visual features.

[0072] In some embodiments, the scene generation network is also used to obtain simulated scenes and scene feature data corresponding to the interior design scheme based on visual features and multimodal features.

[0073] Multimodal features include at least one of the following: text features, speech features, document features, etc.

[0074] In one feasible implementation, the interior design defect identification model is equipped with a spatial scene knowledge base. The scene generation network can obtain the scene type of the interior design scheme input by the user, or determine the scene type of the interior design scheme through visual features and multimodal features. It can then call the corresponding interior design professional basis from the spatial scene knowledge base. Based on the visual features, multimodal features, and interior design professional basis, it obtains the simulated scene and scene feature data corresponding to the interior design scheme.

[0075] Specifically, the process begins by establishing a professional basis for generating simulated scenes based on the scene types of the interior design schemes, such as typical interior layout standards, object size ranges, and spatial relationship constraints. Then, based on this professional basis, interior space structural feature data is generated. Next, interior object layout feature data is generated based on the design rules corresponding to the interior design schemes and the professional basis. Finally, by matching the types of interior objects with the scene type through scene adaptation verification, the interior space structural feature data and interior object layout feature data are transformed into a simulated scene that users can directly observe, and scene feature data is generated simultaneously.

[0076] The following is a detailed introduction to the spatial scene knowledge base.

[0077] In some embodiments, the spatial scene knowledge base is used to provide professional interior design basis for various scene types during the training of the interior design defect identification model. For example, scene types include lounge scenes, office area scenes, warehouse scenes, airport scenes, and room scenes, etc.

[0078] In one feasible implementation, the spatial scene knowledge base can be based on a scene knowledge graph of a graph neural network (GNN), including professional interior design criteria corresponding to various scene types. The nodes of the scene knowledge graph include "scene type—functional area—object—ergonomic constraints," and the edges represent spatial topological relationships and normative constraints.

[0079] Specifically, the knowledge graph corresponding to a scene type is G=(V,E), which adopts a heterogeneous graph structure. The node set V includes three types of nodes: region nodes v s Object category node v o and ergonomic constraint node v c The edge set includes two types of directed edges: spatial topological edges e. topo and normative constraint edge e rule .

[0080] Node embeddings are learned using a GNN neural network and updated using the following formula: Equation (1).

[0081] in, Representative node In the Layer embedding vector; Represents a node In the The embedding vector of the layer; R represents the set of relation types, i.e., the edge set; Represents the nodes under relation r The set of neighbors; The learnable weight matrix corresponding to relation r; This represents the activation function.

[0082] Through multi-layer propagation, various professional constraints in the knowledge graph are encoded into computable embedding vectors. Through a knowledge-injection generation mechanism, the embedding vectors in the knowledge graph are used as additional conditional inputs to the model generation network, thereby enabling professional standards to accurately guide scene generation.

[0083] In one specific implementation, the conditional vectors for node embedding aggregation associated with the current scene type satisfy the following relationship: Equation (2).

[0084] in, Represents the condition vector; This represents the subset of nodes corresponding to the current scene type; L represents the final layer number of the GNN. The conditional vector representing the output is a vector with dimension 1. A real vector.

[0085] The knowledge graph in the spatial scene knowledge base can be dynamically updated, better adapting to newly released industry standards and user-defined constraints, thus improving the practicality and real-time performance of the knowledge base. Furthermore, by calling the standards corresponding to the scene types of the interior design scheme through the scene generation network, layout basis can be provided for the generation of simulated scenes, improving the accuracy and standardization of simulated scene generation.

[0086] (3) The interactive response network is used to obtain scene state data corresponding to the simulated scene from the user's selected perspective based on the operation instructions.

[0087] In some embodiments, the architecture of the interactive response network can be based on the original world model architecture, with the addition of a dual-track prediction architecture and a differentiable RoPE reparameterized view correction layer, thereby enabling a high-frequency interactive process.

[0088] The dual-track prediction architecture can decompose the interactive prediction based on the user's operational intelligence commands into two parallel tracks: the prediction track and the precision track.

[0089] Prediction Track: A lightweight RNN can be used as the user intent prediction network to predict the user's next action intent in advance based on historical interaction sequences, and to generate candidate latent space features in advance. The candidate latent space features satisfy the following relationship: Equation (3).

[0090] in, Represents the characteristics of the candidate latent space; Represents a lightweight prediction network; The operation vector represents the prediction; The candidate latent space features representing the output are an N x d real matrix.

[0091] Mental Track: Performs precise predictions only on keyframes, obtaining latent space features by fusing multimodal features and a knowledge graph. The latent space features satisfy the following relationship: Equation (4).

[0092] in, Represents latent space characteristics; A sequence of latent space features representing the past k frames, used to provide temporal context information; Represents the actual user operation vector at the previous moment; This represents the fused features after processing by a multimodal feature encoder, including visual features, question-and-answer features, document features, and speech features, etc. This represents a conditional vector injected from a spatial scene knowledge base, carrying professional knowledge such as industry standards and ergonomic constraints; The world simulation predictor is responsible for accurately predicting the latent space features at the current time t based on the above inputs. The latent space features representing the output are an N x d real matrix.

[0093] In some embodiments, the features of the two trajectories can be fused using a spatiotemporal interpolation network.

[0094] First, we can determine the time distance between the current frame and the nearest keyframe. The fusion ratio can be dynamically adjusted. The formula for calculating the fusion ratio can be: Equation (5).

[0095] in, Represents the integration ratio; Weight coefficients representing the characteristics of the latent space. and related, , This represents the Sigmoid activation function. When... A smaller value indicates that the current frame is closer to the key frame. The closer to 1, the more the characterization fusion ratio depends on the results of the precise frames; when When the value is large, the characterization fusion ratio depends more on the results of the predicted orbits, which can significantly reduce the overall computational load.

[0096] The differentiable RoPE reparameterized view correction layer of the interactive response network optimizes the attention mechanism by using view weight factors, giving higher attention weights to the scene corresponding to the simulated scene under the user's selected viewpoint, so that the interactive response network pays more attention to the scene features of the user's current field of vision.

[0097] In one feasible implementation, the formula for the attention weight of the patch from the user's selected perspective can be: Equation (6).

[0098] in, This represents the attention weight of the patch from the user's selected perspective; This represents the patch from the current perspective. The coordinates; Represents the coordinates of the center of the current viewpoint; This represents the field of view attenuation coefficient.

[0099] The formula for weighted attention can be: Equation (7).

[0100] in, The query matrix is ​​obtained by multiplying the input patch sequence from the current perspective with the learnable weight matrix corresponding to the query matrix. The key matrix is ​​obtained by multiplying the input patch sequence from the current perspective with the learnable weight matrix corresponding to the key matrix. The value matrix is ​​obtained by multiplying the input patch sequence from the current perspective with the learnable weight matrix corresponding to the value matrix. Represents the normalized exponential function; Represents patch Similarity vectors with other patches; This represents the scaling factor.

[0101] This perspective weight factor optimization attention mechanism assigns higher attention weights to the scene under the user's selected perspective, making the model pay more attention to the scene features of the core area of ​​the user's current field of vision. This significantly improves the response speed and scene continuity in high-frequency interaction scenarios, while also improving the accuracy of prediction and the sense of visual immersion.

[0102] (4) The defect identification network is used to identify defects in scene feature data and scene state data based on design rules, and obtain the defect identification results of the interior design scheme.

[0103] In some embodiments, the defect identification network can adopt a semantic-geometric dual-branch index extraction network. By adding a geometric regression branch on the basis of U-Net semantic segmentation and designing a scene-adaptive weighted defect ranking mechanism, a complete closed loop from index extraction to defect judgment, priority ranking and optimization suggestion output can be realized.

[0104] The semantic-geometric dual-branch indicator extraction network decomposes indicator extraction into two parallel branches: a semantic branch and a geometric branch. The semantic branch is used to extract high-level semantic features from the simulated scene corresponding to the interior design scheme, while the geometric branch is responsible for the accurate regression of physical dimensions.

[0105] Semantic branch: Based on the U-Net encoder, high-level semantic features are extracted from the simulated scene corresponding to the interior design scheme. By combining skip connection technology, the core area boundaries in the simulated scene are accurately located, and object category labels and spatial relationship feature maps are output. It can be understood that the semantic branch is responsible for answering "what is here" and "what is the relationship between them".

[0106] The core area boundary includes passageways, furniture, fire-fighting facilities, walls, etc.

[0107] Specifically, the formula for high-level semantic features can be: Equation (8).

[0108] in, Represents high-level semantic features; Represents the input scene image; Represents the resolution of the feature map; This represents the number of semantic feature channels.

[0109] Geometric Branch: Building upon the semantic branch, an independent geometric regression head is added to directly regress physical size metrics from the feature map. Examples of physical size metrics include channel width, object spacing, and net height.

[0110] Specifically, the formula for physical dimensional indicators can be: Equation (9).

[0111] in, Representing the Predicted values ​​for each physical dimension indicator; The total number of physical indicators to be regressed; This represents a learnable geometric regression head.

[0112] After obtaining the regressed physical size index, the pixel distance can be converted into the actual physical size through the pixel-physical scale mapping system.

[0113] In one specific implementation, the formula for calculating the actual physical dimensions is: Equation (10).

[0114] in, Represents the actual physical dimensions; The ratio of pixels to physical size.

[0115] The semantic and geometric branches share the weights of the U-Net encoder and are separated only at the output head, which ensures both the collaborative training of semantic understanding and geometric regression and avoids parameter redundancy.

[0116] In one feasible implementation, the defect identification network can use a multi-constraint optimization defect determination mechanism to determine the actual physical size, so that the actual physical size meets industry standards and user-defined constraints.

[0117] Specifically, firstly, the design rules and industry standards corresponding to the obtained interior design schemes can be converted into a standardized threshold constraint format: Equation (11).

[0118] in, Represents a standardized threshold constraint format; Representing the The lower limit threshold of the actual physical size of the physical size index; Representing the The upper limit threshold of the actual physical size of a physical size indicator.

[0119] Then, defect determination is achieved through multi-constraint comparison. The actual physical size of each extracted physical size index is compared with the corresponding rule threshold. If the conditions are met... ,or ,but If satisfied ,but .in, Representing the This indicator has flaws. Representing the All indicators are compliant.

[0120] In one feasible implementation, the defect identification network can adjust the weight coefficients of the identified defects through a scene-adaptive weighted defect ranking mechanism, thereby achieving defect priority ranking.

[0121] Specifically, firstly, based on the scene type of the interior design scheme, the corresponding scene weight vector is retrieved from the spatial scene knowledge base. The scene weight vector includes the importance weights of the defects in various indicators under the current scene type.

[0122] Then, calculate the overall priority score for each defect. The formula for calculating the overall priority score for a defect is: Equation (12).

[0123] in, Representing the The overall priority score of a defect indicates that the higher the overall priority score, the higher the priority of the defect that needs to be addressed. Representing the The importance weight of each defect; Representing the The defect determination result of the item; Representing the The defect deviation of a defect.

[0124] Finally, according to Arrange all defects of interior design schemes in descending order, output the priority ranking results, and provide corresponding actionable optimization suggestions based on the defects.

[0125] The scenario-adaptive weighted defect ranking mechanism retains the intuitiveness of the original system, while achieving more refined dynamic adjustment of cross-scenario defect priorities through scenario-adaptive weights. It also provides priority ranking results and corresponding executable optimization suggestions, thereby improving the practicality of the model.

[0126] In some embodiments, the defect deviation can be calculated based on the actual physical dimensions of the defect's corresponding physical dimensional index. The defect deviation is used to quantify the severity of the defect.

[0127] In one specific implementation, the formula for calculating the defect deviation can be: Equation (13).

[0128] in, This represents the amount of defect deviation. The larger the value, the more serious the deviation from the standard.

[0129] Based on the above (1)-(3), the visual encoder first encodes the video data related to the interior design scheme into visual features that the model can understand and calculate; the scene generation network constructs a simulated scene that meets the requirements from scratch based on the visual features and the scene type of the interior design scheme, and generates scene feature data; the interactive response network obtains the scene state data of the user's selected perspective under the operation command based on the operation command used for input; the defect identification network is used to identify defects in the scene feature data and scene state data based on the design rules, realizes a complete closed loop from defect judgment to priority ranking and optimization suggestion output, thereby realizing end-to-end data-driven modeling from user input of information and operation commands related to the interior design scheme to defect identification.

[0130] In some embodiments, step S402 includes steps S4021 and S4022.

[0131] S4021. Based on video data and operation instructions, the interior design defect identification model is trained to obtain the multi-target fusion loss value.

[0132] In some embodiments, the multi-target fusion loss value is obtained by weighted summation of the feature prediction loss value, the interaction continuity loss value, and the defect identification loss value.

[0133] The feature prediction loss value is determined during the training of the visual encoder and is used to constrain the accuracy of visual features.

[0134] In one feasible implementation, the feature prediction loss can be obtained by calculating a smoothed L1 loss, which can ensure the basic accuracy of visual feature prediction.

[0135] The interaction continuity loss value is determined during the training of the scene generation network and the interaction response network, and is used to constrain the continuity of the simulated scene under the operation instructions.

[0136] In one feasible implementation, the interaction continuity loss value can be obtained by weighted summation of the continuity loss values ​​of adjacent frames, the viewpoint correction loss value, and the candidate viewpoint prediction loss value.

[0137] Among them, the adjacent frame continuity loss value is used to constrain the smoothness of changes in adjacent keyframes before and after the interaction, so as to enhance the continuity constraint during fast movement.

[0138] In one specific implementation, the continuity loss between adjacent frames can be achieved using L1 loss. Specifically, the continuity loss values ​​between adjacent frames satisfy the following relationship: Equation (14).

[0139] in, The value represents the continuity loss between adjacent frames; T represents the length of the interactive keyframe sequence; N represents the number of samples. Indicates the first The first sample Scene features of the frame; Indicates the first The first sample Motion weighting factor of a frame.

[0140] The viewpoint correction loss value is used to constrain the difference between the corrected viewpoint features and the true viewpoint features after viewpoint transformation, so as to ensure the accuracy of viewpoint transformation.

[0141] In one specific implementation, the viewpoint correction loss can be expressed as MSE loss. Specifically, the viewpoint correction loss satisfies the following relationship: Equation (15).

[0142] in, This represents the viewpoint correction loss value; N represents the number of samples. Indicates the first Corrected viewpoint features after viewpoint calibration for each sample Indicates the first The true perspective features of each sample.

[0143] The candidate field of view prediction loss is used to constrain the difference between candidate field of view features and true field of view features, in order to improve prediction accuracy and reduce real-time inference latency.

[0144] In one specific implementation, the candidate view prediction loss can be a smoothed L1 loss. Specifically, the candidate view prediction loss satisfies the following relationship: Equation (16).

[0145] in, This represents the prediction loss value for candidate views; Indicates the first Candidate field features predicted by a lightweight network for each sample. Indicates the first The true visual field characteristics of each sample.

[0146] Based on the above-mentioned adjacent frame continuity loss value Viewpoint correction loss value and candidate view prediction loss value This allows us to obtain the interaction continuity loss value.

[0147] Specifically, the formula for calculating the interaction continuity loss value can be: Equation (17).

[0148] in, This represents the interaction continuity loss value; Represents the weighting coefficients corresponding to the continuity loss values ​​between adjacent frames; This represents the weighting coefficient corresponding to the viewpoint correction loss value; This represents the weighting coefficient corresponding to the candidate vision prediction loss value. , , The weighting coefficients are set, and the sum of the three is 1.

[0149] The defect recognition loss value is determined during the training of the defect recognition network and is used to constrain the accuracy of defect recognition.

[0150] In one feasible implementation, the defect identification loss value can be obtained by weighted summation of the scene index regression loss value and the defect classification and priority loss value.

[0151] The scene index regression loss value is used to constrain the difference between the predicted values ​​of scene indices extracted by the interior design defect identification model and the actual labeled values ​​of scene indices, so as to ensure the accuracy of index extraction.

[0152] In one specific implementation, the scene metric regression loss can be a smoothed L1 loss. Specifically, the scene metric regression loss satisfies the following relationship: Equation (18).

[0153] in, P represents the regression loss value of the scenario metric; N represents the number of designed metrics; P represents the number of samples. Indicates the first The first sample The actual labeled value of each scenario indicator The first element extracted by the interior design defect identification model represents the... The first sample Predicted values ​​for indicators in each scenario.

[0154] The defect classification and priority loss value is used to guide the defect recognition network to learn defect classification and priority ranking, so as to ensure that the model focuses on high-priority defects first.

[0155] In one specific implementation, the defect classification and priority loss values ​​can be calculated using a weighted cross-entropy loss. Specifically, the defect classification and priority loss values ​​satisfy the following relationship: Equation (19).

[0156] in, This represents the loss value based on defect classification and priority; Q represents the number of defect categories. Indicates the first The actual defect label corresponding to the q-th defect category of each sample; Indicates the first The defect prediction probability corresponding to the qth defect category of each sample; This represents the defect priority weight corresponding to the q-th defect category.

[0157] The regression loss value is obtained through the above scenario indicators. Defect classification and priority loss value This allows us to obtain the defect identification loss value.

[0158] Specifically, the formula for calculating the defect identification loss value can be: Equation (20).

[0159] in, This represents the defect identification loss value; This represents the weight coefficient corresponding to the regression loss value of the scenario indicator; This represents the weighting coefficients corresponding to the defect classification and priority loss values. , The weighting coefficients are set, and the sum of the two is 1.

[0160] Combining the above-mentioned feature prediction loss value and interaction continuity loss value and defect identification loss value This allows us to obtain the multi-objective fusion loss value.

[0161] Specifically, the formula for calculating the multi-target fusion loss value can be: Equation (21).

[0162] in, This represents the multi-objective fusion loss value; This represents the feature prediction loss value; This represents the adaptive learning weights corresponding to the feature prediction loss values; This represents the adaptive learning weights corresponding to the interaction continuity loss values; This indicates that the defect identification loss value corresponds to the adaptive learning weight.

[0163] S4022. Update the model parameters of the interior design defect identification model based on multi-objective loss values.

[0164] In some embodiments, during the training of the interior design defect recognition model, the model parameters of the interior design defect recognition model are iteratively updated using the multi-objective loss value. When the multi-objective loss value converges to a preset loss threshold or the number of training iterations reaches a preset threshold, it is determined that the visual encoder, scene generation network, interaction response network, and defect recognition network in the interior design defect recognition model have all reached the expected training objectives, thereby obtaining a well-trained interior design defect recognition model.

[0165] Based on the multi-objective loss value mentioned above, a well-trained interior design defect identification model can be constructed. By building high-fidelity simulated scenarios and combining human interaction experience with intelligent analysis, it can identify potential problems such as spatial layout defects, furniture compatibility issues, and insufficient traffic flow in the design scheme in advance, and output targeted optimization suggestions. This can improve the efficiency, comfort, and safety of space use, reduce design rework costs, and provide data support and decision-making basis for the final implementation of the design scheme.

[0166] In some embodiments, the interior design defect identification model further includes a multimodal encoder. The multimodal encoder is used to encode the design data of the interior design scheme into design information features. The scene generation network then uses visual features, design information features, and operational instructions to obtain the simulated scene and scene feature data corresponding to the interior design scheme.

[0167] It is understandable that data related to interior design schemes can be stored in various formats, such as text data, document data, voice data, file data, and so on.

[0168] Text data is used to characterize the design rules corresponding to the natural language descriptions of interior design schemes. For example, the width of the passageway should not be less than 1.8 meters, the color temperature of the lighting should be warm light of 2700K to 3000K, and the fire rating of the interior decoration materials should not be lower than B1.

[0169] Document data refers to static images or documents corresponding to interior design schemes. For example, design draft images, CAD drawings, etc.

[0170] Voice data refers to audio recordings of users verbally describing design requirements, scene types, and operation instructions in interior design schemes.

[0171] In one feasible implementation, the multimodal encoder is a Transformer-based encoder. The Transformer is a deep learning architecture based on a self-attention mechanism. Its core function is to capture global dependencies by calculating the association weights between all elements in the input sequence, without relying on the sequential processing order of recurrent neural networks. It mainly consists of stacked encoders and decoders. The self-attention layer allows the model to dynamically focus on key information at different locations, while the feedforward neural network performs non-linear transformations on the features, thereby achieving efficient context encoding and representation learning for sequential data such as text, speech, and images.

[0172] In some embodiments, the multimodal encoder includes at least one of a text encoder, a speech encoder, and a document encoder.

[0173] The text encoder is used to encode text data into text features.

[0174] In one feasible implementation, the text encoder can use the BERT pre-trained language model to encode the user-input text data into a 768-dimensional semantic vector, and then achieve dimensional alignment with visual latent space features through a learnable linear projection matrix.

[0175] Specifically, a semantic vector can be represented as: Equation (22).

[0176] in, Represents the text feature vector. The first character in the input text data The character, that is, the first character. A sequence of tokens, Represents the length of the input text data sequence; curly braces Represents an ordered sequence of tokens; The text feature vector representing the output belongs to 3D real space, Represents the set of real numbers. Representing the dimension of a vector, for example .

[0177] To achieve alignment with the latent space, a learnable linear projection matrix is ​​introduced. With bias coefficient The projected text features are obtained as follows: Equation (23).

[0178] in, Represents textual features.

[0179] Then, the alignment quality between text features and corresponding visual latent space features is constrained through cross-modal contrastive learning: Equation (24).

[0180] in, The alignment quality between text features and corresponding visual latent space features; The batch size is represented by sim(·,·), the cosine similarity is represented by τ, and the temperature hyperparameter is represented by τ. This represents the visual latent space feature corresponding to the i-th batch.

[0181] This method can significantly improve the semantic mapping accuracy of text constraints to the latent space, enabling the interior design defect identification model to accurately identify design rules, operation instructions, etc. related to the interior design scheme. This allows the model to generate simulated scenarios, scenario feature data, and scenario response data that conform to the text characteristics, thereby improving the model's accuracy.

[0182] In some embodiments, a speech encoder is used to encode speech data into speech features.

[0183] In one feasible implementation, the speech encoder can employ an automatic speech recognition model pre-trained based on Automatic Speech Recognition (ASR) to convert user-input speech commands into text data. The converted text is then reused using the aforementioned BERT encoding and projection process to obtain speech modal features. Voice text contains information related to interior design schemes, enabling the interior design defect identification model to generate simulated scenarios and scene feature data that match the voice data; voice text also contains user operation commands, enabling the interior design defect identification model to generate scene response data that match the voice data.

[0184] In some embodiments, a document encoder is used to encode document data into document features.

[0185] In one feasible implementation, document data is input into a document encoder to obtain geometric and semantic features output by the document encoder. The geometric and semantic features are then concatenated, and the concatenated features are mapped to the latent space via linear projection to obtain the document features.

[0186] Among them, geometric features include spatial boundary features and structural contour features, while semantic features include object category features and spatial relationship features.

[0187] Specifically, the document encoder can employ a geometric-semantic bipath mapping from the layout graph to the latent space: Geometric path: Lightweight convolutional neural networks (CNNs) can be used to extract spatial boundary and structural contour features to obtain the geometric features corresponding to the document data.

[0188] In one specific implementation, the formula for calculating geometric features can be expressed as: Equation (25).

[0189] in, Represents geometric features; This represents the document data corresponding to the input interior design scheme; The geometric feature representing the output is a dimension of A real vector, i.e. yes A point in the feature space of dimension .

[0190] Semantic path: A pre-trained Vision Transformer (VIT) encoder can be used to extract object category features and spatial relationship features to obtain the semantic features corresponding to the document data.

[0191] In one specific implementation, the formula for calculating semantic features can be expressed as: Equation (26).

[0192] in, Represents semantic features; The semantic feature representing the output is a dimensionless feature. A real vector, i.e. yes A point in a dimensional spatial feature.

[0193] Then, the geometric and semantic features are concatenated, and the concatenated features are mapped to the latent space via linear projection to obtain document features.

[0194] In one specific implementation, the formula for calculating document features can be expressed as: Equation (27).

[0195] in, Represents document characteristics; Represents the learnable projection matrix; Representative features are pieced together; Represents the bias coefficient, and .

[0196] In some embodiments, the multimodal encoder may further include a video encoder. This video encoder may be configured as a dedicated encoder for the video modality within the multimodal encoder, or it may be reused as a visual encoder within the world model; this application does not limit its specific implementation.

[0197] In some embodiments, the multimodal encoder is also used to fuse the features of the multimodal structures to obtain the fused features of the interior design scheme.

[0198] In one feasible implementation, visual features, text features, speech features, and document features can be weighted and summed based on preset weight coefficients to obtain the fused features corresponding to the interior design scheme. Subsequently, simulated scenes, scene feature data, and scene response data can be obtained based on the fused features, which can improve the reliability and accuracy of the input. The preset weight coefficients are empirical values.

[0199] In another feasible implementation, a dynamic weighted fusion gating network can be used to perform weighted summation of visual features, text features, voice features, and document features to obtain the fused features corresponding to the interior design scheme.

[0200] Specifically, the fusion weights of visual features, text features, speech features, and document features can be calculated first. The formula for calculating the fusion weights can be expressed as: Equation (28).

[0201] in, Represents the fusion weighting coefficient. , The fusion weight coefficients represent the visual features. The fusion weight coefficients represent the text features. The fusion weight coefficients represent the document features. The fusion weight coefficients representing the speech features are as follows: ; Represents the learnable weight matrix; Represents a learnable bias; The fusion weight coefficients representing the output are a real vector of dimension 4.

[0202] Compared with preset fixed weight coefficients, the dynamic gating mechanism can adaptively adjust the weight coefficients of each modality feature according to different modal combinations of inputs (such as only visual features and text features, or only visual features, speech features and document features, etc.), thereby improving the robustness and flexibility of multimodal fusion.

[0203] This application divides data related to interior design schemes into the aforementioned different modalities for processing, which is merely an exemplary implementation method. Its aim is to achieve unified semantic understanding and fusion of heterogeneous data sources through the collaborative work of multimodal encoders. Those skilled in the art should understand that, depending on specific training needs and data availability, the method of dividing the multimodal data, the specific modal types included, and the selection of their corresponding encoders can be adaptively adjusted and extended.

[0204] The interior design defect model provided in this disclosure can be obtained by retaining the core architecture of the DINO world model and adding a multimodal encoder, a spatial scene knowledge base, an interactive response network, and a defect recognition network.

[0205] Figure 5 This is a schematic diagram of the architecture of an interior design defect identification model provided in an embodiment of this disclosure. Figure 5 As shown, the interior design defect identification model includes a multimodal encoder 501, a Transformer-based predictor 502, an MLP 503, a decoder 504, an interactive response network 505, and a defect identification network 506. The Transformer-based predictor 502 includes RoPE and a Causal Mask (block triangular causal mask).

[0206] It should be noted that the Transformer-based predictor 502, MLP 503, and decoder 504 are used to implement the function of the model generation network.

[0207] The core process of this interior design defect identification model is as follows: First, the multimodal encoder 501 encodes the video data, text data, document data, and voice data corresponding to the user-input interior design scheme into visual features, text features, document features, and voice features, respectively. These features are then weighted and summed based on preset weight coefficients to obtain fused features. Next, the fused features are fed into the Transformer-based predictor 502. The Transformer-based predictor 502 utilizes RoPE spatiotemporal coding and Causal... Mask performs temporal modeling and predicts latent features of specific future spatiotemporal coordinates in parallel. The input operation commands are encoded by MLP503 and interact with the latent features output by Transformer-based predictor 502 to obtain action-condition-guided latent features. Subsequently, decoder 504, based on the action-condition-guided latent features and combined with interior design expertise from a spatial scene knowledge base, constructs a high-fidelity, customized simulation scene and simultaneously outputs scene feature data. Then, interactive response network 505, based on the operation commands, obtains scene state data corresponding to the simulated scene from the user's selected perspective. Finally, defect identification network 506, using user-input design rules, identifies defects in the scene feature data and scene state data to obtain defect identification results for the interior design scheme.

[0208] This application also provides a method for identifying interior design defects, which can be applied to... Figure 3 The interior design defect identification application device shown is equipped with the interior defect identification model provided in the above embodiments.

[0209] Figure 6 This is a flowchart illustrating an interior design defect identification method provided in an embodiment of this disclosure. Figure 6 The method includes: S601. Obtain video data of the interior design scheme input by the user, the design rules corresponding to the interior design scheme, and the user's operation instructions.

[0210] S602. Input the video data, design rules and operation instructions into the interior design defect identification model to obtain the defect identification results output by the interior design defect identification model.

[0211] The interior design defect identification model can be obtained based on the training method of the aforementioned embodiments.

[0212] In some embodiments, the input video data is first encoded into visual features by the visual encoder in the interior design defect model; then the model generation network obtains the simulated scene and scene feature data corresponding to the interior design scheme based on the visual features; the interactive response network obtains the scene state data corresponding to the simulated scene under the user's selected viewpoint based on the user's operation instructions; finally, the defect recognition network performs defect recognition on the scene feature data and scene state data through design rules to obtain the final defect recognition result.

[0213] S603. Based on the defect identification results, an optimization suggestion report corresponding to the interior design scheme is obtained.

[0214] In some embodiments, standard specifications in the spatial scene knowledge base can be combined to output specific and actionable optimization suggestion reports for each defect.

[0215] This application transforms interior design schemes into immersive simulations by inputting the design information into a model. The model directly outputs the flaws of the design, converting it into an immersive environment without actual construction. It can directly recreate the layout, spatial dimensions, lighting, ventilation, furniture placement, and more, and pre-emptively identify problems in the actual design, avoiding post-construction issues that could lead to modifications, delays, and increased costs, thus improving the rationality of the interior design. Furthermore, interactive feedback generates quantifiable and traceable defect identification results and optimization suggestion reports, providing a clear basis for iterative optimization and enhancing the scientific rigor and rationality of the interior design.

[0216] The foregoing primarily describes the solutions of the embodiments of this disclosure from a methodological perspective. It is understood that the training device for the interior design defect identification model, in order to achieve the aforementioned functions, includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure.

[0217] This disclosure embodiment can divide the training device for the interior design defect identification model into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this disclosure embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the example of dividing each functional module according to each function.

[0218] Figure 7 This is a schematic diagram of the structure of a training device for an interior design defect identification model provided in an embodiment of this disclosure. Figure 7 The training device 700 for the interior design defect identification model includes an acquisition module 701 and a training module 702.

[0219] The acquisition module 701 is used to acquire training data. This training data includes video data of the interior design schemes, the corresponding design rules, and user operation instructions.

[0220] Training module 702 is used to train an interior design defect recognition model based on video data and operation instructions.

[0221] The interior design defect identification model comprises a visual encoder, a scene generation network, an interaction response network, and a defect identification network. The visual encoder encodes video data to obtain the visual features of the interior design scheme. The scene generation network, based on the visual features, generates the simulated scene and scene feature data corresponding to the interior design scheme. The interaction response network, based on operation commands, generates the scene state data corresponding to the simulated scene from the user's selected perspective. The defect identification network, based on design rules, identifies defects in the scene feature data and scene state data to obtain the defect identification results for the interior design scheme.

[0222] The training module 702 can also be referred to as a processing module. The processing module can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor can include application processors and baseband processors. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0223] Based on the functions achievable by the interior design defect identification model provided in this disclosure, this disclosure also provides an interior design defect identification device. This device includes an input parsing module, a scene generation module, an interactive response module, and a defect identification module.

[0224] The input parsing module is the entry point of this interior design defect identification device. Its core function is to receive multi-source data, text rules, and operation instructions from users on interior design schemes, and to convert them into a unified and standardized data format through parsing and fusion processing, providing reliable input for subsequent scene generation and interactive response.

[0225] The input parsing module includes video extraction, text parsing, document extraction, speech extraction, and feature fusion functions. The text parsing function transforms design rules and interior design schemes described in natural language into identifiable structured constraints, clarifying key information such as indicator types and requirement standards. The document extraction function accurately extracts core information such as spatial boundaries, object types, and size proportions from design drafts, ensuring that the generated scene aligns with the original design intent. The feature fusion function integrates the parsing results from different inputs to form unified feature data, ensuring information integrity and format compatibility.

[0226] The input parsing module can interact with the scene generation module and the interaction response module. The input parsing module can pass the fused feature data to the scene generation module, providing a foundation for scene generation; the input parsing module can also synchronize operation commands to the interaction response module, ensuring timely responses to operation commands.

[0227] The scene generation module is the core scene construction unit of this interior design defect identification device. It can construct highly reliable and customized simulation scenes based on the fusion features transmitted by the input parsing module and combined with the professional knowledge of interior design in the spatial scene knowledge base.

[0228] The scene generation module includes a scene knowledge management unit, a layered generation and decoding unit, a scene adaptation and verification unit, and a scene visualization unit.

[0229] The scene generation module first uses the scene knowledge management unit to retrieve the corresponding interior design professional knowledge from the spatial scene knowledge base, including typical layout specifications, object size ranges, and spatial relationship constraints, providing a professional basis for scene generation. Then, the layered generation and decoding unit completes the scene construction in stages. First, the overall spatial structure is generated, and then the layout of objects such as furniture and equipment is generated based on the 2008 rule and interior design professional knowledge. Then, the scene adaptation and verification unit ensures that the generated object types and scene types are compatible. Finally, the scene visualization unit transforms the abstract feature data into a simulated scene that users can directly observe, and outputs the scene feature data synchronously.

[0230] The scene generation module can interact with the input parsing module, the interaction response module, and the defect identification module. The scene generation module can receive fused feature data from the input parsing module; it can transmit scene feature data to the interaction response module in real time to support the user interaction experience; and it can also transmit scene feature data to the defect identification module to provide a scene basis for indicator extraction and defect judgment.

[0231] The interactive response module is a key unit in the interior design defect identification device to ensure the user experience. Its core function is to process user operation commands, optimize response latency and scene continuity, and ensure that users' operations such as movement and perspective changes in the simulated scene are smooth and natural.

[0232] The interactive response module includes a parsing unit, an operation prediction and candidate field of view generation unit, a viewpoint correction and optimization unit, and a real-time field of view update unit.

[0233] The interactive response module converts operation commands into standardized operation parameters that can be recognized by the interior design defect device through the parsing unit. Then, it uses keyframe judgment to form a joint decision-making mechanism by combining the operation range, the change in latent space features, and a fixed time interval. Next, the operation prediction and candidate field of view generation unit predicts the next action based on the user's historical operation data and generates candidate field of view features in advance to reduce real-time legend latency. At the same time, the viewpoint correction and optimization unit adjusts the continuity of scene features after viewpoint transformation, giving higher weight to the core areas within the user's field of view to improve observation accuracy. Finally, the real-time field of view update unit uses keyframe sampling and interpolation completion strategies to synchronize the optimized field of view features to the visualization scene, ensuring that the field of view update latency is controlled within a range imperceptible to the user.

[0234] The interactive response module can interact with the input parsing module, the scene generation module, and the defect identification module. It can receive operation commands from the input parsing module; it can receive scene feature data input from the scene generation module; and it can feed back scene state data from the user interaction process to the defect identification module, providing dynamic reference for defect identification.

[0235] The defect identification module is the core of the intelligent analysis of the interior design defect identification device. Its core function is to automatically extract key indicators of the scene, compare them with design rules, identify defects in the interior design scheme, and output positive optimization suggestions.

[0236] The defect identification module includes a scene indicator extraction unit, a rule comparison and defect judgment unit, a defect priority ranking unit, and an optimization suggestion generation unit.

[0237] The defect identification module first extracts design metrics from the scene feature data through the scene metric extraction unit, such as channel width, object spacing, and space utilization, to ensure that the design metric data is consistent with the actual data. Then, through the rule comparison and defect judgment unit, the extracted design metrics are compared one by one with the structured design rules passed by the input parsing module to determine whether there are any design problems that do not meet the requirements and to clarify the defect type and severity. Subsequently, through the defect priority ranking unit, different weights are assigned to defects based on their importance to rank all defects, ensuring that high-priority curves are given priority attention. Finally, through the optimization suggestion generation unit, combined with the standards and specifications passed in the spatial scene knowledge base, specific and actionable optimization suggestions are output for each defect.

[0238] The defect identification module can interact with the input parsing module, the scene generation module, and the interactive response module. It can receive design rules from the input parsing module, scene feature data from the scene generation module, and scene response data from the interactive response module. The module can also output defect identification results and corresponding optimization suggestions to the user (or designer), while simultaneously synchronizing these to the reviewer's interface to support solution iteration and review decisions.

[0239] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0240] This application provides a computer program product containing instructions. When the computer program product is run on an electronic device, it causes the electronic device to execute the training method for the interior design defect identification model in the above method embodiments, or to execute the interior design defect identification method in the above method embodiments.

[0241] This application also provides a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the training method for the interior design defect identification model in the method flow shown in the above method embodiments, or to perform the interior design defect identification method in the above method embodiments.

[0242] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC).

[0243] In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0244] Since the training device, computer-readable storage medium, and computer program product of the interior design defect identification model in the embodiments of the present invention can be applied to the above method, the technical effects that can be obtained can also be referred to the above method embodiments, and the embodiments of the present invention will not be repeated here.

[0245] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0246] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0247] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0248] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training an interior design defect identification model, characterized in that, The method includes: Acquire training data; the training data includes video data of the interior design scheme, the design rules corresponding to the interior design scheme, and user operation instructions. Based on the video data and the operation instructions, the interior design defect recognition model is trained; the interior design defect recognition model includes a visual encoder, a scene generation network, an interaction response network, and a defect recognition network. The visual encoder is used to encode the video data to obtain the visual features of the interior design scheme; The scene generation network is used to obtain the simulated scene and scene feature data corresponding to the interior design scheme based on the visual features; The interactive response network is used to obtain scene state data corresponding to the simulated scene from the user's selected perspective based on the operation command; The defect identification network is used to identify defects in the scene feature data and the scene state data based on the design rules, so as to obtain the defect identification results of the interior design scheme.

2. The method of claim 1, wherein, The step of training the interior design defect identification model based on the video data and the operation instructions includes: Based on the video data and the operation instructions, the interior design defect identification model is trained to obtain a multi-objective fusion loss value. The multi-objective fusion loss value is obtained by weighted summation of the feature prediction loss value, the interaction continuity loss value, and the defect identification loss value. The feature prediction loss value is used to constrain the accuracy of the visual features; the interaction continuity loss value is used to constrain the continuity of the simulated scene under the operation instructions; and the defect identification loss value is used to constrain the accuracy of defect identification. Based on the multi-objective loss value, the model parameters of the interior design defect identification model are updated.

3. The method of claim 2, wherein, The interaction continuity loss value is obtained by weighted summation of the adjacent frame continuity loss value, the viewpoint correction loss value, and the candidate view prediction loss value; the adjacent frame continuity loss value is used to constrain the smoothness of the changes in adjacent keyframes before and after the interaction; the viewpoint correction loss value is used to constrain the difference between the corrected viewpoint features and the real viewpoint features after the viewpoint transformation; the candidate view prediction loss value is used to constrain the difference between the candidate view features and the real view features. The continuity loss values ​​of adjacent frames satisfy the following relationship: wherein, denotes a loss of adjacent frame continuity value; denotes a length of the interactive keyframe sequence; denotes a number of samples; denotes a scene feature of a first frame of a sample; denotes a scene feature of a first frame of a sample; denotes a motion weight factor of a first frame of a sample; The viewpoint correction loss value satisfies the following relationship: wherein, denotes a view correction loss value; denotes a number of samples; denotes a view corrected view feature of the th sample, denotes a true view feature of the th sample; The candidate view prediction loss values ​​satisfy the following relationship: in, This represents the prediction loss value for candidate views; Indicates the first Candidate field features predicted by a lightweight network for each sample. Indicates the first The true visual field characteristics of each sample.

4. The method of claim 2, wherein, The defect identification loss value is obtained by weighted summation of the scene index regression loss value and the defect classification and priority loss value; the scene index regression loss value is used to constrain the difference between the scene index predicted value extracted by the interior design defect identification model and the scene index true labeled value; the defect classification and priority loss value is used to guide the defect identification network to learn defect classification and priority ranking. The regression loss value of the scenario indicator satisfies the following relationship: in, This represents the regression loss value of the scenario metric; Indicates the number of design parameters; Indicates the number of samples; Indicates the first The first sample The actual labeled value of each scenario indicator The first element extracted by the interior design defect identification model represents the... The first sample Predicted values ​​for indicators in each scenario; The defect classification and priority loss value satisfy the following relationship: in, This represents the loss value based on defect classification and priority. Indicates the number of defect categories; Indicates the first The first sample The actual defect label corresponding to each defect category; Indicates the first The first sample Defect prediction probability corresponding to each defect category; Indicates the first Each defect category corresponds to a defect priority weight.

5. The method of claim 1, wherein, The interior design defect identification model also includes a multimodal encoder; the multimodal encoder is used to encode the design data of the interior design scheme into design information features; The scene generation network is used to obtain the simulated scene and scene feature data corresponding to the interior design scheme based on the visual features, the design information features, and the operation instructions.

6. The method of claim 5, wherein, The design information includes at least one of text data, voice data, and document data; The multimodal encoder includes at least one of the following: A text encoder is used to encode the text data into text features; A speech encoder is used to encode the speech data into speech features; A document encoder is used to encode the document data into document features.

7. The method of claim 6, wherein, The document features are determined in the following ways: The document data is input into the document encoder to obtain geometric features and semantic features; the geometric features include spatial boundary features and structural contour features; the semantic features include object category features and spatial relationship features. The geometric and semantic features are concatenated, and the concatenated features are mapped to the latent space via linear projection to obtain document features.

8. The method of claim 1, wherein, The interior design defect identification model is equipped with a spatial scene knowledge base, which is used to provide professional interior design basis for various scene types during the training of the interior design defect identification model.

9. A method for identifying interior design defects, characterized in that, The method includes: Acquire video data of the interior design scheme input by the user, the design rules corresponding to the interior design scheme, and the user's operation instructions; The video data, the design rules, and the operation instructions are input into the interior design defect identification model to obtain the defect identification results output by the interior design defect identification model. Based on the defect identification results, an optimization suggestion report corresponding to the interior design scheme is obtained.

10. An electronic device, comprising: The electronic device includes: a display, a processor, a communication interface, and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the training method of the interior design defect identification model as described in any one of claims 1 to 8, or to implement the interior design defect identification method as described in claim 9.

Images