3D Automatic Modeling Method and System Based on AI Large Model Technology

Abstract

The present invention belongs to the technical field of digital modeling, and relates to a three-dimensional automatic modeling method and system based on AI large model technology, including: obtaining multimodal data input by a user, and converting the multimodal data into feature vectors in a unified coding format; generating a conflict arbitration result based on a priority rule preset by the user or a default physical rationality rule; according to the conflict arbitration result, generating an initial three-dimensional model through a jointly trained large language model and a neural radiance field; inputting the initial three-dimensional model into a physical simulation engine for physical verification, and outputting a corrected model containing structure defect marks; parsing the correction instruction into a model parameterization adjustment command; based on the parameterization adjustment command, performing incremental calculation and re-rendering on the associated area in the corrected model to generate a final three-dimensional model, and updating the verification result of the physical simulation engine. The present invention solves the problem that the multimodal interaction technology stays at the basic parameter adjustment level, and it is difficult to accurately convert fuzzy semantic instructions into model modification actions.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital modeling and relates to a three-dimensional automatic modeling method and system based on AI large model technology. Background Art

[0002] Existing methods mainly rely on manual intervention or multi-software collaborative processing, and there are core problems of insufficient modeling efficiency and intelligence level. Limited by the parsing ability of a single data source, when the input data contains multi-modal information such as images, texts, and videos, traditional algorithms are difficult to effectively identify the semantic associations and conflict relationships between the data, resulting in misaligned model attributes or logical contradictions.

[0003] Traditional solutions mostly focus on single-modal data processing, and solve data conflicts through rule-based feature matching or static priority configuration. The typical modeling process sets data priorities manually or uses an independent physical simulation tool for offline verification after model generation. Although it can handle simple cases, it cannot cope with dynamically changing mixed input scenarios. The phased modeling tool results in the separation of geometric modeling and physical verification, and the lag of defect feedback prolongs the design iteration cycle.

[0004] Based on the above problems, traditional multi-modal interaction technology stays at the basic parameter adjustment level. Fuzzy semantic instructions are difficult to accurately convert into model modification actions, and users need to go through multiple trials and errors to achieve the expected effect, greatly increasing the design and communication costs. Summary of the Invention

[0005] To solve the above problems, the present invention provides a three-dimensional automatic modeling method and system based on AI large model technology.

[0006] In the first aspect, the present invention provides a three-dimensional automatic modeling method based on AI large model technology, adopting the following technical solutions:

[0007] The three-dimensional automatic modeling method based on AI large model technology includes the following steps:

[0008] S1. Obtain multi-modal data input by the user, including at least two types of images, texts, and videos, and convert the multi-modal data into feature vectors in a unified encoding format;

[0009] S2. Use a dynamic arbitration mechanism to perform conflict resolution on the feature vectors. When contradictions in the content of different modal data are detected, generate a conflict arbitration result based on the priority rules preset by the user or the default physical rationality rules;

[0010] S3. According to the conflict arbitration result, generate an initial three-dimensional model through a jointly trained large language model and neural radiance field. The large language model is used to parse semantic constraint conditions, and the neural radiance field is used to reconstruct the three-dimensional volume according to the image space information;

[0011] S4. Input the initial 3D model into the physical simulation engine for real-time physical verification, including balance detection, material load-bearing simulation, and dynamic collision testing, and output a corrected model with structure defect markings;

[0012] S5. Receive dynamic correction instructions input by the user through natural language or gestures, and parse the correction instructions into model parameter adjustment commands;

[0013] S6. Based on the parameter adjustment commands, perform incremental calculation and re-rendering on the associated areas in the model to generate the final 3D model, and update the verification results of the physical simulation engine.

[0014] In a further solution of the present invention, converting multi-modal data into feature vectors in a unified coding format includes the following steps:

[0015] Input channel separation, and allocate the input stream to the corresponding parsing module according to the type of multi-modal data; data dimension alignment, and use a cross-modal embedding layer to convert features from different sources into hidden vectors of the same dimension; perform normalization fusion on the dimension-aligned feature vectors to eliminate the dimensional differences between different modal data.

[0016] In a further solution of the present invention, using a dynamic arbitration mechanism to resolve conflicts in feature vectors includes the following steps:

[0017] Calculate the credibility weights of different modalities using a predefined priority rule library, and the priority rule library includes keyword trigger rules, device type preference rules, and scene default physical rules;

[0018] According to the priority rule library, calculate the credibility weights between different modal data, and the credibility weights are positively correlated with data clarity and user instruction clarity;

[0019] When the difference in credibility weights exceeds the set threshold, automatically mask the conflicting content in the low-weight modal data, otherwise generate a visual conflict prompt interface for the user to manually select.

[0020] In a further solution of the present invention, generating an initial 3D model through a jointly trained large language model and neural radiance field includes the following steps:

[0021] Perform cross-modal attention alignment on the text semantic vector and the image feature vector to establish a weight mapping matrix of semantics and spatial features;

[0022] Use a multiple constraint loss function to jointly optimize the decoding process, and the loss function includes geometric structure loss, semantic matching loss, and physical pre-verification loss.

[0023] A further solution of the present invention is to input the initial three-dimensional model into a physical simulation engine for real-time physical verification, including the following steps:

[0024] Discretize the initial three-dimensional model into grid cells based on finite element analysis;

[0025] Apply virtual mechanical loads to each grid cell, and the load type is automatically selected according to the scenario. Gravity and wind loads are applied in the indoor scenario, and torsion and impact loads are applied in the industrial part scenario;

[0026] When the deformation amount of the model exceeds the safety threshold, a red highlight mark is generated at the corresponding grid position.

[0027] A further solution of the present invention is to parse the correction instruction into a model parameterization adjustment command, including the following steps:

[0028] Map the natural language instruction to a parameter modification type through an intention recognition model, and the parameter modification type includes dimension adjustment, material replacement, and structure addition and deletion;

[0029] Directly standardize the instruction with accurate numerical values into model parameters, and generate an interpolation-style candidate solution for the instruction with fuzzy description.

[0030] A further solution of the present invention is to perform incremental calculation on the associated area in the corrected model and re-render to generate the final three-dimensional model, including the following steps:

[0031] Delimit a local calculation area according to the influence range of the parameter adjustment command, calculate the association strength of adjacent voxels to determine the calculation boundary; set constraint equations for displacement gradient matching and strain energy conservation at the boundary to ensure the continuity of the old and new geometric structures.

[0032] A further solution of the present invention is to re-render to generate the final three-dimensional model, including the following steps:

[0033] Adopt a multi-resolution rendering strategy. First, present the calculation result with a low-precision grid, and simultaneously calculate the physical properties of the high-precision grid asynchronously. Trigger surface parameterization reconstruction to complete the final rendering after the user confirms.

[0034] A further solution of the present invention is to update the verification result of the physical simulation engine, including the following steps:

[0035] Only perform incremental finite element analysis on the calculation area and reuse the simulation data of the unmodified area;

[0036] Dynamically adjust the weight parameters in the multiple constraint loss function according to the verification result to optimize the subsequent model generation process.

[0037] In a second aspect, the present invention provides a three-dimensional automatic modeling system based on AI large model technology, adopting the following technical solutions:

[0038] A multi-modal data acquisition and conversion module, which is used to acquire at least two types of data from the images, texts, and videos input by the user and convert them into feature vectors in a unified encoding format;

[0039] A dynamic arbitration module, which is used to detect content conflicts in different modal feature vectors and generate conflict arbitration results based on the priority rules preset by the user or the default physical rationality rules;

[0040] A joint model generation module, according to the conflict arbitration result, uses the jointly trained large language model to parse semantic constraint conditions, combines the neural radiance field for three-dimensional volume reconstruction of image spatial information, and outputs an initial three-dimensional model;

[0041] A physical simulation verification module, which is used to input the initial three-dimensional model into a physical simulation engine for real-time physical verification, including balance detection, material load-bearing simulation, and dynamic collision testing, and outputs a corrected model with structure defect marks;

[0042] A dynamic correction instruction parsing module, which is used to receive dynamic correction instructions input by the user through natural language or gestures and parse the correction instructions into model parameterized adjustment commands;

[0043] An incremental solution and rendering module, based on the parameterized adjustment command, performs incremental solution on the associated areas in the corrected model and re-renders to generate the final three-dimensional model, and updates the verification result of the physical simulation engine.

[0044] In summary, the present invention includes the following beneficial technical effects:

[0045] 1. Significantly improve the modeling efficiency and reliability in complex scenarios. Through the dynamic arbitration mechanism, it intelligently analyzes potential conflicts in multi-modal input data, automatically applies preset priority rules or physical laws for content adjudication, and effectively avoids the delay and error of manual intervention in decision-making; when there are attribute contradictions between the uploaded image and text description, arbitration results are generated based on quantitative indicators such as data clarity and user instruction clarity, improving the fusion accuracy of cross-modal data;

[0046] 2. Integrate the large language model and the neural radiance field to achieve joint modeling of semantics and space, ensuring double compliance of the generated three-dimensional model in terms of geometric accuracy and semantic consistency. The real-time embedding of the physical simulation engine can identify structural defects in the initial modeling stage in advance;

[0047] 3. In the interactive optimization phase in response to user feedback, an incremental solution and multi-resolution rendering strategy are adopted to achieve efficient response to local modifications and resource optimization. When the user adjusts the model parameters through natural language instructions, only the associated area needs to be resolved instead of reconstructing the entire model. Combining real-time preview of low-precision meshes and asynchronous high-precision calculations, the time-consuming for dynamic correction of complex models is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. The drawings are used to provide a further understanding of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 The flowchart of the 3D automatic modeling method based on the AI large model technology is disclosed.

[0050] Figure 2 The schematic diagrams of the comparison of the original feature distribution and the normalized feature distribution are disclosed.

[0051] Figure 3 The schematic diagram of the structural deformation simulation under finite element analysis is disclosed.

[0052] Figure 4 The schematic diagram of the style transfer optimization curve is disclosed.

[0053] Figure 5 The structural schematic diagram of the 3D automatic modeling system based on the AI large model technology is disclosed. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0055] The following will Figures 1 - 5 make a preferred and detailed description of the present invention.

[0056] Referring to the Figure 1 figures shown, the present invention provides a 3D automatic modeling method based on the AI large model technology, including the following steps:

[0057] S1. Obtain multi-modal data input by the user, including at least two types of image, text, and video, and convert the multi-modal data into feature vectors in a unified encoding format;

[0058] S2. Use a dynamic arbitration mechanism to perform conflict resolution on the feature vectors. When contradictions in the content of different modality data are detected, generate a conflict arbitration result based on the user-predefined priority rules or the default physical rationality rules;

[0059] S3. According to the conflict arbitration result, generate an initial 3D model through a jointly trained large language model and a neural radiance field. The large language model is used to parse semantic constraint conditions, and the neural radiance field is used to reconstruct the 3D volume according to the image space information;

[0060] S4. Input the initial 3D model into a physical simulation engine for real-time physical verification, including balance detection, material load-bearing simulation, and dynamic collision testing, and output a corrected model with structure defect markings;

[0061] S5. Receive dynamic correction instructions input by the user through natural language or gestures, and parse the correction instructions into model parameterization adjustment commands;

[0062] S6. Based on the parameterization adjustment commands, perform incremental calculation on the associated areas in the model and re-render to generate the final 3D model, and update the verification result of the physical simulation engine.

[0063] In one embodiment of the present invention, step S1 includes the following steps:

[0064] Specifically, when the user uploads multi-modal data through a terminal device, the initial processing of the multi-modal data is completed through the following process;

[0065] 1. Input channel separation. According to the type of multi-modal data, allocate the input stream to the corresponding parsing module. For example, image data enters a convolutional neural network to extract spatial features, text data is input into a large language model to extract semantic keywords, and video data is frame-by-frame decomposed and then time series analysis and key frame extraction are performed synchronously;

[0066] 2. Data dimension alignment. Use a cross-modal embedding layer to convert features from different sources into hidden vectors of the same dimension. Text features are compressed into 256-dimensional vectors through attention pooling, image features are reduced to 256 dimensions through global average pooling, and video features are output as 256-dimensional vectors after aggregating time information by a bidirectional long short-term memory network;

[0067] 3. Refer to Appendix Figure 2 , feature normalization and fusion. Perform a linear transformation on the above vectors to satisfy the following formula to achieve dimension unity, , where represents the th normalized feature vector, complete the dimensionless processing, and provide a numerical consistency basis for subsequent conflict resolution. represents the An input feature vector, representing the original data features from different modalities, such as the convolutional features of an image or the semantic embedding vector of text. Represents the mean of the data distribution, calculated by statistically analyzing the feature vectors of all training data or the current processing batch. Represents the standard deviation of the feature data, reflecting the degree of dispersion of the feature values, and is used to eliminate the influence of different measurement units on subsequent processing. Represents a very small constant to prevent division-by-zero errors, and tries to avoid division-by-zero errors when the standard deviation approaches zero.

[0068] Multimodal data, including input forms with more than two interaction media. For example, in an architectural design scenario, a user simultaneously uploads a photo of a room (image), orally describes renovation requirements (text), and takes a panoramic video of the site (video).

[0069] Unified coding format, which is the process of converting heterogeneous data into the same mathematical expression. It realizes the conversion from unstructured data to vectorized representation through the embedding layer of a deep learning model, eliminating the interference of modal differences on subsequent processing.

[0070] Feature vector, a high-dimensional numerical representation abstracted by a neural network. For example, the text description "round glass table" is converted into a 256-dimensional real-valued vector containing shape and material attributes.

[0071] Exemplarily, assume that the user provides a photo of a living room and a voice command "Change the wall to light gray and keep the original wooden floor", and the following process is executed:

[0072] The RGB value of the wall color is identified in the image data as , the floor material texture conforms to the characteristics of wood; after the voice command is converted into text by voice recognition, the keywords "wall", "light gray", and "keep the original wooden floor" are extracted; the weight of the image feature vector , the weight of the text feature vector , and a synthetic vector pointing to "light gray wall + original wooden floor" is generated after weighted fusion.

[0073] In one embodiment of the present invention, step S2 includes the following steps:

[0074] Establish a priority rule library, including keyword trigger rules for user voice commands, device type preference rules, and scene default physical rules.

[0075] Specifically, the priority rule library represents a preset rule set for arbitrating multimodal data conflicts and includes the following core modules.

[0076] Keyword triggering rule, which recognizes the enhanced expression of the user's instruction through natural language processing. For example, when the voice instruction contains "must be retained" or "subject to the text", the priority of the corresponding modality is automatically increased; the keyword triggering rule is jointly implemented through a preset keyword list and a real-time intent recognition model.

[0077] Device type preference rule, which assigns weights according to the characteristics of the hardware of the data source. For example, a wide-angle video taken by a mobile phone is more suitable for spatial scale modeling than a text description, while an image taken by a professional single-lens reflex camera has a higher priority in terms of material details than a sketch.

[0078] Scene default physical rule, which embeds strong constraint conditions of industry specifications. For example, the position of load-bearing walls cannot be modified in a building scene, and the mechanical movement trajectory in an industrial scene must conform to mechanical principles.

[0079] Among them, natural language processing is a technology that converts the voice or text input by the user into structured semantic tags. The real-time intent recognition model is a deep learning model based on the attention mechanism, which is used to extract the core requirements and limiting conditions of the user's instruction.

[0080] Exemplarily, when the user uploads a photo of a bedroom taken by a mobile phone and sends a voice instruction "change this bed to a round one", the following process is executed:

[0081] The keyword triggering rule detects that "change to" in the voice is a mandatory action instruction and sets the priority of the text modality to the highest. The device type preference rule recognizes that the photo is taken by a mobile phone and there may be perspective distortion, and preferentially adopts the bed shape described in the text. The scene default physical rule verifies whether the diameter of the round bed meets the remaining space in the bedroom. If it cannot be satisfied, a secondary confirmation is triggered.

[0082] Calculate the credibility weights of different modality data according to the priority rule library. The calculation of the credibility weights satisfies the following formula: , where represents the credibility weight of the th modality. The larger the value, the higher the priority of the modality in the conflict adjudication. represents the data clarity score, which is quantified by objective indicators such as image resolution, text statement integrity, and video frame rate. For example the image clarity is , and the blurred image is . represents the user instruction clarity score, which is calculated by analyzing the number of specific parameters included in the instruction. For example, the text description "a solid wood table with a length of 2 meters and a width of 1.5 meters" scores , and only the description "a big table" scores . Indicates the scene adaptation score, which is set according to the matching degree between the data modality and the current modeling field. For example, in the mechanical design scene, the adaptation degree of engineering drawings is , and the adaptation degree of hand-drawn sketches is . , , Indicates the dynamic adjustment coefficient, which is a weighted parameter that changes in real time according to the conflict scene type and is determined by the real-time decision result of the priority rule library. For example, when the keyword trigger rule takes effect, the value is increased to to strengthen the influence of the user's instruction.

[0083] When the credibility weight difference exceeds the set threshold, the low-weight conflict content is automatically blocked; otherwise, a visual conflict prompt interface is generated;

[0084] Specifically, the threshold is set using the dynamic double-interval determination method, which satisfies the following conditions:

[0085] The first threshold (absolute threshold), when the weight of a certain modality is higher than this value and exceeds twice the weight of other modalities, directly blocks the low-weight conflict content;

[0086] The second threshold (relative threshold), when the weights of all modalities are lower than the first threshold but the difference between the highest weight and the second-highest weight is greater than , arbitration is automatically executed;

[0087] If the above conditions are not met, a conflict prompt interface is generated. The conflict prompt interface displays the conflict area in a comparison view and provides three options: "Force Application", "Compromise Solution", and "Manual Adjustment".

[0088] Among them, the dynamic double-interval determination method represents a decision-making mechanism that adaptively selects an arbitration strategy according to the weight distribution. The comparison view is an interactive interface that superimposes different modality data and marks the contradictory areas with a semi-transparent mask and a highlighted frame.

[0089] Exemplarily, the user inputs the following data to generate a clothing store display rack.

[0090] Video shooting: "Show a metal shelf with an L-shaped layout", text description: "Need a U-shaped wooden shelf for customers to browse around conveniently". According to the calculation of the credibility weight, the video , the text , the weight difference between the credibility weights of the video and the text is , the first threshold (absolute threshold) is set to , and the second threshold (relative threshold) is set to .

[0091] The weight difference is lower than the second threshold , generate a conflict prompt interface: the three-dimensional model of the L-shaped metal shelf of the video is displayed in the left area; the simulated effect of the U-shaped wooden shelf described by the text is displayed in the right area; the bottom toolbar provides three options: "retain the video structure and use wooden materials instead", "rebuild the shelf according to the text description", and "hybrid scheme: U-shaped metal shelf".

[0092] In one embodiment of the present invention, step S3 includes the following steps:

[0093] Cross-modal attention alignment is performed between the text semantic output vector and the image feature vector;

[0094] Specifically, during the joint training process, the text vector obtained by parsing the text semantics and the extracted image feature vector are subjected to cross-modal feature association. A weight mapping matrix between the text vector and the image vector is established through the self-attention mechanism, and the interaction weights of different modal features are assigned. In the feature dimension alignment stage, the multi-head attention mechanism is used to calculate the spatial matching degree of the text and image features, and a fused joint feature vector is generated. Cross-modal attention alignment establishes a semantic association relationship between different modal data through a weight assignment mechanism, and uses attention scores to measure the spatial matching priority of different modal features.

[0095] Exemplarily, input the text data "a modern-style chair with an arc-shaped waistline" and the corresponding multi-angle chair frame images. The text data is input into the large language model to extract semantic keywords, and semantic encodings of "arc, modern style, chair frame structure" are obtained. The image feature vector extracts edge contour features from the side view of the chair. During the cross-modal attention alignment, the weight matrix of the text data and the image features is calculated, and it is found that the weight of the "arc" semantics and the curve part of the image features is the highest, and the weight of the "chair frame structure" semantics and the support frame of the image features is the second. Finally, a joint vector containing high-weight features is generated by fusion.

[0096] The neural network decoding layer sets a multi-constraint loss function, and the loss function includes a geometric structure loss, a semantic matching loss, and a physical pre-verification loss;

[0097] Specifically, in the decoding stage of the large language model, by defining a geometric structure loss function to constrain the topological integrity of the generation model, a semantic matching loss to ensure the consistency between the model attributes and the input semantics, and a physical pre-verification loss to optimize in advance the local structures that are likely to cause physical failures. The three are combined with dynamic weights in the total loss and satisfy the following formula, , specifically, represents the semantic matching loss, which takes the negative value of the cosine similarity between the text vector and the model attribute vector. represents the physical pre-verification loss, which is a penalty term generated based on the deformation amount of the finite element pre-analysis model. represents the total loss function. Represents the geometric structure loss, specifically the distance , and are the predicted point cloud and the real point cloud respectively. , , represent hyperparameters that are dynamically adjusted.

[0098] Among them, the distance is a metric for measuring the maximum and minimum distances between two point sets in three-dimensional space, used to evaluate the geometric error on the surface of the generation model. Finite element pre-analysis is a simulation method in which the model is discretized into mesh elements and the stress distribution is pre-calculated, used to identify high-stress areas in advance during the training phase.

[0099] Exemplarily, training to generate a three-dimensional model of a robotic arm part: the geometric structure loss, constraining the surface profile error between the generated part and the original design drawing to be less than 2 mm;

[0100] the semantic matching loss, ensuring that the matching degree between the "hinged structure" attribute in the model and the description of "rotatable joint" in the text input exceeds 90%;

[0101] the physical pre-verification loss, discovering stress concentration problems at the joints during the training phase and adjusting the part thickness in advance to reduce the stress peak by 40%.

[0102] In one embodiment of the present invention, step S4 includes the following steps:

[0103] Based on finite element analysis, discretize the initial three-dimensional model into mesh elements;

[0104] Specifically, perform discretization processing on the geometric surface and internal structure of the initial three-dimensional model, divide the model using tetrahedral or hexahedral mesh elements, ensure that the vertex coordinates of each unit are aligned with the feature points of the original model, and increase the mesh density in stress concentration areas (joints, bending parts, etc.). The side length of the mesh element is set according to the accuracy requirements of the initial three-dimensional model. The default side length for indoor scenes is 5 mm, and it is reduced to 1 mm for industrial part scenes.

[0105] Among them, finite element analysis represents a method of decomposing a continuous geometric body into a finite number of mesh elements, and predicting the structural strength by calculating the stress and strain of each element.

[0106] A mesh element represents a closed polyhedron structure composed of geometric vertices and edges, used to approximately express the mechanical behavior of complex shapes.

[0107] Exemplarily, assuming that when a household kitchen island model is meshed, it is detected that the connection between the countertop and the support column is a stress concentration area, and the mesh side length of this area is automatically compressed from 5 mm to 2 mm, while the adjacent areas maintain the original density.

[0108] Virtual mechanical loads are applied to each mesh element. The virtual mechanical load is a set of equivalent external forces applied to the mesh element, which is used to simulate the stress conditions under real working conditions. The type of virtual mechanical load is automatically selected according to the scenario: gravity and wind loads are applied in the indoor scene, and torsion and impact loads are applied in the industrial part scene;

[0109] Specifically, for the indoor furniture model, a vertically downward gravity load is applied to each mesh element. The magnitude of the gravity load is calculated by the product of the volume of the network element, the material density, and the acceleration due to gravity, and satisfies the following formula, , where, represents the gravity load vector; represents the volume of the mesh element, and the volume of the tetrahedron or hexahedron is calculated from the vertex coordinates; represents the material density; represents the acceleration due to gravity; for the outdoor building model, a horizontal wind load is superimposed on each mesh element, and the wind pressure distribution is calculated using the fluid dynamics formula.

[0110] In the industrial part scene, a torque load is applied to parts such as gears, and satisfies the following formula, , where, represents the magnitude of the torque, represents the material torsional stiffness coefficient, represents the applied rotation angle; the impact load needs to be simulated by an instantaneous velocity change, for example, the momentum impact generated by hitting at speed.

[0111] Referring to Appendix Figure 3 , when the deformation of the initial three-dimensional model exceeds the safety threshold, a red highlight mark is generated at the corresponding mesh position;

[0112] Specifically, through the finite element solver, the displacement of each mesh node under the action of the load is calculated, and the deformation is the maximum absolute value of the node displacement. When the deformation exceeds the preset safety threshold (the allowable deformation threshold for indoor furniture materials is of the length dimension, and for steel parts it is ), the alarm mechanism is triggered. In the three-dimensional visualization interface, the over-limit elements are assigned a red highlight material, and the transparency of the material is adjusted according to the severity of the deformation. The greater the deformation, the deeper the red color; a defect report is generated, recording the coordinates of the over-limit elements, the deformation values, and the recommended corrective measures.

[0113] Among them, the safety threshold, which is the maximum allowable deformation of the material under safe working conditions, is preset according to the material mechanics performance table. The red highlight mark represents a visual warning identifier, which is used to guide the user to quickly locate the problem area.

[0114] Exemplarily, when testing the solid wood dining table model of pine wood material, it is detected that the deformation of the grid unit in the middle of the table leg under the load is , and the threshold of the pine wood material is set to . The deformation of the grid unit in the middle of the table leg under the load exceeds the threshold of the pine wood material, and the corresponding area is marked dark red, prompting "It is recommended to add a support crossbeam or replace it with a high-density material".

[0115] In one embodiment of the present invention, step S5 includes the following steps:

[0116] Mapping the natural language instruction to a parameter modification type through an intent recognition model, and the types include size adjustment, material replacement, and structural addition and deletion;

[0117] Specifically, mapping the natural language instruction to a parameter modification type through an intent recognition model includes the following steps:

[0118] The user inputs an instruction in natural language form, such as "Increase the width of the sofa by 20%" or "Replace the wall material with marble", receives the instruction through the speech recognition module or text interface, and performs word segmentation and syntax analysis;

[0119] Use the intent recognition model to parse the preprocessed text. The intent recognition model is trained based on a multi-layer architecture, including a semantic encoder and a classifier module. The encoder converts the input statement into a high-dimensional semantic vector, and the classifier outputs a probability distribution according to the preset modification type labels;

[0120] When the highest probability label output by the classifier belongs to the preset parameter modification type (size adjustment, material replacement, structural addition and deletion), a corresponding modification type code is generated. For a multi-intent mixed statement (such as "Widen the sofa and replace the wood material"), the intent recognition model supports multiple label classification outputs.

[0121] Among them, the parameter modification type represents a set of predefined operation labels. Size adjustment involves changes in the geometric dimensions of the model, material replacement corresponds to modifications of the surface physical properties, and structural addition and deletion refer to topological addition and deletion of the model components. The intent recognition model represents a natural language processing model based on architecture, which is fine-tuned through a large amount of dialogue data in the design field and is used to analyze the core operation intent of the user instruction and extract quantifiable parameters.

[0122] Exemplarily, when the user inputs "Lower the desk height to 75 cm", keywords "Lower", "height", and "75 cm" are extracted. The semantic vector has a matching degree of 0.93 (the highest probability label) with the dimension adjustment type feature, generating a modification type code: DIMENSION-ADJUST, and extracting the target parameter value of 75 cm.

[0123] If the instruction contains a clear numerical description, it is directly converted into an absolute parameter value; if the instruction contains a fuzzy numerical description, a style transfer model is called to generate a set of candidate solutions.

[0124] Specifically, for instructions with clear numerical descriptions (such as "Shrink by 20%" and "Rotate 45 degrees"), the numbers and units are extracted through regular expression matching and standardized into the unit system of the modeling software. For example, "Shrink by 20%" is converted into a scaling factor of 0.8, and "Rotate 45 degrees" is converted into a radian value. For instructions lacking precise numerical descriptions (such as "A more retro look" and "Increase the modern feel"), the style transfer model is launched. Using the adaptive instance normalization technique, the target style is encoded as a direction vector in the feature space. The feature vector of the current model and the style vector are linearly interpolated to generate five intermediate results with different weight ratios for the user to interactively select.

[0125] Among them, the style transfer model refers to a visual conversion model based on the generative adversarial network (GAN), and its hidden layer activation value guiding matrix controls the stylization intensity of materials, colors, and textures. Adaptive instance normalization refers to a technique for dynamically adjusting the input data distribution, which decouples the style features and content features by statistically calculating the feature mean and variance.

[0126] Referring to the appendix Figure 4 , the calculation of the style interpolation equation satisfies the following formula. Among them, represents the output result feature vector, which is superimposed by the difference between the current model feature and the target style feature after weight correction, representing the new state representation of the model after style mixing. represents the current model feature vector, which is the three-dimensional model hidden feature representation extracted through a deep neural network, including the encoded information of geometric structure, material attributes, and light response, and its dimension is determined by the network architecture. represents the feature vector of the target style, which is generated by aggregating the features of a specific style (such as "retro" and "modern") in the pre-trained style library. represents the style mixing coefficient, whose value range is between 0 and 1, and determines the penetration intensity of the target style feature into the original model. When it is 0, the output completely retains the original model features. When it is 1, it is completely replaced with the target style.

[0127] Exemplarily, the user adjusts the industrial part model from the "ordinary steel" style (corresponding to ) to the "polished metal" style (corresponding to ), sets = 0.6, and follows the following process:

[0128] Calculate the difference direction , and the vector includes characteristic change trends such as enhanced metal reflectivity and reduced surface roughness. Apply 60% of the difference amount, and the specular reflection intensity of the original model is increased to 60% of the target value, while retaining 40% of the diffuse reflection attributes of the original material. The parts in the output result model show a semi-polished effect.

[0129] In one embodiment of the present invention, step S6 includes the following steps:

[0130] Delimit the boundary of the local solution area according to the influence range of the parametric adjustment command;

[0131] Specifically, when the user triggers the model modification instruction through natural language or the interaction interface, analyze the set of geometric voxels affected by the modification operation, and determine the solution boundary through the correlation strength calculation formula. The correlation strength is positively correlated with the force conduction coefficient between adjacent voxels and the material connection method. After the boundary is delimited, lock the topological structure of the non-solution area;

[0132] Among them, the boundary of the local solution area is a dynamic range that expands outward with the action point of the modification instruction as the center. The voxels closer to the action point are more sensitive to the adjustment instruction; the calculation of the correlation strength satisfies the following formula, , where represents the correlation strength between adjacent voxels, represents the difference in displacement of adjacent voxels, represents the material elastic modulus adjustment coefficient, represents the voxel spacing.

[0133] Set a transition constraint equation at the boundary to ensure the continuity of the new and old geometric structures;

[0134] Specifically, establish displacement and stress coordination conditions on the boundary surface between the local solution area and the non-solution area, and use a two-parameter constraint equation to ensure a smooth transition between the modified area and the original model, satisfying the following equation,

[0135] Equation 1, the gradient matching condition, requires that the displacement gradient vector at the boundary remains consistent inside and outside the solution area, , represents the gradient vector of displacement, represents the internal displacement of the solution area, Indicates the displacement of the external original model;

[0136] Equation two, the energy conservation condition, restricts the rate of change of strain energy at the boundary not to exceed a preset threshold. , Indicates the strain energy on the boundary. Indicates the time of change of strain energy. Indicates that the maximum allowable rate of energy change of the material is the preset threshold, which is dynamically set according to the structural safety standard.

[0137] Among them, the displacement gradient vector is a vector index describing the direction and intensity of geometric deformation, and its dimension is consistent with the model space coordinate system; the strain energy is the elastic potential energy stored during the deformation of the material, and is obtained by integrating the product of the principal stress and principal strain of all boundary elements.

[0138] Exemplarily, when the user shortens the length of the table leg in the furniture model from 1 meter to 0.8 meters, the above constraint equation is applied to the boundary where the table leg is connected to the tabletop to ensure that no abnormal deformation occurs in the tabletop support structure.

[0139] Adopt a multi-resolution rendering strategy, first present the solution results with a low-precision grid, and perform full-precision optimization after the user confirms.

[0140] Specifically, generate a three-level grid hierarchy for the local solution area: the base layer is a low-face-count grid discretized by an octree, the middle layer uses a quadrilateral subdivision transition structure, and the detail layer retains the NURBS surface parameters of the original model; when rendering, the base layer grid is preferentially displayed, and at the same time, the physical properties of the detail layer are calculated asynchronously in the background; after the user confirms the modification plan, perform surface parameterization reconstruction to complete full-precision rendering.

[0141] Among them, octree discretization represents a data structure that recursively divides three-dimensional space into multiple cubic units, and each cubic unit corresponds to a grid vertex. NURBS surface parameters represent the mathematical expression form of non-uniform rational B-splines, which are used to accurately describe the shape of complex surfaces.

[0142] Exemplarily, when the user adjusts the diameter of the round hole of an industrial part, first display a low-precision ring composed of 24 triangular faces. After the user confirms that the size is correct, then restore the real engineering precision model composed of 256 faces. Hierarchical rendering reduces real-time interaction latency, while retaining high-fidelity details of the final output, and avoiding waste of computing resources during repeated adjustment processes.

[0143] See Appendix Figure 5 As shown, the present invention also proposes a three-dimensional automatic modeling system based on AI large model technology, including the following modules:

[0144] The multimodal data acquisition and conversion module is used to acquire at least two types of data from the images, texts, and videos input by the user and convert them into feature vectors in a unified coding format;

[0145] The dynamic arbitration module is used to detect content conflicts in feature vectors of different modalities and generate conflict arbitration results based on the priority rules preset by the user or the default physical rationality rules;

[0146] The joint model generation module, according to the conflict arbitration results, parses the semantic constraint conditions through a jointly trained large language model, combines the neural radiance field for three-dimensional volume reconstruction of image spatial information, and outputs an initial three-dimensional model;

[0147] The physical simulation verification module is used to input the initial three-dimensional model into a physical simulation engine for real-time physical verification, including balance detection, material load-bearing simulation, and dynamic collision testing, and outputs a corrected model with structure defect marks;

[0148] The dynamic correction instruction parsing module is used to receive dynamic correction instructions input by the user through natural language or gestures and parse the correction instructions into model parameterized adjustment commands;

[0149] The incremental solution and rendering module, based on the parameterized adjustment commands, performs incremental solution on the associated areas in the model and re-renders to generate the final three-dimensional model, and updates the verification results of the physical simulation engine.

[0150] Each of the above-mentioned modules can be implemented in whole or in part by software, hardware, and their combinations, supporting being embedded in the processor of a computer device in hardware form or being independent of it, and also supporting being stored in the memory of the computer device in software form for the processor to call and execute the operations corresponding to each of the above-mentioned modules.

[0151] It should be noted that the human body information (including but not limited to human body device information and personal information, etc.) and data (including but not limited to data for analysis, stored data, and displayed data, etc.) involved in the present invention are all information and data authorized by the human body or fully authorized by all parties. The collection, use, and processing of relevant data require relevant legal standards.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A three-dimensional automatic modeling method based on AI large model technology, characterized in that It includes the following steps: S1. Obtain multimodal data input by the user, including at least two types of images, texts, and videos, and convert the multimodal data into feature vectors in a unified encoding format; S2. Use a dynamic arbitration mechanism to resolve conflicts in the feature vectors. When content contradictions in different modal data are detected, generate a conflict arbitration result based on the priority rules preset by the user or the default physical rationality rules; S3. According to the conflict arbitration result, generate an initial 3D model through a jointly trained large language model and a neural radiance field. The large language model is used to parse semantic constraint conditions, and the neural radiance field is used to reconstruct the 3D volume according to the image space information; S4. Input the initial 3D model into a physical simulation engine for real-time physical verification, including balance detection, material load-bearing simulation, and dynamic collision testing, and output a corrected model with structural defect markings; S5. Receive dynamic correction instructions input by the user through natural language or gestures, and parse the correction instructions into model parameterization adjustment commands; S6. Based on the parameterization adjustment command, perform incremental calculation and re-rendering on the associated areas in the corrected model to generate the final 3D model, and update the verification result of the physical simulation engine.

2. The three-dimensional automatic modeling method based on the AI large model technology according to claim 1, characterized in that Converting the multimodal data into feature vectors in a unified encoding format includes the following steps: Input channel separation, allocate the input stream to the corresponding parsing module according to the type of multimodal data; data dimension alignment, use a cross-modal embedding layer to convert features from different sources into hidden vectors of the same dimension; perform normalization fusion on the dimension-aligned feature vectors to eliminate the dimensional differences between different modal data.

3. The three-dimensional automatic modeling method based on the AI large model technology according to claim 1, characterized in that, Using the dynamic arbitration mechanism to resolve conflicts in the feature vectors includes the following steps: A predefined priority rule library calculates the credibility weights of different modalities. The priority rule library includes keyword trigger rules, device type preference rules, and scene default physical rules; According to the priority rule library, calculate the credibility weights between different modal data. The credibility weights are positively correlated with data clarity and user instruction clarity; When the difference in credibility weights exceeds the set threshold, automatically mask the conflicting content in the low-weight modal data, otherwise generate a visual conflict prompt interface for the user to manually select.

4. The three-dimensional automatic modeling method based on the AI large model technology according to claim 1, wherein, Generating the initial 3D model through a jointly trained large language model and a neural radiance field includes the following steps: Perform cross-modal attention alignment on the text semantic vector and the image feature vector to establish a weight mapping matrix of semantic and spatial features; Use a multiple constraint loss function to jointly optimize the decoding process. The loss function includes geometric structure loss, semantic matching loss, and physical pre-verification loss.

5. The three-dimensional automatic modeling method based on the AI large model technology according to claim 4, wherein Inputting the initial 3D model into a physical simulation engine for real-time physical verification includes the following steps: Discretize the initial 3D model into grid cells based on finite element analysis; Apply virtual mechanical loads to each grid cell. The load type is automatically selected according to the scene. Gravity and wind loads are applied in the indoor scene, and torsion and impact loads are applied in the industrial part scene; When the model deformation amount exceeds the safety threshold, generate a red highlight mark at the corresponding grid position.

6. The three-dimensional automatic modeling method based on the AI large model technology according to claim 5, characterized in that, Parsing the correction instruction into a model parameterization adjustment command includes the following steps: Map the natural language instructions to parameter modification types through the intent recognition model. The parameter modification types include size adjustment, material replacement, and structural addition or deletion. Directly standardize the instructions with exact numerical values into model parameters, and generate interpolation-style candidate solutions for instructions with fuzzy descriptions.

7. The three-dimensional automatic modeling method based on the AI large model technology according to claim 6, wherein Perform incremental solution for the associated regions in the model and re-render to generate the final 3D model, including the following steps: Define the local solution region according to the influence range of the parameter adjustment command, calculate the association strength of adjacent voxels to determine the solution boundary; set the constraint equations of displacement gradient matching and strain energy conservation at the boundary to ensure the continuity of the old and new geometric structures.

8. The three-dimensional automatic modeling method based on the AI large model technology according to claim 7, wherein Re-render to generate the final 3D model, including the following steps: Adopt a multi-resolution rendering strategy. First, present the solution results with a low-precision mesh, and simultaneously calculate the physical properties of the high-precision mesh asynchronously. Trigger surface parameterization reconstruction to complete the final rendering after the user confirmation.

9. The three-dimensional automatic modeling method based on the AI large model technology according to claim 8, wherein, Update the verification results of the physical simulation engine, including the following steps: Only perform incremental finite element analysis on the solution region and reuse the simulation data of the unmodified region; Dynamically adjust the weight parameters in the multiple constraint loss function according to the verification results to optimize the subsequent model generation process.

10. A three-dimensional automatic modeling system based on AI large model technology, characterized in that, Include the following modules: Multi-modal data acquisition and conversion module, which is used to acquire at least two types of data from the images, texts, and videos input by the user and convert them into feature vectors in a unified coding format; Dynamic arbitration module, which is used to detect the content conflicts of feature vectors in different modalities and generate conflict arbitration results based on the priority rules preset by the user or the default physical rationality rules; Joint model generation module, according to the conflict arbitration results, parse the semantic constraint conditions by the jointly trained large language model, and combine the neural radiance field to perform three-dimensional volume reconstruction of the image space information, and output the initial 3D model; Physical simulation verification module, which is used to input the initial 3D model into the physical simulation engine for real-time physical verification, including balance detection, material load-bearing simulation, and dynamic collision test, and output the corrected model with structural defect marks; Dynamic correction instruction parsing module, which is used to receive the dynamic correction instructions input by the user through natural language or gestures and parse the correction instructions into model parameterization adjustment commands; Incremental solution and rendering module, based on the parameterization adjustment command, perform incremental solution for the associated regions in the model and re-render to generate the final 3D model, and update the verification results of the physical simulation engine.

Images