A method, device, medium and product for maintenance auxiliary decision generation

Abstract

The application discloses a maintenance auxiliary decision generation method, device, medium and product, relates to the field of industrial equipment maintenance, and comprises the following steps: a semantic-driven large language model is used to analyze input natural language instructions to obtain a standardized semantic vector; a multi-modal data fusion mechanism is used to perform similarity calculation on the standardized semantic vector and a three-dimensional model index in a bottom knowledge graph to obtain a target three-dimensional model resource; interactive visualization processing is performed on the target three-dimensional model resource to obtain a visualized three-dimensional model; and bidirectional linkage of the three-dimensional model and maintenance knowledge is realized based on a multi-modal data binding mechanism according to the visualized three-dimensional model to generate a maintenance auxiliary decision. The maintenance auxiliary decision comprises strongly associated maintenance knowledge and decision suggestions, and the application improves the spatial perception ability, decision accuracy and overall maintenance efficiency of workers in a complex maintenance scene.

Description

Technical Field

[0001] This application relates to the field of industrial equipment maintenance, and in particular to a maintenance auxiliary decision generation method, equipment, medium, and product. Background Technology

[0002] As industrial equipment becomes increasingly complex, on-site maintenance tasks place extremely high demands on operators' decision-making speed. However, existing maintenance decision support systems suffer from the following technical problems in practical applications: First, existing 3D model visualization tools and knowledge base systems are often isolated from each other. The retrieval methods are usually limited to literal keyword matching, lacking deep semantic understanding of users' natural language questions. This makes it difficult to achieve accurate mapping from "fault phenomenon description" to "specific 3D component", resulting in maintenance personnel facing serious "interaction gaps" and information lag.

[0003] Secondly, the existing system lacks an effective multimodal data binding mechanism, which cannot provide related maintenance parameters, expert experience or tool usage instructions at the same time as the user operates the 3D model. This results in a serious disconnect between textual and graphical information and the 3D model, making it difficult to form an intuitive and coherent maintenance auxiliary decision-making loop. As a result, the decision accuracy is low, and in complex maintenance scenarios, it is easy to cause maintenance personnel to have difficulty in spatial imagination, making it easy to make mistakes in judgment or operation, resulting in low overall maintenance efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a maintenance auxiliary decision generation method, equipment, medium, and product to solve the problems of low spatial perception ability, low decision accuracy, and low overall maintenance efficiency of operators in complex maintenance scenarios.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a maintenance-aided decision generation method, including: The input natural language instructions are parsed using a semantically driven large language model to obtain standardized semantic vectors; By using a multimodal data fusion mechanism, the standardized semantic vector is compared with the three-dimensional model index in the underlying knowledge graph to obtain the target three-dimensional model resource. The target 3D model resource is subjected to interactive visualization processing to obtain a visualized 3D model; Based on the visualized 3D model, and using a multimodal data binding mechanism, a two-way linkage between the 3D model and maintenance knowledge is achieved to generate maintenance auxiliary decisions; the maintenance auxiliary decisions include strongly correlated maintenance knowledge and decision suggestions.

[0006] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described maintenance assistance decision generation method.

[0007] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described maintenance assistance decision generation method.

[0008] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned maintenance assistance decision generation method.

[0009] According to the specific embodiments provided in this application, this application has the following technical effects: This application utilizes a semantically driven large language model to perform deep parsing of input natural language commands to obtain standardized semantic vectors. Combined with a multimodal data fusion mechanism, it calculates similarity with the 3D model index in the underlying knowledge graph, effectively overcoming the limitations of traditional keyword matching. This achieves precise mapping from fault phenomenon descriptions to specific 3D components, eliminating information lag and interaction gaps. Furthermore, this application performs interactive visualization processing on the matched target 3D model resources to obtain a visualized 3D model. Based on a multimodal data binding mechanism, it achieves bidirectional linkage between the 3D model and maintenance knowledge, enabling maintenance personnel to obtain maintenance assistance decisions in real time while operating the visualized 3D model. This maintenance assistance decision includes strongly correlated maintenance knowledge and decision suggestions, completely breaking down the barriers between text / images and models, constructing an immersive maintenance assistance closed loop of "what you see is what you know," significantly improving the spatial perception ability, decision accuracy, and overall maintenance efficiency of operators in complex maintenance scenarios. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart of a maintenance auxiliary decision generation method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the interaction logic flow of a 3D model provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating a 3D model provided in an embodiment of this application; Figure 4 This is a schematic diagram of a front-end question-and-answer interaction interface provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] To make the objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] like Figure 1 As shown in the figure, this application provides a maintenance auxiliary decision generation method, including: S1 parses the input natural language instructions using a semantically driven large language model to obtain standardized semantic vectors.

[0015] S2, using a multimodal data fusion mechanism, the standardized semantic vector is compared with the three-dimensional model index in the underlying knowledge graph to obtain the target three-dimensional model resource.

[0016] S3, Perform interactive visualization processing on the target 3D model resource to obtain a visualized 3D model.

[0017] S4. Based on the visualized 3D model and a multimodal data binding mechanism, a two-way linkage between the 3D model and maintenance knowledge is achieved to generate maintenance auxiliary decisions; the maintenance auxiliary decisions include strongly correlated maintenance knowledge and decision suggestions.

[0018] In an exemplary embodiment, S1 specifically includes: S11, perform text preprocessing on the natural language instructions to remove redundant symbols and obtain the cleaned core text.

[0019] S12, Based on the cleaned core text, the BERT-BiLSTM-CRF model is used for entity recognition to extract maintenance-specific entities and obtain entity features.

[0020] S13. Based on the entity features, the Softmax regression algorithm is used to classify the intent and output the user intent category.

[0021] S14, the entity features are fused with the intent category to generate the standardized semantic vector.

[0022] In an exemplary embodiment, S2 specifically includes: S21, the entity features and intent categories in the standardized semantic vector are converted into word vectors and one-hot encoded vectors respectively and concatenated to obtain the initial semantic feature vector.

[0023] S22, extract the geometric features and attribute features of the three-dimensional model index, and obtain the geometric feature vector and attribute feature vector respectively through dimensionality reduction algorithm and encoding conversion.

[0024] S23. Based on the initial semantic feature vector, geometric feature vector, and attribute feature vector, the attention-weighted fusion algorithm is used to calculate the fusion weights to obtain the final fusion feature vector.

[0025] S24. Using the cosine similarity algorithm, the similarity between the final fused feature vector and the fused vector of the 3D model index is calculated, and the target 3D model resource with the highest matching degree is selected.

[0026] In one exemplary embodiment, after S2, the method further includes: S25, extract the core components and related neighboring parts of the target 3D model resource, and sort them according to the priority of core components, related parts and secondary parts; the related neighboring parts include related parts and secondary parts.

[0027] S26. Load the target 3D model resources in batches according to priority, and use the LRU caching mechanism to manage the cache of the loaded target 3D model resources.

[0028] In practical applications, the first step is to obtain the target 3D model through a semantically driven 3D model (i.e., large language model) precise matching mechanism.

[0029] 1.1. Deeply analyze the natural language commands input by the user using a large language model. For example... Figure 2 As shown, the specific implementation process and algorithm are as follows: It adopts a semantic parsing architecture based on domestically pre-trained large models (such as Qwen-7B / 14B), combined with fine-tuning and optimization in the field of maintenance. The core consists of four steps: "text preprocessing - entity recognition - intent classification - semantic encoding". The parsing accuracy is guaranteed throughout the process through standardized algorithms, with no fuzzy logic.

[0030] The first step is text preprocessing: The unstructured text input by the repair personnel (such as "check the internal structure of the fuel pump on the right side of the engine") is cleaned by removing redundant spaces and interjections. Core text fragments are then extracted using regular expressions, as shown in the following formula: in, For the user's original input text, The core text after cleaning, This is a regular expression replacement function. The function to remove leading and trailing spaces and normalize consecutive whitespace is expressed as " The quotation mark "" indicates that the character is matched except for Chinese characters, English letters and numbers. An empty string "" indicates that the matched irrelevant characters will be deleted. Regular expressions are used to filter meaningless characters and retain only Chinese characters, English letters, numbers and core words in the maintenance field.

[0031] To avoid confusion regarding the meaning of letters throughout the text, the main symbols used in this application are defined as follows: This represents the user's original input text. This represents the core text after cleaning, and X represents the word vector sequence. Indicates the first i There are 1 word vectors, where H represents the sequence of context feature vectors output by BERT. Indicates the first i There are three context feature vectors, where n represents the word sequence length or the number of entries in the matrix dimension, and d represents the feature dimension. Let Y represent the CRF label transition matrix, k represent the number of entity labels, and Y represent the optimal label sequence. Indicates the first i Individual tags.

[0032] In intent classification, m represents the number of training samples, c represents the number of intent categories, and r represents the sample index. Indicates the category number. Indicates that the r-th sample is in the first position. One-hot encoded tags on the class, The model predicts the first... r The sample belongs to the first The probability of the class, where L represents the cross-entropy loss.

[0033] In model matching and loading, Represents semantic feature vectors, Represents geometric eigenvectors. Represents the attribute feature vector. This represents the final fused feature vector. The fusion vector represents the index of the 3D model. , , Indicates the fusion weight. This represents the angle between two vectors in the cosine similarity formula; Indicates the first i Priority weights of class components Indicates the first i The loading completion status of the component class; Progress indicates the overall loading progress. Indicates the number of cache hits. This indicates the number of cache misses.

[0034] In 3D rendering and rigging relationships Represents the model-view projection matrix. , , These represent the model matrix, view matrix, and projection matrix, respectively. , S and S represent the translation matrix, rotation matrix, and scaling matrix, respectively. Indicates the rotation angle of the model interaction. Indicates the scaling factor. and These represent the horizontal displacement of the mouse and the displacement of the scroll wheel, respectively. and These represent the rotation factor and scaling factor, respectively; M represents the incidence matrix. Indicates the first i The model component and the first j The association weight of maintenance knowledge items, Indicates the first j A semantic vector of maintenance knowledge.

[0035] The second step is entity recognition: a finely tuned BERT-BiLSTM-CRF algorithm is used to identify maintenance-specific entities (equipment, components, locations, faults, etc.). In the model structure, the BERT layer is responsible for extracting textual context features, the BiLSTM layer captures sequence dependencies, and the CRF layer outputs standardized entity labels. The core calculation process is as follows: 1. BERT Feature Extraction: Converting the cleaned text into word vectors The BERT encoder outputs the context feature vector. ,in, n For word sequence length, i For word sequence number and i ∈[1, ], For the first i Word vectors, For the first i Context feature vectors, , d The feature dimension is set to 768.

[0036] 2. BiLSTM Feature Enhancement: Bidirectional Long Short-Term Memory computation is performed on H, and the forward LSTM outputs the forward hidden state. The reverse LSTM outputs the backward hidden state. The splicing results in the fusion feature. ;in,f Indicates positive. b Indicates the opposite direction.

[0037] 3. CRF Decoding: Through the transition matrix A∈ (k is the number of entity tags, such as devices, components, etc., 6 categories) Calculate the tag transfer probability, and combine it with the emission probability. The optimal label sequence is solved using dynamic programming. This enables precise entity extraction; among which, For the first i A tag.

[0038] The third step is intent classification: A Softmax regression algorithm is used, inputting the feature vectors after entity recognition into a fully connected layer to output the user intent category (e.g., view, investigate, disassemble, etc.). The loss function is cross-entropy loss. The formula is as follows: Where m is the number of samples and c is the number of intent categories. This is the sample sequence number (distinct from the i that represents the word sequence number in the previous text). For the intent category number; For the first The sample at the th One-hot encoded tags on the class, For the model to predict the first The sample belongs to the first The probability of the class is iteratively optimized using gradient descent to ensure the accuracy of intent recognition. .

[0039] The fourth step is semantic encoding: the identified entities and intentions are fused to generate standardized semantic vectors, which provide input for subsequent model matching.

[0040] The specific implementation process of the repair field prompt word standard: Pre-set repair field prompt word templates (e.g., "User input text is {... "Please extract the device object, specific components, spatial location, and action intent, and output in standardized JSON format." The template and cleaned text are concatenated and input into the large model to constrain the model's output format and avoid ambiguity. The model fine-tuning process uses a maintenance domain corpus (containing over 100,000 maintenance dialogues and fault description texts). The LoRA fine-tuning method freezes the main parameters of the large model, training only the low-rank adaptation matrix to reduce computation while ensuring domain adaptability. The fine-tuning learning rate is set to... -5 The iteration count is 50 rounds, and fine-tuning stops once the accuracy of the validation set meets the target.

[0041] When a repair technician enters a query such as "Check the internal structure of the fuel pump on the right side of the engine", the system's AI intelligent engine initiates the aforementioned entity recognition and intent classification process. Ultimately, the input text is broken down into: equipment object (engine), specific component (fuel pump), spatial location (right side), and action intent (check internal structure). This avoids the literal matching dependency of traditional keyword retrieval, effectively identifies synonyms (such as "fuel pump" and "fuel supply pump"), and improves the error tolerance of the retrieval.

[0042] 1.2. Based on the extracted entities, a multimodal data fusion mechanism is used to vectorize the semantic features and calculate their similarity with the 3D model index in the underlying knowledge graph to obtain the 3D model resource with the highest matching degree. The specific implementation process of the multimodal data fusion mechanism is as follows: A bidirectional fusion strategy of "semantic features - model features" is adopted. The core is to fuse the semantic features (entity + intent) extracted in 1.1 with the geometric features and attribute features of the 3D model to form a unified feature vector. The specific steps are as follows: 1. Semantic Feature Vectorization: The entities (devices, components, etc.) identified in 1.1 are converted into word vectors using the Word2Vec model, and the intent category is converted into a vector using one-hot encoding. Then, the two types of vectors are concatenated to obtain the initial semantic feature vector. ( (The semantic feature dimension is set to 256).

[0043] 2. 3D Model Feature Extraction: Extract geometric features (number of vertices, number of faces, topology) and attribute features (part number, material, dimensions) from the 3D model (OBJ / STL format) in the knowledge graph. Then, convert the geometric features into vectors using the PCA dimensionality reduction algorithm. ( (Geometric feature dimension, taken as 128), attribute features are converted into vectors through encoding. ( (The attribute feature dimension is set to 64).

[0044] 3. Multimodal fusion: An attention-weighted fusion algorithm is used to calculate the association weights between semantic features and model features. The fusion formula is as follows: in, , , To integrate weights, satisfy + + =1, obtained by fitting the training data ( =0.5, =0.3, =0.2), This is the final fused feature vector, used for subsequent similarity calculations.

[0045] The specific implementation process of similarity calculation is as follows: The cosine similarity algorithm is used to calculate the association weight between semantic features and model features. Fusion vector with each 3D model index in the knowledge graph The similarity is calculated using the following formula: Among them, || ||、|| || represents the magnitudes of the two vectors. for and The angle between them The closer the similarity value is to 1, the higher the matching degree.

[0046] The knowledge graph pre-stores the mapping relationship between 3D model component IDs and technical manual entities. Each mapping includes the official name, drawing number, and topological level of the part in 3D space. Through the above cosine similarity calculation, the system executes entity disambiguation logic: when there are multiple different models of "fuel pump" in the system, the algorithm will combine the overall equipment model mentioned in the context and the current maintenance scenario to calculate the similarity between the fusion vector of each fuel pump model and the user semantic vector, and filter out the 3D model resource with the highest similarity (≥0.85). If there are multiple models with similarity ≥0.85, a weighted correction is made based on historical maintenance frequency (the higher the historical usage frequency, the greater the weight), and finally the unique matching 3D model is locked.

[0047] 1.3. Based on the obtained 3D model resources, a dynamic model loading and caching mechanism is used for processing, and the accuracy of the semantic mapping model is optimized through the evaluation and management module. The specific implementation process of the dynamic model loading and caching mechanism is as follows: A "priority loading + LRU caching" strategy is adopted to balance loading speed and resource consumption. The specific steps are as follows: 1. Model Priority Ranking: Based on the "focus" (such as the internal structure of the fuel pump) obtained from the semantic parsing in 1.1, extract the component and its related adjacent parts (such as oil pipes and seals), and sort them according to the priority of "core component → related parts → secondary parts". The priority weights are determined by the analytic hierarchy process (core component weight 0.6, related parts 0.3, secondary parts 0.1).

[0048] 2. Batch Loading: The system backend prioritizes loading the OBJ / STL format files of core components, sending model data in chunks to the frontend via streaming. After the core components are loaded, related and secondary components are loaded asynchronously. The loading progress formula is as follows: in, For the first Priority weights of class components The loading completion rate of the i-th type of component ( Progress represents the overall loading progress, which is reflected in real time to the front-end UI.

[0049] 3. LRU caching mechanism: To avoid repeatedly loading the same model, the system adopts the Least Recently Used (LRU) caching strategy, with a cache capacity of 50 models (which can be dynamically adjusted according to server memory). The cache hit rate is calculated as follows: in, To cache the hit count, To cache the number of cache misses, when the cache hit rate is below 80%, the cache capacity is automatically increased (by 10 models each time). When the cache capacity reaches its limit, the least recently used model is deleted to ensure efficient use of cache resources.

[0050] 4. Temporary Access Link Generation: After the core components are loaded, the system generates a temporary access link (valid for 30 minutes). The encryption algorithm encrypts the link to prevent unauthorized access. The encryption formula is as follows: in, This is the original access link. This is the current timestamp. This is the system key; after encryption, the link is returned to the front-end UI for model rendering.

[0051] The specific implementation process of monitoring and optimizing the evaluation management module: The core is to collect feedback data through user behavior monitoring, continuously fine-tune the semantic mapping model, and form a closed-loop optimization. The specific steps are as follows: 1. User Behavior Monitoring: The system front-end collects user operation behaviors in real time, including the number of model clicks, the number of model switches, "matching success / failure" feedback, and the duration of dwell time, and encapsulates this behavioral data into feature vectors. ; 2. Feedback Weight Calculation: Based on the importance of user behavior, different weights are assigned ("Match Failure" feedback weight 0.4, "Match Success" feedback weight 0.2, Click Count weight 0.2, Dwell Time weight 0.2), and the user feedback score is calculated using the following formula: in, For the first Weights of class behaviors For the first The quantification value of the class behavior (e.g., "match failed" is recorded as 0, and "match succeeded" is recorded as 1).

[0052] 3. Model fine-tuning: When When user feedback is unsatisfactory, the semantic parsing path, user input text, and matching results are used as negative feedback samples and added to the model fine-tuning dataset; when At that time, these samples are used as positive feedback samples to optimize the semantic mapping model. Fine-tuning employs mini-batch gradient descent, with 100 samples per fine-tuning iteration and a learning rate of... -5 The model accuracy is verified after every 10 rounds of fine-tuning to ensure that the accuracy of semantic mapping continues to improve.

[0053] In an exemplary embodiment, S3 specifically includes: S31, Analyze the geometric topology and surface material of the target 3D model resource, and construct... Matrix; the The matrix includes the model matrix, the view matrix, and the projection matrix.

[0054] S32, based on the above The matrix defines the cutting plane based on the profile analysis instructions, and calculates the distance from the fragment to the cutting plane during the clipping test phase in the rendering pipeline, and executes the discard instruction on the fragment located on one side of the cutting plane.

[0055] S33, perform dual-mode rendering of the cut surface after executing the discard command to seal the section, generate filling material, and obtain the visualized 3D model.

[0056] In one exemplary embodiment, after S33, the method further includes: S34, determine the LOD level based on the distance between the visualized 3D model and the camera, and use a cross-fade-in / fade-out algorithm to smoothly switch between different models; wherein, the different models include high-level models, medium-level models and low-level models.

[0057] S35, construct the axis-aligned bounding box for each model; the axis-aligned bounding box includes 8 vertices.

[0058] S36. Based on the view matrix and the projection matrix, construct a view frustum, and detect the positional relationship between the axis-aligned bounding box vertices and the view frustum plane, and eliminate models whose vertices are all located outside any plane of the view frustum; the view frustum includes 6 planes.

[0059] In practical applications, step two involves obtaining a 3D model based on the above steps and then performing interactive visualization processing using WebGL / WebGPU.

[0060] 2.1 Model Loading and Matrix dynamic update implementation.

[0061] After the 3D model link (OBJ / STL format data) obtained from the backend is loaded into the frontend buffer, the model's geometry, topology, and surface materials are calculated in real time using vertex shaders and fragment shaders. The core of this calculation is through the model-view projection matrix. The model is dynamically updated to achieve free rotation and smooth scaling. The specific process and algorithm are as follows: 1. Model Data Parsing: The front-end uses a custom loader to parse OBJ / STL format data and extract vertex coordinates. Normal vector Texture coordinates The material is stored in the VertexBuffer, and the fragment shader calculates the surface color based on material parameters (such as metallicity and roughness).

[0062] 2. Matrix construction and updating: The matrix consists of the model matrix ( ), view matrix ( ), projection matrix ( It consists of three parts, and the calculation formula is as follows: Model matrix (Model): Used to describe the translation, rotation, and scaling of the model in 3D space, the formula is Model = × × Where T is the translation matrix, R is the rotation matrix, and S is the scaling matrix; when the user drags the mouse, the system captures the mouse displacement and converts it into a rotation angle. The scaling factor s is used to update the R and S matrices in real time, which in turn updates the Model matrix.

[0063] View matrix: Used to describe the camera's position and viewing direction, the formula is as follows ,in For camera position, To observe the target point, This is the direction vector on the camera; when the user zooms in on the model, and Distance varies with scaling factor Adjust synchronously to ensure a consistent viewing angle.

[0064] Projection matrix: Using perspective projection, the formula is: in, The distance to the near shear plane. The distance to the far shear plane. These are the left, right, top, and bottom boundaries of the projection plane, which are dynamically adjusted according to the size of the front-end interface to ensure that the model rendering is distortion-free.

[0065] 3. Interaction Protocol Conversion: To adapt to the hardware limitations of single-machine deployment on a local area network, the rendering engine adopts a standardized interaction protocol. This protocol converts physical inputs such as mouse dragging, scrolling, or touch gestures into transformation parameters (rotation angle, scaling factor) in three-dimensional space using a coordinate mapping algorithm. The mapping formula is as follows: in, The rotation angle of the model interaction in this section is the angle between this angle and the angle in the cosine similarity formula mentioned earlier. Different meanings; This represents the horizontal movement of the mouse. For roller displacement, (Rotation factor, taken as 0.01) (Scaling factor, taken as 0.05) is an empirical value to ensure that the operating sensitivity is adapted to industrial terminal equipment, realize free rotation and smooth scaling around the axis center, and help maintenance personnel quickly switch from the macroscopic overall structure to the microscopic part details to establish an intuitive spatial perception.

[0066] 2.2 Implementation of the profile analysis tool.

[0067] To address the need for internal inspection of complex mechanical structures, a cross-sectional analysis tool based on geometric shading is developed. Its core functionality involves generating virtual cutting planes and trimming the rendering pipeline to clearly present the internal structure of the model. The specific implementation steps are as follows: 1. Cutting plane definition: After activating the section function in the interactive interface, maintenance personnel can define the cutting plane in two ways: one is to select three non-collinear reference points on the model surface. , , The plane normal vector is calculated by the cross product of vectors. Thus, the plane equation is obtained. ,in, For normal vector components, Calculations can be performed by substituting any reference point; alternatively, the X / Y / Z axes can be directly specified, and the system will automatically generate the corresponding cutting planes. For example, the equation of the X-axis plane is... , These are the cutting position parameters.

[0068] 2. Rendering pipeline clipping test: The normal vector of the cutting plane... and plane equation parameters ( The fragment is passed as a global variable (Uniforms) to the fragment shader. During the clipping test phase of the rendering pipeline, the spatial position of each fragment to be rendered is checked: the position of the fragment in the world coordinate system is calculated. Distance to the cutting plane Set threshold ,like (i.e., the fragment is located on one side of the cutting plane), then the discard command is executed, and the rendering of that pixel is abandoned; if If the pixel is not found, the rendering process continues.

[0069] 3. Sectional Sealing: To avoid the cut model appearing as an "empty shell," a dual-mode rendering technique is used to ensure the cross-section is clearly visible: First, the stencil buffer on the back side of the cut is rendered, writing the outline of the cut surface into the stencil buffer and setting the stencil value to 1; Second, using stencil testing, only the area with a stencil value of 1 is rendered, generating a specific filling material (dark gray matte texture) on the cut surface. (Value 0.8) simulates the cross-sectional effect of a solid component after it has been cut open, helping maintenance personnel to clearly identify the wall thickness and internal channel routing.

[0070] 4. Interactive Adjustment Function: The system supports multi-angle sectioning adjustment and section transparency compensation. The front-end UI provides a slider control, allowing maintenance personnel to adjust the position parameters of the cutting plane (such as the d value in the plane equation) via the slider, dynamically changing the cutting depth. The mapping relationship between the slider displacement and the parameter d is as follows: ( These are the initial parameters. For adjustment coefficients, (This refers to the displacement of the slider). Simultaneously, the transparency slider can be used to adjust the profile fill material. The value (0.5~1.0) takes into account both internal detail observation and overall structural understanding, assisting maintenance personnel in accurately locating the source of the fault in three-dimensional space.

[0071] 2.3 Specific implementation of dynamic hierarchical loading (LOD) and view frustum culling techniques.

[0072] To improve the response speed of large-scale 3D scenes and adapt to terminal devices with different hardware performance, LOD and frustum culling techniques are introduced. The specific implementation process and calculation are as follows: 1. Implementation of Level-of-Depth (LOD): Model grading: Each 3D model is divided into 3 levels (high, medium and low) according to accuracy. High-level models retain complete vertex data (number of vertices ≥ 10,000), medium-level models reduce the number of vertices by 50% through vertex simplification algorithms (such as edge folding algorithms), and low-level models reduce the number of vertices by 80%. The simplification error is controlled within 0.1mm (to meet the accuracy requirements for maintenance observation).

[0073] Switching threshold calculation: based on the distance between the model and the camera. Determine the LOD level and set two thresholds. , The switching determination formula is as follows: when When, load the high-level model; when At that time, the hierarchical model is loading; when At that time, load the low-level model.

[0074] Smooth transition: When the model distance crosses the threshold, a cross-fade-in / fade-out algorithm is used to achieve a smooth transition between different model levels, avoiding screen stuttering. The transition time is set to 0.3 seconds, and the transition weight formula is as follows: in, For high-level model weights, The lower-level model weights are calculated in real time during the transition, and the two levels of models are rendered in a mixed manner until the switch is completed.

[0075] 2. Implementation of cone removal technology: View frustum construction: Based on the projection matrix and the view matrix, construct the view frustum (consisting of 6 planes: near, far, left, right, top, and bottom), each plane using plane equations. express.

[0076] Bounding box intersection detection: Construct an axis-aligned bounding box (AABB) for each model, calculate the positional relationship between the 8 vertices of the bounding box and the 6 planes of the view frustum, and if all vertices are located outside a certain plane ( If a vertex is not within the view frustum, the model is culled and not rendered; if a vertex is within the view frustum or intersects with a plane, the model is retained and rendered.

[0077] Optimized computation: To reduce computational load, a hierarchical bounding box technique is adopted. The model is divided into parent components and child components according to the topology. The relationship between the parent component's bounding box and the view frustum is detected first. If the parent component is removed, all its child components do not need to be detected and are directly removed, improving detection efficiency by more than 60%.

[0078] In scenarios where multiple servers are deployed in a distributed manner within a local area network, the system automatically adjusts the display precision of the model by detecting the GPU performance of the front-end terminal (such as detecting the GPU memory size and rendering frame rate). When the GPU memory is less than 2GB, low-to-medium level models are loaded by default to ensure smooth rendering.

[0079] In an exemplary embodiment, S4 specifically includes: S41, construct a two-way mapping relationship between the metadata of the visualized 3D model and maintenance knowledge, obtain the association matrix, and assign it to each visualized 3D model component and maintenance knowledge. .

[0080] S42, when a component selection operation is received for the visualized 3D model, according to the... The correlation matrix is ​​queried to extract strongly correlated maintenance knowledge and push it to the front-end interface, realizing a positive linkage between model operation and knowledge push.

[0081] S43, when a maintenance knowledge query instruction is received, the key component entities in the text are identified, the strongly associated model components of the maintenance knowledge are retrieved, and the 3D camera is driven to automatically translate and zoom to the local area, realizing the reverse linkage from knowledge query to model positioning.

[0082] S44, Based on the positive linkage and the negative linkage, a maintenance auxiliary decision is generated.

[0083] In practical applications, step three involves using a multimodal data binding mechanism to achieve bidirectional linkage between the 3D model and maintenance knowledge, thereby assisting in maintenance decision-making.

[0084] This step is the core link connecting Step 1 (semantic-driven model matching) and Step 2 (3D visualization). The logical relationship between the three is as follows: Step 1 matches the target 3D model through semantic parsing, providing a "core carrier" for subsequent linkage; Step 2 presents the model through visualization technology, providing maintenance personnel with an "operation object"; Step 3 associates the model with maintenance knowledge through multimodal data binding, realizing two-way linkage of "model operation → knowledge push" and "knowledge query → model positioning", forming a complete maintenance assistance closed loop and solving the pain point of "model and knowledge disconnect" in traditional systems.

[0085] The specific implementation process and calculation of the multimodal data binding mechanism are as follows: 1. Construction of a unified semantic association layer: Establishing a unified semantic association layer, the core of which is to construct a two-way mapping relationship between "3D model metadata and maintenance knowledge" using an association matrix. To manage, among which The number of components in the 3D model. The number of maintenance knowledge items, matrix elements Indicates the first The model component and the first The association weight of each maintenance knowledge item is calculated using the following formula: in, For the first The fused feature vectors of each model component (same as in step 1.2) ), For the first Semantic vectors of maintenance knowledge (maintenance procedures, troubleshooting, etc.), and association weights. Weight It is considered a strong association and is included in the binding relationship.

[0086] 2. Globally unique entity identifier ( Maintenance: Assign a globally unique value to each 3D model component and each piece of maintenance knowledge. ,use The generation algorithm is in the following format: (Component / knowledge unique identifier + timestamp) ensures the uniqueness and stability of the binding relationship. Association Matrix and Each vector corresponds to a specific vector and is stored in a local vector database, supporting real-time querying and updating.

[0087] 3. Achieve bidirectional linkage: Forward Linkage: In Step 1, the system uses a large model to perform a depth scan on the user-input maintenance-related text (such as "loosen the hydraulic pump fixing bolts"). Through the entity recognition function of semantic parsing in 1.1, it automatically identifies the key component entity (hydraulic pump fixing bolts) in the text. This then triggers the 3D visualization module in Step 2. By adjusting the material properties of the fragment shader (such as setting an edge glow effect with a glow intensity 1.5 times that of a normal material), the corresponding 3D component displays a clear visual feedback (flickering or color change) on the interface. Simultaneously, based on Step 2... The matrix adjustment logic drives the 3D camera to automatically translate and zoom to the local area where the component is located (preset optimal viewing angle parameters). and (Obtained through model metadata queries), this guides maintenance personnel to quickly establish the spatial correspondence between textual descriptions and physical structures, providing intuitive guidance for maintenance operations.

[0088] Reverse linkage: In step two, when the maintenance personnel select a specific 3D part (such as a sensor) by clicking with the mouse, the system will immediately capture the component's... In step three, the multimodal binding mechanism extracts the [data / data] by querying the association matrix M. All corresponding strongly correlated maintenance knowledge (correlation weight) This knowledge is obtained from a vector database deployed on the local area network, including corresponding chapters of the repair manual, historical maintenance records, part parameter tolerances, and disassembly / assembly precautions; subsequently, this information is pushed to the front-end UI interface in real time in the form of a sidebar floating window or augmented reality tile, such as... Figure 4 As shown, this achieves "what you see is what you know"; if maintenance personnel click on the pushed maintenance knowledge, the system can further retrieve other model components associated with the knowledge (such as sensor connection lines and fasteners) through the semantic matching logic in step one, realizing deep linkage between the model and the knowledge.

[0089] 4. Binding Relationship Optimization: Combining with the evaluation management module in step 1.3, user feedback on knowledge pushes (such as "useful / useless") is incorporated into the optimization system. When a user marks a piece of knowledge as "useless," the corresponding association weight is reduced. (Decrease by 0.1 each time), when the weight is <0.5, unbind; when the user marks it as "useful", increase the association weight (increase by 0.05 each time) to ensure the accuracy of the binding relationship and meet the actual maintenance needs.

[0090] The technical solution of this application will be further illustrated below with specific examples.

[0091] 4.1 Example 1: Implementation process of user questions and automatic model retrieval.

[0092] 4.1.1 Initial Trigger and Natural Language Input.

[0093] In actual maintenance and support scenarios, when faced with equipment malfunctions, maintenance personnel input natural language descriptions through the system's user-friendly front-end interface. For example, a maintenance personnel might input: "Inquire about the troubleshooting steps for oil leaks at the fuel regulator housing connection, and display its exploded structure diagram." At this point, the system's front-end UI not only receives the text information but also collects current contextual parameters (such as the model of the equipment being repaired, the maintenance personnel's access level, etc.) and encapsulates this information into a standard request message, sending it to the back-end service module.

[0094] 4.1.2 Semantic parsing and intent recognition based on large models.

[0095] After receiving the request, the backend service first invokes a domestically developed large language model (with parameters ranging from 7B to 14B) deployed on the local area network. The system preprocesses the input text using a preset prompt template, guiding the large model to perform Named Entity Recognition (NER). In this embodiment, the large model will recognize: Core components: fuel regulator (assembly), housing connection (part); Fault symptom: Oil leakage; Interactive intent: query troubleshooting steps and view 3D structure.

[0096] To ensure the accuracy of parsing, the system constrains the model output through prompt word specifications, requiring the model to output standardized JSON format data. If the user's input is ambiguous (e.g., only "regulator leaks oil" is entered, but multiple regulators exist in the system), the large model will perform entity disambiguation based on historical maintenance records and current equipment configuration parameters, prioritizing the specific model with the highest matching degree.

[0097] 4.1.3 Retrieval Enhancement Generation (RAG) and Knowledge Base Retrieval.

[0098] After identifying the core entities, the system enters the RAG processing phase. First, the system converts the parsed entity keywords into vector representations (Embeddings) and then searches the local vector database. The search objects include: Structured data: The hierarchical tree structure of "fuel regulator" in the knowledge graph, including its upper-level systems and lower-level components.

[0099] Unstructured data: PDF format "Maintenance Technical Manual", DOCX format "Fault Case Library" and expert experience summaries.

[0100] The system uses a multi-path recall algorithm to obtain decision suggestions in text form (e.g., check if the seal is aging or if the torque meets the standard), and extracts a unique identifier for the 3D model corresponding to the "fuel regulator breakdown structure" through the association attributes of the knowledge graph. ).

[0101] 4.1.4 Dynamic generation and verification of 3D model links.

[0102] Get Afterwards, the backend service accesses the database storing 3D model resources. This system supports multiple formats such as OBJ and STL. For the exploded structure diagram in this embodiment, the system retrieves the corresponding STL format family file. Before returning the link, the system performs a crucial "viewpoint parameter calculation" step. Based on the user's question focus ("shell connection"), the system queries the pre-stored viewpoint anchor points in the model metadata. If the anchor point exists, the system will calculate the viewpoint coordinates (…). ) and observation center ( The data is encapsulated along with the model link. Furthermore, the system performs real-time encryption on sensitive 3D model data, generating a time-sensitive dynamic access token. This ensures the security of data during transmission over a local area network.

[0103] 4.1.5 Front-end rendering and interactive presentation.

[0104] After receiving the 3D model link and associated text returned by the backend, the front-end UI interface initiates the visualization loading process: Text area presentation: The left side of the interface or a floating window displays "maintenance decision suggestions" integrated from the large model, including possible causes of failure (damaged sealing surface, loose fastening bolts) and suggested operating procedures.

[0105] 3D rendering: The core area of ​​the interface launches the WebGL rendering engine, loading the 3D model of the "fuel regulator" into video memory through a dynamic loading mechanism. Using the previously returned view parameters, the system automatically aligns the camera with the "shell connection" and highlights the fasteners involved in that area (e.g., by setting a special edge-glowing shader).

[0106] At this point, what maintenance personnel see is no longer an isolated model that needs to be manually located, but an "intelligent digital entity" that has been automatically located, automatically labeled, and equipped with in-depth auxiliary decision-making information.

[0107] 4.1.6 Interactive feedback and closed-loop optimization.

[0108] like Figure 3 As shown, during the 3D model display, maintenance personnel can freely rotate and dynamically scale the model using a mouse or touchscreen. If the maintenance personnel deem the currently retrieved model inaccurate and click "Mismatch," the system records this semantic parsing path and uses it as a negative feedback sample. In subsequent vertical domain model training, this feedback data will be used to fine-tune the larger model, generating similar fault description variants through data augmentation techniques, thereby continuously improving the system's robustness in the "question-retrieval" chain.

[0109] 4.1.7 Secure Exit and Data Cleanup.

[0110] When the maintenance task is completed or the user chooses to exit by single sign-out, the system will forcibly clear the 3D model data fragments in the front-end browser cache and revoke the current access token. This mechanism ensures that even on public terminals, subsequent personnel cannot obtain sensitive equipment structural information through caching, fully reflecting the security principles required in the system design specification.

[0111] 4.2 Example 2: Implementation process of interactive operation and cross-sectional analysis of three-dimensional model.

[0112] This embodiment focuses on how the system enables maintenance personnel to perform in-depth interactive operations on the 3D model of complex mechanical equipment through a high-performance real-time rendering engine, especially the application logic of the "section analysis tool" in complex maintenance decision-making.

[0113] 4.2.1 Initialization and environment configuration of the 3D rendering engine.

[0114] After the system frontend receives the 3D model data stream (using the STL format family as an example) returned by the backend, it first initializes the WebGL-based rendering context in the browser environment. The system loads the geometric vertex data, normal vector data, and index buffer into video memory through a custom loader. To ensure the loading efficiency of large-sized models in a single-machine deployment environment, a block loading strategy is adopted in this embodiment, prioritizing the rendering of faulty components of interest to the user. At the same time, the rendering engine is configured with ambient light and directional light, and calculates the lighting and shadow effects on the model surface through the Phong Reflection Model to enhance the metallic texture and spatial hierarchy of mechanical parts, making it easier for maintenance personnel to identify subtle structural features.

[0115] 4.2.2 Quaternion-based free rotation and smooth scaling algorithm.

[0116] To provide an immersive user experience, the system implements a spatial transformation algorithm based on quaternions at the interaction logic layer. Compared to traditional Euler angles, quaternions can effectively avoid the "universal joint deadlock" problem, ensuring that the image remains smooth and undistorted when maintenance personnel observe the troubleshooting path by rotating the model in all directions.

[0117] Rotation logic: When a maintenance user drags the mouse on the screen using the left mouse button, the system captures the starting and ending coordinates, maps them to the surface of the virtual trackball, and calculates the rotation axis and rotation angle. Subsequently, the system generates a rotation quaternion and applies it to the model matrix, enabling the model to rotate sensitively with gestures.

[0118] Dynamic scaling: The system listens for scroll wheel events and adjusts the zoom level between the camera and the target point using smooth interpolation (Lerp) technology. The distance between them. During zooming, the system limits the near and far shear plane parameters of the camera in real time to prevent clipping when observing the interior of micro-parts (such as sensor contacts).

[0119] 4.2.3 Geometric segmentation logic of profile analysis tools.

[0120] When faced with faults such as "wear on gears inside the engine casing" that cannot be observed through external surfaces, the embodiment provides transparent decision support through a "profile analysis tool." Its core technology implementation involves the following steps: Cutting plane definition: After the user activates the profile mode in the interface, by selecting three reference points on the model surface or specifying an axis (X / Y / Z), the system defines a mathematical plane equation in three-dimensional space. .

[0121] Shader-end clipping: The system clips the normal vector of the cutting plane. and offset These are passed to the fragment shader as global variables (Uniforms). During the rendering pipeline, the shader performs spatial position checks on each fragment to be rendered. If the fragment's position in the world coordinate system satisfies... If the condition is met, the discard instruction will be executed, meaning the rendering of that pixel will be abandoned.

[0122] Sectional Capping: To avoid the cut model appearing as an "empty shell," the system employs dual-mode rendering technology. First, the stencil buffer on the back of the cut is rendered. Then, a specific filling material (such as a dark gray matte texture) is generated on the cut surface using stencil testing. This simulates the cross-sectional effect of a solid component after it has been cut open, helping maintenance personnel clearly identify the wall thickness and internal channel routing.

[0123] 4.2.4 Multi-dimensional sectioning and transparency adjustment.

[0124] To further enhance analytical capabilities, this embodiment supports "multi-angle simultaneous sectioning." Maintenance personnel can simultaneously activate two mutually perpendicular cutting planes to form "L-shaped" or "cross-shaped" sections, which is crucial for inspecting blockages in intersecting pipes. The system provides a real-time slider in the front-end UI, allowing maintenance personnel to dynamically change the offset of the cutting planes. As the slider moves, the model matrix updates in real time, allowing maintenance personnel to observe the process of the profile "sliding" inside the equipment, similar to the effect of a medical CT scan.

[0125] Meanwhile, for non-critical external components (such as device enclosures), the system supports "partial transparency" interaction. This can be achieved by adjusting the fragment shader. The channel value allows maintenance personnel to make the outer casing semi-transparent, while the internal faulty parts (such as damaged solenoid valves) remain completely opaque against this background, thus establishing a three-dimensional reference coordinate system of "outer casing-interior-fault point".

[0126] 4.2.5 Interaction Protocol and Data Synchronization.

[0127] All interactive operations follow a standardized protocol. When maintenance personnel section or rotate the model, the current camera matrix information and section parameters are temporarily saved in real time. If the user triggers the "Save Current Decision View" function at this time, the system will save these spatial parameters along with the current fault diagnosis conclusion into the database. When other maintenance personnel retrieve the same case later, the system can automatically restore the same section view, realizing the visual transmission of maintenance decision-making experience.

[0128] 4.2.6 Hardware Adaptation and Performance Guarantee.

[0129] Considering the differences in graphics card performance among terminal devices (such as industrial handheld tablets or ruggedized laptops) within a local area network environment, this embodiment implements a performance adaptive mechanism. The system dynamically adjusts the anti-aliasing (MSAA) level and texture map resolution based on the current frame rate (FPS). When executing complex profiling algorithms, if an increase in rendering latency is detected, the system automatically reduces the rendering frequency of non-operational areas to ensure that the interaction feedback latency is less than 16ms when maintenance personnel perform critical rotation and cutting operations, thereby meeting the requirements of continuous and efficient operation in industrial settings.

[0130] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments. The computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, memory, and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores data to be processed. The I / O interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with an external terminal via a network connection. When the computer program is executed by the processor, it implements the above-described methods.

[0131] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0132] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0134] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by hardware related to computer program instructions. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0135] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0136] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] This application uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. A maintenance auxiliary decision generation method, characterized in that, include: The input natural language instructions are parsed using a semantically driven large language model to obtain standardized semantic vectors; By using a multimodal data fusion mechanism, the standardized semantic vector is compared with the three-dimensional model index in the underlying knowledge graph to obtain the target three-dimensional model resource. The target 3D model resource is subjected to interactive visualization processing to obtain a visualized 3D model; Based on the visualized 3D model, and using a multimodal data binding mechanism, a two-way linkage between the 3D model and maintenance knowledge is achieved to generate maintenance auxiliary decisions; the maintenance auxiliary decisions include strongly correlated maintenance knowledge and decision suggestions.

2. The maintenance auxiliary decision generation method according to claim 1, characterized in that, The input natural language instructions are parsed using a semantically driven large language model to obtain standardized semantic vectors, specifically including: The natural language instructions are preprocessed to remove redundant symbols, resulting in cleaned core text. Based on the cleaned core text, the BERT-BiLSTM-CRF model is used for entity recognition to extract maintenance-specific entities and obtain entity features. Based on the entity characteristics, the Softmax regression algorithm is used to classify intent and output the user intent category; The entity features are fused with the intent category to generate the standardized semantic vector.

3. The maintenance auxiliary decision generation method according to claim 1, characterized in that, By utilizing a multimodal data fusion mechanism, the standardized semantic vector is compared with the 3D model index in the underlying knowledge graph to calculate similarity, thereby obtaining the target 3D model resource, specifically including: The entity features and intent categories in the standardized semantic vector are converted into word vectors and one-hot encoded vectors, respectively, and then concatenated to obtain the initial semantic feature vector. The geometric and attribute features of the three-dimensional model index are extracted, and geometric feature vectors and attribute feature vectors are obtained respectively through dimensionality reduction algorithms and encoding conversion; Based on the initial semantic feature vector, geometric feature vector, and attribute feature vector, the attention-weighted fusion algorithm is used to calculate the fusion weights to obtain the final fusion feature vector. The cosine similarity algorithm is used to calculate the similarity between the final fused feature vector and the fused vector of the 3D model index, and the target 3D model resource with the highest matching degree is selected.

4. The maintenance auxiliary decision generation method according to claim 1, characterized in that, After using a multimodal data fusion mechanism to calculate the similarity between the standardized semantic vector and the 3D model index in the underlying knowledge graph to obtain the target 3D model resource, the method further includes: Extract the core components and related neighboring parts of the target 3D model resource, and sort them according to the priority of core components, related parts, and secondary parts; the related neighboring parts include related parts and secondary parts; Based on priority, target 3D model resources are loaded in batches, and an LRU caching mechanism is used to manage the cache of the loaded target 3D model resources.

5. The maintenance auxiliary decision generation method according to claim 1, characterized in that, Interactive visualization processing is performed on the target 3D model resource to obtain a visualized 3D model, specifically including: Analyze the geometry, topology, and surface material of the target 3D model resource to construct... Matrix; the The matrices include the model matrix, the view matrix, and the projection matrix; Based on the above The matrix defines the cutting plane according to the profile analysis instructions, and calculates the distance from the fragment to the cutting plane during the clipping test phase in the rendering pipeline, and executes the discard instruction on the fragment located on one side of the cutting plane; The cut surface after the discard command is executed is subjected to dual-mode rendering for section sealing, and a filling material is generated to obtain the visualized 3D model.

6. The maintenance auxiliary decision generation method according to claim 5, characterized in that, After performing dual-mode rendering of the cut surface after executing the discard command, generating a filling material, and obtaining the visualized 3D model, the process further includes: The LOD level is determined based on the distance between the visualized 3D model and the camera, and a cross-fade-in / fade-out algorithm is used to smoothly switch between different models; among them, different models include high-level models, medium-level models and low-level models. Construct an axis-aligned bounding box for each model; the axis-aligned bounding box includes 8 vertices; Based on the view matrix and the projection matrix, a view frustum is constructed, and the positional relationship between the axis-aligned bounding box vertices and the view frustum plane is detected. Models in which all vertices are located outside any plane of the view frustum are eliminated. The view frustum comprises 6 planes.

7. The maintenance auxiliary decision generation method according to claim 1, characterized in that, Based on the visualized 3D model and a multimodal data binding mechanism, a two-way linkage between the 3D model and maintenance knowledge is achieved to generate maintenance auxiliary decisions, specifically including: A bidirectional mapping relationship between the metadata of the visualized 3D model and maintenance knowledge is constructed to obtain the association matrix, and then assigned to each visualized 3D model component and maintenance knowledge. ; When a component selection operation is received for the visualized 3D model, according to the... The correlation matrix is ​​queried to extract strongly correlated maintenance knowledge and push it to the front-end interface, realizing a positive linkage between model operation and knowledge push. When a maintenance knowledge query command is received, the key component entities in the text are identified, the strongly associated model components of the maintenance knowledge are retrieved, and the 3D camera is driven to automatically translate and zoom to the local area, realizing the reverse linkage from knowledge query to model positioning. Based on the positive linkage and the negative linkage, maintenance auxiliary decisions are generated.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the maintenance assistance decision generation method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the maintenance auxiliary decision generation method according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the maintenance auxiliary decision generation method according to any one of claims 1-7.

Images