Industrial software user auxiliary system and construction method

Abstract

The invention provides an industrial software user auxiliary system and a construction method, and the system comprises a data collection and preprocessing module which is used for obtaining an industrial software user document; wherein the user document comprises image data and text data; the image analysis module is used for converting the image data into a semantic text; the vectorization coding module is used for carrying out vectorization coding on the semantic text and the text data to obtain a semantic feature vector; the vector index database construction module is used for determining the topological information of the engineering entities according to the topological relation between the engineering entities corresponding to the semantic feature vectors; constructing a vector index database according to the semantic feature vectors and the engineering entity topological information; and the intelligent retrieval and question-answer generation module is used for performing similarity search in the vector index database under the condition of receiving information input by the user and outputting a query result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence and industrial software, and particularly relates to an industrial software user auxiliary system and a construction method. BACKGROUND

[0002] Industrial software (such as CAD, CAE, EDA software, etc.) is a core tool supporting the research and development design of modern high-end manufacturing industry. Due to the characteristics of complex functions, numerous modules, and strong professional operation processes, users (especially novice engineers) often face the problems of high difficulty and high threshold in using the industrial software. In the traditional way, when the user encounters a problem in using the industrial software, the user usually searches the user documents and knowledge base. The user needs to accurately extract keywords and manually search in a large amount of PDF manuals, HTML help documents, internal manuals, and guide documents. This way is inefficient, the information is scattered, the update is lagging, and the complex problems described in natural language (such as "how to fix this twisted mesh?") cannot be understood. For problems that need to be combined with interface screenshots or error pop-up windows, there is no solution, and there is a lack of context association ability. SUMMARY

[0003] Embodiments of the present disclosure provide an industrial software user auxiliary system and a construction method to solve the problems that the existing industrial software is difficult to use and the problems raised by the user are difficult to solve.

[0004] Based on the above problems, in a first aspect, an industrial software user auxiliary system is provided, comprising: a data acquisition and preprocessing module configured to acquire industrial software user documents; wherein the user documents comprise image data and text data; and the operation object of the industrial software comprises a plurality of engineering entities; an image analysis module configured to convert the image data into semantic text; a vectorization encoding module configured to vectorize encode the semantic text and the text data to obtain semantic feature vectors; a vector index database construction module configured to determine engineering entity topological information according to a topological relationship between the semantic feature vectors and the corresponding engineering entities, and construct a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is configured to record a mapping relationship between the semantic feature vectors and semantic source data; and the semantic source data comprises at least one of the following: semantic text, image data, and text data; an intelligent retrieval and question answering generation module configured to, in the case of receiving user input information, perform a similarity search in the vector index database, and output a query result.

[0005] In a second aspect, a construction method of an industrial software user assistance system is provided, comprising: acquiring industrial software user documents; wherein the user documents comprise image data and text data; and the operation objects of the industrial software comprise a plurality of engineering entities; converting the image data into semantic text; vectorizing and encoding the semantic text and the text data to obtain semantic feature vectors; determining engineering entity topological information according to the topological relationships between the corresponding engineering entities of the semantic feature vectors; constructing a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is used to record the mapping relationship between the semantic feature vectors and semantic source data; and the semantic source data at least comprises one of the following: semantic text, image data and text data; in the case of receiving user input information, performing similarity search in the vector index database and outputting query results.

[0006] The beneficial effects of the embodiments of the present disclosure include: The industrial software user assistance system and the construction method provided by the present disclosure comprise: a data acquisition and preprocessing module, which is used to acquire industrial software user documents; wherein the user documents comprise image data and text data; and the operation objects of the industrial software comprise a plurality of engineering entities; an image analysis module, which is used to convert the image data into semantic text; a vectorization and encoding module, which is used to vectorize and encode the semantic text and the text data to obtain semantic feature vectors; a vector index database construction module, which is used to determine engineering entity topological information according to the topological relationships between the corresponding engineering entities of the semantic feature vectors; and construct a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is used to record the mapping relationship between the semantic feature vectors and semantic source data; and the semantic source data at least comprises one of the following: semantic text, image data and text data; and an intelligent retrieval and question answering generation module, which is used to perform similarity search in the vector index database and output query results in the case of receiving user input information. The industrial software user assistance system provided by the embodiments of the present disclosure is based on the retrieval technology of semantic feature vectors, which can accurately associate the related information scattered in the image data and the text data, and greatly improves the accuracy of solving complex problems. In the index and retrieval stage, the topological perception mechanism is introduced, the topological relationships between the engineering entities such as assembly constraints, geometric adjacency and hierarchical parent-child in the industrial software are modeled and integrated into the similarity search, which effectively filters the results that are only similar in form but do not match in engineering logic, and significantly improves the retrieval accuracy and usability in industrial scenarios. Thus, the learning and use threshold of the user for the complex industrial software is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 A structural diagram of an industrial software user assistance system provided by an embodiment of the present disclosure is provided. Figure 2 A flowchart of converting image data into semantic text provided by an embodiment of the present disclosure is provided. Figure 3 A flowchart of vectorizing encoding a text block provided by an embodiment of the present disclosure is provided. Figure 4 A flowchart of an intelligent block splitting strategy provided by an embodiment of the present disclosure is provided. Figure 5 A flowchart of intelligent retrieval and question and answer generation provided by an embodiment of the present disclosure is provided. Figure 6 A schematic diagram of a question and answer robot based on text information retrieval enhancement generation obtaining a query result provided by an embodiment of the present disclosure is provided. Figure 7 A schematic diagram of an industrial software user assistance system obtaining a query result provided by an embodiment of the present disclosure is provided. Figure 8 A schematic diagram of an industrial software user assistance system architecture provided by an embodiment of the present disclosure is provided. Figure 9 A flowchart of a construction method of an industrial software user assistance system provided by an embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0008] The embodiment of the present disclosure provides an industrial software user assistance system and a construction method. The preferred embodiments of the present disclosure are described below in conjunction with the drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure. Furthermore, the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0009] The embodiment of the present disclosure provides an industrial software user assistance system, as shown in Figure 1 includes: A data acquisition and preprocessing module 101 is configured to acquire industrial software user documents. The user documents include image data and text data. The operation objects of the industrial software include a plurality of engineering entities. An image analysis module 102 is configured to convert the image data into semantic text. A vectorization encoding module 103 is configured to vectorize encode the semantic text and the text data to obtain a semantic feature vector. The vector index database construction module 104 is configured to determine engineering entity topological information according to the topological relationship between the engineering entities corresponding to the semantic feature vectors; and construct a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is configured to record a mapping relationship between the semantic feature vectors and semantic source data; and the semantic source data at least includes one of the following: semantic text, image data and text data. The intelligent retrieval and question answering generation module 105 is configured to perform a similarity search in the vector index database in the case of receiving user input information, and output a query result.

[0010] In the embodiments of the present disclosure, industrial software (such as CAD, CAE, EDA software, etc.) is a core tool for supporting the research and design of modern high-end manufacturing industry. Due to the characteristics of complex functions, numerous modules, and strong professional operation processes, users (especially novice engineers) often face the following typical problems in the process of using these industrial software: 1. Deep coupling between industrial software and domain knowledge: software functions are closely related to engineering discipline knowledge (such as finite element theory, fluid dynamics), and professional terms are difficult to understand (such as "turbulence model", "relaxation factor", "modal analysis", etc.); 2. Difficult to understand the operation process: the user is not clear about the operation steps of a specific function (such as meshing, boundary condition setting, post-processing analysis), and needs to cross multiple modules or multiple document queries; 3. Difficulty in parameter configuration: a large number of professional parameters (such as material model parameters, convergence control threshold) in industrial software lack immediate guidance and explanation of their meanings and reasonable value ranges; 4. Obstacle in interpreting error information: software error information (such as solver error codes, geometry check failure prompts) is obscure and difficult to understand, and lacks targeted solutions; 5. Difficulty in understanding interface elements: the user has a fuzzy understanding of the functions of icons, menu items, and dialog box options in the complex interface of industrial software; 6. Difficulty in solving user-proposed problems: user-proposed problems often need to be combined with user documents (manuals / logs), interface screenshots (UI state), code snippets (configuration files / scripts), structured data (parameter tables), and even engineering drawings to accurately answer them.

[0011] To solve the problems encountered in the use of these industrial software, users need to search the user documents and knowledge base. Users need to accurately extract keywords and manually search in massive PDF manuals, HTML help documents, internal manuals, guide documents. This way of searching is inefficient, information is scattered, updating is lagging, and complex problems described in natural language (such as "how to fix this twisted mesh?") cannot be understood. For problems that need to be combined with interface screenshots or error pop-up windows, there is no solution, and there is a lack of context association ability. Users can also solve the problems they face through community forums and expert consultations. However, relying on users to post actively or contact experts, the response delay is high (several hours to several days). Expert experience is difficult to standardize and reuse, and the efficiency of repeated answers to similar problems is low. Unstructured discussion information is difficult to accurately match immediate needs, and there is a risk of outdated or incorrect information. Some industrial software user assistance systems use text information retrieval to enhance the generated question and answer robot during construction, which can solve the problems of low efficiency of traditional user document and knowledge base search, scattered information, etc. However, pure text-based information retrieval enhancement cannot locate related legends, annotations and other related information, and cannot process user-uploaded software interface screenshots, error prompt box images, log file fragments, parameter configuration tables and other multi-modal information, resulting in incomplete understanding and lack of operational guidance for industrial software use.

[0012] In the embodiments of the present disclosure, by fusing multi-modal information such as image data, text data and topological relationship between engineering entities, joint retrieval and semantic understanding are performed to realize accurate analysis and operation guidance of complex problems of industrial software. First, the data collection and preprocessing module 101 collects user documents of industrial software. The user documents can refer to operation manuals, case manuals, user guides and other document data including a large number of picture examples of industrial software. The user documents can be subjected to quality screening (such as clarity, relevance) and format standardization processing to provide high-quality input for subsequent links. The user documents include image data and text data. For the image data, for example, interface screenshots, flowcharts, icons, etc. For the text data, for example, operation manuals, parameter descriptions, term explanations, etc. The image analysis module 102 is configured to analyze the image data, for example, by recognizing characters through OCR, identifying interface elements through target detection, converting the image data into processable semantic text, and realizing unified representation of images and texts. For example, a multi-modal large language model (MLLM, Multimodal Large Language Model) is used to process the image data to generate semantic text (such as icon function explanation, interface state description, error scenario analysis), realizing deep conversion of image information to text information. The vectorization encoding module 103 is configured to convert the semantic text and the text data in the user documents into semantic feature vectors. For example, a pre-trained embedding model is used to convert the semantic text and the text data into high-dimensional semantic feature vectors. This process can effectively capture the deep semantic features of the text and provide support for retrieval based on semantic similarity. The numerical distribution of the vector reflects the semantic meaning of the text, so that the computer can understand the information correlation through vector operation. The engineering entity can refer to an operation object in the industrial software. For example, a three-dimensional model part or assembly created in CAD, a mesh in CAE, etc. There are corresponding topological relationships between engineering entities, such as assembly constraints, geometric adjacency, parent-child hierarchy, etc. The vector index database construction module 104 can obtain the topological relationship between engineering entities through the user documents, and then construct engineering entity topological information. The topological relationship between engineering entities can reveal the internal logic of the operation process of the operation object. Each engineering entity includes a corresponding semantic feature vector representation. The engineering entity topological information constructed based on the semantic feature vector can refer to the association relationship structure between various engineering entities in the industrial software, which reflects the internal connection mode of the various engineering entities in logic, process or function, and can provide support for cross-module query.

[0013] Further, the vector index database construction module 104 constructs a vector index database by integrating the semantic feature vector and the engineering entity topology information. The vector index database records the mapping relationship between the semantic feature vector and the semantic source data (semantic text, image data, and text data), and stores the engineering entity topology information. Through index optimization (such as KD tree), efficient similarity search is realized to ensure fast positioning of relevant information. The intelligent search and question answering generation module 105 is used to convert the input information into a query vector after receiving the input information of the user, search for semantic feature vectors and associated engineering entity topology information in the vector index data, integrate the matched multi-modal data such as semantic text, image data, and text data, and output the solution in an intuitive form. For example, if the retrieved semantic feature vector corresponds to semantic text or image data, the image data in the user document can be quickly referenced and displayed in the answer, thereby realizing accurate and efficient user assistance.

[0014] The embodiments of the present application can accurately associate relevant information scattered in image data and text data based on the retrieval technology of the semantic feature vector, and greatly improve the accuracy of solving complex problems. In the index and retrieval stage, a topology awareness mechanism is introduced to model and integrate the topology relationship between engineering entities such as assembly constraints, geometric adjacency, and hierarchical parent-child in industrial software into similarity search, effectively filtering results that are only similar in form but do not match in engineering logic, and significantly improving the retrieval accuracy and usability in industrial scenarios. Thus, the learning and use threshold of complex industrial software by users is reduced.

[0015] In another embodiment of the present disclosure, the image analysis module 102 is used to: input the image data into a multi-modal large model to obtain image semantic description text; input the image semantic description text and the text data related to the image data in the user document into a pure text large model to obtain semantic text.

[0016] In the embodiments of the present disclosure, an image-based description is generated by a multi-modal large model, and then combined with associated text data to be optimized by a pure text large model, and finally a precise semantic text is obtained. For image data in the user document of industrial software, the image analysis module 102 uses a multi-modal large model with image understanding capability for analysis. The multi-modal large model can generate image semantic description text by recognizing visual elements (such as buttons, icons, menu structures, geometric figures), spatial layout, and text information in the image, thereby realizing the preliminary conversion of image to text and extracting the intuitive visual semantics of the image. The image semantic description text can be in JSON format. Inputting the image data into the multi-modal large model, in order to improve the accuracy of image data recognition, system prompts can be used to guide the multi-modal large model to extract information related to the image and generate a JSON description. The multi-modal large model can generate a JSON format preliminary description containing the key content of the image as the image semantic description text under the guidance of the visual features of the image and the prompts. JSON (JavaScript Object Notation) is a lightweight data exchange format with the characteristics of easy parsing, high readability, and flexible representation of data structure, which is convenient for quick access and processing. Collecting text data associated with the current image data in the user document (such as the explanatory text of the chapter where the image data is located, the operation manual fragment corresponding to the function, etc.), inputting it into a pure text large model (LLM, Large Language Model) together with the image semantic description text. The pure text large model optimizes the image semantic description text by understanding the relevance of the two, such as supplementing professional terms (such as correcting “a certain calculation model” to “k-ε turbulent flow model”), clarifying the function logic (such as supplementing “This interface is used to set the grid discretization parameters for fluid simulation, and needs to be operated after the geometric modeling is completed”), and correcting ambiguous expressions, and finally generates a precise, professional, and contextually appropriate semantic text for industrial software. By fusing image visual information and text context knowledge, the professionalism and completeness of the semantic text are improved, providing high-quality input for subsequent data processing. For example, as shown in Figure 2 , Figure 2 A flowchart for converting image data into semantic text includes the following steps: S201, inputting image data into a multi-modal large model, and using system prompts to guide the multi-modal large model to generate a JSON format image semantic description text; S202, determining whether the JSON text parsing is successful (the JSON text parsing is to ensure the correctness and standardization of the JSON data); if yes, go to step S206; if no, go to step S203; S203, resetting the system prompts; S204, judge whether the number of reset system prompt words is less than a preset value; if yes, go to step S201; if no, go to step S205; S205, record the JSON text analysis error log; the process ends; S206, input the context information related to the image semantic description text and the image data into the pure text large model to obtain a semantic text; the process ends.

[0017] In yet another embodiment of the present disclosure, the vectorization coding module 103 is configured to: perform block processing on the semantic text and the text data after fusion to obtain a plurality of text blocks; input the plurality of text blocks into an embedding model for vectorization coding to obtain a semantic feature vector.

[0018] In the embodiments of the present disclosure, the semantic text and the text data in the industrial software user document are processed by block processing and embedded model coding to obtain semantic feature vectors. The vectorization coding module 103 fuses the semantic text with the original text data in the user document to form a complete text set. Since the text data in the industrial software user document can be long (such as a large section of the operation manual, a detailed description of the parameter document, etc.), it exceeds the processing length limit of the embedded model, and therefore needs to be processed by block processing. The block processing can be performed according to a fixed length (such as 512 tokens per block, and the token is the basic unit of text processing of the model, which can be understood as a fragment of a word or a character), semantic logic (such as according to semantic units such as chapters, function modules, paragraphs, etc.), etc., to ensure that each text block retains local semantic integrity while meeting the input requirements of the embedded model, and obtains multiple text blocks. For example, a long text about grid partitioning of CAE software can be divided into blocks according to semantic paragraphs such as grid type introduction, grid size setting, and grid quality checking. An appropriate embedded model (such as BERT, Sentence-BERT, Doc2Vec, etc., which can be selected or fine-tuned according to the characteristics of the industrial software field) is selected, and the multiple text blocks are sequentially input into the embedded model. In the vector generation stage, the embedded model converts each text block into a high-dimensional vector through semantic analysis of the text. In the high-dimensional space, the semantic distance between text vectors corresponding to similar semantics is closer. For example, inputting the turbulence model parameter setting, the embedded model will generate a high-dimensional vector representing the text, and the high-dimensional vectors of the text blocks related to fluid dynamics analysis will have a high similarity in space. The semantic feature vector can be a high-dimensional vector. Further, in the vector normalization stage, the semantic feature vector can be a high-dimensional vector after vector normalization, for example, L2 norm normalization, to obtain a normalized vector. The semantic feature vector not only retains the semantic features of the text, but also has a unified numerical specification, and can be directly used for cosine similarity calculation (to measure semantic similarity), vector retrieval (to quickly match related text), etc., and is especially suitable for the semantic matching of user queries and document content in the industrial software user assistance system. Block processing can adapt to long text, avoiding information loss or coding effect degradation due to the length of the text exceeding the processing capacity of the embedded model; the embedded model coding converts the text into a semantic feature vector, so that the computer can understand the semantic association of the text through vector operation, providing a basis for subsequent similarity search, knowledge association, etc. For example Figure 3 As shown in Figure 3 The flowchart for vectorizing the text block includes the following steps: S301, input the text block into the embedded model to obtain a high-dimensional vector; S302, perform vector normalization on the high-dimensional vector to obtain a normalized vector.

[0019] In another embodiment of this disclosure, the vectorization encoding module 103 is used for: After fusing semantic text and textual data, the data is input into the inference model for semantic analysis to determine multiple segmentation points. Based on the type of segmentation point, the fused semantic text and text data are divided into blocks according to a preset priority to obtain multiple text blocks; The preset priorities, from highest to lowest, include: the first priority of semantic segmentation points after the current text block reaches the minimum word count threshold; the second priority of paragraph segmentation points after the current text block reaches the minimum word count threshold; and the third priority of forced segmentation points after the current text block reaches the maximum word count threshold. Specifically, if the position of the segmentation point corresponding to the current text block is inside the semantic text, the segmentation point is set at the end of the semantic text.

[0020] In this embodiment, segmentation points are determined through an inference model, and then the text is divided into blocks based on the segmentation point type and preset priority to obtain multiple suitable text blocks. This ensures that each text block has complete semantic units and good contextual coherence, laying the foundation for efficient retrieval. Figure 4 As shown, the vectorization encoding module 103 fuses the semantic text (text converted from image data) with the original text data in the user document to form a complete text set. Then, the fused text is input into the inference model. Figure 4 The input is a long text (S401). The inference model performs semantic analysis on the text, identifying locations in the text that reflect semantic logical pauses and transitions; these locations are the segmentation points. Figure 4 (Semantic analysis of the S402 inference model). For example, in the text of industrial software operation instructions, the inference model may determine the segmentation point at the junction of two different semantic operations: setting boundary conditions and performing mesh partitioning. Preset priorities, from high to low, include: the first priority of semantic segmentation points after the current text block reaches the minimum word count threshold (…). Figure 4 S4031 Priority 1 Semantic Segmentation Point); the second priority of the paragraph segmentation point after the current text block reaches the minimum word count threshold ( Figure 4 S4032 Priority 2 Paragraph Split Point), the third priority of the forced split point after the current text block reaches the maximum word count threshold ( Figure 4 S4033 Priority 3 Forced Segmentation Point). When segmenting the merged text ( Figure 4In the intelligent segmentation strategy S403, firstly, it is checked whether there is a first-priority segmentation point, i.e., the current text block being formed reaches a pre-set minimum word threshold, and there is a semantic segmentation point, and segmentation is performed at the semantic segmentation point. If there is no first-priority segmentation point, it is further checked whether there is a second-priority paragraph segmentation point (also subject to the current text block reaching the minimum word threshold), and if there is, segmentation is performed at the paragraph segmentation point. If neither of the above is satisfied, when the current text block reaches a maximum word threshold, a third-priority forced segmentation point is triggered to perform segmentation. If the position of a segmentation point is located inside a semantic text, since the semantic text is text converted from image data, in order to ensure the integrity of the semantic text, the segmentation point is adjusted to the end position of the semantic text, so as to avoid the semantic text being split into different text blocks, and ensure that the complete semantics carried by the semantic text can be embodied in one text block. Figure 5 In the special processing of picture extension S4034, for example, the semantic text is a complete explanation of a turbulent flow model, and if the segmentation point is inside the explanation text, the segmentation point is moved to the end of the turbulent flow model explanation text, so that the entire explanation is in one text block. After the segmentation processing is completed, a scoring mechanism can be introduced to judge whether the text block is appropriate, for example, scoring is performed in dimensions such as semantic coherence, paragraph boundary fitting degree, integrity of image and text, and length deviation degree. If the score is lower than the expected value, the segmentation processing is re-performed. Figure 5 In the segmentation size appropriateness S404, the segmentation point is determined by using a reasoning model to perform semantic analysis, which can more accurately capture the semantic logic of the text, so that the segmentation is more in line with semantic integrity, and the semantic coherence is avoided from being damaged due to mechanical segmentation. The pre-set priority mechanism comprehensively considers the semantic, paragraph and word limit, balances the semantic integrity and the length requirement of segmentation, and adapts to the demand for text block size in different scenarios. The special processing of the segmentation point inside the semantic text ensures the integrity of the semantic text as a whole, avoids unreasonable splitting, and improves the rationality and professionalism of the segmentation.

[0021] In yet another embodiment of the present disclosure, the vector index database construction module 104 is configured to: According to the topological relationship between the engineering entities corresponding to the semantic feature vectors, a weighted undirected graph between the engineering entities is determined, and the formula is expressed as: ; Wherein, represents a node set, each node represents an engineering entity, and each engineering entity corresponds to a semantic feature vector; represents an edge set, each edge represents that there is a topological association between the engineering entities, represents the weight of each edge, which is used to represent the topological closeness between the engineering entities.

[0022] In the embodiments of the present disclosure, by the engineering entity topological relationship corresponding to the semantic feature vector, a weighted undirected graph is constructed, and the abstract engineering entity association is converted into a structured graph model, and the core is to define the nodes, edges and edge weights of the graph to quantify the topological tightness between engineering entities. Set Each node in the set corresponds to an independent engineering entity in the industrial software operation object, for example, a part operated by CAD software, and each node is a basic unit of topological relationship. Each node (i.e., each engineering entity) is bound to its corresponding semantic feature vector. The semantic feature vector is generated by vectorization encoding of semantic text and text data, and the numerical distribution of the vector implies the semantic attributes of the engineering entity. Set Each edge in the set is a connection between two nodes (engineering entities), and the edge is created only when there is a topological relationship between the two engineering entities, and no edge is created if there is no topological relationship. And because the graph is undirected, the direction of the edge does not affect the association relationship (i.e., the edge between node A and node B indicates that node A is associated with node B, and also indicates that node B is associated with node A). The creation of the edge is based on the actual topological relationship of the engineering entity, and at least one of the following: assembly constraint, geometric adjacency, hierarchical parent-child relationship. Assembly constraint is used to define the relative position and motion relationship between parts, to ensure that the parts are combined together in the correct way in the assembly. Geometric adjacency refers to the spatial adjacency or contact relationship between different geometric elements (such as points, lines, surfaces, and bodies) in a geometric model. For example, in a three-dimensional model of a mechanical part, two adjacent planes may be connected by a common edge, or the side surface of a cylinder may be tangent to a plane. In industrial software, these adjacent and tangent geometric relationships can be geometric adjacency. The hierarchical parent-child relationship is used to describe the hierarchical structure and subordinate relationship between engineering entities, and a parent entity can contain multiple child entities, and the child entities inherit some attributes of the parent entity or are constrained by the parent entity. is the weight of each edge, which is used to represent the topological tightness of the topological association between the corresponding two engineering entities. The greater the weight value, the stronger the association between the two engineering entities in terms of function, process or semantics, and vice versa. The weight value can be assigned based on the type of topological relationship between the two engineering entities, the number of existing topological relationships, the frequency of use, etc. The scattered engineering entities and their topological relationships in industrial software are converted into an intuitive weighted undirected graph, making the complex topological relationship quantifiable and analyzable, avoiding the ambiguity of the association logic, and quantifying the topological tightness between entities by the edge weight. In subsequent similarity search, the engineering entities with stronger association can be matched first by combining the weight, improving the query accuracy.

[0023] In still another embodiment of the present disclosure, the intelligent retrieval and question and answer generation module 105 is configured to: input the input information of the user into the multi-modal large model to obtain an input description text; The input description text is fed into the embedding model for vectorization to obtain the query vector; Based on the query vector, a similarity search is performed in the vector index database to determine the retrieval information; The search information is input into the large language model to obtain the query results.

[0024] In this embodiment, the intelligent retrieval and question-answering generation module 105 unifies the descriptive form of user image and text input through a multimodal large model, generates query vectors through an embedded model, matches relevant information in a vector index database, and then outputs the final result by a large language model, thereby achieving an accurate response to the user's multimodal needs. Figure 5 As shown, in the vector index database construction phase, the vector index database construction module 104 extracts the topological relationships between engineering entities corresponding to semantic feature vectors through S501; S502, constructs an index to record the mapping relationship between semantic feature vectors and semantic source data; and S503, constructs the vector index database. In the retrieval phase, it receives user input information ( Figure 5 The input (S504 user input) can include images and text. For example, a user uploads a screenshot of an error pop-up window from a CAE software solver and enters the text "How do I fix this error?". The image and text are input into a multimodal large-scale model, where information from the image (such as error codes and interface elements) is transformed into machine-processable text semantics. This semantics is then correlated with the user's textual intent to obtain the input description text, providing a unified text foundation for subsequent vectorized encoding. The input description text is then transformed into a high-dimensional semantic vector, i.e., a query vector, through an embedding model. Figure 5 The S505 embedding model is transformed into a query vector, enabling the system to measure the semantic similarity between the query vector and the semantic feature vectors in the vector index database through vector operations. A similarity retrieval algorithm is then invoked to search the vector index database, calculate the similarity between the query vector and the semantic feature vectors in the vector index database, and obtain the retrieval information. Figure 6 (S506 hybrid similarity calculation). If multiple high-similarity results exist, they are sorted from highest to lowest similarity, and the information with the highest relevance is selected as the retrieval information. A large language model is used to process the retrieval information and generate query results. Figure 6 (See S507 query results). This transforms the scattered and technical raw information retrieved (such as manual excerpts and parameter descriptions) into user-friendly, logically clear natural language answers, adapting to the user's actual usage scenarios. It is compatible with multimodal user input information, including images and text, addressing the pain point that users often struggle to clearly express themselves using only text descriptions (e.g., beginners can directly upload screenshots when they cannot accurately describe interface operation issues). Figure 6 As shown, Figure 7 This is a diagram illustrating the query results obtained by a question-answering robot generated based on enhanced text information retrieval.Figure 7 The Q&A robot returned 601 results for the user question, "How to install aifem," but only provided plain text descriptions such as "preparations before installation," "Windows installation steps," "license activation," and "precautions." These descriptions lacked a direct connection to the industrial software's user interface, requiring users to manually interpret the steps. For beginners, this could easily lead to operational errors due to unfamiliarity with interface elements, limiting efficiency and accuracy. Figure 7 As shown, Figure 7 A diagram illustrating how an industrial software user assistance system obtains query results. Figure 7 The document only shows the steps "1. System Preparation", "2. Start Installation", "3. Accept Agreement" and "4. Set Path". Steps "5. Create Shortcut", "6. Confirm Installation", "7. Installation Complete", and "8. Activation Method" are not shown. Figure 8 (Not shown in the image). Because the vector index database records the mapping relationships between semantic feature vectors and semantic text, image data, and text data, therefore, for the user question: "How to install aifem?", by comparing the AI's answer, it can be found that... Figure 8 The industrial software user assistance system obtained query result 701. Based on the installation steps in text form, it can link and display screenshots of the original software operation interface, providing users with intuitive and process-oriented visual guidance, significantly improving understanding efficiency and operational accuracy.

[0025] In another embodiment of this disclosure, the intelligent retrieval and question-answering generation module 105 is used for: Based on the query vector, a similarity search is performed in the vector index database to determine a hybrid distance metric that combines the query vector with semantic feature vectors and engineering entity topology information. The formula is as follows: ; ; ; in, Indicates a mixed distance metric; Indicates semantic distance; Represents topological distance; Represents the query vector; Represents semantic feature vectors; This represents the project entity corresponding to the query vector; This represents the engineering entity corresponding to the semantic feature vector; and Indicates the mixed weighting coefficient; Indicates the connection node and The path; denotes a connection node and an edge on a path; denotes a connection node and a weight of an edge on a path; determine the retrieval information according to the size of the hybrid distance metric.

[0026] In the embodiments of the present disclosure, by constructing a hybrid distance metric model of semantic distance and topological distance, combining a hybrid weight coefficient, the correlation degree of the query vector and the semantic feature vector is comprehensively calculated, and finally the most relevant retrieval information is screened according to the size of the hybrid distance metric. The semantic distance can quantify the semantic similarity between the query vector and the semantic feature vector in the vector index library. The smaller the distance, the more similar the semantics. The topological distance can be based on the weighted undirected graph between engineering entities (formula: ;) denotes an engineering entity corresponding to the semantic feature vector, which can be a node of the weighted undirected graph, thereby quantifying the topological correlation tightness between the engineering entity corresponding to the query vector and the engineering entity corresponding to the semantic feature vector. The smaller the distance, the more closely the topological correlation. The hybrid distance metric is calculated by integrating the semantic distance and the topological distance, and the retrieval information is screened according to the hybrid distance metric from small to large. The top-ranked retrieval information of a predetermined number can be screened and sorted to ensure that the comprehensive correlation degree of the result is the highest. Compared with the search that only depends on the semantic distance, the topological distance of the engineering entity is integrated, which can avoid the false matching caused by the semantic similarity but the topological irrelevance, effectively filter the results that are similar in form but not matched in engineering logic, and make the results more consistent with the function logic of the industrial software.

[0027] In another embodiment of the present disclosure, the intelligent retrieval and question answering generation module 105 is configured to: According to the size of the hybrid distance metric, the hybrid distance metric is pruned by using a topological pruning strategy, and the formula is: ; denotes an invalid hybrid distance metric after pruning; denotes a pruning threshold factor; denotes the smallest hybrid distance metric retrieved; determine the retrieval information according to the hybrid distance metric after pruning.

[0028] In the embodiments of the present disclosure, by using the topological pruning strategy, the smallest hybrid distance metric retrieved is used as the benchmark, and the valid hybrid distance metric is screened by combining the pruning threshold factor. The final retrieval information is determined after eliminating the invalid distance metric. The core is to improve the search efficiency on the premise of ensuring the retrieval accuracy. represents the minimum mixed distance metric retrieved; represents the distance corresponding to the information with the highest comprehensive relevance to the query vector in the current query, which is the benchmark reference for pruning. represents the pruning threshold factor, a parameter that controls the strictness of pruning, the smaller the pruning is more strict (less information is retained), the larger the pruning is more relaxed (more information is retained). represents the mixed distance metric, represents the invalid mixed distance metric after pruning, which can be used as a pruning marker. According to the mixed distance metric after pruning, the retrieval information is determined. Based on the valid mixed distance metric, the corresponding original information is extracted to provide accurate materials for the subsequent large language model generation result. By pruning the invalid distance metric far greater than the minimum mixed distance metric, it avoids the information with extremely low semantic or topological relevance from entering the subsequent process, ensuring that the final retrieval information focuses on the core associated content.

[0029] In yet another embodiment of the present disclosure, the intelligent retrieval and question answering generation module 105 is configured to: According to the query vector, similarity search is performed in the vector index database to determine the mixed similarity of the query vector with the semantic feature vector and the engineering entity topological information, which is expressed by the formula: ; ; ; ; wherein, represents the mixed similarity; represents the semantic similarity; represents the distance similarity; represents the query vector; represents the semantic feature vector; represents the engineering entity corresponding to the query vector; represents the engineering entity corresponding to the semantic feature vector; and represents the mixed weight coefficient; represents the attenuation parameter; represents the topological distance; represents the path connecting nodes and ; represents the edge on the path connecting nodes and ; represents the weight of the edge on the path connecting nodes and ; According to the size of the mixed similarity, the retrieval information is determined.

[0030] In the embodiments of the present disclosure, by constructing a hybrid similarity model of semantic similarity and distance similarity, combining weight coefficient and topological distance decay calculation, the comprehensive correlation degree of the query vector and the semantic feature vector is quantified, and finally the most relevant search information is screened according to the size of the hybrid similarity. The calculation of semantic similarity can quantify the semantic matching degree of the query vector and the semantic feature vector in the vector index library. The calculation of distance similarity converts the topological distance into a distance similarity with the same dimension as the semantic similarity, realizing the additivity of the two-dimensional similarity. By the exponential function (d (x, y) = e ; ), the topological distance is converted into similarity (the closer the distance, the higher the similarity), which not only retains the quantitative characteristics of topological correlation, but also controls the degree of topological influence through the decay parameter and adapts to the difference of topological correlation strength of different industrial software. The topological distance can be based on the weighted undirected graph between engineering entities (formula: ; ), represents the engineering entity corresponding to the semantic feature vector, which can be used as the node of the weighted undirected graph. Based on the semantic similarity and the distance similarity, the neighbor nodes can be screened, so that the search information conforms to the engineering assembly logic. The weight contribution of the comprehensive semantic similarity and the distance similarity is calculated to calculate the hybrid similarity, and the search information is screened according to the hybrid similarity, so as to ensure that the result has the highest comprehensive correlation degree.

[0031] As shown in Figure 9 , ​An auxiliary system architecture for industrial software users is shown in the figure. The user interface layer 801 serves as an entry for users to interact with the system, receives input information from the user, and displays the final query results. Multi-terminal adaptive interaction methods can be provided to meet the needs of users in different scenarios for querying auxiliary information of industrial software. The API service layer 802 is used to process HTTP requests from the user interface layer, perform interface routing, parameter analysis, and result packaging. The API service layer 802 can include multiple components, such as RESTful API, routing management, parameter verification, and other API components, to ensure the normativity and reliability of request processing. The complexity of underlying processing is shielded to provide a simple and unified calling method for the user interface layer 801, and the request parameters are verified to improve system stability. The core processing layer 803 can convert user input information into a query vector to provide a basis for subsequent similarity calculation. The vector index database construction module calculates a hybrid similarity by combining semantic similarity and distance similarity, quantifies the correlation between user input information and information in the vector index database, and considers both semantic matching and the topological relationship of engineering entities in industrial software. Based on the hybrid similarity and topological pruning, data that matches semantics and topology is retrieved from the vector index database, and the index storage, semantic feature vector, and engineering entity topological information of the vector index database are used to quickly locate related information. The recall processing can be used to filter the retrieved data, and the text matching and topological matching are combined to further optimize the accuracy of the recall results. Thus, the semantic and topological dimensions are fused to solve the problem of missing topological associated information in the industrial software field due to the complexity of functions and the close association of modules, and to improve the accuracy and professionalism of the retrieval. The data processing layer 804 can integrate and process the retrieval information from the core processing layer 803, and finally return the query results (including text content and original pictures, etc.) to the API service layer 802 and the user interface layer 801. The vector index database 805 provides index storage, semantic feature vector, and engineering entity topological information, and records the mapping relationship between the semantic feature vector and the semantic source data, to provide data support for the topological awareness retrieval of the core processing layer 803. Through efficient vector storage and retrieval capabilities, combined with engineering entity topological information, the retrieval based on hybrid similarity can be quickly executed.

[0032] Based on the same disclosure concept, the disclosure embodiments also provide a construction method of an industrial software user auxiliary system. Since the principles of the problems solved by these methods are similar to those of the aforementioned industrial software user auxiliary system, the implementation of the method can be referred to the implementation of the aforementioned industrial software user auxiliary system, and the repeated parts will not be described again.

[0033] The disclosure embodiments provide a construction method of an industrial software user auxiliary system, as shown in ​ the figure. S901, obtain industrial software user documents; wherein the user documents include image data and text data; and operation objects of the industrial software include a plurality of engineering entities; S902, convert the image data into semantic text; S903, vectorize encode the semantic text and the text data to obtain a semantic feature vector; S904, determine engineering entity topological information according to a topological relationship between the corresponding engineering entities of the semantic feature vector; S905, construct a vector index database according to the semantic feature vector and the engineering entity topological information; wherein the vector index database is used to record a mapping relationship between the semantic feature vector and semantic source data; and the semantic source data at least includes one of the following: semantic text, image data, and text data; S906, in the case of receiving user input information, perform a similarity search in the vector index database and output a query result.

[0034] In another embodiment of the present disclosure, the conversion of the image data into semantic text includes: inputting the image data into a multi-modal large model to obtain image semantic description text; inputting the image semantic description text and the text data related to the image data in the user documents into a pure text large model to obtain semantic text.

[0035] In another embodiment of the present disclosure, the vectorization encoding of the semantic text and the text data to obtain a semantic feature vector includes: performing block processing on the fusion of the semantic text and the text data to obtain a plurality of text blocks; inputting the plurality of text blocks into an embedding model for vectorization encoding to obtain a semantic feature vector.

[0036] In another embodiment of the present disclosure, the block processing on the fusion of the semantic text and the text data to obtain a plurality of text blocks includes: inputting the fusion of the semantic text and the text data into an inference model for semantic analysis to determine a plurality of segmentation points; according to the type of the segmentation points, block the fused semantic text and text data according to a preset priority to obtain a plurality of text blocks; wherein the preset priority from high to low includes: a first priority of a semantic segmentation point after the current text block reaches a minimum word threshold; a second priority of a paragraph segmentation point after the current text block reaches the minimum word threshold, and a third priority of a forced segmentation point after the current text block reaches a maximum word threshold. In a case where the position of the current text block corresponding to the split point is located inside the semantic text, the split point is set at the position of the end of the semantic text.

[0037] In yet another embodiment of the present disclosure, the determining of the topological information of the engineering entities according to the topological relationship between the engineering entities corresponding to the semantic feature vectors comprises: According to the topological relationship between the engineering entities corresponding to the semantic feature vectors, a weighted undirected graph between the engineering entities is determined, and the formula is expressed as: ; wherein, represents a node set, each node representing an engineering entity, and each engineering entity corresponding to a semantic feature vector; represents an edge set, each edge representing a topological association between engineering entities, represents the weight of each edge, used to represent the topological closeness between engineering entities.

[0038] In yet another embodiment of the present disclosure, in a case where user input information is received, a similarity search is performed in the vector index database, and a query result is output, comprising: inputting the input information of the user into the multi-modal large model to obtain input description text; inputting the input description text into the embedding model for vectorization coding to obtain a query vector; According to the query vector, a similarity search is performed in the vector index database to determine retrieval information; inputting the retrieval information into the large language model to obtain a query result.

[0039] In yet another embodiment of the present disclosure, the determining of the retrieval information according to the query vector in the vector index database comprises: According to the query vector, a similarity search is performed in the vector index database to determine the hybrid distance measure of the query vector and the semantic feature vector and the topological information of the engineering entities, and the formula is expressed as: ; ; ; wherein, represents a hybrid distance measure; represents a semantic distance; represents a topological distance; represents a query vector; represents a semantic feature vector; denotes an engineering entity corresponding to the query vector; denotes an engineering entity corresponding to the semantic feature vector; and denotes a hybrid weight coefficient; denotes a path connecting nodes and ; denotes an edge on a path connecting nodes and ; denotes a weight of an edge on a path connecting nodes and ; determining retrieval information according to the size of the hybrid distance measure.

[0040] In yet another embodiment of the present disclosure, the determining retrieval information according to the size of the hybrid distance measure comprises: cutting the hybrid distance measure using a topological pruning strategy according to the size of the hybrid distance measure, which is expressed by the formula: ; denotes a cut invalid hybrid distance measure; denotes a pruning threshold factor; denotes a retrieved minimum hybrid distance measure; determining retrieval information according to the cut hybrid distance measure.

[0041] In yet another embodiment of the present disclosure, the determining retrieval information according to the query vector, performing a similarity search in the vector index database comprises: performing a similarity search in the vector index database according to the query vector to determine a hybrid similarity between the query vector and the semantic feature vector and the engineering entity topological information, which is expressed by the formula: ; ; ; ; wherein, denotes a hybrid similarity; denotes a semantic similarity; denotes a distance similarity; denotes a query vector; denotes a semantic feature vector; denotes an engineering entity corresponding to the query vector; denotes an engineering entity corresponding to the semantic feature vector; and denotes a mixing weight coefficient; denotes a decay parameter; denotes a topological distance; denotes a path connecting nodes and . denotes an edge on a path connecting nodes and . denotes a weight of an edge on a path connecting nodes and . determines the search information according to the size of the mixed similarity.

[0042] Those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by hardware, or by means of software and necessary universal hardware platforms. Based on such understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present disclosure.

[0043] Those skilled in the art can understand that the drawings are only schematic of a preferred embodiment, and the modules or flows in the drawings are not necessarily essential for implementing the present disclosure.

[0044] Those skilled in the art can understand that the modules in the device in the embodiments can be distributed in the device in the embodiments according to the description of the embodiments, or can be changed and located in one or more devices different from the embodiments. The modules in the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0045] The above sequence numbers of the embodiments of the present disclosure are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0046] Obviously, those skilled in the art can make various modifications and changes to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and changes of the present disclosure belong to the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure also intends to include these modifications and changes.

Claims

1. An industrial software user assistance system, characterized in that, include: The data acquisition and preprocessing module is used to acquire industrial software user documents; wherein, the user documents include: image data and text data; the operation objects of the industrial software include multiple engineering entities; The image analysis module is used to convert the image data into semantic text; The vectorization encoding module is used to perform vectorization encoding on the semantic text and the text data to obtain semantic feature vectors; A vector index database construction module is used to determine the topological information of engineering entities based on the topological relationship between the engineering entities corresponding to the semantic feature vectors; and to construct a vector index database based on the semantic feature vectors and the topological information of the engineering entities; wherein, the vector index database is used to record the mapping relationship between the semantic feature vectors and semantic source data; the semantic source data includes at least one of the following: semantic text, image data, and text data; The intelligent retrieval and question-answering generation module is used to perform similarity searches in the vector index database upon receiving user input information and output the query results.

2. The system as described in claim 1, characterized in that, The image analysis module is used for: The image data is input into a multimodal large model to obtain image semantic description text; The semantic description text of the image and the text data related to the image data in the user document are input into the plain text large model to obtain the semantic text.

3. The system as described in claim 1, characterized in that, The vectorization encoding module is used for: The semantic text and the text data are fused and then segmented to obtain multiple text blocks; Multiple text blocks are input into the embedding model for vectorized encoding to obtain semantic feature vectors.

4. The system as described in claim 3, characterized in that, The vectorization encoding module is used for: The semantic text and the text data are fused and then input into the inference model for semantic analysis to determine multiple segmentation points; Based on the type of the segmentation point, the fused semantic text and text data are divided into blocks according to a preset priority to obtain multiple text blocks; The preset priorities, from high to low, include: the first priority of the semantic segmentation point after the current text block reaches the minimum word count threshold; the second priority of the paragraph segmentation point after the current text block reaches the minimum word count threshold; and the third priority of the forced segmentation point after the current text block reaches the maximum word count threshold. Specifically, if the position of the segmentation point corresponding to the current text block is located inside the semantic text, the segmentation point is set at the end of the semantic text.

5. The system as described in claim 1, characterized in that, The vector index database construction module is used for: Based on the topological relationships between the engineering entities corresponding to the semantic feature vectors, a weighted undirected graph of the engineering entities is determined, expressed by the formula: ； in, This represents a set of nodes, where each node represents an engineering entity, and each engineering entity corresponds to a semantic feature vector. This represents a set of edges, where each edge indicates a topological relationship between engineering entities. This represents the weight of each edge, used to characterize the topological density between engineering entities.

6. The system as described in claim 1, characterized in that, The intelligent retrieval and question-answering generation module is used for: The user's input information is fed into a multimodal large model to obtain the input description text; The input description text is input into the embedding model for vector encoding to obtain the query vector; Based on the query vector, a similarity search is performed in the vector index database to determine the retrieval information; The search information is input into the large language model to obtain the query results.

7. The system as described in claim 6, characterized in that, The intelligent retrieval and question-answering generation module is used for: Based on the query vector, a similarity search is performed in the vector index database to determine a hybrid distance metric between the query vector, the semantic feature vector, and the engineering entity topology information. The formula is as follows: ； ； ； in, Indicates a mixed distance metric; Indicates semantic distance; Represents topological distance; Represents the query vector; Represents semantic feature vectors; This represents the project entity corresponding to the query vector; This represents the engineering entity corresponding to the semantic feature vector; and Indicates the mixed weighting coefficient; Indicates the connection node and The path; Indicates the connection node and Edges on the path; Indicates the connection node and Weights along the path; The retrieval information is determined based on the magnitude of the hybrid distance metric.

8. The system as described in claim 7, characterized in that, The intelligent retrieval and question-answering generation module is used for: Based on the magnitude of the hybrid distance metric, a topology pruning strategy is used to prune the hybrid distance metric, as expressed by the formula: ； Indicates the invalid blending distance metric; Indicates the pruning threshold factor; This represents the minimum mixed distance metric retrieved. The retrieval information is determined based on the post-cropping blending distance metric.

9. The system as described in claim 6, characterized in that, The intelligent retrieval and question-answering generation module is used for: Based on the query vector, a similarity search is performed in the vector index database to determine the mixed similarity between the query vector and the semantic feature vector and the engineering entity topology information, expressed by the formula: ； ； ； ； in, Indicates mixed similarity; Indicates semantic similarity; Indicates distance similarity; Represents the query vector; Represents semantic feature vectors; This represents the project entity corresponding to the query vector; This represents the engineering entity corresponding to the semantic feature vector; and Indicates the mixed weighting coefficient; Indicates the attenuation parameter; Represents topological distance; Indicates the connection node and The path; Indicates the connection node and Edges on the path; Indicates the connection node and Weights along the path; The retrieval information is determined based on the magnitude of the mixed similarity.

10. A method for constructing an industrial software user-aided system, characterized in that, include: Obtain industrial software user documentation; wherein, the user documentation includes: image data and text data; the operation objects of the industrial software include multiple engineering entities; The image data is converted into semantic text; The semantic text and the text data are vectorized and encoded to obtain semantic feature vectors; Based on the topological relationships between the engineering entities corresponding to the semantic feature vectors, the topological information of the engineering entities is determined; Based on the semantic feature vectors and the engineering entity topology information, a vector index database is constructed; wherein, the vector index database is used to record the mapping relationship between the semantic feature vectors and the semantic source data; the semantic source data includes at least one of the following: semantic text, image data, and text data; Upon receiving user input, a similarity search is performed in the vector index database, and the query results are output.

Images