An industrial software user assistance system and a construction method

Abstract

The present disclosure provides an industrial software user auxiliary system and a construction method, comprising: a data acquisition and preprocessing module for obtaining industrial software user documents; wherein the user documents comprise: image data and text data; an image analysis module for converting the image data into semantic text; a vectorization encoding module for vectorizing and encoding the semantic text and the text data to obtain semantic feature vectors; a vector index database construction module for determining engineering entity topological information according to the topological relationship between the engineering entities corresponding to the semantic feature vectors; constructing a vector index database according to the semantic feature vectors and the engineering entity topological information; an intelligent retrieval and question answering generation module for performing similarity search in the vector index database under the condition of receiving user input information, and outputting query results.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence and industrial software, and particularly relates to an industrial software user assistance system and a construction method. BACKGROUND

[0002] Industrial software (such as CAD, CAE, EDA software, etc.) is a core tool supporting the research and development design of modern high-end manufacturing industry. Due to its complex functions, numerous modules, and strong professional operation process, users (especially novice engineers) often face the problems of high difficulty and high threshold in using the industrial software. In the traditional way, when the user encounters a problem in using the industrial software, the user usually searches the user documents and knowledge base. The user needs to accurately extract keywords and manually search in a large amount of PDF manuals, HTML help documents, internal manuals, and guide documents. This way is inefficient, the information is scattered, the update is lagging, and it is difficult to understand the complex problems described in natural language (such as "how to fix this twisted mesh?"). There is no way to solve the problem combined with the interface screenshot or error popup, and there is a lack of context association ability. SUMMARY

[0003] Embodiments of the present disclosure provide an industrial software user assistance system and a construction method to solve the problem that the existing industrial software is difficult to use and the user's problem is difficult to solve.

[0004] Based on the above problems, in a first aspect, an industrial software user assistance system is provided, comprising:

[0005] a data acquisition and preprocessing module configured to acquire industrial software user documents; wherein the user documents comprise image data and text data; and the operation object of the industrial software comprises a plurality of engineering entities;

[0006] an image analysis module configured to convert the image data into semantic text;

[0007] a vectorization encoding module configured to vectorize encode the semantic text and the text data to obtain a semantic feature vector;

[0008] a vector index database construction module configured to determine engineering entity topological information according to a topological relationship between the semantic feature vectors and the corresponding engineering entities, and construct a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is configured to record a mapping relationship between the semantic feature vectors and semantic source data; and the semantic source data comprises at least one of the following: semantic text, image data, and text data;

[0009] An intelligent retrieval and question-answering generation module is configured to perform a similarity search in the vector index database upon receiving user input information and output query results.

[0010] In a second aspect, a construction method of an industrial software user assistance system is provided, including:

[0011] Obtaining industrial software user documents; wherein the user documents include image data and text data; and operation objects of the industrial software include a plurality of engineering entities;

[0012] Converting the image data into semantic text;

[0013] Vectorizing and encoding the semantic text and the text data to obtain semantic feature vectors;

[0014] Determining engineering entity topological information according to topological relationships between engineering entities corresponding to the semantic feature vectors;

[0015] Constructing a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is configured to record mapping relationships between the semantic feature vectors and semantic source data; and the semantic source data at least includes one of the following: semantic text, image data and text data;

[0016] Performing a similarity search in the vector index database upon receiving user input information and outputting query results.

[0017] The beneficial effects of the embodiments of the present disclosure include:

[0018] The industrial software user auxiliary system and construction method provided by the present disclosure comprises a data acquisition and preprocessing module for obtaining industrial software user documents; wherein the user documents comprise image data and text data; the operation objects of the industrial software comprise a plurality of engineering entities; an image analysis module for converting the image data into semantic text; a vectorization coding module for vectorization coding of the semantic text and the text data to obtain semantic feature vectors; a vector index database construction module for determining engineering entity topological information according to the topological relationship between the semantic feature vectors and the engineering entities; and constructing a vector index database according to the semantic feature vectors and the engineering entity topological information; wherein the vector index database is used to record the mapping relationship between the semantic feature vectors and the semantic source data; the semantic source data at least comprises one of the following: semantic text, image data and text data; an intelligent retrieval and question answering generation module for similarity search in the vector index database under the condition of receiving user input information, and outputting query results. The industrial software user auxiliary system provided by the present disclosure is based on the retrieval technology of semantic feature vectors, which can accurately associate the relevant information scattered in the image data and the text data, and greatly improves the accuracy of solving complex problems. In the index and retrieval stage, the topological perception mechanism is introduced, the topological relationship between the engineering entities such as assembly constraints, geometric adjacency and hierarchical parent-child in the industrial software is modeled and integrated into the similarity search, which effectively filters the results that are only similar in form but do not match in engineering logic, and significantly improves the retrieval accuracy and usability in industrial scenarios. Thus, the learning and use threshold of complex industrial software by users is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The structural diagram of the industrial software user auxiliary system provided by the present disclosure is provided.

[0020] Figure 2 The flowchart of converting image data into semantic text provided by the present disclosure is provided.

[0021] Figure 3 The flowchart of vectorization coding of the text block provided by the present disclosure is provided.

[0022] Figure 4 The flowchart of the intelligent block strategy provided by the present disclosure is provided.

[0023] Figure 5 The flowchart of intelligent retrieval and question answering generation provided by the present disclosure is provided.

[0024] Figure 6 The schematic diagram of the question answering robot based on text information retrieval enhancement generation provided by the present disclosure obtaining query results is provided.

[0025] Figure 7A schematic diagram of obtaining a query result by the industrial software user assistance system provided by the embodiments of the present disclosure is shown in FIG. 1.

[0026] Figure 8 A schematic diagram of the architecture of the industrial software user assistance system provided by the embodiments of the present disclosure is shown in FIG. 2.

[0027] Figure 9 A flowchart of the construction method of the industrial software user assistance system provided by the embodiments of the present disclosure is shown in FIG. 3. DETAILED DESCRIPTION

[0028] The embodiments of the present disclosure provide an industrial software user assistance system and a construction method. The preferred embodiments described herein are used to illustrate and explain the present disclosure, and are not used to limit the present disclosure. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] The embodiments of the present disclosure provide an industrial software user assistance system, as shown in FIG. 1, comprising: Figure 1

[0030] A data acquisition and preprocessing module 101 is configured to acquire industrial software user documents. The user documents include image data and text data. The operation objects of the industrial software include a plurality of engineering entities.

[0031] An image analysis module 102 is configured to convert the image data into semantic text.

[0032] A vectorization encoding module 103 is configured to vectorize and encode the semantic text and the text data to obtain a semantic feature vector.

[0033] A vector index database construction module 104 is configured to determine engineering entity topological information according to the topological relationship between the engineering entities corresponding to the semantic feature vector, and to construct a vector index database according to the semantic feature vector and the engineering entity topological information. The vector index database is configured to record the mapping relationship between the semantic feature vector and semantic source data. The semantic source data at least includes one of the following: semantic text, image data and text data.

[0034] An intelligent retrieval and question answering generation module 105 is configured to perform similarity search in the vector index database under the condition of receiving user input information, and to output a query result.

[0035] ​Industrial software (such as CAD, CAE, EDA software, etc.) is the core tool to support modern high-end manufacturing research and development design. Due to its complex functions, numerous modules, and strong professional operation process characteristics, users (especially novice engineers) often face the following typical problems in the process of using these industrial software:

[0036] 1. Deep coupling between industrial software and domain knowledge: software functions are closely related to engineering knowledge (such as finite element theory, fluid dynamics), and professional terms are difficult to understand (such as "turbulence model", "relaxation factor", "modal analysis", etc.);

[0037] 2. Difficult to understand the operation process: the user is not clear about the operation steps of a specific function (such as meshing, boundary condition setting, post-processing analysis), and needs to cross-module or multi-document query;

[0038] 3. Difficulty in parameter configuration: there is a lack of immediate guidance for the meaning and reasonable value range of a large number of professional parameters (such as material model parameters, convergence control threshold) in industrial software;

[0039] 4. Obstacle in error information interpretation: software error information (such as solver error code, geometry check failure prompt) is obscure and difficult to understand, and lacks targeted solutions;

[0040] 5. Difficulty in understanding interface elements: users have a fuzzy understanding of the functions of icons, menu items, and dialog box options in the complex interface of industrial software;

[0041] 6. Difficulty in solving user problems: user problems often need to be combined with user documents (manuals / logs), interface screenshots (UI status), code snippets (configuration files / scripts), structured data (parameter tables), and even engineering drawings to accurately solve them.

[0042] To solve the problems encountered in the use of these industrial software, users need to search the user documents and knowledge base. Users need to accurately extract keywords and manually search in massive PDF manuals, HTML help documents, internal manuals, guide documents. This way of searching is inefficient, information is scattered, updating is lagging, and complex problems described in natural language (such as "how to fix this twisted mesh?") cannot be understood. For problems that need to be combined with interface screenshots or error pop-up windows, there is no solution, and there is a lack of context association ability. Users can also solve the problems they face through community forums and expert consultations. However, relying on users to post actively or contact experts, the response delay is high (several hours to several days). Expert experience is difficult to standardize and reuse, and the efficiency of repeated answers to similar problems is low. Unstructured discussion information is difficult to accurately match immediate needs, and there is a risk of outdated or incorrect information. Some industrial software user assistance systems use text information retrieval to enhance the generated question and answer robot during construction, which can solve the problems of low efficiency of traditional user document and knowledge base search, scattered information, etc. However, pure text-based information retrieval enhancement cannot locate related legends, annotations and other related information, and cannot process user-uploaded software interface screenshots, error prompt box images, log file fragments, parameter configuration tables and other multi-modal information, resulting in incomplete understanding and lack of operational guidance for industrial software use.

[0043] In the embodiments of the present disclosure, by fusing multi-modal information such as image data, text data and topological relationship between engineering entities, joint retrieval and semantic understanding are performed to realize accurate analysis and operation guidance of complex problems of industrial software. First, the data collection and preprocessing module 101 collects user documents of industrial software. The user documents can refer to operation manuals, case manuals, user guides and other document data including a large number of picture examples of industrial software. The user documents can be subjected to quality screening (such as clarity, relevance) and format standardization processing to provide high-quality input for subsequent links. The user documents include image data and text data. For the image data, for example, interface screenshots, flowcharts, icons, etc. For the text data, for example, operation manuals, parameter descriptions, term explanations, etc. The image analysis module 102 is configured to analyze the image data, for example, by recognizing characters through OCR, identifying interface elements through target detection, converting the image data into processable semantic text, and realizing unified representation of images and texts. For example, a multi-modal large language model (MLLM, Multimodal Large Language Model) is used to process the image data to generate semantic text (such as icon function explanation, interface state description, error scenario analysis), realizing deep conversion of image information to text information. The vectorization encoding module 103 is configured to convert the semantic text and the text data in the user documents into semantic feature vectors. For example, a pre-trained embedding model is used to convert the semantic text and the text data into high-dimensional semantic feature vectors. This process can effectively capture the deep semantic features of the text and provide support for retrieval based on semantic similarity. The numerical distribution of the vector reflects the semantic meaning of the text, so that the computer can understand the information correlation through vector operation. The engineering entity can refer to an operation object in the industrial software. For example, a three-dimensional model part or assembly created in CAD, a mesh in CAE, etc. There are corresponding topological relationships between engineering entities, such as assembly constraints, geometric adjacency, parent-child hierarchy, etc. The vector index database construction module 104 can obtain the topological relationship between engineering entities through the user documents, and then construct engineering entity topological information. The topological relationship between engineering entities can reveal the internal logic of the operation process of the operation object. Each engineering entity includes a corresponding semantic feature vector representation. The engineering entity topological information constructed based on the semantic feature vector can refer to the association relationship structure between various engineering entities in the industrial software, which reflects the internal connection mode of the various engineering entities in logic, process or function, and can provide support for cross-module query.

[0044] Further, the vector index database construction module 104 constructs a vector index database by integrating the semantic feature vector and the engineering entity topology information. The vector index database records the mapping relationship between the semantic feature vector and the semantic source data (semantic text, image data, and text data), and stores the engineering entity topology information. Through index optimization (such as KD tree), efficient similarity search is realized to ensure fast positioning of relevant information. The intelligent search and question answering generation module 105 is used to convert the input information into a query vector after receiving the input information of the user, search for semantic feature vectors and associated engineering entity topology information in the vector index data, integrate the matched multi-modal data such as semantic text, image data, and text data, and output the solution in an intuitive form. For example, if the retrieved semantic feature vector corresponds to semantic text or image data, the image data in the user document can be quickly referenced and displayed in the answer, thereby realizing accurate and efficient user assistance.

[0045] Based on the retrieval technology of the semantic feature vector, the embodiments of the present application can accurately associate relevant information scattered in image data and text data, and greatly improve the accuracy of solving complex problems. In the index and retrieval stage, a topology awareness mechanism is introduced to model and integrate the topology relationship between engineering entities such as assembly constraints, geometric adjacency, and hierarchical parent-child in industrial software into similarity search, effectively filtering results that are only similar in form but do not match in engineering logic, and significantly improving the retrieval accuracy and usability in industrial scenarios. Thus, the learning and use threshold of complex industrial software by users is reduced.

[0046] In another embodiment of the present disclosure, the image analysis module 102 is configured to:

[0047] input the image data into the multi-modal large model to obtain image semantic description text;

[0048] input the image semantic description text and the text data related to the image data in the user document into the pure text large model to obtain semantic text.

[0049] In the embodiments of the present disclosure, an image-based description is generated by a multi-modal large model, and then combined with associated text data to be optimized by a pure text large model, and finally a precise semantic text is obtained. For image data in the user document of industrial software, the image analysis module 102 uses a multi-modal large model with image understanding capability for analysis. The multi-modal large model can generate image semantic description text by recognizing visual elements (such as buttons, icons, menu structures, geometric figures), spatial layout, and text information in the image, thereby realizing the preliminary conversion of image to text and extracting the intuitive visual semantics of the image. The image semantic description text can be in JSON format. To improve the accuracy of image data recognition, system prompts can be used to guide the multi-modal large model to extract information related to the image and generate a JSON description. The multi-modal large model can generate a JSON format preliminary description containing the key content of the image as the image semantic description text under the guidance of the visual features of the image and the prompts. JSON (JavaScript Object Notation) is a lightweight data exchange format with the characteristics of easy parsing, high readability, and flexible representation of data structure, which facilitates quick access and processing. Collect the text data associated with the current image data in the user document (such as the explanatory text of the chapter where the image data is located, the operation manual fragment corresponding to the function, etc.), and input it together with the image semantic description text into a pure text large model (LLM, Large Language Model). The pure text large model optimizes the image semantic description text by understanding the relevance of the two, such as supplementing professional terms (such as correcting "a certain calculation model" to "k-ε turbulent flow model"), clarifying the function logic (such as supplementing "this interface is used to set the grid discretization parameters for fluid simulation, and needs to be operated after the geometric modeling is completed"), and correcting ambiguous expressions, and finally generates a precise, professional, and contextually appropriate semantic text for industrial software. By fusing image visual information and text context knowledge, the professionalism and completeness of the semantic text are improved, providing high-quality input for subsequent data processing. For example, as shown in Figure 2 , Figure 2 The flowchart for converting image data into semantic text includes the following steps:

[0050] S201, input the image data into the multi-modal large model, and use the system prompts to guide the multi-modal large model to generate a JSON format image semantic description text;

[0051] S202, determine whether the JSON text parsing is successful (the JSON text parsing is to ensure the correctness and standardization of the JSON data); if yes, go to step S206; if no, go to step S203;

[0052] S203, reset the system prompts;

[0053] S204, judge whether the number of reset system prompt words is less than a preset value; if yes, go to step S201; if no, go to step S205;

[0054] S205, record the JSON text parsing error log; the process ends;

[0055] S206, input the context information related to the image semantic description text and the image data into the pure text large model to obtain a semantic text; the process ends.

[0056] In yet another embodiment of the present disclosure, the vectorization encoding module 103 is configured to:

[0057] perform block processing on the semantic text and the text data after fusion to obtain a plurality of text blocks;

[0058] input the plurality of text blocks into an embedding model for vectorization encoding to obtain a semantic feature vector.

[0059] In this embodiment, semantic text and text data from industrial software user documents are segmented and encoded using an embedding model to obtain semantic feature vectors. The vectorization encoding module 103 merges the semantic text with the original text data in the user document to form a complete text set. Since the text data in industrial software user documents may be quite long (e.g., large sections of instructions in operation manuals, detailed descriptions in parameter documents, etc.), exceeding the processing length limit of the embedding model, segmentation is necessary. Segmentation can be performed according to fixed lengths (e.g., 512 tokens per segment, where a token is the basic unit of text processing by the model, which can be understood as a fragment of words or characters) or semantic logic (e.g., according to chapters, functional modules, paragraphs, etc.), ensuring that each text block retains its local semantic integrity while meeting the input requirements of the embedding model, resulting in multiple text blocks. For example, a long text about CAE software mesh partitioning can be segmented according to semantic paragraphs such as mesh type introduction, mesh size settings, and mesh quality checks. Select a suitable embedding model (such as BERT, Sentence-BERT, Doc2Vec, etc., which can be selected or fine-tuned according to the characteristics of the industrial software domain), and input multiple text blocks into the embedding model sequentially. In the vector generation stage, the embedding model transforms each text block into a high-dimensional vector through semantic analysis of the text. In this high-dimensional space, texts with similar semantics have a closer semantic distance. For example, when inputting turbulence model parameter settings, the embedding model will generate a high-dimensional vector representing the text, and the high-dimensional vectors of text blocks related to fluid dynamics analysis will have a high similarity in space. Semantic features can be vectors, which can also be high-dimensional vectors. Further, in the vector standardization stage, the semantic feature vectors can be high-dimensional vectors that have undergone vector normalization, such as L2 norm normalization, to obtain standardized vectors. The semantic feature vectors retain the semantic features of the text and have a unified numerical standard, making them directly applicable to tasks such as cosine similarity calculation (measuring semantic similarity) and vector retrieval (quickly matching related text), especially suitable for semantic matching scenarios between user queries and document content in industrial software user assistance systems. Chunking can adapt to long texts, avoiding information loss or reduced encoding performance caused by text length exceeding the processing capacity of the embedding model; the embedding model encodes text into semantic feature vectors, enabling computers to understand semantic relationships through vector operations, providing a foundation for subsequent similarity searches, knowledge associations, and other operations. For example... Figure 3 As shown, Figure 3 The flowchart for vectorizing text blocks includes the following steps:

[0060] S301. The text block is input into the embedding model to obtain a high-dimensional vector;

[0061] S302. Normalize the high-dimensional vector to obtain a standardized vector.

[0062] In yet another embodiment of the present disclosure, the vectorization coding module 103 is configured to:

[0063] input the fused semantic text and text data into an inference model for semantic analysis to determine a plurality of segmentation points;

[0064] According to the type of the segmentation points, the fused semantic text and text data are blocked according to a preset priority to obtain a plurality of text blocks;

[0065] The preset priority includes, from high to low, a first priority of a semantic segmentation point after a current text block reaches a minimum word threshold, a second priority of a paragraph segmentation point after the current text block reaches the minimum word threshold, and a third priority of a forced segmentation point after the current text block reaches a maximum word threshold.

[0066] In a case where the position of the segmentation point corresponding to the current text block is located inside the semantic text, the segmentation point is set at the end position of the semantic text.

[0067] In the embodiment of the present disclosure, the segmentation points are determined by the inference model, and then the segmentation point types and the preset priority are combined for blocking to obtain a plurality of appropriate text blocks. Thus, each text block has a complete semantic unit and good context coherence, laying a foundation for efficient retrieval. Figure 4 As shown in FIG. 1, the vectorization coding module 103 fuses the semantic text (text converted from image data) and the original text data in the user document to form a complete text set. Then, the fused text is input into the inference model (S401 in FIG. 1). Figure 4 The inference model performs semantic analysis on the text to identify positions that can reflect semantic logical pauses and transitions in the text, which are the segmentation points (S402 in FIG. 1). Figure 4 For example, in the text of an industrial software operation instruction, the inference model can determine a segmentation point at the junction of setting boundary conditions and grid division, which are two different semantic operations. The preset priority includes, from high to low, a first priority of a semantic segmentation point after a current text block reaches a minimum word threshold (S4031 in FIG. 1), a second priority of a paragraph segmentation point after the current text block reaches the minimum word threshold (S4032 in FIG. 1), and a third priority of a forced segmentation point after the current text block reaches a maximum word threshold (S4033 in FIG. 1). Figure 4 Figure 4 Figure 4 Figure 4 ​​​In the intelligent segmentation strategy S403, firstly, it is checked whether there is a first-priority segmentation point, i.e., the current text block being formed reaches a pre-set minimum word threshold, and there is a semantic segmentation point, and segmentation is performed at the semantic segmentation point. If there is no first-priority segmentation point, it is further checked whether there is a second-priority paragraph segmentation point (also subject to the current text block reaching the minimum word threshold), and if there is, segmentation is performed at the paragraph segmentation point. If neither of the above is satisfied, when the current text block reaches a maximum word threshold, a third-priority forced segmentation point is triggered to perform segmentation. If the position of a segmentation point is located inside a semantic text, since the semantic text is text converted from image data, in order to ensure the integrity of the semantic text, the segmentation point is adjusted to the end position of the semantic text, so as to avoid the semantic text being split into different text blocks, and ensure that the complete semantics carried by the semantic text can be embodied in one text block. Figure 5 In the special processing of picture extension S4034, for example, the semantic text is a complete explanation of a turbulent flow model, and if the segmentation point is inside the explanation text, the segmentation point is moved to the end of the turbulent flow model explanation text, so that the entire explanation is in one text block. After the segmentation processing is completed, a scoring mechanism can be introduced to judge whether the text block is appropriate, for example, by scoring in dimensions such as semantic coherence, paragraph boundary fitting degree, image-text integrity, length deviation degree, etc. If the score is lower than the expected value, the segmentation processing is re-performed. Figure 5 In the segmentation size appropriateness S404, the segmentation point is determined by using a reasoning model to perform semantic analysis, which can more accurately capture the semantic logic of the text, so that the segmentation is more in line with semantic integrity, and the semantic coherence is avoided to be destroyed due to mechanical segmentation. The pre-set priority mechanism comprehensively considers the semantic, paragraph and word limit, balances the semantic integrity and the length requirement of segmentation, and adapts to the demand for text block size in different scenarios. The special processing of the segmentation point inside the semantic text ensures the integrity of the semantic text as a whole, avoids unreasonable splitting, and improves the rationality and professionalism of the segmentation.

[0068] In yet another embodiment of the present disclosure, the vector index database construction module 104 is configured to:

[0069] According to the topological relationship between the engineering entities corresponding to the semantic feature vectors, a weighted undirected graph between the engineering entities is determined, and the formula is expressed as:

[0070] ;

[0071] wherein, represents a node set, each node represents an engineering entity, and each engineering entity corresponds to a semantic feature vector; represents an edge set, each edge represents that there is a topological association between the engineering entities, represents the weight of each edge, used to represent the topological density between engineering entities.

[0072] In the embodiments of the present disclosure, the weighted undirected graph is constructed through the topological relationship of the engineering entity corresponding to the semantic feature vector, the abstract engineering entity association is converted into a structured graph model, and the core is to define the nodes, edges and edge weights of the graph to quantify the topological density between engineering entities. Each node in the set Each node in the set corresponds to an independent engineering entity in the industrial software operation object, for example, a part operated by CAD software, and each node is a basic unit of topological relationship. Each node (i.e., each engineering entity) is bound to its corresponding semantic feature vector. The semantic feature vector is generated by the vectorization encoding of the semantic text and the text data, and the numerical distribution of the vector implies the semantic attributes of the engineering entity. The set Each edge in the set is a connection between two nodes (engineering entities), and the edge is created only when there is a topological relationship between the two engineering entities, and no edge is created if there is no topological relationship. And because the graph is an undirected graph, the direction of the edge does not affect the association relationship (i.e., the edge between node A and node B represents node A associating node B, and also represents node B associating node A). The creation of the edge is based on the actual topological relationship of the engineering entity, and at least one of the following: assembly constraint, geometric adjacency, hierarchical parent-child relationship. Assembly constraint is used to define the relative position and motion relationship between parts, to ensure that the parts are combined together in the correct way in the assembly. Geometric adjacency refers to the spatial adjacency or contact relationship between different geometric elements (such as points, lines, surfaces, and bodies) in a geometric model. For example, in a three-dimensional model of a mechanical part, two adjacent planes may be connected by a common edge, or the side surface of a cylindrical body may be tangent to a plane. In industrial software, these adjacent and tangent geometric relationships can be geometric adjacency. The hierarchical parent-child relationship is used to describe the hierarchical structure and subordinate relationship between engineering entities, and a parent entity can contain multiple child entities, and the child entities inherit some attributes of the parent entity or are constrained by the parent entity. is the weight of each edge, used to represent the topological density of the topological association between the two corresponding engineering entities. The greater the weight value, the stronger the association between the two engineering entities in terms of function, process or semantics, and vice versa. The weight value can be assigned based on the type of topological relationship between the two engineering entities, the number of existing topological relationships, the frequency of use, etc. The scattered engineering entities and their topological relationships in industrial software are converted into an intuitive weighted undirected graph, making the complex topological relationship quantifiable and analyzable, avoiding the ambiguity of the association logic, and quantifying the topological density between entities through the edge weight. In subsequent similarity search, the engineering entities with stronger association can be matched first by combining the weight, improving the query accuracy.

[0073] In still another embodiment of the present disclosure, the intelligent retrieval and question and answer generation module 105 is configured to:

[0074] Input the input information of the user into the multi-modal large model to obtain input description text;

[0075] Input the input description text into the embedding model for vectorization coding to obtain a query vector;

[0076] According to the query vector, perform similarity search in the vector index database to determine retrieval information;

[0077] Input the retrieval information into the large language model to obtain a query result.

[0078] In the embodiments of the present disclosure, the intelligent retrieval and question and answer generation module 105 unifies the description forms of user images and text inputs through the multi-modal large model, generates a query vector through the embedding model, matches related information in the vector index database, and outputs the final result by the large language model, thereby realizing accurate response to the multi-modal needs of the user. Figure 5 As shown in the vector index database construction stage, the vector index database construction module 104 extracts the topological relationship between the semantic feature vectors corresponding to the engineering entities through S501; constructs an index and records the mapping relationship between the semantic feature vectors and the semantic source data through S502; and constructs the vector index database through S503. In the retrieval stage, the user input information (S504 in the Figure 5 ) is received. The input information can include images and text. For example, the user uploads a screenshot of the CAE software solver error popup window and inputs the text "How to solve this error?". The image and text are input into the multi-modal large model, the information in the image (such as error code, interface element) is converted into machine-processable text semantics, and the user's text intention is associated to obtain input description text, which provides a unified text basis for subsequent vectorization coding. The input description text is converted into a high-dimensional semantic vector, i.e., a query vector (S505 in the Figure 5 embedding model is converted into a query vector), so that the system can measure the semantic similarity between the query vector and the semantic feature vector in the vector index database through vector operation. The similarity retrieval algorithm is called to retrieve in the vector index database, the similarity between the query vector and the semantic feature vector in the vector index database is calculated, and retrieval information is obtained (S506 in the Figure 6 mixed similarity calculation). If there are multiple high-similarity results, the information with the highest correlation degree is selected as the retrieval information in priority according to the similarity from high to low. The retrieval information is processed by the large language model to generate a query result (S507 in the Figure 6The query result in the middle S507). Thus, the retrieved scattered and professional original information (such as manual fragments, parameter descriptions) is converted into natural language answers understandable by users and clear in logic, which is adapted to the actual use scenarios of users. Multi-modal user input information of images and texts is compatible, and the pain points of users in clearly expressing problems by single text description (such as when a novice cannot accurately describe an interface operation problem, a screenshot can be directly uploaded) are solved. As shown in Figure 6 Figure 7 An illustrative view of a query result obtained by a question and answer robot based on text information retrieval and enhanced generation; Figure 7 In the middle, the question and answer robot obtains the query result 601, and for the "user question: how to install aifem", only the answer description in the form of pure text is included, for example, the text form description of "installation preparation", "Windows installation steps", "license activation", and "notes", which lacks direct association with the operation interface of the industrial software, and the user needs to understand the steps by comparing the software interface, and for a novice, operation deviation may occur due to fuzzy recognition of interface elements, and the improvement of understanding efficiency and operation accuracy is limited. As shown in Figure 7 Figure 7 An illustrative view of a query result obtained by an industrial software user auxiliary system Figure 7 In the middle, only the steps of "1. System preparation", "2. Start installation", "3. Accept agreement", and "4. Set path" are shown, and for the steps of "5. Create shortcut", "6. Confirm installation", "7. Installation completed", and "8. Activation method", Figure 8 In the middle, the mapping relationship between the semantic feature vector and the semantic text, image data, and text data recorded in the vector index database is not shown). Therefore, for the "user question: how to install aifem", by comparing the AI answer, it can be found that Figure 8 The query result 701 of the industrial software user auxiliary system in the middle is shown, based on the text form installation steps, the original software operation interface screenshot can be associated and displayed, providing intuitive and flow-based visual guidance for users, and significantly improving the understanding efficiency and operation accuracy.

[0079] In yet another embodiment of the present disclosure, the intelligent retrieval and question and answer generation module 105 is configured to:

[0080] According to the query vector, similarity search is performed in the vector index database, and the mixed distance measure of the query vector, the semantic feature vector, and the engineering entity topology information is determined, which is expressed by the formula:

[0081] ;

[0082] ;

[0083] ; ​​

[0084] wherein, denotes a hybrid distance metric; denotes a semantic distance; denotes a topological distance; denotes a query vector; denotes a semantic feature vector; denotes an engineering entity corresponding to the query vector; denotes an engineering entity corresponding to the semantic feature vector; and denotes a hybrid weight coefficient; denotes a path connecting nodes and ; denotes an edge on the path connecting nodes and ; denotes a weight of the edge on the path connecting nodes and ;

[0085] determining the retrieval information according to the size of the hybrid distance metric.

[0086] In the embodiments of the present disclosure, by constructing a hybrid distance metric model of semantic distance and topological distance, combining a hybrid weight coefficient, the correlation degree of the query vector and the semantic feature vector is comprehensively calculated, and finally the most relevant retrieval information is screened according to the size of the hybrid distance metric. The semantic distance can quantify the semantic similarity between the query vector and the semantic feature vector in the vector index library, and the smaller the distance, the more similar the semantics. The topological distance can be based on the weighted undirected graph between engineering entities (formula: ; ), denotes an engineering entity corresponding to the semantic feature vector, which can be used as a node of the weighted undirected graph, thereby quantifying the topological correlation tightness between the engineering entity corresponding to the query vector and the engineering entity corresponding to the semantic feature vector, and the smaller the distance, the tighter the topological correlation. The hybrid distance metric is calculated by comprehensively considering the semantic distance and the topological distance, and the retrieval information is screened according to the hybrid distance metric from small to large, so that the retrieval information with a preset number of sorting in front can be screened and sorted, and the highest comprehensive correlation degree of the result is ensured. Compared with the search relying only on the semantic distance, the topological distance of the engineering entity is integrated, which can avoid the false matching caused by the semantic similarity but the topological irrelevance, effectively filter the results that are similar in form but not matched in engineering logic, and make the results more consistent with the function logic of the industrial software.

[0087] In another embodiment of the present disclosure, the intelligent retrieval and question answering generation module 105 is configured to:

[0088] According to the size of the hybrid distance metric, a topological pruning strategy is used to prune the hybrid distance metric, and the formula is:

[0089] ;

[0090] represents a pruning invalid mixed distance measure; represents a pruning threshold factor; represents a retrieved minimum mixed distance measure;

[0091] According to the mixed distance measure after pruning, the retrieval information is determined.

[0092] In the embodiments of the present disclosure, through the topology pruning strategy, the effective mixed distance measure is screened in combination with the pruning threshold factor based on the retrieved minimum mixed distance measure, and the final retrieval information is determined after eliminating the invalid distance measure. The core is to improve the search efficiency on the premise of ensuring the retrieval accuracy. represents a pruning invalid mixed distance measure, and represents the distance corresponding to the information with the highest comprehensive correlation degree with the query vector in the current query, which is the reference for pruning. represents a pruning threshold factor, which is a parameter for controlling the strictness of pruning. The smaller the pruning threshold factor is, the stricter the pruning is (the less information is retained), and the larger the pruning threshold factor is, the looser the pruning is (the more information is retained). represents a mixed distance measure, represents a pruning invalid mixed distance measure, which can be used as a pruning marker. According to the mixed distance measure after pruning, the retrieval information is determined. Based on the effective mixed distance measure, the corresponding original information is extracted to provide accurate materials for the subsequent large language model generation result. By pruning the invalid distance measure far greater than the minimum mixed distance measure, it is avoided that the information with extremely low semantic or topological correlation degree enters the subsequent process, and it is ensured that the final retrieval information focuses on the core associated content.

[0093] In still another embodiment of the present disclosure, the intelligent retrieval and question answering generation module 105 is configured to:

[0094] According to the query vector, similarity search is performed in the vector index database to determine the mixed similarity of the query vector and the semantic feature vector and the engineering entity topological information, which is expressed by the formula:

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] wherein, represents a mixed similarity; represents a semantic similarity; represents a distance similarity; represents a query vector; denotes a semantic feature vector; denotes an engineering entity corresponding to the query vector; denotes an engineering entity corresponding to the semantic feature vector; and denotes a mixing weight coefficient; denotes a decay parameter; denotes a topological distance; denotes a path connecting nodes and ; denotes an edge on the path connecting nodes and ; denotes a weight of the edge on the path connecting nodes and ;

[0100] determining the retrieval information according to the size of the mixed similarity.

[0101] In the embodiments of the present disclosure, by constructing a mixed similarity model of semantic similarity and distance similarity, combining weight coefficient and topological distance decay calculation, the comprehensive correlation degree of the query vector and the semantic feature vector is quantified, and finally the most relevant retrieval information is screened according to the size of the mixed similarity. The calculation of semantic similarity can quantify the semantic matching degree of the query vector and the semantic feature vector in the vector index library. The calculation of distance similarity converts the topological distance into a distance similarity with the same dimension as the semantic similarity, realizing the additivity of the two-dimensional similarity. By the exponential function , the topological distance is converted into similarity (the closer the distance, the higher the similarity), which not only retains the quantization characteristics of topological correlation, but also flexibly controls the degree of topological influence through the decay parameter , adapting to the differences in topological correlation strength of different industrial software. The topological distance can be based on a weighted undirected graph between engineering entities (formula: ; denotes an engineering entity corresponding to the semantic feature vector, which can be used as a node of the weighted undirected graph. Based on the semantic similarity and the distance similarity, neighbor nodes can be screened, so that the retrieval information conforms to the engineering assembly logic. The weight contribution of the semantic similarity and the distance similarity is integrated to calculate the mixed similarity, and the retrieval information is screened according to the mixed similarity, so as to ensure that the comprehensive correlation degree of the result is the highest.

[0102] As shown in Figure 9 , ​An auxiliary system architecture for industrial software users is shown. The user interface layer 801 serves as an entry for users to interact with the system, receives input information from the user, and displays the final query results. Multi-terminal adaptive interaction methods can be provided to meet the needs of users in different scenarios for querying auxiliary information of industrial software. The API service layer 802 is used to process HTTP requests from the user interface layer, perform interface routing, parameter analysis, and result packaging. The API service layer 802 can include multiple components, such as RESTful API, routing management, parameter verification, and other API components, to ensure the normativity and reliability of request processing. The complexity of the underlying processing is shielded to provide a simple and unified calling method for the user interface layer 801, and the request parameters are verified to improve system stability. The core processing layer 803 can convert user input information into a query vector to provide a basis for subsequent similarity calculation. The vector index database construction module calculates a hybrid similarity by combining semantic similarity and distance similarity, quantifies the correlation between user input information and information in the vector index database, and considers both semantic matching and the topological relationship of engineering entities in industrial software. Based on the hybrid similarity and topological pruning, data that matches semantics and topology is retrieved from the vector index database, and the index storage, semantic feature vector, and engineering entity topological information of the vector index database are used to quickly locate related information. The recall processing can be used to filter the retrieved data, and the text matching and topological matching are combined to further optimize the accuracy of the recall results. Thus, the semantic and topological dimensions are fused to solve the problem of missing topological associated information in the industrial software field due to the complexity of functions and the close association of modules, and to improve the accuracy and professionalism of the retrieval. The data processing layer 804 can integrate and process the retrieval information from the core processing layer 803, and finally return the query results (including text content and original pictures, etc.) to the API service layer 802 and the user interface layer 801. The vector index database 805 provides index storage, semantic feature vector, and engineering entity topological information, and records the mapping relationship between the semantic feature vector and the semantic source data, to provide data support for the topological awareness retrieval of the core processing layer 803. Through efficient vector storage and retrieval capabilities, combined with engineering entity topological information, the retrieval based on hybrid similarity can be quickly executed.

[0103] Based on the same disclosure concept, the disclosure embodiments also provide a construction method of an industrial software user auxiliary system. Since the principles of the problems solved by these methods are similar to those of the aforementioned industrial software user auxiliary system, the implementation of the method can be referred to the implementation of the aforementioned industrial software user auxiliary system, and the repeated parts will not be described again.

[0104] The disclosure embodiments provide a construction method of an industrial software user auxiliary system, as shown in ​ ​

[0105] S901, obtain industrial software user documents; wherein the user documents include image data and text data; and operation objects of the industrial software include a plurality of engineering entities;

[0106] S902, convert the image data into semantic text;

[0107] S903, vectorize and encode the semantic text and the text data to obtain a semantic feature vector;

[0108] S904, determine engineering entity topological information according to a topological relationship between the corresponding engineering entities of the semantic feature vector;

[0109] S905, construct a vector index database according to the semantic feature vector and the engineering entity topological information; wherein the vector index database is used to record a mapping relationship between the semantic feature vector and semantic source data; and the semantic source data at least includes one of the following: semantic text, image data, and text data;

[0110] S906, in the case of receiving user input information, perform a similarity search in the vector index database and output a query result.

[0111] In another embodiment of the present disclosure, the conversion of the image data into semantic text includes:

[0112] inputting the image data into a multi-modal large model to obtain image semantic description text;

[0113] inputting the image semantic description text and text data related to the image data in the user documents into a pure text large model to obtain semantic text.

[0114] In another embodiment of the present disclosure, the vectorization and encoding of the semantic text and the text data to obtain a semantic feature vector includes:

[0115] performing block processing on the fusion of the semantic text and the text data to obtain a plurality of text blocks;

[0116] inputting the plurality of text blocks into an embedding model for vectorization and encoding to obtain a semantic feature vector.

[0117] In another embodiment of the present disclosure, the block processing on the fusion of the semantic text and the text data to obtain a plurality of text blocks includes:

[0118] inputting the fusion of the semantic text and the text data into an inference model for semantic analysis to determine a plurality of segmentation points;

[0119] According to the type of the segmentation point, the fused semantic text and the text data are blocked according to a preset priority, to obtain a plurality of text blocks;

[0120] The preset priority comprises, from high to low, a first priority of a semantic segmentation point after a current text block reaches a minimum word number threshold, a second priority of a paragraph segmentation point after the current text block reaches the minimum word number threshold, and a third priority of a forced segmentation point after the current text block reaches a maximum word number threshold.

[0121] In a case where the position of the segmentation point corresponding to the current text block is located inside the semantic text, the segmentation point is set at a position where the semantic text ends.

[0122] In another embodiment of the present disclosure, the determination of the topological relationship between the engineering entities according to the semantic feature vectors comprises:

[0123] According to the topological relationship between the engineering entities according to the semantic feature vectors, a weighted undirected graph between the engineering entities is determined, and the formula is expressed as:

[0124] ;

[0125] wherein, represents a node set, each node represents an engineering entity, and each engineering entity corresponds to a semantic feature vector; represents an edge set, each edge represents a topological association between engineering entities, represents the weight of each edge, used to represent the topological closeness between engineering entities.

[0126] In another embodiment of the present disclosure, in a case where user input information is received, similarity search is performed in the vector index database, and a query result is output, which comprises:

[0127] The input information of the user is input into a multi-modal large model to obtain an input description text;

[0128] The input description text is input into an embedding model for vectorization coding to obtain a query vector;

[0129] According to the query vector, similarity search is performed in the vector index database to determine retrieval information;

[0130] The retrieval information is input into a large language model to obtain a query result.

[0131] In another embodiment of the present disclosure, the determination of the retrieval information according to the query vector in the vector index database comprises:

[0132] According to the query vector, a similarity search is performed in the vector index database to determine a mixed distance measure of the query vector and the semantic feature vector and the engineering entity topology information, which is expressed as follows:

[0133] ;

[0134] ;

[0135] ;

[0136] wherein, denotes the mixed distance measure; denotes the semantic distance; denotes the topology distance; denotes the query vector; denotes the semantic feature vector; denotes the engineering entity corresponding to the query vector; denotes the engineering entity corresponding to the semantic feature vector; and denotes a mixed weight coefficient; denotes a path connecting nodes and ; denotes an edge on the path connecting nodes and ; denotes a weight of the edge on the path connecting nodes and ;

[0137] According to the size of the mixed distance measure, the retrieval information is determined.

[0138] In yet another embodiment of the present disclosure, the determination of the retrieval information according to the size of the mixed distance measure comprises:

[0139] According to the size of the mixed distance measure, a topology pruning strategy is used to clip the mixed distance measure, which is expressed as follows:

[0140] ;

[0141] denotes a clipped invalid mixed distance measure; denotes a pruning threshold factor; denotes a retrieved minimum mixed distance measure;

[0142] According to the clipped mixed distance measure, the retrieval information is determined.

[0143] In yet another embodiment of the present disclosure, the determining the retrieval information according to the query vector and performing the similarity search in the vector index database comprises:

[0144] According to the query vector, the similarity search is performed in the vector index database to determine the mixed similarity of the query vector and the semantic feature vector and the engineering entity topology information, which is expressed by a formula as follows:

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] wherein, the mixed similarity is represented by S; the semantic similarity is represented by Ss; the distance similarity is represented by Sd; the query vector is represented by q; the semantic feature vector is represented by f; the engineering entity corresponding to the query vector is represented by E; the engineering entity corresponding to the semantic feature vector is represented by E'; and the mixed weight coefficient is represented by w; the attenuation parameter is represented by a; the topology distance is represented by d; the path connecting the nodes and is represented by P; the edge on the path connecting the nodes and is represented by e; the weight of the edge on the path connecting the nodes and is represented by w;

[0150] The retrieval information is determined according to the size of the mixed similarity.

[0151] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by hardware, or by means of software and necessary universal hardware platforms. Based on such understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present disclosure.

[0152] Those skilled in the art can understand that the modules or flows in the drawings are not necessarily required for implementing the present disclosure.

[0153] Those skilled in the art can understand that the modules in the devices in the embodiments can be distributed in the devices in the embodiments according to the embodiment description, or can be changed to be located in one or more devices different from the embodiments. The modules in the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0154] The above sequence numbers of the embodiments of the present disclosure are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0155] Obviously, those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure belong to the scope of the claims of the present disclosure and the equivalent technologies thereof, the present disclosure also intends to include these modifications and variations.

Claims

1. An industrial software user assistance system, characterized in that, include: The data acquisition and preprocessing module is used to acquire industrial software user documents; wherein, the user documents include: image data and text data; The image analysis module is used to convert the image data into semantic text; The vectorization encoding module is used to perform vectorization encoding on the semantic text and the text data to obtain semantic feature vectors; The vector index database construction module is used to determine the topological information of engineering entities based on the topological relationships between engineering entities corresponding to the semantic feature vectors. This includes determining a weighted undirected graph between engineering entities based on the topological relationships between them, expressed by the formula: ； in, This represents a set of nodes, where each node represents an engineering entity, and each engineering entity corresponds to a semantic feature vector. This represents a set of edges, where each edge indicates a topological relationship between engineering entities. This represents the weight of each edge, used to characterize the topological density between engineering entities; Wherein, the engineering entity is the operation object of the industrial software; the engineering entity includes at least one of the following: a 3D model part created in CAD, an assembly, or a mesh in CAE; the topological relationship between the engineering entities includes at least one of the following: assembly constraint, geometric adjacency, or parent-child hierarchy. Based on the semantic feature vectors and the engineering entity topology information, a vector index database is constructed; wherein, the vector index database is used to record the mapping relationship between the semantic feature vectors and the semantic source data; the semantic source data includes at least one of the following: semantic text, image data, and text data; The intelligent retrieval and question-answering generation module is used to perform similarity searches in the vector index database upon receiving user input and output query results, including: The user's input information is fed into a multimodal large model to obtain the input description text; the input information includes: images and text; The input description text is input into the embedding model for vector encoding to obtain the query vector; Based on the query vector, a similarity search is performed in the vector index database to determine the retrieved information, including: based on the query vector, a similarity search is performed in the vector index database to determine a hybrid distance metric between the query vector, the semantic feature vector, and the engineering entity topology information, expressed by the formula: ； ； ； in, Indicates a mixed distance metric; Indicates semantic distance; Represents topological distance; Represents the query vector; Represents semantic feature vectors; This represents the project entity corresponding to the query vector; This represents the engineering entity corresponding to the semantic feature vector; and Indicates the mixed weighting coefficient; Indicates the connection node and The path; Indicates the connection node and Edges on the path; Indicates the connection node and The weights along the path; based on the magnitude of the hybrid distance metric, determine the retrieval information; The search information is input into the large language model to obtain the query results.

2. The system as described in claim 1, characterized in that, The image analysis module is used for: The image data is input into a multimodal large model to obtain image semantic description text; The semantic description text of the image and the text data related to the image data in the user document are input into the plain text large model to obtain the semantic text.

3. The system as described in claim 1, characterized in that, The vectorization encoding module is used for: The semantic text and the text data are fused and then segmented to obtain multiple text blocks; Multiple text blocks are input into the embedding model for vectorized encoding to obtain semantic feature vectors.

4. The system as described in claim 3, characterized in that, The vectorization encoding module is used for: The semantic text and the text data are fused and then input into the inference model for semantic analysis to determine multiple segmentation points; Based on the type of the segmentation point, the fused semantic text and text data are divided into blocks according to a preset priority to obtain multiple text blocks; The preset priorities, from high to low, include: the first priority of the semantic segmentation point after the current text block reaches the minimum word count threshold; the second priority of the paragraph segmentation point after the current text block reaches the minimum word count threshold; and the third priority of the forced segmentation point after the current text block reaches the maximum word count threshold. Specifically, if the position of the segmentation point corresponding to the current text block is located inside the semantic text, the segmentation point is set at the end of the semantic text.

5. The system as described in claim 1, characterized in that, The intelligent retrieval and question-answering generation module is also used for: Based on the magnitude of the hybrid distance metric, a topology pruning strategy is used to prune the hybrid distance metric, as expressed by the formula: ； Indicates the invalid blending distance metric; Indicates the pruning threshold factor; This represents the minimum mixed distance metric retrieved. The retrieval information is determined based on the post-cropping blending distance metric.

6. The system as described in claim 1, characterized in that, The intelligent retrieval and question-answering generation module is also used for: Based on the query vector, a similarity search is performed in the vector index database to determine the mixed similarity between the query vector and the semantic feature vector and the engineering entity topology information, expressed by the formula: ； ； ； ； in, Indicates mixed similarity; Indicates semantic similarity; Indicates distance similarity; Represents the query vector; Represents semantic feature vectors; This represents the project entity corresponding to the query vector; This represents the engineering entity corresponding to the semantic feature vector; and Indicates the mixed weighting coefficient; Indicates the attenuation parameter; Represents topological distance; Indicates the connection node and The path; Indicates the connection node and Edges on the path; Indicates the connection node and Weights along the path; The retrieval information is determined based on the magnitude of the mixed similarity.

7. A method for constructing an industrial software user-aided system, characterized in that, include: Obtain industrial software user documentation; wherein, the user documentation includes: image data and text data; The image data is converted into semantic text; The semantic text and the text data are vectorized and encoded to obtain semantic feature vectors; Based on the topological relationships between the engineering entities corresponding to the semantic feature vectors, the topological information of the engineering entities is determined, including: determining a weighted undirected graph between the engineering entities based on the topological relationships between the engineering entities corresponding to the semantic feature vectors, expressed by the formula: ； in, This represents a set of nodes, where each node represents an engineering entity, and each engineering entity corresponds to a semantic feature vector. This represents a set of edges, where each edge indicates a topological relationship between engineering entities. This represents the weight of each edge, used to characterize the topological density between engineering entities; Wherein, the engineering entity is the operation object of the industrial software; the engineering entity includes at least one of the following: a 3D model part created in CAD, an assembly, or a mesh in CAE; the topological relationship between the engineering entities includes at least one of the following: assembly constraint, geometric adjacency, or parent-child hierarchy. Based on the semantic feature vectors and the engineering entity topology information, a vector index database is constructed; wherein, the vector index database is used to record the mapping relationship between the semantic feature vectors and the semantic source data; the semantic source data includes at least one of the following: semantic text, image data, and text data; Upon receiving user input, a similarity search is performed in the vector index database, and the query results are output, including: The user's input information is fed into a multimodal large model to obtain the input description text; the input information includes: images and text; The input description text is input into the embedding model for vector encoding to obtain the query vector; Based on the query vector, a similarity search is performed in the vector index database to determine the retrieved information, including: based on the query vector, a similarity search is performed in the vector index database to determine a hybrid distance metric between the query vector, the semantic feature vector, and the engineering entity topology information, expressed by the formula: ； ； ； in, Indicates a mixed distance metric; Indicates semantic distance; Represents topological distance; Represents the query vector; Represents semantic feature vectors; This represents the project entity corresponding to the query vector; This represents the engineering entity corresponding to the semantic feature vector; and Indicates the mixed weighting coefficient; Indicates the connection node and The path; Indicates the connection node and Edges on the path; Indicates the connection node and The weights along the path; based on the magnitude of the hybrid distance metric, determine the retrieval information; The search information is input into the large language model to obtain the query results.

Images