Intelligent agent-based virtual field operator interaction system and method

Abstract

The invention relates to the technical field of virtual field operator interaction, and particularly discloses a virtual field operator interaction system and method based on an intelligent agent. A standard question-answer library is constructed, and then interaction questions input to the intelligent agent by field operators are utilized; therefore, a plurality of matched template questions and matched reference answers are fuzzily queried in the standard question-answer library, and furthermore, a semantic comprehension algorithm based on artificial intelligence and natural language processing is introduced at the back end to analyze the interaction question, the plurality of matched template questions and the matched reference answers. And decoding to generate an interaction answer. Therefore, the reply more conforming to the actual scene can be generated by utilizing the modes of interactive library fuzzy query and semantic weighted aggregation of a plurality of matched answers, and the defects of flexibility and real-time performance of a traditional mode can be avoided in the interactive process of the field operation personnel and the intelligent agent through a more intelligent interactive mode.

Description

Technical Field

[0001] The present application relates to the technical field of virtual on-site worker interaction, and more specifically, to an agent-based virtual on-site worker interaction system and method. Background Art

[0002] In modern manufacturing and industrial scenarios, effective interaction between on-site workers and virtual agents (such as intelligent agents or virtual assistants) is increasingly crucial. Virtual agents are often used in scenarios such as training, remote guidance, and troubleshooting. They can provide real-time information and support, helping on-site workers better complete their tasks. For example, in complex production processes, virtual characters can answer questions from on-site workers in real time and offer professional advice, thereby reducing operational errors and improving production efficiency.

[0003] However, in traditional industrial environments, the way workers interact with virtual characters often relies on fixed instruction sets or graphical user interfaces (GUIs). Although this simple preset dialogue question-and-answer method can provide virtual interaction with on-site workers, it has limitations in flexibility and ease of use. Especially for complex production environments, it is difficult to meet the flexibility and real-time requirements of on-site workers in complex environments.

[0004] Therefore, an optimized virtual on-site worker interaction solution is desired. Summary of the Invention

[0005] The present application provides an agent-based virtual on-site worker interaction system and method, which can use fuzzy query of the interaction library and semantic weighted aggregation of multiple matching answers to generate responses that are more in line with actual scenarios. Through a more intelligent interaction method, the interaction process between on-site workers and intelligent agents can avoid the flexibility and real-time defects of traditional methods, which helps on-site workers solve various problems in complex production environments, and also provides better support and training methods for workers.

[0006] In a first aspect, a method for interacting with virtual on-site workers based on an agent is provided, comprising:

[0007] Constructing a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions;

[0008] Obtain interactive questions input by field workers to the agent;

[0009] Based on the standard question-answer library, performing a fuzzy query on the interactive question to obtain a plurality of matching template questions and reference answers;

[0010] Performing semantic encoding on the multiple matching template questions and reference answers and the interactive question to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and an interactive question semantic encoding feature vector;

[0011] The interactive question semantic coding feature vector and the multiple matching template question semantic coding vectors are subjected to feature fusion based on self-supervised pseudo clustering to obtain a cluster contribution field modulation weight vector, including: calculating cluster contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector; performing cluster contribution field explicit feature recognition on the matching template question semantic feature cluster contribution field distribution vector to obtain the cluster contribution field modulation weight vector;

[0012] After weighting the multiple matching reference answer semantic encoding vectors using the cluster contribution field modulation weight vector to obtain a matching reference answer semantically significant aggregation representation, the matching reference answer semantically significant aggregation representation is used to generate an answer text to obtain an interactive answer.

[0013] In one possible implementation, semantic encoding is performed on the multiple matching template questions and reference answers and the interactive question to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors, and an interactive question semantic encoding feature vector, including:

[0014] Semantically encoding the multiple matching template questions and reference answers respectively to obtain the multiple matching template question semantic encoding vectors and the multiple matching reference answer semantic encoding vectors;

[0015] The interaction question is semantically encoded to obtain a semantic encoding feature vector of the interaction question.

[0016] In one possible implementation, calculating cluster contribution information of the multiple matching template question semantic encoding vectors relative to the interaction question semantic encoding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector includes:

[0017] Calculating the implicit clustering contribution factor of each matching template question semantic encoding vector in the multiple matching template question semantic encoding vectors relative to the interactive question semantic encoding feature vector to obtain a sequence of matching template question semantic feature implicit clustering contribution factors;

[0018] The sequence of implicit clustering contribution factors of the matching template question semantic features is vectorized and arranged to obtain the matching template question semantic feature clustering contribution field distribution vector.

[0019] In one possible implementation, calculating the implicit clustering contribution factor of each matching template question semantic encoding vector in the multiple matching template question semantic encoding vectors relative to the interaction question semantic encoding feature vector to obtain a sequence of implicit clustering contribution factors of matching template question semantic features includes:

[0020] Calculating the absolute value of positional division between the matching template question semantic encoding vector and the interaction question semantic encoding feature vector to obtain a matching template question semantic self-supervised clustering interaction correlation feature vector;

[0021] Calculating the logarithmic function value of each position feature value in the matching template problem semantic self-supervised clustering interaction correlation feature vector with base 2 to obtain the matching template problem semantic self-supervised clustering interaction logarithmic representation vector;

[0022] Using the feature values of each position in the matching template question semantic self-supervised clustering interaction logarithm representation vector as weighting coefficients, calculating the weighted sum of the feature values of each position in the matching template question semantic encoding vector to obtain a matching template question semantic feature clustering contribution representation factor;

[0023] The exponential function value of the matching template question semantic feature cluster contribution representation factor is calculated with the natural constant e as the base to obtain the matching template question semantic feature implicit cluster contribution factor.

[0024] In one possible implementation, cluster contribution field explicit feature recognition is performed on the matching template problem semantic feature cluster contribution field distribution vector to obtain the cluster contribution field domain modulation weight vector, including: inputting the matching template problem semantic feature cluster contribution field distribution vector into a cluster contribution field explicit modeling module based on a self-attention mechanism to obtain the cluster contribution field domain modulation weight vector.

[0025] In one possible implementation, the matching template problem semantic feature cluster contribution field distribution vector is input into a cluster contribution field explicit modeling module based on a self-attention mechanism to obtain the cluster contribution field domain modulation weight vector, including:

[0026] Calculating matrix multiplications between the matching template question semantic feature cluster contribution field distribution vector and the query weight matrix, the key weight matrix, and the value weight matrix respectively to obtain a matching template question semantic cluster contribution field distribution query vector, a matching template question semantic cluster contribution field distribution key vector, and a matching template question semantic cluster contribution field distribution value vector;

[0027] After calculating the multiplication between the matching template question semantic cluster contribution field distribution query vector and the transposed vector of the matching template question semantic cluster contribution field distribution key vector, the obtained matrix is divided by the square root of the length of the matching template question semantic cluster contribution field distribution key vector by position to obtain a query-key matching template question semantic cluster contribution field semantic interaction representation matrix;

[0028] Normalizing the query-key matching template problem semantic clustering contribution field semantic interaction representation matrix using a Softmax function to obtain a normalized query-key matching template problem semantic clustering contribution field semantic interaction representation matrix;

[0029] The multiplication between the matching template problem semantic cluster contribution field distribution value vector and the query-key matching template problem semantic cluster contribution field semantic interaction representation matrix is calculated to obtain the cluster contribution field domain modulation weight vector.

[0030] In one possible implementation, after weighting the multiple matching reference answer semantic encoding vectors using the cluster contribution field modulation weight vector to obtain a matching reference answer semantically significant aggregate representation, generating an answer text using the matching reference answer semantically significant aggregate representation to obtain an interactive answer, including:

[0031] Using each eigenvalue in the cluster contribution field modulation weight vector as a weight, calculating the position-weighted sum of the multiple matching reference answer semantic encoding vectors to obtain a matching reference answer semantically significant aggregation encoding vector as the matching reference answer semantically significant aggregation representation;

[0032] The semantically significant aggregated encoding vector of the matching reference answer is input into a decoder-based answer text generator to obtain the interactive answer.

[0033] In a second aspect, an agent-based virtual on-site worker interaction system is provided, comprising:

[0034] A standard question-answer library construction module is used to construct a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions;

[0035] The interactive question acquisition module is used to obtain the interactive questions input by the on-site operators to the intelligent agent;

[0036] An interactive question fuzzy query module, configured to perform a fuzzy query on the interactive question based on the standard question-answer library to obtain a plurality of matching template questions and reference answers;

[0037] A semantic encoding module, configured to perform semantic encoding on the plurality of matching template questions and reference answers and the interactive question to obtain a plurality of matching template question semantic encoding vectors, a plurality of matching reference answer semantic encoding vectors and an interactive question semantic encoding feature vector;

[0038] A cluster contribution field modulation module is used to perform feature fusion based on self-supervised pseudo-clustering on the interactive question semantic coding feature vector and the multiple matching template question semantic coding vectors to obtain a cluster contribution field modulation weight vector, including: a cluster contribution information calculation unit, used to calculate the cluster contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector; a cluster contribution field explicit feature recognition unit, used to perform cluster contribution field explicit feature recognition on the matching template question semantic feature cluster contribution field distribution vector to obtain the cluster contribution field modulation weight vector;

[0039] An interactive answer generation module is used to use the cluster contribution field modulation weight vector to weight the multiple matching reference answer semantic encoding vectors to obtain a matching reference answer semantically significant aggregation representation, and then use the matching reference answer semantically significant aggregation representation to generate an answer text to obtain an interactive answer.

[0040] In a possible implementation, the semantic encoding module is used to:

[0041] Semantically encoding the multiple matching template questions and reference answers respectively to obtain the multiple matching template question semantic encoding vectors and the multiple matching reference answer semantic encoding vectors;

[0042] The interaction question is semantically encoded to obtain a semantic encoding feature vector of the interaction question.

[0043] In a possible implementation, the cluster contribution information calculation unit is configured to:

[0044] Calculating the implicit clustering contribution factor of each matching template question semantic encoding vector in the multiple matching template question semantic encoding vectors relative to the interactive question semantic encoding feature vector to obtain a sequence of matching template question semantic feature implicit clustering contribution factors;

[0045] The sequence of implicit clustering contribution factors of the matching template question semantic features is vectorized and arranged to obtain the matching template question semantic feature clustering contribution field distribution vector.

[0046] The present application provides an agent-based virtual on-site worker interaction system and method, which can use virtual technology combined with artificial intelligence to create a simulated environment. In the process of interaction between on-site workers and the agent, the agent can understand human language in the virtual scene and support the automatic generation of dialogue content between production site workers and the agent. This agent-based virtual on-site worker interaction method allows the creation of highly realistic simulation scenes and can dynamically adapt to user input and behavior. This means that the interaction process requires not only that the agent can understand and respond to user instructions, but also that it can automatically generate contextual responses that are consistent with different situations, so as to improve the flexibility and adaptability of the interaction process, so as to better assist production work or training processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings of the embodiments of the present application. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0048] Figure 1 This is a schematic flowchart of an agent-based virtual on-site worker interaction method according to an embodiment of the present application.

[0049] Figure 2 Schematic diagram of data flow of the agent-based virtual on-site worker interaction method according to an embodiment of the present application.

[0050] Figure 3 In the agent-based virtual on-site worker interaction method of an embodiment of the present application, a schematic flowchart of semantically encoding the multiple matching template questions and reference answers and the interaction questions to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and interaction question semantic encoding feature vectors.

[0051] Figure 4 In the agent-based virtual on-site worker interaction method of an embodiment of the present application, a schematic flowchart of performing feature fusion based on self-supervised pseudo-clustering on the semantic encoding feature vector of the interaction problem and the semantic encoding vectors of the multiple matching template problems to obtain a cluster contribution field modulation weight vector.

[0052] Figure 5 This is a schematic flowchart of an agent-based virtual on-site worker interaction method in an embodiment of the present application, in which the cluster contribution field modulation weight vector is used to weight the multiple matching reference answer semantic encoding vectors to obtain a matching reference answer semantically significant aggregation representation, and then the matching reference answer semantically significant aggregation representation is used to generate an answer text to obtain an interactive answer.

[0053] Figure 6 This is a schematic block diagram of an agent-based virtual on-site worker interaction system according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts also fall within the scope of protection of this application.

[0055] In traditional industrial scenarios, workers' interactions with virtual assistants or other digital systems are often limited to a fixed set of commands or rely on graphical user interfaces. While this approach can achieve a certain degree of human-machine dialogue and support basic task requirements, its rigid framework and limited interaction modes make it inadequate when faced with changing and complex production conditions. This traditional interaction method is particularly inadequate when rapid response to changes and handling of unforeseen issues are required, as it cannot effectively meet the actual needs of field workers for immediate feedback and flexible communication.

[0056] In response to the above technical problems, the technical solution of this application proposes an agent-based virtual on-site worker interaction method, which can use virtual technology combined with artificial intelligence to create a simulated environment, and then in the process of interaction between on-site workers and the agent, the agent can understand human language in the virtual scene, and support the automatic generation of dialogue content between production site workers and the agent. This agent-based virtual on-site worker interaction method allows the creation of highly realistic simulation scenes and can dynamically adapt to user input and behavior. This means that the interaction process requires not only that the agent can understand and respond to user instructions, but also that it can automatically generate contextual responses that are consistent with different situations, so as to improve the flexibility and adaptability of the interaction process, so as to better assist production work or training processes.

[0057] Based on this, the technical concept of this application is to construct a standard question-answer library, and then use the interactive questions input by on-site personnel to the intelligent agent to fuzzily query multiple matching template questions and matching reference answers in the standard question-answer library. Furthermore, a semantic understanding algorithm based on artificial intelligence and natural language processing is introduced at the back end to analyze the interactive questions and the multiple matching template questions and matching reference answers, so as to capture the semantic encoding features therein. Furthermore, the interactive question semantics is used to query the modulation weight based on the cluster contribution from the multiple matching template question semantics, and this weight is used to adaptively weight and aggregate the multiple matching reference answer semantics, so as to decode and generate the interactive answer. In this way, it is possible to use the interactive library fuzzy query and the weighted aggregation of multiple matching answer semantics to generate a response that is more in line with the actual scenario. Through a more intelligent interactive method, the interactive process between on-site operators and intelligent agents can avoid the flexibility and real-time defects of traditional methods, which helps on-site operators solve various problems in complex production environments, and also provides better support and training means for operators.

[0058] Figure 1 This is a schematic flowchart of an agent-based virtual on-site worker interaction method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the agent-based virtual on-site operator interaction method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the agent-based virtual on-site worker interaction method includes: S1, constructing a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions; S2, obtaining the interactive questions input by the on-site workers to the intelligent agent; S3, based on the standard question-answer library, performing fuzzy query on the interactive questions to obtain multiple matching template questions and reference answers; S4, semantically encoding the multiple matching template questions and reference answers and the interactive questions to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and interactive question semantic encoding feature vectors; S5, performing feature fusion based on self-supervised pseudo-clustering on the interactive question semantic encoding feature vectors and the multiple matching template question semantic encoding vectors to obtain a cluster contribution field modulation weight vector; S6, using the cluster contribution field modulation weight vector to weight the multiple matching reference answer semantic encoding vectors to obtain a matching reference answer semantic saliency aggregation representation, and then using the matching reference answer semantic saliency aggregation representation to generate an answer text to obtain an interactive answer.

[0059] In the above-mentioned agent-based virtual on-site worker interaction method, the S1 constructs a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions. It should be understood that by pre-constructing a library of questions and answers, it is possible to quickly respond to common questions raised by on-site workers, reducing the query time and the time waiting for a response. When an on-site worker asks a question, the system can fuzzily query multiple matching template questions and reference answers in the standard question-answer library, and perform semantic understanding through natural language processing technology to generate answers that are closer to the actual scenario. The design of the standard question-answer library takes into account various situations that may arise in actual production and operation, so it can provide answers that are closer to on-site needs and improve the user experience.

[0060] In the above-mentioned agent-based virtual on-site worker interaction method, the S2 obtains the interaction questions input by the on-site workers to the agent. Optionally, in one embodiment of the present application, the acquisition of the interaction questions input by the on-site workers to the agent can be achieved in the following ways, depending on the design and deployment environment of the agent. 1. If the agent has voice recognition capabilities, it can capture the voice of the on-site workers through a microphone and convert it into text form as an interaction question. 2. Through a keyboard or touch screen, the worker can directly input text to ask the agent. This method is suitable for work environments with screens and input devices. 3. In some scenarios, workers may use mobile devices (such as smartphones or tablets) to interact with the agent. In this case, questions are submitted through mobile applications or dedicated software. 4. The agent can also be integrated with IoT devices to receive sensor data or other forms of data input from these devices as part of the question or as a prerequisite for triggering the agent to perform certain actions. 5. For some integrated applications, requests from other systems can be accepted through API interfaces. These requests may contain questions that the worker wants to solve.

[0061] In the above-mentioned agent-based virtual on-site worker interaction method, S3, based on the standard question-answer library, performs a fuzzy query on the interactive question to obtain multiple matching template questions and reference answers. It should be understood that by fuzzy matching the interactive question input by the user with the template question in the standard question-answer library, a question instance close to the user's question can be found, thereby obtaining the most relevant reference answer. This ensures that the answer given by the agent is more accurate and better meets the needs of the user. At the same time, the fuzzy query allows the agent to understand that the user may express the same question in a variety of different ways, and even if the expression method is different, the corresponding answer can be found. This makes the system more flexible and can adapt to the different expression habits of different users. By matching multiple template questions, the system can comprehensively consider the answers in different situations and generate more comprehensive and context-appropriate answers, which is especially important for solving problems in complex production environments.

[0062] Optionally, in one embodiment of the present application, based on the standard question-answer library, a fuzzy query is performed on the interactive question to obtain multiple matching template questions and reference answers, including: preprocessing each question in the question-answer library, including operations such as word segmentation, removal of stop words, and standardization. Based on the preprocessed question, a template is generated using pattern matching technology. The template can be in the form of a regular expression or a description based on certain keywords and structures. An index of the question library can be constructed, and an inverted index can be used to map the keywords in the question to the question itself to speed up retrieval. A suitable fuzzy matching algorithm, such as Levenshte in Distance, cosine similarity, Jaccard similarity, etc., is selected to calculate the similarity between the question entered by the user and the templates in the library. When a user asks a question, the user's question is first queried, and then a fuzzy matching algorithm is used to search for similar question templates in the question library. According to the similarity score obtained by the matching algorithm, the matching template questions are sorted, and the most similar ones are selected as the results. The sorted question templates and their corresponding reference answers are output to the user.

[0063] In the above-mentioned agent-based virtual on-site worker interaction method, the S4 performs semantic encoding on the multiple matching template questions and reference answers and the interactive questions to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and interactive question semantic encoding feature vectors. It should be understood that in order to be able to more deeply understand the interactive questions and the multiple matching template questions and reference answers obtained through fuzzy matching, and to convert these semantic information into data types that are convenient for subsequent processing, in the technical solution of the present application, the multiple matching template questions and reference answers are semantically encoded respectively to extract the semantic encoding features of the multiple matching template questions and reference answers, thereby obtaining multiple matching template question semantic encoding vectors and multiple matching reference answer semantic encoding vectors. In addition, the interactive questions are semantically encoded to extract the semantic encoding features in the interactive questions, thereby obtaining the interactive question semantic encoding feature vectors.

[0064] Optionally, in one embodiment of the present application, Figure 3 This is a schematic flow chart of the process of semantically encoding the multiple matching template questions and reference answers and the interactive questions in the agent-based virtual on-site worker interaction method according to an embodiment of the present application to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and interactive question semantic encoding feature vectors. Figure 3 As shown, the multiple matching template questions and reference answers and the interactive questions are semantically encoded to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and interactive question semantic encoding feature vectors, including: S41, semantically encoding the multiple matching template questions and reference answers respectively to obtain the multiple matching template question semantic encoding vectors and the multiple matching reference answer semantic encoding vectors; S42, semantically encoding the interactive question to obtain the interactive question semantic encoding feature vector.

[0065] In the above-mentioned agent-based virtual on-site worker interaction method, the S5 performs feature fusion based on self-supervised pseudo-clustering on the interactive question semantic encoding feature vector and the multiple matching template question semantic encoding vectors to obtain a cluster contribution field modulation weight vector. It should be understood that the interactive question semantic encoding feature vector represents the feature representation of the question raised by the on-site worker after semantic encoding. The multiple matching template question semantic encoding vectors are the question semantic encoding feature representations of multiple matching templates fuzzy matched from the standard question-answer library. In order to be able to perform correlation recognition and semantic fusion on the semantics of these matching template questions around the interactive question semantics, so as to use the answers to these different matching template questions in the subsequent interactive answer generation process to generate answers that are more in line with the actual application scenario, in the technical solution of the present application, the interactive question semantic encoding feature vector is further used as a self-supervised pseudo-cluster representation vector, and the interactive question semantic encoding feature vector and the multiple matching template question semantic encoding vector are subjected to feature fusion based on self-supervised pseudo-clustering to obtain a cluster contribution field modulation weight vector.

[0066] Optionally, in one embodiment of the present application, Figure 4 This is a schematic flow chart of the process of performing feature fusion based on self-supervised pseudo-clustering on the semantic encoding feature vector of the interaction question and the semantic encoding vectors of the multiple matching template questions in the agent-based virtual on-site worker interaction method of the embodiment of the present application to obtain the cluster contribution field modulation weight vector. Figure 4 As shown, the interactive question semantic coding feature vector and the multiple matching template question semantic coding vectors are subjected to feature fusion based on self-supervised pseudo-clustering to obtain a cluster contribution field modulation weight vector, including: S51, calculating the cluster contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector; S52, performing cluster contribution field explicit feature recognition on the matching template question semantic feature cluster contribution field distribution vector to obtain the cluster contribution field modulation weight vector.

[0067] Optionally, in one embodiment of the present application, the clustering contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector is calculated to obtain a matching template question semantic feature clustering contribution field distribution vector, including: calculating the implicit clustering contribution factor of each matching template question semantic coding vector in the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a sequence of implicit clustering contribution factors of matching template question semantic features; vectorizing and arranging the sequence of implicit clustering contribution factors of matching template question semantic features to obtain the matching template question semantic feature clustering contribution field distribution vector.

[0068] Optionally, in one embodiment of the present application, the implicit clustering contribution factor of each matching template problem semantic coding vector in the multiple matching template problem semantic coding vectors relative to the interactive problem semantic coding feature vector is calculated to obtain a sequence of implicit clustering contribution factors of matching template problem semantic features, including: calculating the absolute value of the positional division between the matching template problem semantic coding vector and the interactive problem semantic coding feature vector to obtain a matching template problem semantic self-supervised clustering interaction association feature vector; calculating the logarithmic function value of each position feature value in the matching template problem semantic self-supervised clustering interaction association feature vector with a base of 2 to obtain a matching template problem semantic self-supervised clustering interaction logarithmic representation vector; using each position feature value in the matching template problem semantic self-supervised clustering interaction logarithmic representation vector as a weighting coefficient, calculating the weighted sum of the feature values of each position in the matching template problem semantic coding vector to obtain a matching template problem semantic feature clustering contribution representation factor; calculating the exponential function value of the matching template problem semantic feature clustering contribution representation factor with a natural constant e as the base to obtain a matching template problem semantic feature implicit clustering contribution factor.

[0069] Optionally, in one embodiment of the present application, cluster contribution field explicit feature recognition is performed on the matching template problem semantic feature cluster contribution field distribution vector to obtain the cluster contribution field domain modulation weight vector, including: inputting the matching template problem semantic feature cluster contribution field distribution vector into a cluster contribution field explicit modeling module based on the self-attention mechanism to obtain the cluster contribution field domain modulation weight vector.

[0070] Optionally, in one embodiment of the present application, the matching template problem semantic feature cluster contribution field distribution vector is input into the cluster contribution field explicit modeling module based on the self-attention mechanism to obtain the cluster contribution field domain modulation weight vector, including: calculating the matrix multiplication between the matching template problem semantic feature cluster contribution field distribution vector and the query weight matrix, key weight matrix and value weight matrix respectively to obtain the matching template problem semantic cluster contribution field distribution query vector, matching template problem semantic cluster contribution field distribution key vector and matching template problem semantic cluster contribution field distribution value vector; calculating the matching template problem semantic cluster contribution field distribution query vector and the matching template problem semantic cluster contribution field distribution key vector. After multiplying the transpose vectors of the key vectors, the obtained matrix is divided by the square root of the length of the matching template problem semantic clustering contribution field distribution key vector by position to obtain the query-key matching template problem semantic clustering contribution field semantic interaction representation matrix; the query-key matching template problem semantic clustering contribution field semantic interaction representation matrix is normalized using the Softmax function to obtain the normalized query-key matching template problem semantic clustering contribution field semantic interaction representation matrix; the multiplication between the matching template problem semantic clustering contribution field distribution value vector and the query-key matching template problem semantic clustering contribution field semantic interaction representation matrix is calculated to obtain the cluster contribution field modulation weight vector.

[0071] In summary, in the embodiment of the present application, the interactive question semantic encoding feature vector is used as a self-supervised pseudo clustering representation vector, and the interactive question semantic encoding feature vector and the multiple matching template question semantic encoding vectors are subjected to self-supervised pseudo clustering feature fusion using the following feature fusion formula to obtain the cluster contribution field modulation weight vector;

[0072] Among them, the feature fusion formula is:

[0073] X={x1,x2,...,x k ,...,x n}

[0074]

[0075] v q =W q v d

[0076] v k =W k v d

[0077] v v =W b v d

[0078] Where X is the semantic encoding vector of the multiple matching template questions, x1, x2, x k , x n are the first, second, kth and nth matching template question semantic encoding vectors in the multiple matching template question semantic encoding vectors, respectively, c The semantic encoding feature vector for the interaction question, x j is the jth matching template question semantic encoding vector among the multiple matching template question semantic encoding vectors, is the feature value of the kth position in the semantic encoding vector of the jth matching template question, The kth position eigenvalue in the semantic encoding feature vector of the interaction question, log represents the logarithmic function value with base 2, exp represents the natural exponential function value, d j is the implicit clustering contribution factor of the semantic features of the matching template problem corresponding to the j-th matching template problem semantic encoding vector, v d The semantic feature clustering contribution field distribution vector of the matching template problem obtained by quantizing the implicit clustering contribution factors of multiple matching template problem semantic features, W q 、W k and W b are the query weight matrix, key weight matrix and value weight matrix respectively, v q 、v k and v v are the query vector of the matching template problem semantic cluster contribution field distribution, the key vector of the matching template problem semantic cluster contribution field distribution, and the value vector of the matching template problem semantic cluster contribution field distribution. L is the length of the matching template problem semantic cluster contribution field distribution key vector. Softmax(·) is the Softmax function. is vector multiplication, v w A field modulation weight vector is contributed to the cluster.

[0079] Specifically, the feature fusion process based on self-supervised pseudo-clustering first uses the interactive question semantic encoding feature vector as a self-supervised pseudo-cluster representation vector, which means that it will become a reference point in the clustering process, helping the system understand the correlation between the different matching template question semantics obtained by fuzzy matching and the cluster reference point. Then, the cluster contribution information of the multiple matching template question semantic encoding vectors relative to the interactive question semantic encoding feature vector is calculated to obtain the matching template question semantic feature cluster contribution field distribution vector. Specifically, by calculating the implicit cluster contribution factor of each matching template question semantic encoding vector in the multiple matching template question semantic encoding vectors relative to the interactive question semantic encoding feature vector, it is used to reflect the contribution of each matching template question semantic encoding vector to the cluster to which it belongs, that is, the correlation between each matching template question semantic and the interactive question semantic. Accordingly, the calculation of the implicit cluster contribution factor helps to identify and quantify the role of each matching template question semantic encoding vector in the clustering process, providing a basis for subsequent feature selection and weight allocation, so that the model can pay more attention to the matching template semantic features that have a significant contribution to the clustering. Next, the sequence of implicit cluster contribution factors of the matching template question semantic features is vectorized and arranged to obtain the matching template question semantic feature cluster contribution field distribution vector. Specifically, data integration is used to integrate the sequence of implicit cluster contribution factors of the matching template question semantic features into a vector field to construct the cluster contribution field. Subsequently, the cluster contribution field distribution vector is explicitly modeled using a self-attention mechanism to generate a cluster contribution field modulation weight vector. Specifically, the self-attention mechanism is used to explicitly model the cluster contribution of each fine-grained factor in the cluster contribution field based on a global analysis of field autocorrelation. This strengthens attention to feature vectors with high cluster contributions related to the interaction question semantics while suppressing unimportant features, thereby improving the quality of the aggregated representation of each matching template feature. Through self-supervised pseudo-clustering, the feature vector of the interaction question is fused with the feature vectors of multiple matching template questions. This process helps identify the semantic similarities and differences between the current interaction question and the matching template question, thereby determining which existing answer templates are most suitable for the current question and performing contribution weight analysis on these suitable question semantics. The result of feature fusion is a modulation weight vector, which reflects the correlation between the semantic encoding vectors of the different matching template questions and the semantic encoding feature vectors of the interaction question. In other words, it reflects the degree of influence of the semantics of these matching template questions on the final answer generation. In this way, the system can highlight the features most relevant to the current question while reducing the influence of less relevant features. This helps generate more accurate and personalized answers.

[0080] In the above-mentioned agent-based virtual on-site worker interaction method, the S6, after weighting the multiple matching reference answer semantic encoding vectors using the cluster contribution field modulation weight vector to obtain a matching reference answer semantic significant aggregate representation, uses the matching reference answer semantic significant aggregate representation to generate a response text to obtain an interactive answer. Optionally, in one embodiment of the present application, Figure 5 This is a schematic flow chart of the method for interacting with virtual on-site workers based on an intelligent agent in an embodiment of the present application, wherein after weighting the semantic encoding vectors of the multiple matching reference answers using the cluster contribution field modulation weight vector to obtain a semantically significant aggregate representation of the matching reference answers, the semantically significant aggregate representation of the matching reference answers is used to generate an answer text to obtain an interactive answer. Figure 5 As shown, after using the cluster contribution field modulation weight vector to weight the multiple matching reference answer semantic coding vectors to obtain the matching reference answer semantic significant aggregation representation, the matching reference answer semantic significant aggregation representation is used to generate an answer text to obtain an interactive answer, including: S61, using each eigenvalue in the cluster contribution field modulation weight vector as a weight, calculating the position-weighted sum of the multiple matching reference answer semantic coding vectors to obtain the matching reference answer semantic significant aggregation coding vector as the matching reference answer semantic significant aggregation representation; S62, inputting the matching reference answer semantic significant aggregation coding vector into a decoder-based answer text generator to obtain the interactive answer.

[0081] It should be understood that since the individual eigenvalues in the cluster contribution field modulation weight vector represent the contribution correlation between the semantics of each matching template question and the semantics of the interactive question, these correlations can reflect the correlation between the semantics and the importance and significance of these matching reference answers for the generation of subsequent interactive answers. Therefore, by using these weight values, the features that are most explanatory of the current question can be highlighted, while those unimportant features can be ignored. This ensures that the generated answers are more focused on the core elements of the question. Based on this, in order to be able to adaptively generate answers that conform to the actual scenario of the interactive question, in the technical solution of the present application, the individual eigenvalues in the cluster contribution field modulation weight vector are further used as weights to calculate the position-weighted sum of the semantic encoding vectors of the multiple matching reference answers to obtain the semantically significant aggregate encoding vector of the matching reference answers. By calculating the weighted sum of the semantic encoding vectors of multiple matching reference answers, the advantages of different reference answers can be integrated, and their contribution and correlation weights can be used to generate a comprehensive answer. The answer generated in this way not only contains the common points of multiple answers, but also takes into account the specific circumstances of the current question, thereby improving the relevance and quality of the answer. The semantically significant aggregated encoding vector of the matching reference answer is then fed into a decoder-based answer text generator to generate an interactive answer. This allows for automatic generation of contextual responses tailored to different situations, improving the flexibility and adaptability of the interactive process and better assisting production or training.

[0082] In the technical solution of the present application, the semantically significant aggregated coding vector of the matched reference answer represents the semantically interactive aggregated coding features of multiple matched template questions and reference answers that are fuzzy matched with the interactive question. It should be understood that although semantic aggregation is performed using the individual eigenvalues in the cluster contribution field modulation weight vector as weights, the application unit of this attention mechanism is the feature level, which makes it possible that when the decoder-based answer text is generated for the semantically significant aggregated coding vector of the matched reference answer, the decoder-based answer text generator does not adequately decode or unbalances the fine-grained semantic part of the semantically significant aggregated coding vector of the matched reference answer, thereby affecting the full-text semantic quality of the interactive answer.

[0083] In response to the above technical problems, in the technical solution of the present application, before the semantically significant aggregated coding vector of the matching reference answer is input into the decoder-based answer text generator to obtain the interactive answer, the semantically significant aggregated coding vector of the matching reference answer is subjected to local semantic saliency modulation, and the process includes the following steps:

[0084] Calculate the mutual benchmark index and divergence characteristic index between the eigenvalues of any two positions in the matching reference answer semantically significant aggregation encoding vector to obtain the matching reference answer semantically significant aggregation main vibration configuration fitting matrix and the matching reference answer semantically significant aggregation pan-harmonic energy distribution matrix, which are expressed as:

[0085]

[0086] v i ,v j ∈V

[0087] Where V represents the semantically significant aggregated encoding vector of the matching reference answer, v i and v j Respectively represent the feature values of any two positions in the semantically significant aggregated encoding vector of the matching reference answer, Represents the value of the (i, j) position of the semantically significant aggregated main vibration configuration fitting matrix that matches the reference answer, Represents the value at position (i, j) of the semantically significant aggregated pan-harmonic energy distribution matrix that matches the reference answer.

[0088] The matching reference answer semantically significant aggregation main vibration configuration fitting matrix and the matching reference answer semantically significant aggregation pan-harmonic energy distribution matrix are respectively applied to the matching reference answer semantically significant aggregation encoding vector to obtain the matching reference answer semantically significant aggregation main oscillation mode vector and the matching reference answer semantically significant aggregation pseudo high-frequency component vector, which are expressed as:

[0089] V m =Sigmoid[W1PCA(D1)+b1]

[0090] V f =Sigmoid[W2PCA(D2)+b2]

[0091] V1=V m ⊙V

[0092] V2=V f ⊙V

[0093] Wherein, ⊙ represents the position point multiplication, D1 represents the semantically significant aggregated main vibration configuration fitting matrix of the matching reference answer, D2 represents the semantically significant aggregated pan-harmonic energy distribution matrix of the matching reference answer, W1 represents the first weight matrix, W2 represents the second weight matrix, b1 represents the first bias vector, b2 represents the second bias vector, Sigmoid represents the S-type activation function, PCA represents the principal component analysis, V m Represents the semantically significant aggregated main vibration configuration fitting activation vector matching the reference answer, V fIt represents the activation vector of the semantically significant aggregated pan-harmonic energy distribution of the matching reference answer, V1 represents the main oscillation mode vector of the semantically significant aggregated matching reference answer, and V2 represents the imitation high-frequency component vector of the semantically significant aggregated matching reference answer.

[0094] The semantically significant aggregated main oscillation modal vector of the matching reference answer and the semantically significant aggregated imitation high-frequency component vector of the matching reference answer are quantified to obtain the semantically significant aggregated dynamic modal synergy matrix of the matching reference answer, which is expressed as:

[0095]

[0096] Where T represents the transpose symbol, L represents the length of the vector, represents matrix multiplication, M represents the semantically significant aggregation dynamic modal collaboration matrix of the matching reference answer,

[0097] After feature fusion of the matching reference answer semantically significant aggregated main oscillation modal vector and the matching reference answer semantically significant aggregated imitation high-frequency component vector, they are input into the matching reference answer semantically significant aggregated dynamic modal synergy matrix for saliency demodulation to obtain an optimized matching reference answer semantically significant aggregated encoding vector, which is expressed as:

[0098]

[0099] in, represents vector addition, and V' represents the optimized semantically significant aggregated encoding vector of the matching reference answer.

[0100] Accordingly, in the technical solution of the present application, a pseudo-resonance superposition interference architecture based on statistical analysis is constructed for the semantically significant aggregated coding vector of the matching reference answer, and the state space attraction domain boundary of each feature data in the semantically significant aggregated coding vector of the matching reference answer is determined by the pseudo-resonance superposition interference architecture. Furthermore, low-rank component extraction is applied to connect and analyze the characteristic rotation phase synchronization relationship measure between the local unit characteristics and the overall vector, and then the feature manifold iterative trajectory tracking is clarified through autocorrelation analysis, so as to construct a description of the interactive relationship between the construction elements and an analysis of the disturbance sensitivity, thereby realizing a cross-dimensional multi-resolution feature joint decomposition expression from the structural micro-unit characteristics to the high-level system state. In this way, the global feature expression of the semantically significant aggregated coding vector of the matching reference answer is improved relative to the targeted feature mapping interpretation capability of the target generation domain, so as to improve the generation accuracy of the interactive answer. In this way, it is possible to use fuzzy queries in the interaction library and semantically weighted aggregation of multiple matching answers to generate responses that are more in line with actual scenarios. Through more intelligent interaction methods, the interaction process between on-site workers and intelligent agents can avoid the flexibility and real-time deficiencies of traditional methods, helping on-site workers to solve various problems in complex production environments, and also providing better support and training methods for workers.

[0101] In summary, according to the embodiment of the present application, the agent-based virtual on-site worker interaction method is explained, which can use virtual technology combined with artificial intelligence to create a simulation environment, and then in the process of interaction between on-site workers and the agent, the agent can understand human language in the virtual scene, and support the automatic generation of dialogue content between production site workers and the agent. This agent-based virtual on-site worker interaction method allows the creation of highly realistic simulation scenes and can dynamically adapt to user input and behavior. This means that the interaction process requires not only that the agent can understand and respond to user instructions, but also that it can automatically generate contextual responses that are consistent with different situations, so as to improve the flexibility and adaptability of the interaction process, so as to better assist production work or training processes.

[0102] Figure 6 FIG is a schematic block diagram of an agent-based virtual on-site worker interaction system according to an embodiment of the present application. Figure 6As shown, the agent-based virtual on-site worker interaction system 100 includes: a standard question-answer library construction module 110, which is used to construct a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions; an interactive question acquisition module 120, which is used to obtain interactive questions input by on-site workers to the agent; an interactive question fuzzy query module 130, which is used to perform fuzzy query on the interactive questions based on the standard question-answer library to obtain multiple matching template questions and reference answers; a semantic encoding module 140, which is used to semantically encode the multiple matching template questions and reference answers and the interactive questions to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and interactive question semantic encoding feature vectors; a clustering contribution field modulation module 150, which is used to perform semantic encoding feature vectors on the interactive question semantic encoding. The feature vector and the multiple matching template question semantic coding vectors are subjected to feature fusion based on self-supervised pseudo clustering to obtain a cluster contribution field modulation weight vector, including: a cluster contribution information calculation unit 151, used to calculate the cluster contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector; a cluster contribution field explicit feature identification unit 152, used to perform cluster contribution field explicit feature identification on the matching template question semantic feature cluster contribution field distribution vector to obtain the cluster contribution field modulation weight vector; an interactive answer generation module 160, used to use the cluster contribution field modulation weight vector to weight the multiple matching reference answer semantic coding vectors to obtain a matching reference answer semantic significant aggregation representation, and then use the matching reference answer semantic significant aggregation representation to generate an answer text to obtain an interactive answer.

[0103] Optionally, in one embodiment of the present application, the semantic encoding module is used to: perform semantic encoding on the multiple matching template questions and reference answers respectively to obtain the multiple matching template question semantic encoding vectors and the multiple matching reference answer semantic encoding vectors; and perform semantic encoding on the interactive question to obtain the interactive question semantic encoding feature vector.

[0104] Optionally, in one embodiment of the present application, the cluster contribution information calculation unit is used to: calculate the implicit cluster contribution factor of each matching template question semantic coding vector in the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a sequence of implicit cluster contribution factors of matching template question semantic features; and vectorize the sequence of implicit cluster contribution factors of matching template question semantic features to obtain the cluster contribution field distribution vector of the matching template question semantic features.

[0105] Furthermore, in one embodiment of the present application, a virtual on-site worker role-playing technology based on an intelligent agent is provided. Based on the system's application scenarios and application functions, the technology analyzes and studies the operational behaviors of on-site workers and other relevant personnel during the production command process, and establishes an operational behavior model for the relevant personnel. The technology also studies the interaction process modeling and intelligent interaction technology between virtual workers and production command personnel, and establishes a spatiotemporal model of the interaction process that includes elements such as interaction scenarios, interaction entities, and interaction behaviors. The technology also studies virtual worker role-playing technology, supporting the automatic generation of interaction content between virtual workers and production command personnel.

[0106] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0108] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0109] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0110] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units recited in a device claim can also be implemented by one unit through software or hardware.

[0111] Finally, it should be noted that the above description has been provided for purposes of illustration and description. Furthermore, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A virtual on-site worker interaction method based on an intelligent agent, characterized in that: include: Constructing a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions; Obtain interactive questions input by field workers to the agent; Based on the standard question-answer library, performing a fuzzy query on the interactive question to obtain a plurality of matching template questions and reference answers; Performing semantic encoding on the multiple matching template questions and reference answers and the interactive question to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors and an interactive question semantic encoding feature vector; The interactive question semantic coding feature vector and the multiple matching template question semantic coding vectors are subjected to feature fusion based on self-supervised pseudo clustering to obtain a cluster contribution field modulation weight vector, including: calculating cluster contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector; performing cluster contribution field explicit feature recognition on the matching template question semantic feature cluster contribution field distribution vector to obtain the cluster contribution field modulation weight vector; After weighting the multiple matching reference answer semantic encoding vectors using the cluster contribution field modulation weight vector to obtain a matching reference answer semantically significant aggregation representation, the matching reference answer semantically significant aggregation representation is used to generate an answer text to obtain an interactive answer.

2. The agent-based virtual on-site worker interaction method according to claim 1, characterized in that: Semantically encoding the multiple matching template questions and reference answers and the interactive question to obtain multiple matching template question semantic encoding vectors, multiple matching reference answer semantic encoding vectors, and an interactive question semantic encoding feature vector, including: Semantically encoding the multiple matching template questions and reference answers respectively to obtain the multiple matching template question semantic encoding vectors and the multiple matching reference answer semantic encoding vectors; The interaction question is semantically encoded to obtain a semantic encoding feature vector of the interaction question.

3. The agent-based virtual on-site worker interaction method according to claim 2, characterized in that: Calculating cluster contribution information of the plurality of matching template question semantic encoding vectors relative to the interaction question semantic encoding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector, including: Calculating the implicit clustering contribution factor of each matching template question semantic encoding vector in the multiple matching template question semantic encoding vectors relative to the interactive question semantic encoding feature vector to obtain a sequence of matching template question semantic feature implicit clustering contribution factors; The sequence of implicit clustering contribution factors of the matching template question semantic features is vectorized and arranged to obtain the matching template question semantic feature clustering contribution field distribution vector.

4. The agent-based virtual on-site worker interaction method according to claim 3, characterized in that: Calculating the implicit clustering contribution factor of each matching template question semantic encoding vector in the plurality of matching template question semantic encoding vectors relative to the interaction question semantic encoding feature vector to obtain a sequence of matching template question semantic feature implicit clustering contribution factors, including: Calculating the absolute value of positional division between the matching template question semantic encoding vector and the interaction question semantic encoding feature vector to obtain a matching template question semantic self-supervised clustering interaction correlation feature vector; Calculating the logarithmic function value of each position feature value in the matching template problem semantic self-supervised clustering interaction correlation feature vector with base 2 to obtain the matching template problem semantic self-supervised clustering interaction logarithmic representation vector; Using the feature values of each position in the matching template question semantic self-supervised clustering interaction logarithm representation vector as weighting coefficients, calculating the weighted sum of the feature values of each position in the matching template question semantic encoding vector to obtain a matching template question semantic feature clustering contribution representation factor; The exponential function value of the matching template question semantic feature cluster contribution representation factor is calculated with the natural constant e as the base to obtain the matching template question semantic feature implicit cluster contribution factor.

5. The agent-based virtual on-site worker interaction method according to claim 4, characterized in that: The cluster contribution field explicit feature recognition is performed on the matching template problem semantic feature cluster contribution field distribution vector to obtain the cluster contribution field domain modulation weight vector, including: inputting the matching template problem semantic feature cluster contribution field distribution vector into the cluster contribution field explicit modeling module based on the self-attention mechanism to obtain the cluster contribution field domain modulation weight vector.

6. The agent-based virtual on-site worker interaction method according to claim 5, characterized in that: Inputting the matching template problem semantic feature cluster contribution field distribution vector into the cluster contribution field explicit modeling module based on the self-attention mechanism to obtain the cluster contribution field modulation weight vector, including: Calculating matrix multiplications between the matching template question semantic feature cluster contribution field distribution vector and the query weight matrix, the key weight matrix, and the value weight matrix respectively to obtain a matching template question semantic cluster contribution field distribution query vector, a matching template question semantic cluster contribution field distribution key vector, and a matching template question semantic cluster contribution field distribution value vector; After calculating the multiplication between the matching template question semantic cluster contribution field distribution query vector and the transposed vector of the matching template question semantic cluster contribution field distribution key vector, the obtained matrix is divided by the square root of the length of the matching template question semantic cluster contribution field distribution key vector by position to obtain a query-key matching template question semantic cluster contribution field semantic interaction representation matrix; Normalizing the query-key matching template problem semantic clustering contribution field semantic interaction representation matrix using a Softmax function to obtain a normalized query-key matching template problem semantic clustering contribution field semantic interaction representation matrix; The multiplication between the matching template problem semantic cluster contribution field distribution value vector and the query-key matching template problem semantic cluster contribution field semantic interaction representation matrix is calculated to obtain the cluster contribution field domain modulation weight vector.

7. The agent-based virtual on-site worker interaction method according to claim 6, characterized in that: After weighting the multiple matching reference answer semantic encoding vectors using the cluster contribution field modulation weight vector to obtain a matching reference answer semantically significant aggregate representation, generating an answer text using the matching reference answer semantically significant aggregate representation to obtain an interactive answer, including: Using each eigenvalue in the cluster contribution field modulation weight vector as a weight, calculating the position-weighted sum of the multiple matching reference answer semantic encoding vectors to obtain a matching reference answer semantically significant aggregation encoding vector as the matching reference answer semantically significant aggregation representation; The semantically significant aggregated encoding vector of the matching reference answer is input into a decoder-based answer text generator to obtain the interactive answer.

8. An agent-based virtual on-site worker interaction system, characterized in that: include: A standard question-answer library construction module is used to construct a standard question-answer library, wherein the standard question-answer library includes a series of template questions and a series of reference answers corresponding to the template questions; The interactive question acquisition module is used to obtain the interactive questions input by the on-site operators to the intelligent agent; An interactive question fuzzy query module, configured to perform a fuzzy query on the interactive question based on the standard question-answer library to obtain a plurality of matching template questions and reference answers; A semantic encoding module, configured to perform semantic encoding on the plurality of matching template questions and reference answers and the interactive question to obtain a plurality of matching template question semantic encoding vectors, a plurality of matching reference answer semantic encoding vectors and an interactive question semantic encoding feature vector; A cluster contribution field modulation module is used to perform feature fusion based on self-supervised pseudo-clustering on the interactive question semantic coding feature vector and the multiple matching template question semantic coding vectors to obtain a cluster contribution field modulation weight vector, including: a cluster contribution information calculation unit, used to calculate the cluster contribution information of the multiple matching template question semantic coding vectors relative to the interactive question semantic coding feature vector to obtain a matching template question semantic feature cluster contribution field distribution vector; a cluster contribution field explicit feature recognition unit, used to perform cluster contribution field explicit feature recognition on the matching template question semantic feature cluster contribution field distribution vector to obtain the cluster contribution field modulation weight vector; An interactive answer generation module is used to use the cluster contribution field modulation weight vector to weight the multiple matching reference answer semantic encoding vectors to obtain a matching reference answer semantically significant aggregation representation, and then use the matching reference answer semantically significant aggregation representation to generate an answer text to obtain an interactive answer.

9. The agent-based virtual on-site worker interaction system according to claim 8, characterized in that: The semantic encoding module is used to: Semantically encoding the multiple matching template questions and reference answers respectively to obtain the multiple matching template question semantic encoding vectors and the multiple matching reference answer semantic encoding vectors; The interaction question is semantically encoded to obtain a semantic encoding feature vector of the interaction question.

10. The agent-based virtual on-site worker interaction system according to claim 9, characterized in that: The cluster contribution information calculation unit is used to: Calculating the implicit clustering contribution factor of each matching template question semantic encoding vector in the multiple matching template question semantic encoding vectors relative to the interactive question semantic encoding feature vector to obtain a sequence of matching template question semantic feature implicit clustering contribution factors; The sequence of implicit clustering contribution factors of the matching template question semantic features is vectorized and arranged to obtain the matching template question semantic feature clustering contribution field distribution vector.

Images