Human-computer interaction method and system based on AI large model

Abstract

The invention relates to the technical field of artificial intelligence, and provides a man-machine interaction method and system based on an AI large model. Comprising an interaction scene modeling and data acquisition module, an interaction context and joint interaction risk assessment module, a natural language intention analysis and task abstraction module, a hierarchical interaction arrangement module and an interaction presentation and control request generation module. According to the method, the joint interaction risk is evaluated, the natural language intention is analyzed, the hierarchical interaction process is generated, the adaptive interface presentation is adopted, and the control request is generated, so that the misoperation probability under the high-risk working condition is reduced, and the operation and maintenance safety, the decision controllability and the man-machine cooperation efficiency are improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, and particularly relates to a man-machine interaction method and system based on an AI large model. BACKGROUND

[0002] The production and operation of a new energy plant have perfect automatic control systems and intelligent analysis capabilities, but the existing man-machine interaction still mainly depends on traditional pictures and buttons, and lacks a technical solution capable of considering work condition data, intelligent analysis results and an operator's own state, and supporting safe and efficient operation through natural language interaction and automatic unfolding of task-level operation steps.

[0003] In Chinese application No. CN202511056568.X, a man-machine interaction system and method based on an AI large model. The configuration method comprises a natural language real-time extraction module, a natural language node segmentation module, an initial portrait modeling identification module, a dynamic portrait adjustment identification module and an AI feature identification result output module. Through NLP technology, the natural language input of a user is extracted in real time when the user first interacts with the system. The initial portrait of the user and the initial portrait of the default AI entity of the user are constructed based on the first interaction language. The user dynamically adjusts the initial portrait of the user and the initial portrait of the default AI entity of the user through a long short-term memory method to obtain the dynamic portrait of the user and the dynamic portrait of the default AI entity of the user, and outputs the corresponding AI feature identification result. Each AI entity has unique personality characteristics, and the efficiency and satisfaction of man-machine interaction are improved.

[0004] In the man-machine interaction scene of the production and operation of a new energy plant, although attempts have been made to use an AI large model for code generation, parameter configuration and operation and maintenance knowledge question and answer, a large number of applications still remain at the level of assisting engineers in writing codes and querying manuals, and the man-machine interaction interface itself still mainly uses traditional buttons, tables and menus, resulting in that fault diagnosis and work condition adjustment still highly depend on experience, and there is a lack of a solution capable of obtaining operation suggestions through a unified interaction entrance; meanwhile, when multiple alarms occur on site, there is a lack of the ability to comprehensively analyze the alarms in combination with the current production state and historical processing records, and to actively guide an operator to dispose according to the priority.

[0005] To solve the above problems, the application provides a man-machine interaction method and system based on an AI large model. The method adaptively adjusts an interaction process and an interface based on a joint interaction risk index, and converts a natural language instruction into a control request constrained by a risk level. In addition, the system comprises an interaction scene modeling and data acquisition module, an interaction context and joint interaction risk assessment module, a natural language intention analysis and task abstraction module, a hierarchical interaction arrangement module and an interaction presentation and control request generation module, and is used for implementing the method in a new energy plant. SUMMARY

[0006] In view of the above existing problems, a human-computer interaction method and system based on an AI large model are proposed.

[0007] The technical solution adopted by the present application to solve the above technical problems is: a human-computer interaction method based on an AI large model, comprising:

[0008] By modeling and labeling the process flow, equipment objects and monitoring interface, an interaction scene model is obtained, and the collection parameters of process measurement point data and operation behavior data are configured to establish an interaction data model and a real-time collection channel;

[0009] The production process data and operation behavior data obtained by the real-time collection channel are subjected to feature extraction and joint coding, and the working condition risk index and the interaction load index are calculated, and the two are combined to obtain the joint interaction risk index and the real-time interaction context information;

[0010] Based on natural language instructions and real-time interaction context information, a pre-trained language model is used for intent analysis and task abstraction to obtain an operation and maintenance task abstraction result, and the operation and maintenance task abstraction result is classified and interactively arranged according to the risk level to which the joint interaction risk index belongs;

[0011] Under the control of the joint interaction risk index, the interaction step sequence is subjected to interaction presentation processing to obtain a hierarchical interaction interface and a prompt, and the parameter adjustment and confirmation operation of the operator in the interaction interface is analyzed and protocol mapped to generate a control request corresponding to the natural language intent and subject to the risk level constraint.

[0012] As a preferred embodiment, the specific steps of establishing the interaction data model and the real-time collection channel are:

[0013] According to the actual process, the process line is divided into multiple process nodes, and the connection relationship between the nodes is established to form a process topology structure, the field equipment is abstracted as equipment objects, and the monitoring quantity is abstracted as process measurement points, the interface controls in the monitoring picture are identified and numbered, and the labeling relationship between each interface element and the corresponding process measurement point is established to jointly represent the process topology, equipment objects and interface elements in the same data structure to form an interaction scene model; based on the interaction scene model, the collection parameters of the process measurement point data and the operation behavior data to be monitored are configured, and the interaction data model and the corresponding real-time collection channel are established accordingly;

[0014] As a preferred embodiment, the specific steps of feature extraction and joint coding of the production process data and operation behavior data obtained by the real-time collection channel are:

[0015] The production process data is normalized and the change rate is calculated within a preset time window, the normalized value is taken as the reference value of the current working condition, the remaining safety margin between the reference value and the upper and lower safety limits is calculated, the change rate of adjacent sampling points and the maximum change rate within the window are calculated according to the sampling time, and the working condition feature vector representing the stability of the current working condition and the degree of approaching the safety boundary is formed by splicing in a fixed order, and the operation behavior data is counted, the type, occurrence time and belonging interface of each interaction event are stored in the operation behavior log, the occurrence frequency of each type of event is calculated by the window length, the mean and variance are calculated according to the time interval sequence between adjacent operation events, the interface stay time distribution is constructed according to the proportion of stay time on different monitoring interfaces, and the normalized entropy value of the distribution is calculated, and the features obtained by the above calculation are spliced to form the interaction load feature vector representing the operation frequency, operation rhythm and attention dispersion degree;

[0016] As a preferred embodiment, the specific steps of calculating the working condition risk index and the interaction load index are:

[0017] The working condition feature vector is input into the working condition risk assessment model to calculate the working condition risk index, wherein the working condition risk assessment model is obtained by offline training of historical operation data and historical abnormal events, the current working condition feature vector is converted into a working condition risk index in the interval of 0 to 1, and the working condition risk index calculation formula is:

[0018] ,

[0019] Wherein represents the working condition risk index of the current time window t, i represents the subscript of the working condition feature, and n represents the total number of working condition features, represents the weight parameter corresponding to the i-th working condition feature, represents the value of the i-th working condition feature in the current time window t, represents the bias term of the working condition risk assessment model;

[0020] The interaction load feature vector is input into the interaction load assessment model to calculate the interaction load index, wherein the interaction load assessment model is obtained by supervised learning training based on historical operation logs and manual load labeling, the interaction load feature vector is converted into a scoring model of the interaction load index in the interval of 0 to 1, and the interaction load index calculation formula is:

[0021] ,

[0022] Wherein represents the interaction load index of the current time window t, j represents the subscript of the interaction load feature, and m represents the total number of interaction load features, denotes the weight parameter corresponding to the jth interaction load feature, denotes the value of the jth interaction load feature at the current time window t, denotes the bias term of the interaction load evaluation model;

[0023] The working condition risk index and the interaction load index are weighted and nonlinearly combined to obtain a joint interaction risk index, and the working condition features, the interaction load features, and the three indexes are packaged as real-time interaction context information;

[0024] As a preferred embodiment, the specific steps of using the pre-trained language model for intent analysis and task abstraction are:

[0025] First, the voice instruction is recognized to obtain a natural language instruction in text form, and context elements are extracted from the real-time interaction context information; the natural language instruction and the context elements are uniformly encoded and input into the pre-trained language model, and a structured operation and maintenance task abstraction result is output by the language model, which converts the originally fuzzy natural language intent into a machine-understandable task description;

[0026] In the central control room, a voice acquisition terminal is arranged to acquire the voice instruction spoken by the operator, after echo cancellation, noise suppression and endpoint detection processing, the continuous voice signal is divided into time frames of fixed length, and acoustic features are extracted from each frame to form an acoustic feature sequence arranged in time sequence; the acoustic feature sequence is input into a pre-trained language model as an input, and a voice recognition sub-model is trained offline in advance, the voice recognition sub-model preferably adopts an end-to-end structure of an encoder and a decoder, the encoder is used to map the acoustic feature sequence into a high-dimensional hidden representation, the decoder internally calls the language modeling capability of the pre-trained language model, and gradually outputs a pinyin, Chinese character or sub-word unit sequence in a conditional generation manner, and a beam search and a domain word list are used to constrain the decoding process, so as to preferentially match and correct the names of special equipment and process nouns in the new energy power plant, and obtain a text string corresponding to the voice content; after obtaining the preliminary recognition result, the text correction capability of the same pre-trained language model is called to correct the words that do not conform to the operation and maintenance context of the new energy power plant, and a corrected natural language instruction text is obtained.

[0027] As a preferred embodiment, the specific steps of classifying and interacting the operation and maintenance task abstraction result according to the risk level to which the joint interaction risk index belongs are:

[0028] The calculated joint interaction risk index is mapped to low risk, medium risk and high risk levels according to preset risk classification rules, and the risk level is used as a control variable for interaction strategy selection, and the operation and maintenance task abstraction result is classified and interaction is arranged, in the low risk level, a plurality of subtasks are combined to generate an interaction step sequence with few steps and concentrated parameters, allowing a recommended scheme to be given at one time and completed through single confirmation; in the medium risk level, the task is split into several logical subtasks, each interaction step involves only a set of related parameters, and the operator is required to explicitly confirm before proceeding to the next step after each step; in the high risk level, the complex task is further refined into smaller interaction steps, each step only adjusts a single key parameter, and additional reason explanation, risk prompt and multiple confirmation mechanism are added;

[0029] As a preferred embodiment, the specific steps of analyzing and protocol mapping the parameter adjustment and confirmation operations of the operator in the interaction interface are as follows:

[0030] For each obtained interaction step, a display template corresponding to the interaction step is selected from a preset interface template library according to the target device, the parameter to be adjusted and the process constraint, and the interaction step and the corresponding template are associated to form a candidate interface; for each interaction step, voice and text prompts are generated by combining a pre-trained language model with a fixed prompt template;

[0031] The parameter input, option selection and confirmation operations of the operator in the interaction interface are structurally analyzed to obtain the actual operation intention of the step, and the operation intention is mapped to a standard control instruction supported by the underlying control system according to the target object and the parameter type in the interaction step, and the adjustment amplitude and executable instruction type are automatically limited according to the current risk level during the mapping process, the control request after risk constraint processing is encapsulated into a data packet conforming to the communication protocol, and the execution result and key parameter change are written back to the real-time interaction context for the next cycle of joint interaction risk assessment and subsequent interaction adjustment;

[0032] In order to achieve the same purpose of the above technical scheme, the present application also provides a man-machine interaction system based on AI large model, comprising: an interaction scene modeling and data acquisition module, an interaction context and joint interaction risk assessment module, a natural language intention analysis and task abstraction module, a hierarchical interaction arrangement module, an interaction presentation and control request generation module.

[0033] As a preferred embodiment, the specific steps of the man-machine interaction system based on AI large model are as follows:

[0034] An interactive scene modeling and data collection module, which internally has a process topology library, a device object library and an interface mapping table, divides the process flow into process nodes, represents the field devices as device objects, records the corresponding relationship between the monitoring interface elements and the process measurement points and the device objects in the interface mapping table, and forms an interactive scene model; collects the identification, sampling period and trigger condition of each process measurement point and each type of operation behavior collection item through a collection configuration table, and establishes a real-time collection channel in the control network that matches the interactive scene model;

[0035] An interactive context and joint interactive risk assessment module, which is connected with the real-time collection channel, writes the collected production process data into a working condition feature buffer, and writes the collected interface switching, mouse clicking, parameter confirmation and voice instruction logs into an interactive behavior buffer; a working condition feature calculation unit and an interactive load feature calculation unit, which are respectively used to normalize, calculate the rate of change and the fluctuation degree of the production process data within a preset time window to generate a working condition feature vector, and to calculate the frequency, rhythm fluctuation degree and interface residence distribution of the operation behavior data to generate an interactive load feature vector; input the working condition feature vector into a working condition risk assessment model to obtain a working condition risk index, input the interactive load feature vector into an interactive load assessment model to obtain an interactive load index, and by a joint risk synthesis submodule, the two are weighted and nonlinearly combined to generate a joint interactive risk index and real-time interactive context information containing working condition features, interactive load features and the above-mentioned indexes;

[0036] A natural language intent analysis and task abstraction module, which is based on a large-scale pre-training language model of deep learning, is connected with the voice recognition module and the interactive context and joint interactive risk assessment module, receives the natural language instruction text obtained by voice recognition and the real-time interactive context information, encodes the two into semantic vectors through a context coding unit, and calls the large-scale pre-training language model to output structured operation and maintenance task abstraction results;

[0037] A hierarchical interactive arrangement module, which is connected with the natural language intent analysis and task abstraction module and the interactive context and joint interactive risk assessment module, internally has an interactive strategy library and a step template library, reads the operation and maintenance task abstraction results and the joint interactive risk index, selects the corresponding strategy from the interactive strategy library according to the risk level to which the joint interactive risk index belongs, and generates an interactive step sequence according to the step template library, wherein a coarse-grained step sequence containing multiple operation parameters is generated at a low risk level, and a fine-grained step sequence containing only a single operation parameter is generated at a medium and high risk level;

[0038] The interactive presentation and control request generation module is connected with the hierarchical interactive arrangement module and the industrial control system interface, and is internally provided with an interface template library, a prompt generation unit and a protocol adaptation unit, under the control of the joint interactive risk index, selects different interface templates for the interactive step sequence and limits the number of variables displayed on each interface, generates hierarchical interactive interfaces and corresponding voice prompts and text prompts, analyzes the parameter adjustment, option selection and confirmation operation of the operator in the hierarchical interactive interface, forms a structured operation intention, and the control constraint unit restricts the available instruction type and parameter adjustment amplitude according to the current risk level, and the protocol adaptation unit converts the control request into a control request conforming to the communication protocol, and generates a control request corresponding to the natural language intention and subjected to the risk level constraint.

[0039] Advantages

[0040] Compared with the prior art, the present application has the following advantages:

[0041] 1. By adopting the method of normalization, change rate and fluctuation degree calculation on the production process data, the working condition feature vector representing the working condition stability and the degree of approaching the safety boundary is obtained, whether the equipment working condition is close to the dangerous state and whether the operator is in the high load state are reflected in a clear numerical index, which provides a quantifiable risk control variable for subsequent hierarchical interaction and interface self-adaptation.

[0042] 2. By adopting the method of joint encoding of the natural language instruction of the operator and the real-time interactive context information and inputting a large-scale pre-trained language model, the large model considers the current production state and safety constraint when understanding the natural language instruction, which solves the problem that the existing voice interaction system can only perform template matching and cannot accurately map the colloquial instruction to a structured operation and maintenance task that can be directly used for control decision. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope of the present application.

[0044] Figure 1 is a flowchart of the present application;

[0045] Figure 2 is a module diagram of the present application;

[0046] Figure 3 is an effect comparison diagram of the present application and the prior art, wherein the gray column represents the prior art and the black column represents the present application. DETAILED DESCRIPTION

[0047] In order to make the technical means, creative features, purposes and effects of the present application easy to understand, the following further describes the present application in combination with specific embodiments, but the following embodiments are only preferred embodiments of the present application, not all. Based on the embodiments in the embodiments, other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the present application.

[0048] Embodiment 1:

[0049] To achieve the above purpose, please refer to Figure 1 The embodiment of the present application provides a human-computer interaction method based on an AI large model, which comprises the following steps:

[0050] Step S1: constructing an interaction scene model, collecting configuration data;

[0051] Step S2: calculating real-time interaction context, evaluating joint interaction risk;

[0052] Step S3: analyzing natural language intention, generating hierarchical interaction process;

[0053] Step S4: taking adaptive interface presentation, generating control request.

[0054] The method is implemented in the order of S1-S4, and the overall process is as follows:

[0055] By modeling and labeling the process flow, equipment objects and monitoring interface, the interaction scene model is obtained, and the collection parameters of process measurement point data and operation behavior data are configured to establish the interaction data model and real-time collection channel;

[0056] The production process data and operation behavior data obtained by the real-time collection channel are subjected to feature extraction and joint coding, and the working condition risk index and interaction load index are calculated, and the two are combined to obtain the joint interaction risk index and real-time interaction context information;

[0057] Based on the natural language instruction and the real-time interaction context information, the pre-trained language model is used for intention analysis and task abstraction to obtain the operation and maintenance task abstraction result, and according to the risk level to which the joint interaction risk index belongs, the operation and maintenance task abstraction result is subjected to hierarchical interaction arrangement;

[0058] Under the control of the joint interaction risk index, the interaction step sequence is subjected to interaction presentation processing to obtain the hierarchical interaction interface and prompt, and the parameter adjustment and confirmation operation of the operator in the interaction interface is analyzed and protocol mapped to generate the control request corresponding to the natural language intention and constrained by the risk level.

[0059] The specific steps of constructing the interaction scene model and collecting the configuration data are:

[0060] According to the actual process of the new energy plant, the process line is split into multiple process nodes, and the connection relationship between the nodes is established to form a process topology structure. The field devices are abstracted as device objects, and the monitoring quantities such as pressure, temperature, flow, and SOC are abstracted as process measuring points. The numerical display box, trend curve, alarm row, and button control in the monitoring picture are identified and numbered. The annotation relationship between each interface element and the corresponding process measuring point is established to jointly represent the process topology, device object, and interface element in the same data structure to form an interactive scene model. On the basis of the interactive scene model, the collection parameters of the process measuring point data and operation behavior data to be monitored are configured. The collection parameters are used to clearly define the collection items, sampling period, and trigger conditions of various data, and an interactive data model and the corresponding real-time collection channel are established accordingly to realize the synchronous collection and unified coding of process data flow and human-computer interaction behavior.

[0061] Specifically, the collection parameters of the process measuring point data and operation behavior data to be monitored are configured. The monitoring purpose and importance are determined for each process measuring point and each type of operation behavior to form a set of objects to be configured. The objects are classified according to the process section, device type, and safety importance into high-priority monitoring objects and ordinary monitoring objects. The collection parameter vector is configured for each monitoring object to complete the collection parameter configuration. For operation behavior data, the time window length parameter is configured for interface dwell time and picture switching times, and the event trigger collection parameter is configured for mouse clicks and voice instructions. The operation frequency is counted in each time window, and the operation behavior is recorded immediately when each interaction event occurs. The collection parameter vector configured according to the objects and priorities is associated with the process measuring points and interface elements in the interactive scene model to generate an interactive data model.

[0062] Through the above unified modeling, annotation, and collection parameter configuration, the process physical objects, control measuring points, interface elements, and operation behaviors can be associated in the unified interactive scene model, solving the problem that the process modeling and interface modeling in the prior art are mutually fragmented and cannot depict the joint state of human, machine, interface, and process from a holistic perspective.

[0063] The specific steps of calculating the real-time interaction context and evaluating the joint interaction risk are as follows:

[0064] The production process data is normalized and the change rate is calculated within a preset time window, the normalized value is taken as the reference value of the current working condition, and the remaining safety margin between the reference value and the upper and lower safety limits is calculated, the change rate of adjacent sampling points and the maximum change rate within the window are calculated according to the sampling time, and the working condition feature vector representing the stability of the current working condition and the degree of approaching the safety boundary is formed by splicing in a fixed order, and the operation behavior data is counted, the type, occurrence time and belonging interface of each interaction event are stored in the operation behavior log, the occurrence frequency of each type of event is calculated by the window length, the mean and variance are calculated according to the time interval sequence between adjacent operation events, the interface stay time distribution is constructed according to the proportion of stay time on different monitoring interfaces, and the normalized entropy value of the distribution is calculated, and the features obtained by the above calculation are spliced to form the interaction load feature vector representing the operation frequency, operation rhythm and attention dispersion degree;

[0065] The working condition feature vector is input into the working condition risk assessment model to calculate the working condition risk index, wherein the working condition risk assessment model is obtained by offline training of historical operation data and historical abnormal events, and is used to convert the current working condition feature vector into a working condition risk index in the interval of 0 to 1 during operation, and the working condition risk index calculation formula is:

[0066] ,

[0067] Among them, indicates the working condition risk index of the current time window t, i indicates the subscript of the working condition feature, and n indicates the total number of working condition features, indicates the weight parameter corresponding to the i-th working condition feature, indicates the value of the i-th working condition feature in the current time window t, indicates the bias term of the working condition risk assessment model;

[0068] Specifically, the establishment of the working condition risk assessment model includes an offline training phase and an online calculation phase. In the offline training phase, the historical operation data of the new energy plant in the past year is selected, the continuous data is divided according to the preset time window, and the corresponding working condition feature vector is calculated for each time window , The operating condition characteristics include the normalized values, safety margins, maximum rate of change, and volatility of each key measuring point at the end of the window. Based on whether an over-limit alarm occurs, whether an interlocking action is triggered, or whether an event is recorded as a fault or hazard within the same time window, the time window is manually labeled as normal, slightly abnormal, significantly abnormal, or dangerous. Each level is then converted into a numerical label between 0 and 1 using a preset mapping, for example, corresponding to 0, 0.33, 0.67, and 1 respectively, resulting in a large number of operating condition feature vectors and risk label samples. Using the operating condition feature vectors as input and the risk labels as the desired output, a supervised learning algorithm is used to adjust the weight parameters in the formula. and bias terms Training is performed to ensure that the calculated work condition risk index on all training samples approximates the corresponding risk label as closely as possible within the mean squared error range. During training, predicted values ​​exceeding the 0-1 range are truncated to ensure the model output consistently falls within this range. When the loss function converges to below a preset threshold, the resulting weight parameters are... and bias terms The parameters are solidified into the working condition risk assessment model. During the online calculation phase, the system calculates the current working condition feature vector in the same way as in the training phase within each time window, and substitutes it into the above formula to obtain the working condition risk index of the current time window t. When the working condition risk index is close to 0, it indicates that the current working condition is in a safe state. When the working condition risk index is close to 1, it indicates that the current working condition is close to or has entered a dangerous state.

[0069] The interaction load index is calculated by inputting the interaction load feature vector into the interaction load assessment model. This model is trained through supervised learning based on historical operation logs and manually labeled load data. It is used to convert the interaction load feature vector into a scoring model with an interaction load index ranging from 0 to 1. The formula for calculating the interaction load index is as follows:

[0070] ,

[0071] in This represents the interactive load index for the current time window t, where j represents the index of the interactive load feature, and m represents the total number of interactive load features. This represents the weight parameter corresponding to the j-th interactive load feature. This represents the value of the j-th interactive load feature within the current time window t. This represents the bias term in the interactive load assessment model;

[0072] Specifically, the establishment of the interactive load assessment model includes a model training phase and an online calculation phase. In the model training phase, the control room operation logs of the new energy plant for three consecutive months are selected, with each 30-second interval serving as a time window. The corresponding interactive load feature vector is then statistically obtained for each time window. ,..., ,in This refers to the number of screen transitions per unit of time. This represents the number of mouse clicks per unit time. Simultaneously, a subjective evaluation of the operational load within the same time window is performed, mapping easy, normal, stressful, and significantly overloaded to 0, 0.3, 0.7, and 1 respectively, serving as the artificial load label for that time window. Using the interactive load feature vector of each time window as input and the artificial load label as the desired output, a supervised learning algorithm is employed to adjust the weight parameters. and bias terms Training is performed to make the interaction load index calculated on the training samples as close as possible to the artificial load label. During the online operation phase, the current interaction load feature vector is calculated in each time window using the same statistical method as in the training phase, and then substituted into the formula to calculate the interaction load index of the current time window t. If the interaction load index is close to 0, it indicates that the operation load is low. If the interaction load index is close to 1, it indicates that the operator is in a significantly high load state.

[0073] The operating condition risk index and the interactive load index are weighted and nonlinearly combined to obtain the joint interactive risk index. The operating condition characteristics, interactive load characteristics and the three indices are encapsulated as real-time interactive context information.

[0074] By jointly modeling and nonlinearly combining operating condition risks and interactive loads, this invention differs from existing technologies that rely solely on process parameters for risk assessment. It can simultaneously reflect both equipment hazards and operator cognitive overload in the same risk indicator, enabling refined control over interaction depth, rhythm, and information density in subsequent human-machine interactions.

[0075] The specific steps for parsing natural language intent and generating a hierarchical interaction flow are as follows:

[0076] First, the operator's voice commands are recognized to obtain natural language commands in text form. Then, contextual elements such as working condition category, key process variables, alarm overview, and the most recent round of interaction steps are extracted from the real-time interaction context information. The natural language commands and contextual elements are uniformly encoded and input into a pre-trained language model. Preferably, the pre-trained language model is a large-scale pre-trained language model based on the Transformer structure. The language model outputs a structured operation and maintenance task abstraction result, which includes task type, target object set, parameters to be operated and expected change direction, as well as process constraints that must be met, thus converting the originally vague natural language intent into a machine-understandable task description.

[0077] Specifically, the process involves recognizing speech commands using a large-scale pre-trained language model to obtain natural language commands in text form. A voice acquisition terminal is set up in the central control room to collect the speech commands spoken by the operator. After echo cancellation, noise suppression, and endpoint detection, the continuous speech signal is divided into fixed-length time frames, and acoustic features, such as Mel-frequency cepstral coefficients and log-Mel filter bank energy, are extracted from each frame to form a temporally ordered acoustic feature sequence. Subsequently, this acoustic feature sequence is fed as input into a speech recognition sub-model within a pre-trained, offline large-scale pre-trained language model. Preferably, the speech recognition sub-model uses... Using an end-to-end structure integrating encoder and decoder, the encoder maps acoustic feature sequences to high-dimensional hidden representations, while the decoder internally invokes the language modeling capabilities of a pre-trained language model to progressively output sequences of pinyin, Chinese characters, or sub-word units through conditional generation. During the decoding process, bundle search and domain vocabulary constraints are employed to prioritize matching and correcting names of proprietary equipment and process terms in new energy factories, ultimately yielding a text string corresponding to the speech content. After obtaining the preliminary recognition results, the text correction capabilities of the same pre-trained language model are invoked to correct words that clearly do not conform to the operational context of new energy factories, resulting in corrected natural language instruction text.

[0078] The calculated joint interaction risk index is mapped to low-risk, medium-risk, and high-risk levels according to preset risk grading rules. The risk level is used as the control variable for interaction strategy selection. The abstract results of operation and maintenance tasks are arranged in a hierarchical manner. Under the low-risk level, multiple sub-tasks are merged to generate an interaction step sequence with fewer steps and more parameters, allowing a recommended solution to be provided at once and completed with a single confirmation. Under the medium-risk level, the task is broken down into several logical sub-tasks, with each interaction step involving only one set of relevant parameters. After each step, the operator is required to make a clear confirmation before proceeding to the next step. Under the high-risk level, complex tasks are further refined into smaller-granular interaction steps. Each step only adjusts a single key parameter and includes explanations of reasons, risk warnings, and multiple confirmation mechanisms.

[0079] Specifically, the thresholds mapped by the preset risk classification rules are used to replay historical operating data of the new energy plant. For each time window, the corresponding operating condition risk index, interactive load index, and joint interactive risk index are calculated to form a historical risk sequence. Simultaneously, various types of events are extracted from alarm records, interlocking action records, fault work orders, and shift reports. After time alignment, each time window is marked as a different category, such as safe samples, early warning samples, and high-risk samples. The distribution of the joint interactive risk index corresponding to each category of samples is statistically analyzed to obtain the distribution intervals for safe samples, early warning samples, and high-risk samples. Indicators such as the maximum inter-class interval and the minimum false positive rate are used to evaluate different candidate threshold combinations. The threshold combination that can distinguish between safe, early warning, and high-risk samples is selected as the initial threshold. Based on this, and combined with expert understanding of specific operating conditions, the thresholds are fine-tuned. During the trial operation phase, the number of false alarms and missed alarms is continuously recorded, and the thresholds are iteratively corrected a limited number of times, ultimately solidifying into the risk classification threshold parameters for the current site.

[0080] By combining the above natural language with context, the task is abstracted into an interactive process arranged according to risk level. This invention differs from the existing technology that only performs simple keyword matching on natural language. It realizes the joint parsing of complex natural language instructions and working condition information into structured tasks using a pre-trained language model. With the joint interaction risk index as the core control variable, it adaptively generates operation and maintenance interaction processes of different depths and rhythms, which improves the adaptability of human-computer interaction to complex working conditions and high-pressure states.

[0081] The specific steps for generating control requests using an adaptive interface are as follows:

[0082] For each interaction step obtained, based on the target device, parameters to be adjusted, and process constraints, a corresponding display template is selected from the preset interface template library, and the interaction step is associated with the corresponding template to form a candidate interface; for each interaction step, voice and text prompts are generated by combining a pre-trained language model with a fixed prompt template.

[0083] The system performs structured analysis on the parameter inputs, option selections, and confirmation operations of operators in the interactive interface to obtain the actual operational intent of this step. Based on the target object and parameter type in the interactive step, the operational intent is mapped to the standard control instructions supported by the underlying control system, such as setpoint adjustment instructions and start / stop instructions. During the mapping process, the single adjustment range and executable instruction types are automatically limited according to the current risk level. The control request after risk constraint processing is encapsulated into data messages that conform to communication protocols such as EMS and PLC and sent for execution. The execution results and key parameter changes are written back to the real-time interactive context for joint interactive risk assessment and subsequent interactive adjustments in the next cycle.

[0084] Through the above-mentioned interactive presentation processing and risk constraint control request generation under the joint interactive risk index control, the present invention simultaneously constrains interface display and control issuance under the same mechanism. This is different from the existing technology where the interface is fixed and the control commands are disconnected from the operation load. It realizes a risk-adaptive human-computer interaction closed loop from natural language intent to underlying control.

[0085] like Figure 3 This is a comparison chart of the effects of a human-computer interaction method and system based on a large AI model. The horizontal axis lists key performance indicators, and the vertical axis represents the exemplified performance scores, ranging from 0 to 100. The higher the value, the better the performance. The aim is to intuitively demonstrate the expected improvement of the present invention in key capabilities compared to typical existing technologies.

[0086] Example 2, as Figure 2 This is a human-computer interaction system based on a large AI model, including modules for interaction scene modeling and data acquisition, interaction context and joint interaction risk assessment, natural language intent parsing and task abstraction, hierarchical interaction orchestration, and interaction presentation and control request generation. The specific implementation steps are as follows:

[0087] The interactive scenario modeling and data acquisition module has a built-in process topology library, equipment object library, and interface mapping table. It divides the process flow into process nodes and represents field equipment as equipment objects. The interface mapping table records the correspondence between monitoring interface elements, process measurement points, and equipment objects to form an interactive scenario model. The acquisition configuration table identifies each process measurement point and various operation behavior acquisition items, sets the sampling period, and trigger conditions, and establishes a real-time acquisition channel in the control network that matches the interactive scenario model.

[0088] The interaction context and joint interaction risk assessment module is connected to the real-time acquisition channel. It writes the acquired production process data into the operating condition feature buffer and writes the acquired interface switching, mouse click, parameter confirmation, and voice command logs into the interaction behavior buffer. The operating condition feature calculation unit and the interaction load feature calculation unit are used to normalize, calculate the rate of change and volatility of the production process data within a preset time window to generate the operating condition feature vector, and to calculate the frequency statistics, rhythm volatility, and interface dwell distribution of the operation behavior data to generate the interaction load feature vector. The operating condition feature vector is input into the operating condition risk assessment model to obtain the operating condition risk index, and the interaction load feature vector is input into the interaction load assessment model to obtain the interaction load index. The joint risk synthesis submodule performs weighted and nonlinear combination of the two to generate the joint interaction risk index and real-time interaction context information containing the operating condition features, interaction load features, and the above indices.

[0089] The Natural Language Intent Parsing and Task Abstraction Module, based on a large-scale pre-trained language model of deep learning, is connected to the speech recognition module and the interaction context and joint interaction risk assessment module. It receives the natural language instruction text obtained from speech recognition and real-time interaction context information, encodes the two into semantic vectors through the context encoding unit, and calls the large-scale pre-trained language model to output structured operation and maintenance task abstraction results.

[0090] The hierarchical interaction orchestration module is connected to the natural language intent parsing and task abstraction module, as well as the interaction context and joint interaction risk assessment module. It has a built-in interaction strategy library and step template library. It reads the operation and maintenance task abstraction results and the joint interaction risk index. Based on the risk level of the joint interaction risk index, it selects the corresponding strategy from the interaction strategy library and generates an interaction step sequence according to the step template library. At the low risk level, it generates a coarse-grained step sequence containing multiple parameters to be operated. At the medium and high risk levels, it generates a fine-grained step sequence containing only a single parameter to be operated.

[0091] The interactive presentation and control request generation module is connected to the hierarchical interactive orchestration module and the industrial control system interface. It has a built-in interface template library, prompt generation unit, and protocol adaptation unit. Under the control of the joint interactive risk index, it selects different interface templates for the interactive step sequence and limits the number of variables displayed on each interface to generate hierarchical interactive interfaces and corresponding voice and text prompts. It parses the parameter adjustment, option selection, and confirmation operations of the operator in the hierarchical interactive interface to form a structured operation intention. The control constraint unit constrains the available instruction types and parameter adjustment ranges according to the current risk level. The protocol adaptation unit converts the control request into a control request that conforms to the communication protocol, generating a control request that corresponds to the natural language intention and is constrained by the risk level.

[0092] The embodiments of the present invention described above are subject to modification and change of method by those skilled in the art without departing from the embodiments of the present invention and its broader aspects. The appended claims are intended to include all such modifications and changes of method that do not depart from the present invention.

Claims

1. A human-computer interaction method based on a large AI model, characterized in that, include: By modeling and labeling the process flow, equipment objects, and monitoring interfaces, an interactive scenario model is obtained. The acquisition parameters for process measurement point data and operation behavior data are configured to establish an interactive data model and a real-time acquisition channel. The production process data and operation behavior data obtained from the real-time acquisition channel are subjected to feature extraction and joint encoding to calculate the working condition risk index and interaction load index. The two are then combined to obtain the joint interaction risk index and real-time interaction context information. Based on natural language instructions and real-time interactive context information, a pre-trained language model is used for intent parsing and task abstraction to obtain the operation and maintenance task abstraction results. The operation and maintenance task abstraction results are then hierarchically and interactively arranged according to the risk level of the joint interaction risk index. Under the control of the joint interaction risk index, the interaction step sequence is processed by interaction presentation to obtain a hierarchical interaction interface and prompts. The parameter adjustment and confirmation operations of the operator in the interaction interface are parsed and mapped by protocol to generate control requests that correspond to natural language intent and are subject to risk level constraints.

2. The human-computer interaction method based on a large AI model according to claim 1, characterized in that: The specific steps for establishing the interactive data model and real-time acquisition channel are as follows: Based on the actual process, the process line is divided into multiple process nodes, and the connection relationships between each node are established to form a process topology. Field equipment is abstracted as equipment objects, and the monitored quantities are abstracted as process measurement points. Interface controls in the monitoring screen are identified and numbered, and a labeling relationship is established between each interface element and its corresponding process measurement point. This allows the process topology, equipment objects, and interface elements to be jointly represented in the same data structure, forming an interactive scenario model. Based on the interactive scenario model, the acquisition parameters for the process measurement point data and operation behavior data that need to be monitored are configured, and an interactive data model and corresponding real-time acquisition channels are established accordingly.

3. The human-computer interaction method based on a large AI model according to claim 1, characterized in that: The specific steps for feature extraction and joint encoding of production process data and operational behavior data obtained from the real-time acquisition channel are as follows: Within a preset time window, production process data is normalized and its rate of change is calculated. The normalized value is used as the baseline value for the current operating condition, and the remaining safety margin between the baseline value and the upper and lower safety limits is calculated. The rate of change of adjacent sampling points and the maximum rate of change within the window are calculated according to the sampling time. These are then concatenated in a fixed order to form a working condition feature vector representing the stability of the current operating condition and its proximity to the safety boundary. Simultaneously, operational behavior data is statistically analyzed. The type, occurrence time, and associated interface of each interaction event are stored in the operation behavior log. The ratio of the occurrence frequency of each type of event to the window duration is calculated. Based on the time interval sequence between two adjacent operation events, the mean and variance are calculated. Based on the proportion of dwell time on different monitoring interfaces, an interface dwell time distribution is constructed, and the normalized entropy value of this distribution is calculated. The features obtained from the above calculations are concatenated to form an interaction load feature vector representing the frequency of operation, the rhythm of operation, and the degree of attention distraction.

4. The human-computer interaction method based on a large AI model according to claim 1, characterized in that: The specific steps for calculating the operating condition risk index and the interactive load index are as follows: The operating condition feature vector is input into the operating condition risk assessment model to calculate the operating condition risk index. This model is obtained through offline training using historical operating data and historical abnormal events. The current operating condition feature vector is converted into an operating condition risk index within the range of 0 to 1. The formula for calculating the operating condition risk index is as follows: ， in This represents the operating condition risk index for the current time window t, where i represents the index of the operating condition feature, and n represents the total number of operating condition features. This represents the weight parameter corresponding to the i-th working condition feature. This represents the value of the i-th operating condition characteristic within the current time window t. This represents the bias term of the operating condition risk assessment model; The interaction load index is calculated by inputting the interaction load feature vector into the interaction load assessment model. This model is trained through supervised learning based on historical operation logs and manually labeled load data. The interaction load feature vector is then converted into a scoring model for the interaction load index, which ranges from 0 to 1. The formula for calculating the interaction load index is as follows: ， in This represents the interactive load index for the current time window t, j represents the index of the interactive load feature, and m represents the total number of interactive load features. This represents the weight parameter corresponding to the j-th interactive load feature. This represents the value of the j-th interactive load feature within the current time window t. This represents the bias term in the interactive load assessment model.

5. The human-computer interaction method based on a large AI model according to claim 4, characterized in that: The specific steps for calculating the operating condition risk index and the interactive load index also include: The operating condition risk index and the interactive load index are weighted and nonlinearly combined to obtain the joint interactive risk index. The operating condition characteristics, interactive load characteristics, and the three indices are then encapsulated as real-time interactive context information.

6. The human-computer interaction method based on a large AI model according to claim 1, characterized in that: The specific steps for using a pre-trained language model for intent parsing and task abstraction are as follows: First, the voice commands are recognized to obtain natural language commands in text form, and contextual elements are extracted from real-time interactive context information. The natural language commands and contextual elements are uniformly encoded and input into a pre-trained language model. The language model outputs structured abstract results of operation and maintenance tasks, transforming the originally vague natural language intent into a machine-understandable task description.

7. The human-computer interaction method based on a large AI model according to claim 6, characterized in that: The specific steps for using a pre-trained language model for intent parsing and task abstraction also include: A voice acquisition terminal is set up in the central control room to collect the voice commands spoken by the operators. After echo cancellation, noise suppression, and endpoint detection, the continuous voice signal is divided into fixed-length time frames, and acoustic features are extracted from each frame to form a sequence of acoustic features arranged in chronological order. This acoustic feature sequence is then fed into a speech recognition sub-model within a pre-trained, large-scale pre-trained language model. The speech recognition sub-model preferably adopts an end-to-end structure integrating an encoder and decoder. The encoder maps the acoustic feature sequence into a high-dimensional hidden representation, while the decoder internally invokes the language modeling capabilities of the pre-trained language model to gradually output sequences of pinyin, Chinese characters, or sub-word units through conditional generation. During the decoding process, bundle search and domain vocabulary constraints are used to prioritize matching and correcting the names of proprietary equipment and process terms in the new energy plant, resulting in a text string corresponding to the voice content. After obtaining the preliminary recognition results, the text correction capabilities of the same pre-trained language model are invoked to correct words that do not conform to the operational context of the new energy plant, resulting in corrected natural language command text.

8. The human-computer interaction method based on a large AI model according to claim 1, characterized in that: The specific steps for hierarchical interactive orchestration of the abstract results of operation and maintenance tasks based on the risk level of the joint interaction risk index are as follows: The calculated joint interaction risk index is mapped to low-risk, medium-risk, and high-risk levels according to preset risk grading rules. The risk level serves as the control variable for interaction strategy selection. The abstract results of operation and maintenance tasks are then hierarchically arranged for interaction. At the low-risk level, multiple sub-tasks are merged to generate a sequence of interaction steps with fewer steps and more parameters, allowing for a recommended solution to be provided at once and completed with a single confirmation. At the medium-risk level, tasks are broken down into several logical sub-tasks, with each interaction step involving only one set of relevant parameters. After each step, operators are required to explicitly confirm before proceeding to the next step. At the high-risk level, complex tasks are further refined into smaller-granularity interaction steps, with each step adjusting only a single key parameter, and accompanied by explanations of reasons, risk warnings, and multiple confirmation mechanisms.

9. The human-computer interaction method based on a large AI model according to claim 1, characterized in that: The specific steps for parsing and protocol mapping of the parameter adjustment and confirmation operations performed by the operator in the interactive interface are as follows: For each interaction step obtained, based on the target device, parameters to be adjusted, and process constraints, a corresponding display template is selected from the preset interface template library, and the interaction step is associated with the corresponding template to form a candidate interface; for each interaction step, voice and text prompts are generated by combining a pre-trained language model with a fixed prompt template. The system performs structured analysis on the parameter inputs, option selections, and confirmations made by operators in the interactive interface to obtain the actual operational intent of each step. Based on the target object and parameter type in the interactive step, the operational intent is mapped to the standard control instructions supported by the underlying control system. During the mapping process, the single adjustment range and executable instruction types are automatically limited according to the current risk level. The control request after risk constraint processing is encapsulated into a data message conforming to the communication protocol and sent for execution. The execution results and changes in key parameters are written back to the real-time interactive context for joint interactive risk assessment and subsequent interactive adjustments in the next cycle.

10. A human-computer interaction system based on a large AI model, comprising a human-computer interaction method based on a large AI model according to any one of claims 1-9, characterized in that, Includes the following steps: The interactive scenario modeling and data acquisition module has a built-in process topology library, equipment object library, and interface mapping table. It divides the process flow into process nodes and represents field equipment as equipment objects. The interface mapping table records the correspondence between monitoring interface elements, process measurement points, and equipment objects to form an interactive scenario model. The acquisition configuration table identifies each process measurement point and various operation behavior acquisition items, sets the sampling period, and trigger conditions, and establishes a real-time acquisition channel in the control network that matches the interactive scenario model. The interaction context and joint interaction risk assessment module is connected to the real-time acquisition channel. It writes the acquired production process data into the working condition feature buffer and writes the acquired interface switching, mouse click, parameter confirmation and voice command logs into the interaction behavior buffer. The operating condition feature calculation unit and the interactive load feature calculation unit are respectively used to normalize, calculate the rate of change and volatility of production process data within a preset time window to generate operating condition feature vectors, and to calculate the frequency statistics, rhythm volatility and interface dwell distribution of operation behavior data to generate interactive load feature vectors. The operating condition feature vectors are input into the operating condition risk assessment model to obtain the operating condition risk index, and the interactive load feature vectors are input into the interactive load assessment model to obtain the interactive load index. The joint risk synthesis submodule performs weighted and nonlinear combination of the two to generate a joint interactive risk index and real-time interactive context information including operating condition features, interactive load features and the above indices. The Natural Language Intent Parsing and Task Abstraction Module, based on a large-scale pre-trained language model of deep learning, is connected to the speech recognition module and the interaction context and joint interaction risk assessment module. It receives the natural language instruction text obtained from speech recognition and real-time interaction context information, encodes the two into semantic vectors through the context encoding unit, and calls the large-scale pre-trained language model to output structured operation and maintenance task abstraction results. The hierarchical interaction orchestration module is connected to the natural language intent parsing and task abstraction module, as well as the interaction context and joint interaction risk assessment module. It has a built-in interaction strategy library and step template library. It reads the operation and maintenance task abstraction results and the joint interaction risk index. Based on the risk level of the joint interaction risk index, it selects the corresponding strategy from the interaction strategy library and generates an interaction step sequence according to the step template library. At the low risk level, it generates a coarse-grained step sequence containing multiple parameters to be operated. At the medium and high risk levels, it generates a fine-grained step sequence containing only a single parameter to be operated. The interactive presentation and control request generation module is connected to the hierarchical interactive orchestration module and the industrial control system interface. It has a built-in interface template library, prompt generation unit, and protocol adaptation unit. Under the control of the joint interactive risk index, it selects different interface templates for the interactive step sequence and limits the number of variables displayed on each interface to generate hierarchical interactive interfaces and corresponding voice and text prompts. It parses the parameter adjustment, option selection, and confirmation operations of the operator in the hierarchical interactive interface to form a structured operation intention. The control constraint unit constrains the available instruction types and parameter adjustment ranges according to the current risk level. The protocol adaptation unit converts the control request into a control request that conforms to the communication protocol, generating a control request that corresponds to the natural language intention and is constrained by the risk level.

Images