Application program native interaction middleware communication architecture and method based on AI large model

Abstract

The invention provides an application program native interactive middleware communication architecture and method based on an AI large model, and the architecture is used in a user APP (ERP (Enterprise Resource Planning)), and aims to understand a human natural language and convert the human natural language into an interactive communication protocol. And driving the native app to interact through the interactive communication protocol. A user dialogue language or text communication is input through a globally suspended AI dialogue window on an APP interface, then an AI large model carries out intention recognition on a natural language based on fed training big data (application software knowledge), the human natural language is understood, and contents to be operated by the user next are obtained. After the operation content is converted into a target protocol format, the protocol content is interpreted by a protocol interpreter, a corresponding native interaction event is obtained and output, and interaction is triggered. The problems that an existing ERP system is complex in operation, low in efficiency and lack of natural language interaction are solved, user experience and operation efficiency are improved, and high commercial value is achieved.

Description

Technical Field

[0001] This invention relates to the field of APP communication technology, and in particular to an application-native interaction middleware communication architecture based on an AI big model, as well as an application-native interaction middleware communication method, electronic device, and computer-readable storage medium based on an AI big model. Background Technology

[0002] In the daily use of Enterprise Resource Planning (ERP) systems (such as Xiaokuaibang ERP), users (such as sales representatives and financial personnel) face complex operational processes: manually navigating menus (such as "Sales → Orders → Create") and manually entering customer / product data (such as customer name, product quantity, and price), which is tedious and prone to errors. The interaction mode of traditional ERP systems suffers from the following pain points:

[0003] High operational complexity: Users need to memorize multiple menu paths, resulting in a high learning cost for beginners (usually requiring 1-2 weeks of training).

[0004] Inefficient: Manually entering data is time-consuming (an average of 3-5 minutes per sales order), and errors can easily lead to rework in subsequent processes;

[0005] Lack of natural language interaction: Existing systems rely heavily on click / input operations and cannot understand natural language requests such as "Help me open a sales order for three beef products";

[0006] Insufficient native interaction support: Some third-party natural language interaction solutions use WebView or H5 pages, which have high response latency (≥500ms) and cannot match the smooth experience of native apps. Summary of the Invention

[0007] To address the technical problems existing in the prior art, the present invention provides the following technical solution:

[0008] On the one hand, a communication architecture for application-native interaction middleware based on a large AI model is provided, including:

[0009] The APP input interface is connected to the AI ​​large model service to provide a globally floating input window, receive users' voice or text input, and preprocess the input content;

[0010] AI large model service is used to receive and understand natural language input from applications, identify user intent and extract structured slot information;

[0011] An interaction protocol engine, connected to the AI ​​large model service, is used to map the user intent and structured slot information into a standardized interaction protocol.

[0012] A protocol interpreter, connected to the interaction protocol engine, is used to parse the interaction protocol and find the corresponding application native controls and page paths based on the parsing results;

[0013] The trigger interaction module is connected to the protocol interpreter and is used to generate and execute native interaction events based on the parsing results to drive the application to complete the corresponding operations.

[0014] Preferably, the AI ​​large model service includes:

[0015] The pre-trained model, built on the Transformer architecture, is used to extract contextual semantic features from the input text.

[0016] The fine-tuning module uses training big data from a preset application domain to fine-tune the pre-trained model;

[0017] The intent classification layer, connected to the output of the pre-trained model, is used to classify user intents based on the extracted features: based on the token output by BERT ([CLS] token features), intent classification is achieved through a linear layer + Softmax (activation function, which converts the output of the linear layer into a probability distribution);

[0018] The slot-filling layer, connected to the output of the pre-trained model, is used to extract structured information corresponding to the user's intent from the input text: a Conditional Random Field (CRF) model is used to extract structured information from the user input.

[0019] Preferably, the formula for the pre-trained model to extract the contextual semantic features of the input text is as follows:

[0020] ,

[0021] ,

[0022] in:

[0023] Q (query matrix), K (key matrix), and V (value matrix): are obtained from the input embedding through linear transformation;

[0024] : Dimensions of the key (base version is 64);

[0025] h: Number of heads (base version is 12);

[0026] Single-head attention output;

[0027] Output projection matrix;

[0028] The operating mechanism is as follows: First, the input embedding is transformed into a query matrix Q, a key matrix K, and a value matrix V through a linear transformation; then, the dot product of the transposes of Q and K is calculated and scaled by the square root of the key dimension dk to avoid excessively large values; next, Softmax (an activation function that transforms the linear layer output into a probability distribution) is applied to the result to obtain attention weights (representing the importance of each token to other tokens); finally, the weights are multiplied by V to obtain the single-head attention output, which is then projected through the output matrix. Integration ( The result of multi-head attention (integration function) is used to obtain the final multi-head self-attention output ( ).

[0029] Preferably, the user intent probability distribution in the intent classification layer is calculated as follows:

[0030] ,

[0031] in:

[0032] : Weight matrix of the intended linear layer;

[0033] : The bias of the intended linear layer (obtained through experimental correction);

[0034] IntentProb: The probability distribution of the intent category;

[0035] This refers to outputting the semantic features of the entire sentence;

[0036] The operating mechanism is as follows: The embedding of the [CLS] token goes through a linear layer (weight matrix). and bias After transformation, the result is converted into a probability distribution using the Softmax function. Each probability value corresponds to the likelihood of an intent category, and the category with the highest probability is the user intent predicted by the model.

[0037] Preferably, the interaction protocol engine includes:

[0038] A protocol definition tool is used to define the standard format of the interaction protocol, the format of which includes at least the target control identifier, the interaction type, and the interaction content;

[0039] The protocol conversion engine is used to convert the user intent and structured slot information into an interaction protocol that conforms to the standard format, according to predefined mapping rules.

[0040] Preferably, the protocol interpreter includes:

[0041] A syntax analyzer is used to verify and parse the structure and fields of the interaction protocol;

[0042] The control mapping table stores the correspondence between control identifiers defined in the protocol and native control resource identifiers of the application.

[0043] The page path mapping table stores the correspondence between page paths defined in the protocol and the application's native page classes.

[0044] Preferably, the triggering interaction module includes:

[0045] The native interaction engine is used to call the native control interaction interface provided by the operating system.

[0046] An event scheduler is used to control the execution order of multiple native interactive events.

[0047] Preferably, the APP input interface includes:

[0048] A global floating window component is used to overlay and display on the front of the application interface and receive user input; a speech-to-text module is used to convert the user's speech input into text.

[0049] According to claim 1, the application-native interaction middleware communication architecture based on AI large model is characterized in that the AI ​​large model service, interaction protocol engine, protocol interpreter and trigger interaction module are all deployed locally on the user terminal device.

[0050] On the other hand, a native interaction middleware communication method for applications based on large AI models is provided, including:

[0051] S1: Receive natural language commands submitted by the user through the application's input interface;

[0052] S2: The natural language instructions are processed through the AI ​​large model service to identify user intent and extract structured slot information. Specifically, this includes: extracting contextual features of the input text through the Transformer encoder of the BERT model; inputting the CLSCLS identifier features in the contextual features into the intent classification layer to obtain the user intent category; inputting each token feature in the contextual features into the slot filling layer, and using a conditional random field model for sequence labeling to obtain structured slot information.

[0053] S3: The user intent and structured slot information are converted into a standardized interaction protocol through the interaction protocol engine;

[0054] S4: The interaction protocol is parsed by the protocol interpreter to determine the target application page, the target native control, and the type and content of the interaction to be executed;

[0055] S5: By triggering the interaction module, the corresponding native interaction event is generated and executed to drive the application to complete the operation corresponding to the user command.

[0056] On the other hand, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method described above.

[0057] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the above method.

[0058] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0059] (a) Reduce user learning costs and improve operational efficiency

[0060] Natural language interaction: Users do not need to remember menu paths and can complete operations with "one sentence" (such as "help me open a beef sales order for Zhang San"). The learning cost for beginners has been reduced from 1-2 weeks to less than 1 hour.

[0061] Automated workflow processing: The system automatically completes steps such as page redirection, data entry, and submission, reducing the processing time for a single sales order from 3-5 minutes to less than 30 seconds, improving efficiency by 80%.

[0062] (ii) High-accuracy natural language understanding

[0063] Targeted fine-tuning: The BERT model was fine-tuned using the Xiaokuaibang ERP operation manual and user dialogue data, achieving an intent recognition accuracy of ≥95% and a slot filling F1 value of ≥92%, which is far higher than that of general NLP models (such as the untuned BERT with an accuracy of about 85%).

[0064] Structured extraction: Extract structured information from user input using a slot-filling model (e.g., "Zhang San" → customer, "2 boxes of beef for 120 yuan" → product list) to ensure data accuracy (≤1% error rate).

[0065] (iii) Native interactive experience with fast response speed

[0066] Native event execution: Using native controls and events (such as View.performClick(), EditText.setText()), the response time is ≤50ms (much faster than WebView's ≥500ms);

[0067] Low-latency communication: Data is transmitted via HTTPS protocol, combined with TorchServe's high-concurrency inference (≥1000QPS), with end-to-end latency ≤300ms (based on 5G network), meeting the needs of real-time interaction.

[0068] (iv) Strong scalability and maintainability

[0069] Middleware architecture: The AI ​​large model service, interaction protocol engine, and protocol interpreter are all independent components, isolating the AI ​​model from the native APP (e.g., AI model updates do not require modification of the native code).

[0070] Dynamic configuration: The control mapping table and page path mapping table are stored in JSON format and support dynamic updates (such as downloading via the APP configuration file), so that they can adapt to changes in control positions without redeploying the APP;

[0071] Multi-platform support: The architecture supports multiple platforms including Android, iOS, and PC (through a unified interaction protocol), reducing cross-platform development costs (≤50%).

[0072] (v) A sound user feedback mechanism

[0073] Real-time result prompts: Display interaction results to users via Toast and UIAlertController (such as "Sales order has been successfully created") to increase user confidence.

[0074] Error handling: When protocol parsing fails or the control does not exist, a clear error message is displayed (such as "Control not found: add-sales-order-btn") to help users quickly locate the problem. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a schematic diagram of a communication architecture provided in an embodiment of the present invention;

[0077] Figure 2 This is a schematic diagram of a global floating window on the APP interface provided by an embodiment of the present invention;

[0078] Figure 3 This is a flowchart illustrating the steps of a communication method provided in an embodiment of the present invention;

[0079] Figure 4 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0080] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0081] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0082] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0083] In this embodiment of the invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0084] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0085] This solution proposes an application-native interaction middleware communication architecture based on an AI large model. It transforms user needs into native APP interaction events through Natural Language Understanding (NLU), achieving an efficient experience of "completing the operation with just one sentence".

[0086] I. Architecture Introduction

[0087] This architecture centers on the "XiaoKuaiBang ERP User APP," integrating five major components: the APP input interface, a lightweight AI large-scale model service (deployed locally on the APP), an interaction protocol engine, a protocol interpreter, and a trigger interaction module. The entire architecture executes locally on the APP, covering the entire process from user input to native interaction. The functions of each component are described below:

[0088] (1) APP input interface

[0089] Through the AI ​​dialogue window that floats globally on the APP interface, users can click on the window to input dialogue, communicate with the AI ​​in normal language or text, and transmit natural language to the AI ​​model.

[0090] (2) AI large model

[0091] Based on the training big data (which typically includes massive amounts of application software knowledge data, such as software operation manuals containing instructions like clicking the "Add Sales Order" button and entering customer product information (e.g., "Help me open a sales order; this is for Zhang San, who ordered 2 boxes of beef for 120 yuan and 50 jin of pig's trotters for 450 yuan"), after this customer product information is fed into the AI ​​model for training, the AI ​​understands the semantics of this application software knowledge data and converts it into the protocol "{orderType: salesOrder; customer: Zhang San;goodsList:[{"niurou": "2 boxes for 120 yuan"},{"zhuti": "50 jin for 450 yuan"}]}"), the AI ​​model performs intent recognition on the input human natural language, understands human natural language, and obtains the content the user wants to perform next.

[0092] (3) Interaction Protocol

[0093] The interaction protocol is used to convert the "content that the user will operate next" identified by the AI ​​model into the target protocol format.

[0094] The interaction protocol consists of the following: target control (page path, page elements (buttons, labels, diu, etc.)) + control interaction type (click type, input type, selection type, etc.) + interaction content (add, delete, change, query, confirm, etc.).

[0095] (4) Protocol interpreter

[0096] The AI ​​model transforms the "content that the user will operate next" into the corresponding target protocol according to the preset interaction protocol and submits it to the "protocol interpreter". The protocol interpreter interprets the protocol content, obtains the corresponding native interaction events and outputs them.

[0097] (5) Triggering interaction

[0098] After receiving a native interaction event, the system triggers an interaction, executes the native interaction event (such as automatically redirecting to create a new sales order and enter data), and completes the interaction.

[0099] The application implementation schemes for each component will be described in detail below with reference to the accompanying drawings.

[0100] II. APP Architecture Composition (For basic architecture, please refer to the architecture design of existing marketing APPs, such as Xiaokuaibang ERP)

[0101] As attached Figure 1 As shown, this architecture centers on the "XiaoKuaiBang ERP User APP," integrating five major components: the APP input interface, a lightweight AI large-scale model service (deployed locally on the APP), an interaction protocol engine, a protocol interpreter, and a trigger interaction module. The entire architecture executes locally on the APP, covering the entire process from user input to native interaction. The following is a detailed breakdown of each component of the APP architecture:

[0102] (a) APP Input Interface: Natural Language Interaction Entry Point

[0103] 1. Hardware components of the APP:

[0104] Terminal devices: smartphones (Android / iOS), tablets, PCs (Windows / macOS);

[0105] Input hardware: Touch screen (supports gesture operation), microphone (supports voice input), keyboard (supports text input);

[0106] Output hardware: Monitor (displays floating windows and input boxes), speaker (plays voice feedback).

[0107] The aforementioned hardware can directly access the hardware of existing smart terminals, referring to the operation of existing mobile phone apps (the same applies below).

[0108] 2. Software components:

[0109] Global floating window component: Implemented based on Android's WindowManager or iOS's UIWindow, it supports dragging, hiding / showing, and is always placed on top of the APP interface;

[0110] Input box component: Android's EditText or iOS's UITextField, supporting auto-completion and history (storing frequently used user requests);

[0111] Speech-to-Text (STT) module: Integrates Android's SpeechRecognizer (supports Mandarin and Cantonese) or iOS's SFSpeechRecognizer (supports multiple languages) to convert speech input into text;

[0112] Input preprocessing module: Removes redundant information from the text (such as interjections "um" and "oh"), and standardizes the format (such as converting "2 boxes of beef 120 pieces" to "2 boxes of beef 120 yuan").

[0113] (II) AI Large Model Service: Core of Natural Language Understanding

[0114] 1. Hardware components of the APP:

[0115] Compute Node: A lightweight AI inference engine integrated into the APP (supporting terminal GPU acceleration, such as Android's NNAPI and iOS's Core ML), used for local model inference;

[0116] Storage nodes: Local storage for the app (such as SQLite database, sandbox file system), storing lightweight model parameters and training data snapshots;

[0117] Network node: APP-side network component (supports 5G / Wi-Fi) to achieve low-latency communication (≤100ms) with the cloud model update server.

[0118] 2. Software components:

[0119] Pre-trained model: BERT-base (110 million parameters) is used as the basic model. Its pre-training process is systematically designed to build a powerful contextual semantic understanding capability. The specific steps are as follows: 1) Data collection and preprocessing: The pre-training data comes from a fusion corpus of Wikipedia (16GB of structured text) and BookCorpus (800M of multi-domain book text), covering more than 20 fields such as natural science, business, and law. The preprocessing stage includes text deduplication (based on the SimHash algorithm, the repetition rate is reduced to ≤0.5%), sentence segmentation (using spaCy word segmenter, accuracy ≥99%), Unicode normalization (to solve the problem of inconsistent character encoding), and construction of a text sequence of length 512 (truncated if the length exceeds the limit, and padded with zeros if the length is insufficient). 2). Pre-training task design: (1) Masked Language Modeling (MLM): Randomly select 15% of the tokens for masking operations, of which 80% are replaced with [MASK] token, 10% are replaced with random tokens (such as replacing "sales order" with "purchase order"), and 10% retain the original tokens; the model needs to predict the masked tokens, and the training objective is to minimize the cross-entropy loss. This task forces the model to learn context-dependent lexical semantics. (2) Next Sentence Prediction (NSP): Extract consecutive sentence pairs (positive samples, accounting for 50%) and random sentence pairs (negative samples, accounting for 50%) from the corpus. The model needs to determine whether the two sentences are consecutive contexts. The training objective is binary classification cross-entropy loss, which aims to enhance the model's sentence-level semantic understanding ability. 3) Model Structure and Parameter Configuration: BERT-base contains a 12-layer Transformer encoder, each layer consisting of a multi-head self-attention mechanism (12 heads, each with a dimension of 64) and a feedforward neural network (FFN, with a hidden layer dimension of 3072); the hidden layer dimension is 768, and the vocabulary size is 30522 (including common Chinese characters, English words, and special tokens); the GELU activation function is used (which reduces the gradient vanishing problem compared to ReLU), and layer normalization and residual connections are added to each layer. 4) Training Strategy and Optimizer: The Adam optimizer is used with parameters set to β1=0.9, β2=0.999, and ε=1e-6. The learning rate adopts a linear warm-up strategy (increasing from 0 to 2e-5 in the first 10,000 steps) followed by a linear decay strategy. The batch size is set to 256 (processing 256 sequences per step), and the total number of training steps is 1 million. Gradient clipping (max_grad_norm=1.0) is used to prevent gradient explosion, while dynamic masks (regenerating mask positions for each epoch) are used to improve the model's generalization ability.5) Hardware and Training Efficiency: Training is conducted using a Google TPU v2 cluster (16 TPU chips), with a single-step training time of approximately 0.3 seconds and a total training time of approximately 347 hours. Mixed Precision Training (FP16) reduces memory usage and improves training speed, increasing memory utilization by approximately 50%. 6) Pre-training Objective Function: The total loss is the weighted sum of the MLM loss and the NSP loss (weight ratio 1:1), expressed as: The definitions of each item are as follows: : Masked language modeling loss, expressed as M is the set of masked tokens, t is a single masked token, C is the context text, and P(t|C) is the probability that the model predicts for token t. The next line predicts the loss; the expression is: y represents the continuity label of a sentence pair (1 indicates continuity, 0 indicates discontinuity), P(y=1) is the probability that the model predicts a continuous sentence pair, and P(y=0) is the probability that it predicts a discontinuous sentence pair. 7). Pre-trained model validation: After pre-training, the model was validated on the GLUE benchmark set, achieving an average accuracy of 80.5%. Specifically, the Pearson correlation coefficient on the sentence similarity task (STS-B) was 0.85, and the accuracy on the natural language inference task (MNLI) was 84.6%, validating the model's strong contextual understanding capabilities and laying a solid foundation for subsequent fine-tuning in ERP scenarios.

[0120] Fine-tuning module: Use the training big data of Xiaokuaibang ERP (100,000 operation manual entries and 50,000 user dialogue records) to fine-tune BERT and optimize intent recognition and slot filling performance;

[0121] Intent classification layer: Based on the [CLS] token features output by BERT, intent classification (such as "create_sales_order" and "delete_sales_order") is achieved through a linear layer + Softmax.

[0122] Slot filling layer: The Conditional Random Field (CRF) model is used to extract structured information from user input (such as "Zhang San" → customer, "2 boxes of beef for 120 yuan" → product list).

[0123] Model deployment tools: Deploy quantized BERT models using lightweight deployment frameworks (such as TensorFlow Lite and PyTorch Mobile) to support local inference on the terminal;

[0124] (III) Interaction Protocol Engine: Conversion of Intent to Protocol

[0125] 1. Software components:

[0126] Protocol definition tool: The JSON Schema is used to define the standard format of the interaction protocol (see “Interaction Protocol Format” below) to ensure protocol consistency;

[0127] Protocol conversion engine: Based on the Python-based Jinja2 template engine, it maps the intent and slot information output by large AI models into a standardized interaction protocol;

[0128] Version management module: Uses Git to manage protocol versions and supports backward compatibility (e.g., the V1 version protocol can be parsed by the V2 version interpreter).

[0129] Interaction protocol format (JSON example):

[0130] {

[0131] "targetWidget": {

[0132] "pagePath": " / sales / order / new", / / Target page path (corresponding to the native Activity / Fragment)

[0133] "elementId": "add-sales-order-btn" / / Target control ID (corresponding to the resource ID of the native control)

[0134] },

[0135] "interactionType": "click", / / Interaction type (click / input / select, etc.)

[0136] "interactionContent": { / / Interaction content (structured data)

[0137] "customer": "Zhang San",

[0138] "goodsList": [

[0139] {"name": "beef", "quantity": "2 boxes", "price": "120 yuan"},

[0140] {"name": "pig's trotters", "quantity": "50 jin", "price": "450 yuan"} ]

[0142] }

[0143] }

[0144] (4) Protocol Interpreter: Mapping from Protocol to Native Events

[0145] 1. Software Components:

[0146] Syntax Analyzer: Uses ANTLR4 to parse the JSON structure of the interaction protocol and checks the integrity of fields (e.g., whether targetWidget contains pagePath);

[0147] Control Mapping Table: Stores the correspondence between elementId and native control ID (e.g., add-sales-order-btn → R.id.add_sales_order_btn), and supports dynamic updates (through the APP configuration file);

[0148] Page Path Mapping Table: Stores the correspondence between pagePath and native page classes (e.g., / sales / order / new → com.xiaokuaibang.sales.OrderNewActivity);

[0149] Exception Handling Module: When the protocol format is incorrect or the control does not exist, returns a standardized error message (e.g., {"code": 400, "msg": "Control not found: add-sales-order-btn"}).

[0150] (5) Trigger Interaction Module: Execution and Feedback of Native Events

[0151] 1. Hardware Components on the APP Side:

[0152] Terminal CPU: Snapdragon 8 Gen 3 (Android) / A17 Pro (iOS), used to execute native events;

[0153] Terminal Memory: 16GB LPDDR5 (Android) / 8GB LPDDR4X (iOS), used to cache page and control information;

[0154] Terminal Storage: UFS 4.0 (Android) / NVMe SSD (iOS), used to store configuration files such as the control mapping table.

[0155] 2. Software Components:

[0156] Native Interaction Engine: Android's View system (e.g., Button, EditText), iOS's UIKit (e.g., UIButton, UITextField), supports native events such as clicks, inputs, selections, etc.;

[0157] Event dispatchers: Android's Handler and iOS's Grand Central Dispatch (GCD) are used to schedule the execution order of interactive events (such as navigating to a page before inputting data).

[0158] Results feedback module: Toast for Android, UIAlertController for iOS, to display the interaction results to the user (such as "Sales order has been successfully created").

[0159] III. Detailed Introduction of Core Algorithm Model: Intent Recognition and Slot Filling Based on BERT

[0160] This solution uses a fine-tuned BERT model to achieve natural language understanding, focusing on solving two core problems: "intent recognition" (what the user wants to do) and "slot filling" (what object the user wants to interact with). The following are the technical details of the model:

[0161] (a) Model Input Representation

[0162] The input to BERT consists of three linearly added parts: token embedding (capturing the semantics of individual words), segment embedding (distinguishing between different sentences), and position embedding (capturing the order information of words), as shown in the following formula:

[0163] ,

[0164] : The (i)th token in the input text (e.g., “help”, “me”, “open”, “sales order”);

[0165] : segment identifier (0 for single-line input, 1 for multi-line input);

[0166] The position of the token (e.g., the 0th position is [CLS], and the 1st position is "bang").

[0167] The formula works by summing the word embedding, the sentence embedding, and the position embedding in the sequence for each input token element by element to obtain the final input representation, enabling the model to understand word meaning, sentence boundaries, and word order simultaneously.

[0168] Example: A user inputs "Help me open a beef sales order for Zhang San", and the tokenizer outputs the following after word segmentation:

[0169] [CLS], Help, Me, Open, Zhang San, of, Beef, Sales Order, [SEP]

[0170] The corresponding input embedding is the sum of the embeddings of each token (e.g., the embedding of "Zhang San" = the token embedding of "Zhang San" + 0 (segment) + the embedding of position 4).

[0171] (ii) Transformer Encoder: Contextual Feature Extraction

[0172] BERT employs a 6-layer Transformer encoder (base version), each layer containing a self-attention mechanism (capturing contextual dependencies between tokens) and a feedforward neural network (FFN, performing a non-linear transformation on the features of each token). The formula for the self-attention mechanism is as follows:

[0173] ,

[0174] ,

[0175] in:

[0176] (Q) (query matrix), (K) (key matrix), (V) (value matrix): obtained from the input embedding through linear transformation;

[0177] : Dimensions of the key (base version is 64);

[0178] (h): Number of heads (base version is 12);

[0179] Single-head attention output;

[0180] : Output projection matrix.

[0181] The mechanism operates as follows: First, the input embedding is transformed into a query matrix Q, a key matrix K, and a value matrix V through a linear transformation. Then, the dot product of the transposes of Q and K is calculated and scaled by the square root of the key dimension dk to avoid excessively large values. Next, Softmax (an activation function that transforms the linear layer output into a probability distribution) is applied to the result to obtain attention weights (representing the importance of each token to other tokens). Finally, the weights are multiplied by V to obtain the single-head attention output, which is then projected through the output matrix. Integration ( The result of multi-head attention (integration function) is used to obtain the final multi-head self-attention output ( ).

[0182] Through self-attention mechanisms, the model can capture contextual relationships such as between "Zhang San" and "customer", or between "beef" and "product", thereby improving the accuracy of feature representation.

[0183] (iii) Intent Classification Layer: Outputs user intent

[0184] The [CLS] token embedding (containing the semantic information of the entire sentence) output by the Transformer encoder is input into a linear layer + Softmax layer to obtain the probability distribution of the intent category:

[0185] ,

[0186] in:

[0187] : Weight matrix of the intended linear layer; : The bias of the intended linear layer (obtained through experimental correction);

[0188] IntentProb: The probability distribution of the intent category (e.g., the probability of "create_sales_order" is 0.95).

[0189] [CLS]token: A special token in the BERT model, referring in this context to the semantic features of the entire output sentence. (e.g., the overall intention of “help me open a beef sales order for three days”).

[0190] The formula works as follows: the embedding of the [CLS] token goes through a linear layer (weight matrix). and bias After transformation, the result is converted into a probability distribution using the Softmax function. Each probability value corresponds to the likelihood of an intent category, and the category with the highest probability is the user intent predicted by the model.

[0191] Training objective: Minimize cross-entropy loss.

[0192] ,

[0193] in:

[0194] (N): Number of intent categories (e.g., 10 categories);

[0195] : A one-hot vector of the true intent (e.g., corresponding to "create_sales_order") (and the rest are 0).

[0196] : The probability of the intention predicted by the model.

[0197] The purpose of minimizing the cross-entropy loss function is to guide the model to optimize the weight matrix and bias parameters by measuring the difference between the model's predicted intent probability distribution and the one-hot vector of the true intent, thereby improving the accuracy of intent recognition. The closer the predicted distribution is to the true distribution, the smaller the loss value; conversely, the closer it is to the true distribution, the larger the loss value. This loss function enables the model to learn a more accurate intent classification ability, ensuring that user requests (such as "help me open a sales order") are accurately identified as the corresponding business intent (such as "create_sales_order"), providing a reliable foundation for subsequent interaction protocol conversion and native operation triggering.

[0198] (iv) Slot filling layer: Extracting structured information

[0199] A CRF model (Conditional Random Field, which captures the dependencies between slot label sequences) is used to annotate the slots of each token embedding output by the Transformer encoder, as shown in the following formula:

[0200] ,

[0201] in:

[0202] (x): The token sequence of the input text;

[0203] (y): Slot label sequence (e.g., “B-customer”, “I-customer”, “O”);

[0204] (Z(x)): Normalization factor (partition function);

[0205] : Launch probability (token) Belongs to the label The probability of "Zhang San" belonging to "B-customer" is such as the probability of "Zhang San" belonging to "B-customer".

[0206] Transition probability (label) Transferred to The probability, such as the probability of a "B-customer" switching to an "I-customer".

[0207] The operating mechanism of this formula is as follows: The CRF model calculates the sum of probabilities of all possible slot tag sequences as a normalization factor Z(x), then calculates the probability of the true tag sequence y (the product of the emission probability and the transition probability), and finally obtains the conditional probability of the sequence. The emission probability represents the probability that token i belongs to tag j, and the transition probability represents the probability of transitioning from tag i to tag j. The combination of the two ensures the rationality of the tag sequence (e.g., B-customer can only be followed by I-customer or O, and cannot be directly followed by B-quantity).

[0208] Slot label definition (using IOB format):

[0209] : The beginning of an entity (e.g., "Zhang San" → B-customer);

[0210] : The middle of an entity (e.g., "2 boxes" → I-quantity);

[0211] (O): Non-entity (e.g., “of” → O).

[0212] Training objective: Minimize the negative log-likelihood loss.

[0213] ,

[0214] in It is the actual slot label sequence.

[0215] The mechanism of this loss is as follows: calculate the negative logarithm of the conditional probability P(y|x) of the true slot label sequence y, and minimize this loss to enable the model to learn to maximize the probability of generating the true label sequence, thereby improving the accuracy and rationality of slot filling.

[0216] (V) Model fine-tuning process (using the training big data of Xiaokuaibang ERP (100,000 operation manual entries and 50,000 user dialogue records) to fine-tune BERT, and combine the LoRA low-rank adaptation algorithm and AdamW optimizer to optimize intent recognition and slot filling performance. The specific process is as follows)

[0217] Training data preparation:

[0218] 1) Data Collection: Collect 50,000 records of operation manual data from Xiaokuaibang ERP (covering structured operation processes of 12 business modules such as sales, procurement, and inventory) and 50,000 records of user dialogue data (including unstructured scenario data such as real-time consultation and historical work orders).

[0219] 2) Data Labeling: The BIO labeling method (Begin-Inside-Outside) is used to label intents and slots. Intent labels include 20 categories such as "Create Sales Order" and "Query Inventory" (e.g., "Help me open a sales order for Zhang San's beef" is labeled as "Create Sales Order"). Slot labels include 15 categories such as "Customer Name" and "Product Name" (e.g., "Zhang San" is labeled as B-customer, and "Beef" is labeled as B-goodsName).

[0220] 3) Data partitioning: The data is divided into a training set (120,000 records), a validation set (15,000 records), and a test set (15,000 records) in an 8:1:1 ratio to ensure even data distribution.

[0221] Fine-tuning algorithm and parameter settings:

[0222] 1) Algorithm selection: The LoRA (Low-Rank Adaptation) algorithm is adopted. A low-rank matrix (rank r=8) is injected into the self-attention layer of BERT's Transformer. The pre-trained model parameters are frozen, and only the low-rank matrix is ​​trained to reduce the computational cost (the number of parameters is only 1.05% of the original model). The algorithm principle is as follows: the weight matrix W (d×d) of the self-attention layer is decomposed into: W1= W + BA, where B∈Rd×r and A∈Rr×d are low-rank matrices. Model adaptation is achieved by training B and A.

[0223] 2) Optimizer: Use the AdamW optimizer, the formula is:

[0224] ,

[0225] in For the model parameters at step t, The learning rate is (2e-5). For first-order moment estimation, For second-order moment estimation, For the smoothing term (1e-8). Weight decay (0.01);

[0226] 3) Training parameters: batch size=32 (adapting to GPU memory), epoch=10 (using early stopping method, patience=3), learning rate adopts linear warm-up (increasing from 0 to 2e-5 in the first 1000 steps) followed by cosine annealing strategy;

[0227] 3. Model training process:

[0228] 1) Input Construction: Convert the text sequence into BERT input format (including [CLS] and [SEP] tokens), with a fixed length of 128, and preserve contextual information;

[0229] 2) Loss Function: The joint loss function Loss = Loss_{intent} + Loss_{slot} is adopted, where Loss_{intent} is the cross-entropy loss for intent classification (such as predicting the probability of intents like "create a sales order"), and Loss_{slot} is the CRF (Conditional Random Field) loss for slot filling (ensuring the rationality of the slot label sequence, such as B-customer can only be followed by I-customer or O).

[0230] 3) Training strategy: Evaluate on the validation set after each training round, save the best model, and use gradient clipping (max_grad_norm=1.0) to prevent gradient explosion;

[0231] 4. Model Evaluation:

[0232] 1) Evaluation metrics: Intent classification accuracy = number of correctly classified samples / total number of samples; Slot filling F1 score = 2 * (precision * recall) / (precision + recall), where precision = number of correctly identified slots / total number of identified slots, and recall = number of correctly identified slots / total number of actual slots.

[0233] 2) Evaluation Results: On the validation set (15,000 entries), the intent classification accuracy was 95.2%, and the slot filling F1 score was 92.8%; on the test set (15,000 entries), the intent classification accuracy was 94.8%, and the slot filling F1 score was 92.3%.

[0234] 3) Inference performance: Single input inference latency ≤100ms (based on NVIDIA T4 GPU), supporting 100+ requests per second;

[0235] Model deployment optimization: The trained model is quantized into INT8 format (reducing the size by 75%) and deployed to the cloud inference service of Xiaokuaibang ERP. TensorRT is used to accelerate inference and dynamic batch processing is supported to improve concurrency.

[0236] Model iteration mechanism: Collect new user dialogue data (≥10,000 records) every quarter for incremental fine-tuning, update low-rank matrix parameters, and ensure continuous optimization of model performance.

[0237] IV. Data Communication and Interaction Process of Hardware and Software Systems

[0238] The data flow in this architecture begins with user input, proceeds through five stages: "APP input interface → AI large model service → interaction protocol engine → protocol interpreter → trigger interaction module," and finally executes the native interaction event. The following are the data communication details for each stage:

[0239] (I) Step 1: User inputs natural language (APP input interface → AI large model service)

[0240] 1. Input method: The user clicks the global floating window on the APP interface (see...). Figure 2 (See the diagram), select "Text Input" or "Voice Input";

[0241] o Text input: Directly enter "Help me open a beef sales order for Zhang San" in the input box;

[0242] o Voice input: Call the system's speech recognition API (Android's SpeechRecognizer, iOS's SFSpeechRecognizer) to convert speech into text.

[0243] 2. Data Transmission: The app sends a POST request to the AI ​​large model service via Retrofit (Android) or Alamofire (iOS). The request format is as follows:

[0244] URL: https: / / api.xiaokuaibang.com / ai / intent;

[0245] o Request header: Content-Type: application / json;

[0246] o Request body: {"text": "Help me open a beef sales order for Zhang San"}.

[0247] (II) Stage 2: AI Large Model Inference (AI Large Model Service Internal)

[0248] Input preprocessing: Use BERT's tokenizer to segment and encode the input text to obtain token IDs and attention masks;

[0249] Model inference: Input the token IDs and attention mask into the fine-tuned BERT model to obtain the embedding of the [CLS] token (for intent classification) and the embedding of each token (for slot filling).

[0250] Intent classification: Embed [CLS] into the input intent classification layer to obtain the intent probability distribution (e.g., the probability of "create_sales_order" is 0.95).

[0251] Slot filling: Embed each token into the CRF model to obtain the slot label sequence (e.g., "Zhang San" → B-customer, "beef" → B-goodsName).

[0252] Output: Encapsulate the intent and slot information into JSON (e.g., "Intent: create_sales_order, Slot: Customer=Zhang San, Product List=[{"name":"Beef","quantity":"2 boxes","price":"120 yuan"}]").

[0253] (III) Step 3: Protocol Conversion (AI Large Model Service → Interactive Protocol Engine)

[0254] Protocol Mapping: The interaction protocol engine uses Jinja2 templates to map the intent and slot information output by the large AI model into an interaction protocol that conforms to JSON Schema (see “Interaction Protocol Format” above).

[0255] Protocol validation: Use the jsonschema library to validate the protocol format and ensure that fields such as targetWidget and interactionType exist;

[0256] Data Return: The AI ​​large model service encapsulates the interaction protocol into a response (status code 200) and returns it to the APP. Example response body: {

[0257] "code": 0,

[0258] "msg": "success",

[0259] "data": {

[0260] "targetWidget": {

[0261] "pagePath": " / sales / order / new",

[0262] "elementId": "add-sales-order-btn"

[0263] },

[0264] "interactionType": "click",

[0265] "interactionContent": {

[0266] "customer": "Zhang San",

[0267] "goodsList": [{"name": "beef", "quantity": "2 boxes", "price": "120 yuan"}]

[0268] }

[0269] }

[0270] }

[0271] (iv) Step 4: Protocol Interpretation (APP → Protocol Interpreter)

[0272] Protocol parsing: The app sends the data field from the response body to the protocol interpreter, which uses Jackson (Android) or SwiftyJSON (iOS) to parse the JSON;

[0273] Page redirection check: Parse targetWidget.pagePath (e.g., " / sales / order / new") to determine if the current app is on the target page.

[0274] If not, the protocol interpreter generates a page navigation event (such as Android's Intent: Intent intent = new Intent(context, OrderNewActivity.class); context.startActivity(intent);).

[0275] Control lookup: Parse targetWidget.elementId (e.g., "add-sales-order-btn"), query the control mapping table (stored in the APP's assets directory), and obtain the corresponding native control ID (e.g., R.id.add_sales_order_btn).

[0276] Interaction type mapping: Parse the interactionType (e.g., "click") and map it to the native event (e.g., "click" → View.performClick()).

[0277] (V) Step 5: Triggering native interaction (protocol interpreter → trigger interaction module)

[0278] Event execution: The triggering interaction module executes native interaction events based on the output of the protocol interpreter.

[0279] Click event: Calling the performClick() method of the control (such as findView). The `.performClick()` method triggers the `onClick()` callback function of the button, opening the new sales order page.

[0280] Input event: On the new sales order page, locate the customer input box (elementId="customer-input"), call the setText("Zhang San") method to set the customer name;

[0281] Selection event: If the product list requires selection from a dropdown menu, call the Spinner.setSelection() method to select "Beef";

[0282] Results feedback: After the interaction is completed, the triggering module displays the results to the user via Toast (Android) or UIAlertController (iOS) (e.g., "Sales order has been successfully created, ID: 12345").

[0283] V. Examples

[0284] like Figure 3 As shown, the method of this solution consists of six steps: user input, natural language understanding, protocol conversion, protocol interpretation, native interaction, and result feedback. The implementation process of each step is as follows:

[0285] (a) Step 1: User inputs natural language

[0286] Floating window trigger: When the user clicks the global floating window (icon is "AI Assistant") in the lower right corner of the APP interface, the floating window pops up, displaying the input box and voice button (see below). Figure 1 );

[0287] Input method selection: Users can choose "text input" and enter "Help me open a sales order for beef from the store, 2 boxes for 120 yuan, 50 jin of pig trotters for 450 yuan" in the input box; or choose "voice input" and speak the same content.

[0288] Speech to text: If voice input is selected, the APP calls the system's speech recognition API (such as Android's SpeechRecognizer) to convert speech into text (accuracy ≥ 98%).

[0289] Input preprocessing: The APP removes redundant information (such as "um" and "oh") from the text and standardizes the format (such as converting "2 boxes of beef 120 pieces" to "2 boxes of beef 120 yuan").

[0290] The global floating window is implemented using WindowManager (Android), always placed on top of the APP interface, and supports dragging and hiding; speech-to-text conversion uses the client-side API to avoid network latency.

[0291] Users can directly input their needs through natural language without needing navigation menus, reducing the learning cost by 80%.

[0292] (2) Step 2: Transfer natural language to the large AI model

[0293] Data encapsulation: The APP encapsulates the preprocessed text into JSON (such as {"text": "Help me create a beef sales order for Zhang San, 2 boxes for 120 yuan, 50 catties of pig's trotters for 450 yuan"});

[0294] Network transmission: Send a POST request to the large AI model service (URL: https: / / api.xiaokuaihelp.com / ai / intent) through Retrofit (Android);

[0295] Request interception: The Nginx reverse proxy of the large AI model service intercepts the request and forwards it to the uWSGI server, and the uWSGI calls the processing function of the Flask application.

[0296] Use the HTTPS protocol to transmit data to ensure data security; The Nginx reverse proxy implements load balancing and supports high concurrency (≥1000 QPS).

[0297] The data transmission delay ≤ 200ms (based on the 5G network), meeting the real-time interaction requirements.

[0298] (3) Step 3: Intent recognition and protocol generation of the large AI model

[0299] Input encoding: The Flask application calls the tokenizer of BERT to tokenize the input text (such as "Help", "me", "create", "Zhang San's", "beef", "sales order", ",", "2", "boxes", "120", "yuan", ",", "pig's trotters", "50", "catties", "450", "yuan"), and encodes them into token IDs (such as [101, 2339, 2769, 3323, 3749, 4638, 2815, 3116, 117, 1922, 3851, 3342, 3711, 117, 3678, 2345, 3851, 3342, 3711, 102]);

[0300] Model inference: Input the token IDs and attention mask into the fine-tuned BERT model to obtain the embedding of the [CLS] token (768 dimensions) and the embeddings of each token (768 dimensions);

[0301] Intent classification: [CLS] is embedded into the input intent classification layer (linear layer + Softmax) to obtain the intent probability distribution (e.g., the probability of "create_sales_order" is 0.96 and the probability of "delete_sales_order" is 0.02), and the intent with the highest probability ("create_sales_order") is selected.

[0302] Slot filling: Input the embedding of each token into the CRF model to obtain the slot label sequence (e.g., "Zhang San" → B-customer, "Beef" → B-goodsName, "2 boxes" → I-quantity, "120 yuan" → I-price, "Pig's trotters" → B-goodsName, "50 jin" → I-quantity, "450 yuan" → I-price).

[0303] Protocol Generation: The interaction protocol engine maps intents and slot information to interaction protocols (see "Interaction Protocol Format" above), where:

[0304] targetWidget.pagePath: Mapped to " / sales / order / new" based on the intent "create_sales_order";

[0305] targetWidget.elementId: Mapped to “add-sales-order-btn” (New Sales Order button) based on intent;

[0306] interactionType: Mapped to "click" (button click) based on intent;

[0307] interactionContent: Populates the customer and product list based on slot information.

[0308] The BERT model captures contextual information through a self-attention mechanism, improving the accuracy of intent recognition and slot filling; the CRF model solves the dependency problem of slot labels (e.g., "B-customer" can only be followed by "I-customer" or "O").

[0309] The intent recognition accuracy is ≥95%, and the slot filling F1 value is ≥92%, ensuring the accuracy of protocol generation.

[0310] (iv) Step 4: Protocol interpreter parses the protocol

[0311] Protocol reception: When the APP receives the response from the AI ​​large model service (see “Response Body Example” above), it passes the data field to the protocol interpreter.

[0312] Format validation: The protocol interpreter uses the jsonschema library to validate the protocol format, ensuring that targetWidget contains pagePath and elementId, and that interactionType is an allowed value (click / input / select).

[0313] Page navigation: Parse targetWidget.pagePath(" / sales / order / new") to determine if the current app is on the OrderNewActivity page:

[0314] If not, the protocol interpreter generates an Intent (Android) and calls startActivity(intent) to jump to OrderNewActivity;

[0315] Control lookup: Parse targetWidget.elementId(“add-sales-order-btn”), query the control mapping table (assets / widget_mapping.json), and get the corresponding native control ID (R.id.add_sales_order_btn);

[0316] Interaction type mapping: Parse interactionType("click") and map it to the native event (View.performClick()).

[0317] The control mapping table is stored in JSON format and supports dynamic updates (such as downloading via the APP configuration file); page navigation uses native Intents to ensure navigation speed (≤100ms).

[0318] Protocol parsing accuracy is ≥99%, avoiding interaction failures caused by incorrect protocol format.

[0319] (v) Step 5: Trigger native interaction events

[0320] Event scheduling: The triggering interaction module uses Handler (Android) to schedule the execution order of interactive events (e.g., first jump to the page, then input data);

[0321] Click event execution: On the OrderNewActivity page, find the R.id.add_sales_order_btn control, call the performClick() method, and trigger the button's onClick() callback function (open the form page for creating a new sales order);

[0322] Input event execution: On the form page, locate the customer input box (elementId="customer-input"), call the setText("Zhang San") method to set the customer name; locate the product name input box (elementId="goods-name-input"), set it to "beef"; set the quantity input box (elementId="quantity-input") to "2 boxes"; set the price input box (elementId="price-input") to "120 yuan";

[0323] Multi-product processing: For "50 jin of pig's trotters for 450 yuan", trigger the interaction module to repeatedly execute the input event (set the product name to "pig's trotters", the quantity to "50 jin", and the price to "450 yuan");

[0324] Save event execution: Clicking the "Save" button (elementId="save-btn") calls the performClick() method to submit the sales order data (sent to the server via API).

[0325] Native events (such as performClick() and setText()) directly manipulate the underlying methods of the control, with a response time of ≤50ms (much faster than WebView); event scheduling ensures the sequential nature of the interaction (such as jumping before input).

[0326] The processing time for a single sales order has been reduced from 3-5 minutes to less than 30 seconds, improving efficiency by 80%.

[0327] (vi) Step 6: Feedback on Interactive Results

[0328] Results: After a sales order is successfully submitted, the server returns the sales order ID (e.g., "12345").

[0329] Feedback trigger: Triggers the interaction module to send a broadcast (Android) or notification (iOS) to inform the APP of the interaction result;

[0330] The results show that after receiving the broadcast, the APP interface displays a Toast ("Sales order has been successfully created, ID: 12345") and updates the sales order list (by calling the API to get the latest list and refreshing the RecyclerView).

[0331] Broadcast (Android) is used for communication between components to ensure real-time feedback of results; Toast is used for brief notifications without interfering with user operations.

[0332] Users can understand the results of their interactions in a timely manner, which increases their confidence in using the service.

[0333] Therefore, the application-native interaction middleware communication architecture proposed in this solution, based on an AI large-scale model, transforms user needs into native APP interaction events through natural language understanding, addressing the pain points of existing ERP systems such as "complex operation, low efficiency, and lack of natural language interaction." Its core advantages include:

[0334] High efficiency: The processing time for a single sales order is reduced to less than 30 seconds, improving efficiency by 80%;

[0335] Accuracy: Intent recognition accuracy ≥ 95%, slot filling F1 value ≥ 92%;

[0336] Native experience: Response time ≤300ms, much faster than WebView solution;

[0337] Scalability: Supports multiple platforms and dynamic configuration, reducing system maintenance costs.

[0338] This solution can be widely applied to enterprise-level applications such as ERP, CRM, and OA, improving user experience and operational efficiency, and has high commercial value.

[0339] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, electronic device 410 may include a first processor 2001.

[0340] Optionally, the electronic device 410 may also include a memory 2002 and a transceiver 2003.

[0341] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0342] The following is combined with Figure 4 A detailed description of each component of the electronic device 410 is provided below:

[0343] The first processor 2001 is the control center of the electronic device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0344] Optionally, the first processor 2001 can perform various functions of the electronic device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0345] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0346] In a specific implementation, as one example, the electronic device 410 may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0347] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0348] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be connected via the interface circuit of the electronic device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0349] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0350] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0351] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected via the interface circuit of the electronic device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0352] It should be noted that, Figure 4 The structure of the electronic device 410 shown does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0353] Furthermore, the technical effects of the electronic device 410 can be referred to the technical effects of the XXX method described in the above method embodiments, and will not be repeated here.

[0354] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0355] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0356] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0357] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0358] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0359] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0360] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A communication architecture for application-native interaction middleware based on a large AI model, characterized in that, include: The APP input interface is connected to the AI ​​large model service to provide a globally floating input window, receive users' voice or text input, and preprocess the input content; AI large model service is used to receive and understand natural language input from applications, identify user intent and extract structured slot information; An interaction protocol engine, connected to the AI ​​large model service, is used to map the user intent and structured slot information into a standardized interaction protocol. A protocol interpreter, connected to the interaction protocol engine, is used to parse the interaction protocol and find the corresponding application native controls and page paths based on the parsing results; The trigger interaction module is connected to the protocol interpreter and is used to generate and execute native interaction events based on the parsing results to drive the application to complete the corresponding operations.

2. The application-native interaction middleware communication architecture based on AI large model according to claim 1, characterized in that, The AI ​​large model service includes: The pre-trained model, built on the Transformer architecture, is used to extract contextual semantic features from the input text. The fine-tuning module uses training big data from a preset application domain to fine-tune the pre-trained model; The intent classification layer, connected to the output of the pre-trained model, is used to classify user intents based on the extracted features: based on the token output by BERT ([CLS] token features), intent classification is achieved through a linear layer + Softmax (activation function, which converts the output of the linear layer into a probability distribution); The slot-filling layer, connected to the output of the pre-trained model, is used to extract structured information corresponding to the user's intent from the input text: a Conditional Random Field (CRF) model is used to extract structured information from the user input.

3. The application-native interaction middleware communication architecture based on AI large model according to claim 2, characterized in that, The formula for the pre-trained model to extract the contextual semantic features of the input text is as follows: ， ， in: Q (query matrix), K (key matrix), and V (value matrix): are obtained from the input embedding through linear transformation; : Dimensions of the key (base version is 64); h: Number of heads (base version is 12); Single-head attention output; Output projection matrix; The operating mechanism is as follows: First, the input embedding is transformed into a query matrix Q, a key matrix K, and a value matrix V through a linear transformation; then, the dot product of the transposes of Q and K is calculated and scaled by the square root of the key dimension dk to avoid excessively large values; next, Softmax (an activation function that transforms the linear layer output into a probability distribution) is applied to the result to obtain attention weights (representing the importance of each token to other tokens); finally, the weights are multiplied by V to obtain the single-head attention output, which is then projected through the output matrix. Integration ( The result of multi-head attention (integration function) is used to obtain the final multi-head self-attention output ( ).

4. The application-native interaction middleware communication architecture based on AI large model according to claim 2, characterized in that, The formula for calculating the user intent probability distribution in the intent classification layer is as follows: ， in: : Weight matrix of the intended linear layer; : The bias of the intended linear layer (obtained through experimental correction); IntentProb: The probability distribution of the intent category; This refers to outputting the semantic features of the entire sentence; The operating mechanism is as follows: The embedding of the [CLS] token goes through a linear layer (weight matrix). and bias After transformation, the result is converted into a probability distribution using the Softmax function. Each probability value corresponds to the likelihood of an intent category, and the category with the highest probability is the user intent predicted by the model.

5. The application-native interaction middleware communication architecture based on AI large model according to claim 2, characterized in that, The interaction protocol engine includes: A protocol definition tool is used to define the standard format of the interaction protocol, the format of which includes at least the target control identifier, the interaction type, and the interaction content; The protocol conversion engine is used to convert the user intent and structured slot information into an interaction protocol that conforms to the standard format, according to predefined mapping rules.

6. The application-native interaction middleware communication architecture based on AI large model according to claim 1, characterized in that, The protocol interpreter includes: A syntax analyzer is used to verify and parse the structure and fields of the interaction protocol; The control mapping table stores the correspondence between control identifiers defined in the protocol and native control resource identifiers of the application. The page path mapping table stores the correspondence between page paths defined in the protocol and the application's native page classes.

7. The application-native interaction middleware communication architecture based on AI large model according to claim 1, characterized in that, The triggering interaction module includes: The native interaction engine is used to call the native control interaction interface provided by the operating system. An event scheduler is used to control the execution order of multiple native interactive events.

8. The application-native interaction middleware communication architecture based on AI large model according to claim 1, characterized in that, The APP input interface includes: A global floating window component is used to overlay and display on the front of the application interface and receive user input; a speech-to-text module is used to convert the user's speech input into text.

9. The application-native interaction middleware communication architecture based on AI large model according to claim 1, characterized in that, The AI ​​large model service, interaction protocol engine, protocol interpreter, and trigger interaction module are all deployed locally on the user terminal device.

10. A communication method for application-native interaction middleware based on a large AI model, implemented based on the communication architecture of any one of claims 1-8, characterized in that, include: S1: Receive natural language commands submitted by the user through the application's input interface; S2: The natural language instructions are processed through the AI ​​large model service to identify user intent and extract structured slot information. Specifically, this includes: extracting contextual features of the input text through the Transformer encoder of the BERT model; inputting the CLSCLS identifier features in the contextual features into the intent classification layer to obtain the user intent category; inputting each token feature in the contextual features into the slot filling layer, and using a conditional random field model for sequence labeling to obtain structured slot information. S3: The user intent and structured slot information are converted into a standardized interaction protocol through the interaction protocol engine; S4: The interaction protocol is parsed by the protocol interpreter to determine the target application page, the target native control, and the type and content of the interaction to be executed; S5: By triggering the interaction module, the corresponding native interaction event is generated and executed to drive the application to complete the operation corresponding to the user command.

Images