Edge gateway north-south communication configuration method, device and medium

Abstract

The invention discloses an edge gateway north-south communication configuration method and device and a medium, and relates to the technical field of edge gateways. The method comprises the following steps: in response to a natural language instruction of a user, analyzing the natural language instruction by adopting a natural language intention analysis model to obtain a user intention; the user intention is converted into a configuration command, and north-south communication of the edge gateway is configured based on the configuration command; and based on a configuration result obtained by completing the configuration command, generating a multi-modal feedback result by using a multi-modal generation model, and displaying the multi-modal feedback result to the user. Therefore, the north-south communication of the edge gateway can be configured.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of edge gateway, and in particular to an edge gateway north-south communication configuration method, device and medium. BACKGROUND

[0002] In the field of Internet of Things, edge gateway plays an important role, which is embodied in reducing data transmission delay, reducing core network bandwidth occupation, and improving data security and privacy protection. However, with the increase of on-site points and the improvement of data reporting requirements, it is difficult to meet the project delivery requirements by relying on manual configuration of edge gateway. In particular, the traditional edge gateway configuration relies on fixed instructions or manual parameter input, which is difficult to adapt to complex north-south communication scenarios. SUMMARY

[0003] The edge gateway north-south communication configuration method, device and medium provided by the embodiments of the application solve the technical problem of how to configure the edge gateway north-south communication in a complex scenario.

[0004] In a first aspect, the embodiments of the application provide an edge gateway north-south communication configuration method, which comprises: in response to a natural language instruction of a user, analyzing the natural language instruction by using a natural language intention analysis model to obtain a user intention, wherein the natural language instruction is used to instruct to configure the north-south communication of the edge gateway; converting the user intention into a configuration command, and configuring the north-south communication of the edge gateway based on the configuration command; based on a configuration result obtained by completing the configuration command, generating a multi-modal feedback result by using a multi-modal generation model; and showing the multi-modal feedback result to the user.

[0005] In a second aspect, the embodiments of the application further provide an edge gateway north-south communication configuration device, which comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the edge gateway north-south communication configuration method of the first aspect.

[0006] In a third aspect, the embodiments of the application further provide a computer storage medium storing computer executable instructions, and the computer executable instructions are executed to implement the edge gateway north-south communication configuration method of the first aspect.

[0007] The edge gateway north-south communication configuration method, device and medium provided by the embodiments of the application have the following beneficial effects: In the embodiment of the present application, in response to the natural language instruction of the user, the natural language intention analysis model can be used to analyze the user's intention, and then the corresponding configuration operation is performed according to the analyzed user's intention. Finally, the feedback result is generated through the multi-modal generation model and is shown to the user. In this way, the south-north communication of the edge gateway can be configured without relying on fixed instructions and without manually inputting parameters by the user. Moreover, even in a complex scenario, the south-north communication configuration can also be performed. Moreover, multi-modal feedback results can also be generated and shown to the user for confirmation, so as to improve the understanding efficiency of the user and the accuracy of the configuration. In this way, not only is the efficiency high, but also the satisfaction of the user can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0008] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate embodiments of the present application and the description thereof, which serve to explain the present application and do not constitute improper limitations on the present application. In the drawings: Figure 1 A flowchart of an edge gateway south-north communication configuration method provided by an embodiment of the present application; Figure 2 An internal structure schematic diagram of an edge gateway south-north communication configuration device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0009] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in conjunction with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0010] Natural language intention analysis based on context awareness is a technology that accurately judges the real intention of user speech by combining user dialogue history, scene information and language context. The core is to break through the limitations of single sentence understanding and make the analysis more consistent with human communication logic. Its advantages are that it can handle ambiguous or omitted expressions and improve the continuity of interaction; it can also adapt to multi-round dialogue and meet the complex needs of intelligent assistants, customer service and other scenarios. Its disadvantage is that the processing capability for ambiguity scenarios is still limited and is easily disturbed by irrelevant information.

[0011] Natural language generation technology is a technology of converting structured data, logical rules or semantic representation into human understandable natural language. The core is to make the machine generate coherent, grammatically correct and contextually appropriate text or speech like a human being, covering various scenarios from simple summary to complex dialogue. Its advantages are obvious. It can efficiently process massive data to generate reports and support intelligent dialogue gateway interaction scenarios, greatly reducing the cost of manual creation. However, it is difficult to combine generated text with images. For example, when the gateway configuration fails, it is difficult to show users more intuitively with images.

[0012] Knowledge distillation technology is a core method of model compression through a "teacher-student" model architecture. The core is to let the "student model" with simple structure and small number of parameters learn the generalization knowledge of the "teacher model" with complex and excellent performance. Finally, while greatly reducing the model size and computing cost, the accuracy is close to that of the teacher model. Its advantages are that, compared with directly training a small model, distillation can make the student model inherit the decision logic of the teacher model, avoid a significant drop in accuracy, and deploy the compressed model on mobile phones, Internet of Things devices and other resource-constrained scenarios to meet real-time inference requirements. At the same time, it can reduce redundant parameters of the model, and reduce storage and energy consumption costs. However, it also has obvious shortcomings. First, it depends on a high-quality teacher model. If the teacher model has poor performance, the distillation effect will be directly limited. Second, the distillation process requires additional training of the teacher model, increasing the upfront computing cost. Third, in some complex tasks, the student model may not be able to fully learn the deep knowledge of the teacher model, resulting in a loss of accuracy.

[0013] In practical applications, the south-north communication of the edge gateway is a core concept for describing the data flow and functional positioning. Southbound communication refers to the data interaction between the edge gateway and the underlying terminal devices (such as sensors, PLCs, industrial controllers, etc.), with the direction being from the edge gateway downward. It is responsible for collecting and transmitting device data from the physical world to the gateway, and is the bridge between the "perception layer" and the "network layer" of the Internet of Things. Northbound communication refers to the data interaction between the edge gateway and the cloud platform or the upper-level system, with the direction being from the edge gateway upward. It is responsible for transmitting processed data to the cloud to support remote decision-making and global management.

[0014] The embodiment of the present application provides a south-north communication configuration scheme of an edge gateway. The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0015] Figure 1 A method flowchart is provided for the embodiments of the present application. As shown in Figure 1 The method provided by the embodiments of the present application specifically includes the following steps: Step 101, in response to a natural language instruction of a user, a natural language intent analysis model is used to analyze the natural language instruction to obtain the user's intent.

[0016] The natural language instruction is used to instruct to configure the north-south communication of the edge gateway.

[0017] In the embodiments of the present application, in the case of needing to configure the north-south communication, the user does not need to use fixed instructions or manually input parameters, but can configure through a natural language instruction. The natural language instruction can be presented in the form of text or language, both of which essentially use natural language to convey instructions. For example, the user can say "configure north-south communication, allow sensors to send data to the cloud server", or input the above natural language instruction on the interface. In the case of voice natural language instruction, the voice needs to be converted into text and then input into the natural language intent analysis model. In the case of text natural language instruction, it can be directly input into the natural language intent analysis model. The natural language intent analysis model can analyze the natural language instruction and understand the user's intent. In this way, through the natural language intent analysis model, the real intent behind the user's instruction can be accurately understood, and the service that meets the user's needs can be provided.

[0018] Step 102, converting the user intent into a configuration command, and configuring the north-south communication of the edge gateway based on the configuration command.

[0019] In actual application, the above user intent can include an intent and a slot. The intent refers to the goal or core requirement expressed by the user through the natural language instruction. The slot, also known as a semantic slot or a parameter slot, is specific information that needs to be extracted from the user instruction in order to fully realize the user intent, and is a further refinement and supplement to the intent. The intent can be regarded as an action, and the slot is various parameters required for the action. For example, the natural language instruction is "configure north-south communication, allow all edge devices to send data to the cloud server", and the intent can be identified as "set north communication" through the natural language intent analysis model. The slot includes source=all edge devices, target=cloud server, and protocol=default.

[0020] In the embodiments of the present application, after obtaining the user intent, it can be converted into a configuration command. In actual application, the determination process from the structured intent to the configuration command is a multi-stage, rule-based conversion process. First, it is necessary to ensure that the intent is reasonable and feasible, and to check whether the parsed slots are complete and whether they meet the specifications of network configuration. If there are missing or non-compliant requirements, the user may need to be requested to supplement the information or make default inferences. Then, the configuration policy is generated, and according to the intent and slot information, the configuration items and specific configuration values that need to be modified are determined. This may include firewall rules, routing tables, access control lists, etc. Finally, according to the configuration policy, the corresponding command template is selected, and the slot values are filled into the template to generate a specific configuration command. After generating the configuration command, the generated configuration command can be sent to the execution module of the edge gateway to execute the configuration change. In this way, the user does not need to follow a specific format or use professional terms to configure the north-south communication of the edge gateway. Even for complex north-south communication scenarios, the user does not need to handle it, and the corresponding configuration command that meets the actual edge gateway situation can be generated according to the user intent, which can improve the response capability to complex scenarios, avoid errors caused by user configuration, and improve the accuracy of configuration.

[0021] Step 103, based on the configuration result obtained by completing the configuration command, a multi-modal generation model is used to generate a multi-modal feedback result.

[0022] In actual application, in the process of feedback with the user, it is difficult to combine images using natural language generation technology to generate text. In the feedback process, pure text description may not be enough to show complex network changes, and combining images (such as topology graphs, flow graphs) can enhance user understanding. Therefore, in the embodiments of the present application, multi-modal collaborative generation can be introduced to improve natural language generation technology, that is, a multi-modal generation model is used to output text and images at the same time, which can improve the quality of feedback. For example, the natural language instruction is "allow device A to access cloud server port B", the multi-modal generation model can integrate information, such as obtaining the configuration result, successfully adding the firewall rule, allowing device A to access the B port of the cloud server. The network topology can also be obtained, and it is known that device A belongs to the internal network and the cloud server is external. Then, the text is generated, "the firewall rule has been successfully configured: allow device A to access the B port of the cloud server." A network topology graph can also be generated, including: internal network nodes (including device A), external cloud server nodes, firewall nodes, and green arrows can be set from device A to port B to represent allowed traffic. Finally, multi-modal alignment can be performed to check that the IP and port mentioned in the text are reflected in the image and the rules are consistent.

[0023] In the embodiments of the present application, the abstract configuration result (such as success, failure, specific rule change) can be converted into a multi-modal feedback result combining accurate textual description and intuitive visual graphics by using a multi-modal generation model, which can significantly improve the understanding efficiency and accuracy of the user.

[0024] Step 104, presenting the multi-modal feedback result to the user.

[0025] After generating the multi-modal feedback result, the multi-modal feedback result can be displayed on the user interface at the same time, that is, the text and image of this edge gateway north-south communication configuration are displayed at the same time.

[0026] In the embodiments of the present application, through the multi-modal generation process, the originally obscure command line configuration result can be converted into an intuitive report that can be easily and accurately understood by any administrator, and is displayed for the user to confirm, which can greatly reduce the risk of misreading and operation. Moreover, the user can learn the specific edge gateway north-south communication configuration, and if the configuration result does not meet the user's expectation, the user can modify it, and if the configuration is wrong, the user can check it and then output natural language instructions again to complete the configuration again.

[0027] In the embodiments of the present application, in response to the natural language instruction of the user, the natural language intention analysis model can be used to analyze the user's intention, and then the corresponding configuration operation is performed according to the analyzed user's intention. Finally, the feedback result is generated by the multi-modal generation model and presented to the user. In this way, the south-north communication of the edge gateway can be configured without relying on fixed instructions or manually inputting parameters by the user. Moreover, even in a complex scenario, the south-north communication can still be configured. The multi-modal feedback result can also be generated and presented to the user for confirmation, which can improve the understanding efficiency and accuracy of the user. In this way, not only is the efficiency high, but also the user's satisfaction can be improved.

[0028] In actual application, the natural language intention analysis model can analyze the natural language instruction based on context awareness, and at the same time, in order to solve the ambiguity problem in natural language, disambiguation can be performed. The context awareness and disambiguation mechanism can reduce errors and improve the accuracy of the configuration.

[0029] In one possible implementation, the natural language intention analysis model uses a tree recursive neural network to model the context part, uses a tree recursive neural network with an attention mechanism to encode the document part, and uses a fine-grained preposition type system to determine the meaning of the preposition in the natural language instruction.

[0030] In practical applications, in terms of context representation, tree recursive neural networks are introduced to exploit the linguistic structure of text to compose sentence representations. By computing the representations of two candidate children, the network will compute the representation of its parent and a plausibility score for the new node. This process is then repeated recursively to obtain a semantic representation of a phrase. In the above embodiment, a tree recursive neural network with long short-term memory functionality can be selected to encode context information in order to obtain better representations of text units by considering the semantic composition of small elements. When connected with a tree recursive neural network, the tree recursive neural network with long short-term memory functionality can also retain sequential information over time. The tree recursive neural network can be considered as a superset of the tree recursive neural network with long short-term memory functionality with a linear structure. The tree recursive neural network with long short-term memory functionality requires previous context to capture a phrase and will typically capture too much of the last word in the final vector. Unlike the tree recursive neural network with long short-term memory functionality, the tree recursive neural network can capture larger structures similar to linguistic structures from small pieces. The goal of the tree recursive neural network model is to strengthen the high-level representation of tree nodes by recursively tracking the propagation on different branches in the tree. In the tree recursive neural network structure, if the color of a word is deeper than that of another word, it means that the word is more relevant to the context vector than the other word.

[0031] In one possible implementation, the determining the meaning of the preposition in the natural language instruction using the fine-grained preposition type system comprises: generating at least one candidate preposition for the preposition, wherein the candidate preposition is used to indicate a meaning of the preposition; obtaining a previous score of the candidate preposition by using a frequency dictionary calculated in advance; converting context keywords in the natural language instruction and the candidate preposition into word vectors and decomposing into multiple sub-components; calculating cosine similarity scores of the context keywords and the candidate preposition at multiple granularities; selecting the candidate preposition with the highest combined score of the previous score and the cosine similarity scores as the meaning of the preposition.

[0032] In practical applications, fine-grained types can provide structured information for prepositions. Fine-grained types refer to a method of dividing and classifying targets more meticulously in classification tasks, which introduces more sub-classes or more meticulous classification labels in classification tasks, making the classification more accurate and detailed. For example, for animal classification tasks, the target can be further divided into feline, canine, bird, etc. Such fine-grained classification can more accurately capture the characteristics and attributes of the target and provide more rich semantic information. By introducing more fine-grained types, the accuracy and discrimination ability of the classification model can be improved, so that it can distinguish targets with similar characteristics.

[0033] In the above embodiment, some candidate prepositions need to be generated, which have previous scores that have been scored in previous evaluations, and the scores are obtained by using a frequency dictionary calculated in advance. Then, the semantic similarity between the candidate prepositions can be calculated from multiple fine-grained. The feature F(w, e) of the formula represents multiple granularities of semantic similarity.

[0034] F(w, e) = [cos(w c , e c ), cos(w c , e d ), cos(w c , e n ), cos(w d , e c ), cos(w d , e d ), cos(w d , e n ), cos(w n , e c ), cos(w n , e d ), cos(w n , e n )] The above formula calculates the cosine similarity of different components between two word vectors w and e. Specifically, w and e are decomposed into their sub-components, respectively labeled as w c , w d , w n and e c , e d , e n . The final score is a combination of the previous score and the semantic similarity score. Then, the candidate preposition with the highest final score is selected as the result.

[0035] In practical applications, the multi-modal generative model can simulate the interdependence between text and vision. The multi-modal generative model can generate corresponding images (such as network topology diagrams, traffic diagrams, etc.) while generating natural language text feedback to enhance the user's understanding of the configuration results.

[0036] In the training process of the multi-modal generative model described above, a specific multi-modal data set can be used.

[0037] In one possible implementation, in the training process of the multi-modal generative model, the data set used includes a set of dialogue scenes wherein, is a sequence of dialogue turns in the text, is a sequence of images, each dialogue turn and the corresponding image are paired, is the visual content in the context, n represents the length of the dialogue segment, and each dialogue turn is a sequence composed of tokens, wherein , represents the length of the text turn .

[0038] In practical applications, a large multi-modal data set can include millions of dialogue turns, each associated with a specific visual context, and dialogue sentences occur in these images, rather than using a single image for the entire dialogue set. This better simulates multi-modal dialogue generation in the real world. Regarding this part of the dialogue set, the average number of turns per dialogue set in the above data set is 14, which is the most in existing open-domain multi-modal dialogue data sets. The above data set D consists of many dialogue scenes, each represented by (X, Z), that is, each scene contains both text dialogue information X and image information Z, which are interrelated and together constitute a complete dialogue context. X is a set, and the elements to represent individual dialogue turns, which are arranged in a certain order to form a sequence. For example, in a complete dialogue, there can be multiple rounds of speaking, and each round of speaking is a dialogue turn, may be the first round of speaking, is the second round of speaking, and so on.

[0039] In the above dialogue scene, each dialogue turn is also a sequence composed of a series of tokens (which can be understood as words, morphemes, etc. Language units). For example, may be a sentence, which is composed of multiple words, denotes the first word in the jth dialogue turn, denotes the second word, and so on, denotes the total number of words in this dialogue turn, i.e., the length of the text in this turn. Z is also a set, and the elements in the set represent a series of images arranged in order, forming an image sequence. In the dataset, each dialogue turn has a corresponding image , which matches each other. This image can visually present the specific context of the dialogue turn . For example, if the dialogue turn is discussing a park scene, the corresponding image may be a real photo of the park, helping people better understand the background of the dialogue. n refers to the number of dialogue turns in a dialogue scene, i.e., the number of elements in the set X, and also corresponds to the number of images in the image sequence Z. For example, if a dialogue scene contains 5 rounds of dialogue, then n = 5, there are to 5 dialogue turns in the set X, and to 5 corresponding images in the set Z.

[0040] In a possible implementation, the multi-modal generation model is used to learn the probability of generating the next dialogue sentence given the previous dialogue sentence and the visual environment, and to learn the probability of predicting the visual features of the dialogue sentence, so that the generated dialogue sentence is related to a specific visual environment.

[0041] In practical applications, in the task of multi-modal dialogue generation, the goal is to generate the next dialogue turn in an autoregressive manner according to the previous dialogue turn , the previous visual context and the current visual context .

[0042]

[0043] The formula describes the probability calculation logic of generating the next dialogue turn , and the core is to decompose the joint probability into the sum of conditional probabilities of word-by-word generation through autoregressive decomposition. Each step of generation needs to refer to the historical dialogue text, historical and current visual information, which can realize text-visual fusion. At the same time, through word-by-word generation, the complexity is reduced, which conforms to the sequential characteristics of natural language generation, and can ensure that the generated Both consistent with the historical dialogue logic and matching the semantic of the current visual scene.

[0044] The above is the probability of generating the next dialogue sentence. The multi-modal generation model also needs to predict the probability of the visual feature of the dialogue sentence. The current visual feature can be predicted based on the complete history and the current dialogue text, historical visual information , so as to strengthen the understanding of the visual context through the reverse constraint model.

[0045] In a possible implementation, before the response to the user's natural language instruction, the natural language instruction is parsed by using a natural language intent parsing model to obtain the user's intent, the method further comprises: obtain an expert teacher model and a temporary teacher model; based on the expert teacher model and the temporary teacher model, fuse global and local information by using a double-teacher distillation architecture, and collaboratively train a student model, wherein the student model is a lightweight natural language intent parsing model; deploy the natural language intent parsing model on a terminal device configured with the edge gateway.

[0046] In actual application, the edge gateway has limited resources (such as limited computing power and memory), and a lightweight natural language intent parsing model is needed to run locally to avoid relying on cloud services, reduce latency and privacy risks. In actual application, model compression and optimization can be achieved through knowledge distillation technology, but traditional knowledge distillation has defects. Therefore, in the above embodiment, a multi-level feature distillation loss can be used to improve the knowledge distillation technology, a double-teacher distillation architecture can be used to fuse global and local information, and a student model can be collaboratively trained to obtain a lightweight natural language intent parsing model. In this way, not only the model is lightweight, but also the generalization ability of the student model is improved through knowledge transfer, which becomes a key bridge connecting high precision and high efficiency. The temporary teacher model needs to be randomly initialized and trained from scratch together with the student model; and the expert teacher model is a pre-trained model.

[0047] In a possible implementation, based on the expert teacher model and the temporary teacher model, the double-teacher distillation architecture is used to fuse global and local information, and the student model is collaboratively trained, comprising: process the input sample into a global view and a local view through data enhancement; input the global view into the temporary teacher model and the student model respectively to obtain temporary teacher probability prediction and student probability prediction, and calculate the classification loss of the temporary teacher model and the student model; inputting the local view into the expert teacher model to obtain an expert teacher probability prediction; based on decomposing the expert teacher probability prediction and the student probability prediction into probabilities of source categories and other categories, calculating a knowledge distillation loss; based on the classification loss of the temporary teacher model, the classification loss of the student model, and the knowledge distillation loss, using a loss function to calculate a total loss; updating the student model and the temporary teacher model using the total loss; repeating the above steps until the training is completed.

[0048] In practical applications, this double-teacher distillation method enables the student model to learn both global features and fine-grained knowledge from the expert teacher through collaborative training, ultimately obtaining a lightweight yet powerful model that is well suited for edge computing scenarios.

[0049] In past pioneering knowledge distillation research, knowledge was transferred from a higher-performing teacher to a compact student. However, in this case, the diversity of knowledge and the ability of a single teacher are limited. However, different teachers can provide their own unique and useful knowledge. Therefore, a double-teacher distillation architecture can be used to fuse global and local information to collaboratively train a student model, which consists of two parts. Specifically, for a training sample x, the original input data can be processed into a global overall input image (global view) x g and a local input image (local view) x l through data augmentation.

[0050] The global view is input into a randomly initialized temporary teacher model T1 and a student model S. T1 and S extract global information features, respectively, and calculate the standard cross-entropy between the probability prediction p i output by the temporary teacher model and the student model softmax and the true label y i as their classification loss L CE . The calculation process is as follows:

[0051]

[0052] where N is the number of categories, y i is the true label, is the prediction probability value of the temporary teacher model, is the prediction probability value of the student model.

[0053] The temporary teacher model T1 and the student model S are both trained from scratch, in this way, the teacher network will constantly guide the student model, not just with the final trained response pairs. However, the temporary teacher trained from scratch may provide incorrect guidance to the student model at the early stage of training. In this case, it is difficult to train a deeper student with knowledge distillation without introducing local feature constraints. Therefore, an expert teacher model can be introduced to process the local input image, which can guide the student to focus on richer source class knowledge and other class knowledge to provide more comprehensive supervised prediction information.

[0054] In the above embodiment, an expert teacher T2 can be introduced to co-train the student model S with the temporary teacher T1. First, an expert teacher model is pre-trained, and then the local view x l is input into the expert teacher model and feature extraction is performed to obtain the local output response g T2 . Then the local response g T2 of the expert teacher model and the global response g S of the student model processing the global view are divided into two parts: binary prediction for the source class and all other classes; and multi-class prediction for each other class. Specifically, in order to distinguish the prediction of the source class and the other classes, the local response g T2 of the expert teacher model and the global response g S of the student model can be feature separated to separate the source class knowledge and the other class knowledge. In this part, the classic KD uses the KL divergence as the loss function, which is expressed as:

[0055] Wherein, represents the similarity between the binary probability of the source class predicted by the expert teacher model and the student model, represents the similarity between the prediction probability of the other classes predicted by the expert teacher model and the student model, represents the source class probability prediction of the teacher network.

[0056] In the above possible implementation, the formula of the loss function L is as follows:

[0057] Wherein, is used to represent the classification loss of the temporary teacher model, is used to represent the classification loss of the student model, is used to represent the mean square error loss between the temporary teacher model and the student model, is used to represent the KL divergence loss between the student model and the expert teacher model about the source class in the knowledge distillation loss, for representing the KL divergence loss between the student model and the expert teacher model in the knowledge distillation loss about other classes, and a, b, g are hyperparameters for balancing different loss terms.

[0058] L CE Partly guarantee the teacher and student to train independently with the original classification loss, and give full play to the importance of global features. In L MSF In the mean square error loss, since the difference is squared, the penalty for abnormal values is greater, and the model is more robust to abnormal values. The effectiveness of the mean square error loss can be proved in the experimental part. and Combining the global feature output from the student model with the local feature output from the expert teacher model makes the source class probability value and the probability value of other classes output by the teacher and student as similar as possible. In this way, the global and local features are fused, the student model can be trained according to the classification task loss, and can also be supervised and predicted globally by the temporary teacher and the source class and other class knowledge provided by the expert teacher at each step of the training process. This multi-loss combination strategy enables the student model to maintain lightweight characteristics while having expert-level understanding ability in complex scenarios such as edge gateway configuration.

[0059] In practical applications, the multi-modal generative model can also be distilled through the above double-teacher distillation architecture, and then deployed on the terminal where the edge gateway is located. The specific steps are the same as above, and are not repeated here.

[0060] The above is the method embodiment of the present application. Based on the same inventive concept, the present application also provides a device, the structure of which is shown in Figure 2 .

[0061] Figure 2 An edge gateway north-south communication configuration device internal structure schematic diagram is provided in the embodiments of the present application. As shown in Figure 2 , the device comprises: at least one processor 201; and a memory 202 in communication connection with the at least one processor; wherein the memory 202 stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor 201 to enable the at least one processor 201 to perform the above edge gateway north-south communication configuration method.

[0062] Some embodiments of the present application provide a non-volatile computer storage medium corresponding to Figure 1 , which stores computer executable instructions, and the computer executable instructions are set to execute the above edge gateway north-south communication configuration method.

[0063] The various embodiments in the present application are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the IoT device and medium embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0064] The system and medium provided by the embodiments of the present application are one-to-one corresponding to the method, and therefore, the system and medium also have similar beneficial technical effects to the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be described here.

[0065] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0066] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device implemented in the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the function specified in one block or multiple blocks.

[0067] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the function specified in one block or multiple blocks.

[0068] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1

[0069] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0070] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.

[0071] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition provided herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0072] It is also important to note that the terms "comprises", "comprising", or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0073] ​​The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A method for configuring north-south communication in an edge gateway, characterized in that, include: In response to a user's natural language command, the natural language command is parsed using a natural language intent parsing model to obtain the user's intent, wherein the natural language command is used to instruct the configuration of north-south communication of the edge gateway; The user intent is converted into a configuration command, and the north-south communication of the edge gateway is configured based on the configuration command; Based on the configuration result obtained from completing the configuration command, a multimodal feedback result is generated using a multimodal generation model; The multimodal feedback results are displayed to the user.

2. The method according to claim 1, characterized in that, The natural language intent parsing model uses a tree recurrent neural network to model the context, a tree recurrent neural network with an attention mechanism to encode the document, and a fine-grained preposition typing system to determine the meaning of prepositions in the natural language instructions.

3. The method according to claim 2, characterized in that, The process of determining the meaning of prepositions in the natural language instruction using a fine-grained preposition type system includes: Generate at least one candidate preposition for the preposition, wherein the candidate preposition is used to indicate a meaning of the preposition; The previous scores of the candidate prepositions are obtained by using a pre-calculated frequency dictionary; The context keywords and candidate prepositions in the natural language instructions are converted into word vectors and decomposed into multiple sub-components; Calculate the cosine similarity scores of the context keywords and the candidate prepositions at multiple granularities; The candidate preposition with the highest combined score of previous score and cosine similarity score is selected as the meaning of the preposition.

4. The method according to claim 1, characterized in that, The training process of the multimodal generative model uses a dataset that includes a set of dialogue scenarios. ,in, It is a sequence of dialogue rounds in the text. It is a sequence of images, for each dialogue round. and corresponding images Being paired yes The contextual visual content, where n represents the length of the dialogue segment, and each dialogue round It is a sequence composed of tags, where , Indicates text round The length.

5. The method according to claim 4, characterized in that, The multimodal generation model is used to generate dialogue statements that are relevant to a specific visual environment by learning the probability of generating the next dialogue statement given a previous dialogue statement and a visual environment, as well as learning the probability of predicting the visual features in which the dialogue statement occurs.

6. The method according to claim 1, characterized in that, Before parsing the natural language instruction using a natural language intent parsing model in response to the user's natural language instruction to obtain the user's intent, the method further includes: Obtain expert teacher models and temporary teacher models; Based on the expert teacher model and the temporary teacher model, a dual-teacher distillation architecture is used to fuse global and local information and collaboratively train the student model, wherein the student model is a lightweight natural language intent parsing model. The natural language intent parsing model is deployed on the terminal device configured with the edge gateway.

7. The method according to claim 6, characterized in that, The method of collaboratively training a student model based on the expert teacher model and the temporary teacher model, using a dual-teacher distillation architecture to fuse global and local information, includes: Data augmentation is used to process input samples into global and local views; The global view is input into the temporary teacher model and the student model respectively to obtain the temporary teacher probability prediction and the student probability prediction, and the classification loss of the temporary teacher model and the student model is calculated. Input the local view into the expert teacher model to obtain the expert teacher probability prediction; The knowledge distillation loss is calculated by decomposing the expert teacher probability prediction and the student probability prediction into probabilities of the source category and other categories. Based on the classification loss of the temporary teacher model, the classification loss of the student model, and the knowledge distillation loss, the total loss is calculated using a loss function. Update the student model and the temporary teacher model using the total loss; Repeat the above steps until training is complete.

8. The method according to claim 7, characterized in that, The formula for the loss function L is as follows: in, Used to represent the classification loss of the temporary teacher model, Used to represent the classification loss of the student model, This is used to represent the mean squared error loss between the temporary teacher model and the student model. This is used to represent the KL divergence loss between the student model and the expert teacher model with respect to the source category in the knowledge distillation loss. The KL divergence loss between the student model and the expert teacher model with respect to other categories is used to represent the knowledge distillation loss. α, β, and γ are hyperparameters used to balance different loss terms.

9. A north-south communication configuration device for an edge gateway, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform an edge gateway north-south communication configuration method as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement the edge gateway north-south communication configuration method as described in any one of claims 1-8.

Images