Method and device for generating feedback information, electronic equipment and storage medium

Abstract

The invention provides a method and device for generating feedback information, electronic equipment and a storage medium. The method comprises the following steps: acquiring query information input by a user, wherein the query information is multi-modal information; according to the query information and historical information, a prediction result is obtained, and the prediction result comprises topics related to the query information; obtaining a program function corresponding to the prediction result according to the prediction result and a function mapping table; executing the program function to convert the prediction result into input data of a target model; and generating feedback information corresponding to the query information based on the input data and the target model. By predicting the query information and executing the program function corresponding to the prediction result, the more accurate input data of the target model is obtained, and the feedback information generated based on the input data better meets the requirements of a user.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for generating feedback information. Background Art

[0002] Artificial intelligence technology is dedicated to developing computer systems that can simulate human intelligence. Natural language processing technology, as an important branch of artificial intelligence, focuses on the interaction between computers and human language. Intelligent question-answering systems are an important manifestation of natural language processing technology in practical applications, and have gone through a development process from simple rule matching to complex machine learning algorithms.

[0003] In related technologies, intelligent question-answering systems are unable to understand complex user intentions, and thus are unable to accurately generate feedback information that meets user needs. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose a method, device, electronic device and storage medium for generating feedback information, so as to solve or partially solve the above-mentioned problems.

[0005] Based on the above purpose, in a first aspect, the present application provides a method for generating feedback information, comprising:

[0006] Acquire query information input by a user, wherein the query information is multimodal information;

[0007] Obtaining a prediction result according to the query information and the historical information, wherein the prediction result includes a topic related to the query information;

[0008] According to the prediction result and the function mapping table, a program function corresponding to the prediction result is obtained;

[0009] Executing the program function to convert the prediction result into input data of the target model;

[0010] Based on the input data and the target model, feedback information corresponding to the query information is generated.

[0011] In a second aspect of the present application, a device for generating feedback information is also provided, including:

[0012] A multimodal input processing module, configured to obtain query information input by a user, wherein the query information is multimodal information;

[0013] a dynamic topic prediction module, configured to obtain a prediction result according to the query information and the historical information, wherein the prediction result includes a topic related to the query information;

[0014] A program function mapping module, configured to obtain a program function corresponding to the prediction result according to the prediction result and a function mapping table;

[0015] An execution module, configured to execute the program function to convert the prediction result into input data of a target model;

[0016] The reasoning module is configured to generate feedback information corresponding to the query information based on the input data and the target model.

[0017] In a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0018] In a fourth aspect of the present application, a non-transitory computer-readable storage medium is further provided, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.

[0019] From the above, it can be seen that the present application provides a method, device, electronic device and storage medium for generating feedback information. The method includes: obtaining query information input by a user, the query information being multimodal information; obtaining a prediction result based on the query information and historical information, the prediction result including topics related to the query information; obtaining a program function corresponding to the prediction result based on the prediction result and a function mapping table; executing the program function to convert the prediction result into input data for a target model; and generating feedback information corresponding to the query information based on the input data and the target model. By predicting the query information and executing the program function corresponding to the prediction result, more accurate input data for the target model is obtained, and the feedback information generated based on the input data better meets the needs of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart diagram of an exemplary rule-based method is shown.

[0022] Figure 2 A flowchart of an exemplary machine learning-based method is shown.

[0023] Figure 3 A schematic diagram of an exemplary information feedback system according to an embodiment of the present application is shown.

[0024] Figure 4 A schematic diagram of an exemplary multimodal input processing module according to an embodiment of the present application is shown.

[0025] Figure 5 A schematic diagram of a dynamic topic prediction module according to an embodiment of the present application is shown.

[0026] Figure 6 A schematic diagram of an exemplary program function mapping module according to an embodiment of the present application is shown.

[0027] Figure 7 A schematic diagram of a guarantee module of an exemplary execution module according to an embodiment of the present application is shown.

[0028] Figure 8 A schematic diagram of an exemplary reasoning module according to an embodiment of the present application is shown.

[0029] Fig. 9 A flow chart of an exemplary method for generating feedback information according to an embodiment of the present application is shown.

[0030] Fig.10 A schematic diagram of an exemplary electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0032] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0033] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0034] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0037] Artificial intelligence technology is dedicated to developing computer systems that can simulate human intelligence. Natural language processing technology, as an important branch of artificial intelligence, focuses on the interaction between computers and human languages.

[0038] The intelligent question-answering system is an important manifestation of natural language processing technology in practical applications. It has gone through a development process from simple rule matching to complex machine learning algorithms. Rule matching method: Match user questions through a preset rule base to obtain answers, but it is difficult to cover all questions and the rule base needs to be constantly updated. Machine learning algorithm: Use a large amount of training data to train the model to automatically answer questions. Although it can cover more questions, it requires a large amount of data and computing resources.

[0039] Existing intelligent question-answering systems are mainly divided into two categories: rule-based methods: rely on predefined rules and templates, and when users ask questions, the system generates answers based on them. Machine learning-based methods: use a large amount of training data and machine learning algorithms to automatically learn language features to generate more natural and accurate answers.

[0040] Figure 1 A flow chart of an exemplary rule-based method 100 is shown.

[0041] The rule-based approach is to match the questions input by the user through a series of preset rules and logic, and return the corresponding answers. Figure 1As shown, method 100 generally includes the following steps:

[0042] 1. Question classification: Classify the questions input by users into preset categories. For example, questions can be classified into simple factual queries, complex factual queries, opinion questions, etc.

[0043] 2. Rule matching: Match the corresponding rules according to the question classification. For example, for simple factual queries, you can match the database query rules to query information from the database; for opinion questions, you can match the preset answer template and use the preset template to generate answers; for complex question queries, you can match the complex query rules and execute the complex query logic; for other queries, you can match other rules and execute other rule logic.

[0044] 3. Generate answers: Generate corresponding answers according to the matching rules. For example, for database query rules, you can query relevant information from the database and generate answers; for answer templates, you can directly use the preset templates.

[0045] Figure 2 A flowchart of an exemplary machine learning-based method 200 is shown.

[0046] The machine learning-based approach is to enable the system to automatically understand and answer user questions by training the machine learning model. Figure 2 As shown, method 200 generally includes the following steps:

[0047] 1. Data collection and preprocessing: Collect a large amount of question-answering data and preprocess the data, including word segmentation, stop word removal, stemming, etc.

[0048] 2. Model selection and training: Select a suitable machine learning model, for example, a deep learning model (such as the Transformer model), a traditional machine learning model (such as SVM (Support Vector Machine), decision tree, etc.), and use the preprocessed data for training.

[0049] 3. Model evaluation and optimization: Use the validation set or test set to evaluate the performance of the model, and optimize the model based on the evaluation results, for example, adjusting model parameters, changing the model architecture, etc.

[0050] 4. Deployment and application: Deploy the trained model to the mobile question-answering system, and generate answers through model reasoning based on the questions entered by the user.

[0051] However, the methods 100 and 200 in the related art have the following problems:

[0052] 1. Poor flexibility: rules need to be constantly updated to deal with new types of problems, and the adaptability is poor;

[0053] 2. Limited coverage: It is difficult to cover all possible types of questions, and it is easy to have questions that cannot be answered;

[0054] 3. Insufficient generalization ability: A large amount of data and computing resources are required for model training and optimization. It is not possible to answer or generate content well for scenarios that are not reflected in the training data.

[0055] In order to at least solve the above problems, the present application provides a method, device, electronic device and storage medium for generating feedback information. The method includes: obtaining query information input by a user, the query information being multimodal information; obtaining a prediction result based on the query information and historical information, the prediction result including topics related to the query information; obtaining a program function corresponding to the prediction result based on the prediction result and a function mapping table; executing the program function to convert the prediction result into input data of a target model; and generating feedback information corresponding to the query information based on the input data and the target model. By predicting the query information and executing the program function corresponding to the prediction result, more accurate input data of the target model is obtained, and the feedback information generated based on the input data better meets the needs of the user.

[0056] Figure 3 A schematic diagram of an exemplary information feedback system 300 according to an embodiment of the present application is shown. The system 300 may be a mobile question-answering system for the power industry oriented to a large power model.

[0057] like Figure 3 As shown, the system 300 may include a multimodal input processing module 302 , a dynamic topic prediction module 304 , a program function mapping module 306 , an execution module 308 , and a reasoning module 310 .

[0058] The multimodal input processing module 302 can receive various query information input by the user, including natural language queries and possible multimodal information such as images and audio, and pass the basic information to other modules after processing.

[0059] The dynamic topic prediction module 304 can predict the topic trend based on the information transmitted by the multimodal input processing module 302 by using a reinforcement learning algorithm combined with data in a small database, and pass the prediction result to the program function mapping module 306 .

[0060] The program function mapping module 306 may map the action to a specific program function for execution through a function mapping table and an execution engine according to the prediction result of the dynamic topic prediction module 304 .

[0061] During the execution of the program function, it can be managed and optimized by the execution module 308.

[0062] The reasoning module 310 can receive information from the previous modules, and through a series of processes such as training and fine-tuning of the large language model, construction and optimization of the intelligent question-answering system, introduction of logical reasoning and causal reasoning, implementation of personalized services, provision of services beyond expectations, and implementation of personified services, it can ultimately provide users with accurate, personalized, and natural answers.

[0063] It should be noted that if Figure 3 As shown, when the content of the query information input by the user received by the multimodal input processing module 302 is accurate enough, the multimodal input processing module 302 can directly input the query information to the reasoning module 310 to generate feedback information, without the need for other modules to process the query information.

[0064] Figure 4 A schematic diagram of an exemplary multimodal input processing module 302 according to an embodiment of the present application is shown.

[0065] like Figure 4 As shown, in some embodiments, the multimodal input processing module 302 may further include a natural language processing unit 3022 and a multimodal input processing unit 3024.

[0066] The multimodal input processing module 302 can obtain the query information input by the user. Since the query information input by the user can be multimodal information, the query information can be converted into text (for example, audio to text, image to text) through the natural language processing unit 3022. The multimodal input processing unit 3024 receives the information transmitted by the natural language processing unit 3022 as well as multimodal information such as action lists, topic stack status, and chat history records. For image, audio and other data, after increasing the sample diversity through operations such as rotation, scaling, and adding noise, they are processed using big data analysis technology to extract relevant information and upper and lower layer information, and integrated with the information transmitted by the natural language processing unit 3022, and finally the comprehensive information obtained after processing is passed to the dynamic topic prediction module 304.

[0067] The natural language processing unit 3022 can perform semantic analysis through deep learning technology. The specific process is: first, perform lexical analysis on the input query information (natural language) to decompose the sentence into words; then perform syntactic analysis to determine the grammatical relationship between words; finally, extract keywords and intentions, and pass the processed information to the multimodal input processing unit 3024.

[0068] In terms of sample processing, the natural language processing unit 3022 can use data enhancement techniques, such as synonym replacement, random insertion, random deletion, etc. to increase the diversity of corpus samples. For example, for a sentence "The stability of the power system is very important", "stability" can be replaced with "robustness" through synonym replacement to generate new samples. In terms of algorithm options, adaptive learning rate algorithms, such as Adagrad algorithm, Adadelta algorithm, etc., can be used to automatically adjust the learning rate according to the gradient information during the training process to improve the model convergence speed and performance.

[0069] The multimodal input processing unit 3024 can use big data analysis technology to process action lists, topic stack status, chat history, etc., extract relevant information and context, and cooperate with the natural language processing unit to provide comprehensive input information for the system. For the multimodal input processing unit 3024, the number and diversity of samples are increased by rotating, scaling, adding noise, and other operations to image, audio and other data. In terms of algorithms, fusion algorithms such as early fusion and late fusion can be used to select appropriate fusion methods according to the characteristics of different modal data to improve the effect of multimodal information processing.

[0070] The multimodal input processing module 302 can comprehensively process multiple types of input information, and process natural language and other related information respectively through different technical units to achieve effective integration of multiple inputs. This integration improves the system's ability to understand user input, enabling it to more comprehensively obtain user intent and related context information, thereby better adapting to the language preferences and styles of different users, and providing a more sufficient basis for accurately answering questions.

[0071] Figure 5 A schematic diagram of the dynamic topic prediction module 304 according to an embodiment of the present application is shown.

[0072] like Figure 5 As shown, in some embodiments, the dynamic topic prediction module 304 may include a prediction model 3042 and a small database 3044 .

[0073] The prediction model 3042 can receive the processed query information (i.e., comprehensive information) and historical information (for example, the historical information can be the historical query information of the user obtained from the database, etc.) from the multimodal input processing module 302, and use the reinforcement learning algorithm for analysis. The deep Q network (DQN) and its variants, such as Double DQN, DuelingDQN, etc., can be used to improve the performance of the prediction model by improving the algorithm structure and updating strategy. At the same time, techniques such as experience replay and target network can be used to reduce the variance in the training process and improve the stability and convergence speed of the model. In the training process of the model, techniques such as experience replay and target network are used to reduce the variance. At the same time, the curriculum learning (CL) strategy is used to train the prediction model from simple dialogue scenes and topics. As the training progresses, the complexity of the dialogue and the diversity of topics are gradually increased, so that the model can better adapt to various situations in practical applications. The prediction model 3042 predicts the topic trend by analyzing the information (for example, if the query information input by the user is to query the weather, the predicted topic trend may be to query the weather in the city where the user is currently located), and transmits the prediction result to the program function mapping module 306. The prediction model 3042 can also interact with the small database 3044 to obtain common topics and related information as reference.

[0074] The small database 3044 can store common topics and related information, provide data support for the prediction model 3042, and assist the prediction model to more accurately predict the trend of topics.

[0075] The dynamic topic prediction module 304 performs topic prediction by combining the database information with the reinforcement learning algorithm, and can dynamically predict the direction of the topic according to the actual situation of the conversation, improve the coherence and consistency of the conversation, avoid users asking the same question multiple times, and improve the user experience.

[0076] Figure 6 FIG. 3 is a schematic diagram of an exemplary program function mapping module 306 according to an embodiment of the present application.

[0077] like Figure 6 As shown, the program function mapping module 306 may include a function mapping table 3062 and an execution module 3064 .

[0078] The function mapping table 3062 can store each action and its corresponding program function. After receiving the prediction result from the dynamic topic prediction module 304, the corresponding program function can be searched in the function mapping table 3062. To improve the search efficiency, a data structure such as a hash table can be used to optimize the search operation.

[0079] The program function mapping module 306 can find the corresponding program function (such as query table structure, merge data and other operations) according to the intention and keywords in the prediction results: the function mapping table 3062 can be compared to a dictionary, which records various "actions" (which can be understood as user needs or system tasks) and corresponding "program functions" (that is, the specific codes used in computer programs to complete these tasks). When the system receives a task (the information received by the previous module), it will look in its dictionary to see which program function should be executed for this task. In order to quickly find the corresponding program function, this module uses a "hash table".

[0080] The execution module 3064 can execute program functions, use parallel computing technology (such as multi-threading, multi-process) to execute multiple program functions at the same time, and pass the execution results as the system's response to user interaction to subsequent modules for further processing. At the same time, during the operation of the system 300, the mapping relationship in the function mapping table is dynamically adjusted according to user feedback and actual execution conditions to optimize the calling order and parameter settings of the program functions.

[0081] After finding the corresponding program function, the execution module 3064 executes multiple program functions simultaneously through concurrent technology, and performs further processing of the obtained structured data (converted intent and keyword information).

[0082] It should be noted that what is outputted by the multimodal input processing module 302 is a high-dimensional vector after the keywords and intentions in the query information are converted into text, and then the high-dimensional vector is also obtained through the dynamic topic prediction module 304, and finally the structured data is obtained through the program function mapping module 306. This data is the structured data after executing the query, traversal, merging, and dependency (it is the basic data subsequently input into the target model).

[0083] In addition, the execution module 3064 can dynamically adjust the mapping relationship in the "function mapping table" according to user feedback and actual execution conditions, and optimize the calling order and parameter settings of the program functions. This dynamic adjustment of the mapping relationship is self-optimized through continuous feedback from user input, thereby improving the accuracy of the system 300.

[0084] The program function mapping module 306 establishes a mapping relationship between actions and program functions, so that the system 300 can respond to complex user interactions in a programmable and extensible manner. This mapping method improves the flexibility and extensibility of the system, facilitates system maintenance and updates, and can provide solutions more accurately and timely to meet the diverse needs of users.

[0085] Figure 7 A schematic diagram of a guarantee module 700 of an exemplary execution module 3064 according to an embodiment of the present application is shown.

[0086] like Figure 7 As shown, in some embodiments, the guarantee module 700 may include a decorator function 702 and an action manager 704 .

[0087] The guarantee module 700 can be a guarantee module of the execution module 3064, and its work is mainly divided into two parts: the decorator function 702 and the action manager 704. Considering the robustness of the entire dialogue system and improving the user experience, the decorator function does not change the program function of the execution module 3064, but adds a log and exception handling mechanism to achieve that the failure of a program function does not affect the flow of the entire dialogue process. Considering the situation of multiple rounds of dialogue, the purpose of the action manager 704 is to ensure that these functions can be executed in the correct order and control the number of tasks executed at the same time to avoid system downtime.

[0088] The decorator function 702 can decorate multiple program functions. Multiple decorator functions can be applied in sequence using the decorator chain method, such as applying a logging decorator first and then applying an exception handling decorator, etc., to add additional functions such as logging, exception handling, etc. to enhance function behavior.

[0089] The action manager 704 is responsible for scheduling and managing multiple decorated program functions to ensure the execution order and concurrency control of actions. It dynamically adjusts the parameters and application order of the decorator function according to the system's operation log and error report, optimizes the system's behavior, and ensures the correct execution of system actions.

[0090] The decorator function 702 can be implemented using Python decorator technology, which can enhance the function behavior without modifying the function code itself. This technology improves the flexibility and maintainability of the system, facilitates the development and upgrading of the system, and ensures the stability and reliability of the system when executing actions.

[0091] Figure 8 A schematic diagram of an exemplary reasoning module 310 according to an embodiment of the present application is shown.

[0092] like Figure 8 As shown, in some embodiments, the reasoning module 310 may include a large model algorithm module 3102 and a context understanding and reasoning module 3104.

[0093] The core of the large model algorithm module 3102 is to use advanced large language models, such as models based on the Transformer architecture, for pre-training and domain-adaptive fine-tuning. In the pre-training stage, the model learns the general features of the language on a large-scale general corpus. In the fine-tuning stage, the model is adjusted on the professional corpus of the power industry to adapt to the terminology and expression habits of specific fields. In addition, the large model algorithm module 3102 also uses a variety of sample enhancement technologies, such as back-translation, generative adversarial networks (GANs), and variational autoencoders (VAEs), to enhance the model's generalization ability for domain data.

[0094] In the training and fine-tuning of the large language model, first, a large amount of corpus data is collected, including but not limited to daily conversations, technical documents, news articles, etc. Then, the large language model is trained using this data so that it can understand and process complex natural language problems. Finally, the large language model is fine-tuned according to specific application scenarios and domain data so that it can better adapt to specific application requirements.

[0095] For the training of large language models, a variety of sample enhancement methods are used. For example, back-translation technology translates one language into another language and then translates it back to generate new samples; generative models such as generative adversarial networks (GANs) or variational autoencoders (VAEs) are used to generate new samples similar to the original corpus. Text augmentation techniques can also be used, such as rule-based augmentation (such as adding modifiers, synonym replacement, etc.) and deep learning-based augmentation (such as automatic generation using language models). In the training of large language models, optimization algorithms such as Adam, Adagrad, and Adadelta can be used to automatically adjust the learning rate according to the gradient information during the training process to improve the model convergence speed and performance. For logical reasoning and causal reasoning, graph neural networks (GNNs) or rule-based reasoning algorithms can be used to improve the accuracy and efficiency of reasoning. For personalized services, cluster analysis algorithms can be used to divide users into different groups according to their behavior and preferences to provide more accurate personalized services. An early stopping strategy is used to monitor the performance of the model on the validation set, and training is stopped when the performance no longer improves to avoid overfitting. At the same time, a stratified sampling method can be used to extract samples from corpus data from different sources in a certain proportion for training to ensure the diversity and representativeness of the training data. For the fine-tuning process, a weighted fine-tuning strategy can be used according to the characteristics of the application scenario and domain data to give different weights to different data and improve the effect of fine-tuning.

[0096] The innovation of the large model algorithm module 3102 is that it combines the large model algorithm with the specific needs of the power industry. Through domain adaptability fine-tuning, the model's understanding and answer quality of power professional questions are significantly improved. In addition, the application of sample enhancement technology further improves the robustness and accuracy of the model when processing domain data.

[0097] In some embodiments, in order to improve the adaptability of the target model in a specific domain, a weighted cross entropy loss function L can be introduced into the target model, where the loss of domain-specific samples is given a higher weight:

[0098]

[0099] Among them, ω i is a domain-specific weight factor used to enhance the learning effect of domain-related samples, y i is the one-hot encoding of the true label, p i is the probability distribution predicted by the model. N is the total number of samples.

[0100] In some embodiments, the target model may also include a learning rate adjustment strategy α based on model performance feedback. t+1 , which dynamically adjusts the learning rate to optimize the training process:

[0101] α t+1 =α t ·f(ΔL t )

[0102] Among them, α t is the current learning rate, ΔL t is the loss change between two consecutive training cycles, f(ΔL t ) is a feedback function designed based on loss changes and is used to adjust the learning rate.

[0103] The context understanding and reasoning module 3104 achieves in-depth analysis and logical reasoning of the problem context by introducing graph neural networks (GNNs) and rule-based reasoning algorithms, combined with the semantic understanding capabilities of large language models. GNNs are used to mine the potential logical structure and causal relationships in text data, while rule-based reasoning algorithms process explicit logical relationships and causal rules. In addition, the context understanding and reasoning module 3104 also establishes a semantic understanding system and a knowledge retrieval system, using technologies such as inverted indexes to quickly retrieve relevant knowledge from large-scale text data in the power industry to support the answering of complex questions.

[0104] In the context understanding and reasoning module 3104, considering the dynamic nature of the context, in some embodiments, a time decay factor Sim(u,v) may be introduced to calculate the semantic similarity to reflect the timeliness of the information. The time decay factor is as follows:

[0105]

[0106] Among them, λ is the decay rate parameter used to control the similarity decay over time, t is the time difference, which represents the difference between the current moment and the moment when the information is generated, a and b are two vectors, a·b is the dot product of the vectors, and ||a|| and ||b|| are the Euclidean norms of the vectors.

[0107] In the context understanding and reasoning module 3104, in some embodiments, a knowledge retrieval optimization strategy based on user query intent is proposed, which improves the relevance of retrieval by query expansion:

[0108]

[0109] Among them, E u represents the query word set expanded based on the user's query intent, s j represents the weight of the extended word j, c j represents the document coverage of the extended word j, I u represents the set of documents containing the query word u, ω i represents the weight of word u in document i, d i Represents the length of document i.

[0110] In logical reasoning, in some embodiments, a causal analysis mechanism may be used to enhance the model's understanding of the causal relationship of events:

[0111]

[0112] Where CA represents the causal analysis function, which is used to identify and emphasize the causal paths directly related to node α, and W (l) and layer-specific weights, b (l) is the layer-specific bias, H (l) is the hidden state of layer l, is the set of neighbor nodes of node α, σ represents the activation function, which is used to introduce nonlinear characteristics in the neural network, and v represents a neighbor node of node α in the set.

[0113] The innovation of the context understanding and reasoning module 3104 is that it combines graph neural networks with rule-based reasoning algorithms to improve the logical reasoning and causal reasoning capabilities for power industry issues. Through the introduction of semantic understanding and knowledge retrieval systems, it can provide more accurate and in-depth answers.

[0114] The internal structure and processing flow of the large model algorithm module 3102 and the context understanding and reasoning module 3104 are described as follows.

[0115] Training and fine-tuning of large language models: First, collect a large amount of corpus data, including but not limited to daily conversations, technical documents, news articles, etc. Then, use this data to train the large language model so that it can understand and process complex natural language problems. Finally, fine-tune the large language model according to specific application scenarios and domain data so that it can better adapt to specific application needs. For the training of large language models, a variety of sample enhancement methods are used. For example, back-translation technology translates one language into another language and then translates it back to generate new samples; use generative models such as generative adversarial networks (GANs) or variational autoencoders (VAEs) to generate new samples similar to the original corpus. Text expansion techniques can also be used, such as rule-based expansion (such as adding modifiers, synonym replacement, etc.) and deep learning-based expansion (such as using language models for automatic generation). In the training of large language models, optimization algorithms such as Adam, Adagrad, Adadelta, etc. can be used to automatically adjust the learning rate according to the gradient information during the training process to improve the model convergence speed and performance. For logical reasoning and causal reasoning, graph neural networks (GNN) or rule-based reasoning algorithms can be used to improve the accuracy and efficiency of reasoning. For personalized services, cluster analysis algorithms can be used to divide users into different groups based on their behaviors and preferences, and provide more accurate personalized services. An early stopping strategy is used to monitor the performance of the model on the validation set, and training is stopped when the performance no longer improves to avoid overfitting. At the same time, a stratified sampling method can be used to extract samples from corpus data from different sources in a certain proportion for training to ensure the diversity and representativeness of the training data. For the fine-tuning process, a weighted fine-tuning strategy can be used according to the characteristics of the application scenario and domain data to assign different weights to different data and improve the effect of fine-tuning.

[0116] System architecture construction: Based on the trained and fine-tuned large language model, determine the basic architecture of the intelligent question-answering system. The architecture should include input layer, processing layer and output layer. The input layer is responsible for receiving user questions, the processing layer performs reasoning and analysis based on the large language model, and the output layer generates and presents answers. The input layer pre-processes user questions, such as lexical analysis and syntactic analysis, and converts natural language questions into a form that can be processed by the model. The large language model in the processing layer encodes and decodes the input information, and uses the learned language knowledge and semantic information for reasoning. The output layer post-processes the answers generated by the model, such as format adjustment and language polishing, to make them more in line with the user's reading habits.

[0117] Model correction and optimization algorithm: Use real-time correction algorithms, such as gradient descent-based methods, to dynamically adjust the parameters of the question-answering model. During the optimization process, you can use adaptive learning rate algorithms (such as Adagrad, Adadelta, etc.) to automatically adjust the learning rate according to the gradient changes during model training to improve the model convergence speed and performance. Assuming that the question-answering model makes an error when answering a question in the power industry, the real-time correction algorithm is used to calculate the error between the predicted answer and the correct answer, and then adjust the model parameters based on the error back propagation. During the optimization process, if it is found that the model learns too slowly at a certain stage, the adaptive learning rate algorithm can be used to increase the learning rate and accelerate model convergence.

[0118] Information acquisition mechanism: Use guided dialogue and proactive questioning strategies to obtain more effective information. Guided dialogue can further guide the direction of questions based on the user's answers through preset question templates. Proactive questions are based on the preliminary analysis of the user's questions and proactively ask for relevant information to supplement the completeness of the questions. When a user asks about a power equipment failure, the guided dialogue may ask about the model of the equipment, the age of the equipment, and other information. Proactive questions may further ask about the manifestation of the failure, whether there is an alarm message, etc. based on the keywords in the question, such as "failure".

[0119] Introduction of logical reasoning and causal reasoning: Introducing logical reasoning and causal reasoning into large-scale intelligent question answering, deeply analyzing questions and knowledge, and providing more accurate answers. Retrieve relevant knowledge from large-scale text data through semantic understanding to improve retrieval accuracy.

[0120] In large-scale intelligent question answering, a combination of rule-based reasoning algorithms and graph neural network (GNN) algorithms is introduced to perform logical reasoning and causal reasoning. The rule-based reasoning algorithm is used to process some clear logical relationships and causal rules, and the GNN algorithm is used to mine the potential logical structure and causal relationship in text data. At the same time, a semantic understanding system is established, and word vector models (such as Word2Vec, GloVe, etc.) are used to convert text data into vector representations for better semantic matching and knowledge retrieval. The knowledge retrieval system uses technologies such as inverted indexing to quickly retrieve relevant knowledge from large-scale text data. Example of the final reasoning process: When a user asks "Why does power equipment overheat?", the rule-based reasoning algorithm first checks whether there are clear rules about power equipment overheating, such as "overload can cause power equipment to overheat." At the same time, the GNN algorithm analyzes the relevant text data and mines potential causal relationships such as "poor heat dissipation can also cause power equipment to overheat." Then the two reasoning results are combined to provide a more accurate answer. Final semantic understanding and retrieval example: For the keywords "power equipment" and "overheating" in the user's question, the semantic understanding system converts them into vector representations, and then searches for text paragraphs similar to these vectors in the knowledge retrieval system. For example, relevant knowledge content is retrieved from technical documents and maintenance manuals in the power industry to answer questions.

[0121] Implementation of personalized services: By analyzing user behavior and preferences, we can provide answers that better meet user needs. The large language model can understand and adapt to the language preferences and styles of different users. Through fine-tuning, it can control the style, expectations, emotional color and other aspects of the generated text, and provide more accurate personalized services. The user behavior and preference analysis algorithm uses a combination of cluster analysis algorithm and deep learning algorithm to analyze user behavior and preferences. The cluster analysis algorithm divides users into different groups according to their behavioral characteristics, and the deep learning algorithm further analyzes the preference characteristics of each group. In terms of model fine-tuning strategy, the large language model is fine-tuned according to the preference characteristics of the user group. In the fine-tuning process, a weighted fine-tuning strategy can be used to give different weights to different preference characteristics in order to better control the style, expectations, emotional color and other aspects of the generated text. Analysis process example: Through the cluster analysis algorithm, users are divided into two groups: professionals in the power industry and ordinary users. The deep learning algorithm further analyzed and found that professionals in the power industry are more concerned about technical parameters and solutions, while ordinary users are more concerned about equipment use and safety issues. Fine-tuning process example: For the group of professionals in the power industry, when fine-tuning the large language model, a higher weight is given to the parts involving technical parameters and solutions, so that the generated answers are more in line with their needs. For example, when answering questions about power equipment selection, more detailed technical parameters and selection basis are provided.

[0122] Providing services beyond expectations: Using continuous interaction and associative fusion capabilities, we provide more intelligent and complete answers, achieve services beyond expectations, and improve user satisfaction. The large language model has the ability of logical reasoning, draws conclusions based on the learned logical rules and the language structure and logical relationships in the text, and provides more constructive answers. Selecting continuous interaction and associative fusion algorithms: Using a combination of attention mechanism and memory network to achieve continuous interaction and associative fusion capabilities. The attention mechanism is used to focus on current issues and related knowledge, and the memory network is used to store and call historical interaction information and related knowledge. In terms of logical reasoning applications, we use the logical reasoning capabilities of the large language model to reason based on the learned logical rules and the language structure and logical relationships in the text, and provide more constructive answers. Interaction and fusion example: When a user asks several questions about the stability of the power system in succession, the attention mechanism will focus on these questions and related power system knowledge, and the memory network will store previous interaction information. For example, the first question is about the factors affecting the stability of the power system, and the second question is how to improve the stability of the power system. The system will provide a more intelligent and complete answer based on the previously stored information and the currently focused knowledge. Reasoning application example: If a user asks "how to solve a certain fault in the power system", the large language model reasons based on the learned logical rules (such as the general steps of troubleshooting) and the language structure and logical relationships in the text (such as the correspondence between the fault phenomenon and the possible causes) to provide a detailed solution.

[0123] Implementation of anthropomorphic service: A large-scale natural language corpus is used to train the model by combining unsupervised learning and supervised learning. Unsupervised learning is used to learn general features of the language, such as grammar, semantics, and language style. Supervised learning is used to optimize the model according to specific tasks and requirements, and to establish a colloquial modal vocabulary and a language polishing rule library. When generating answers, according to the language style and task requirements, appropriate colloquial modal particles are selected from the modal vocabulary, and the language is polished according to the rules in the polishing rule library. During the training process: In the unsupervised learning stage, the model learns the grammar, semantics, and language style patterns in a large amount of natural language text. In the supervised learning stage, for example, for the question-and-answer task in the power industry, the model is adjusted according to the correct answers and user feedback to make it more in line with the task requirements. When answering users' questions about the use of power equipment, modal particles such as "um" and "oh" are selected from the colloquial modal vocabulary, and the answers are polished according to the rules in the language polishing rule library, such as adjusting the sentence structure, adding modifiers, etc., to make the answers more natural and more personalized.

[0124] The reasoning module 310 can receive information from the previous modules. After training and fine-tuning, the large language model can perform intelligent question and answer based on the input information. Through logical reasoning and causal reasoning, the questions are analyzed, and combined with the realization of personalized services, unexpected services and anthropomorphic services, accurate, personalized and natural answers are provided to users. At the same time, this module cooperates with other modules. For example, the realization of personalized services requires the analysis of user behavior and preference information, which may come from the multimodal input processing module.

[0125] In the reasoning module 310, a large amount of corpus data is used for training and fine-tuned in combination with specific application scenarios. In this way, the large language model can better understand and process complex natural language problems, adapt to specific application requirements, and improve the accuracy of understanding user problems. Introduce logical reasoning and causal reasoning in large model intelligent question and answer. This method can conduct in-depth analysis of questions and knowledge, provide more accurate answers, and improve the accuracy of answers. Personalized services are achieved by analyzing user behavior and preferences. The large language model can provide personalized answers based on the different needs of users, improve the accuracy and satisfaction of services. Utilize continuous interaction and prospect fusion capabilities to provide services beyond expectations. This method can provide smarter and more complete answers and improve user satisfaction. Anthropomorphic services are achieved through natural language training. The model can generate output answers that conform to human language habits, provide more natural user experience and services, and improve user experience and interaction quality.

[0126] In the system 300, the modules cooperate with each other, and the multimodal input processing module 302 provides the input information basis for other modules. The dynamic topic prediction module 304 predicts the topic trend based on the input information and provides a decision basis for the program function mapping module 306. The program function mapping module 306 performs the corresponding action and ensures the correct execution of the action by using the execution module 308. The reasoning module 310 receives information from the previous module to provide intelligent question and answer and related services, and also cooperates with other modules to obtain the required information. For example, personalized services may require user behavior and preference information provided by the multimodal input processing module. Through the close collaboration of various modules, the entire system achieves accurate understanding and answers to user questions, improves the coherence and consistency of the dialogue, and provides a personalized, natural interactive experience and beyond-expectation services.

[0127] It should be noted that the method for generating feedback information provided in this application can be applied to the following industries in addition to the power industry:

[0128] Medical industry: It can be used in medical intelligent question-answering systems to provide medical knowledge consulting services to doctors and patients. For example, patients can ask questions such as disease symptoms and treatment methods, and the system provides accurate answers through similar multimodal input processing, topic prediction, model reasoning and other processes.

[0129] Financial industry: In the intelligent customer service system in the financial field, help customers answer questions about financial products, investment strategies, financial regulations, etc. The system uses its ability to learn and process knowledge in the financial field to provide customers with personalized and accurate services.

[0130] Fig. 9 A flow chart of an exemplary method 900 for generating feedback information according to an embodiment of the present application is shown. The method 900 may be executed by the system 300 and may include the following steps.

[0131] In step 902, query information input by a user is obtained, where the query information is multimodal information.

[0132] In step 904, a prediction result is obtained based on the query information and the historical information, and the prediction result includes topics related to the query information.

[0133] In some embodiments, common topics related to the query information are obtained from a small database; and the prediction result is obtained based on the query information, the historical information and the common topics.

[0134] In step 906, a program function corresponding to the prediction result is obtained according to the prediction result and the function mapping table.

[0135] In step 908, the program function is executed to convert the prediction result into input data of the target model.

[0136] In step 910, feedback information corresponding to the query information is generated based on the input data and the target model.

[0137] In some embodiments, the target model is trained by the following method: obtaining corpus data and extracting first sample data from the corpus data in a certain proportion; obtaining second sample data based on the first sample data, and the second sample data is similar to the corpus data; using the first sample data and the second sample data to train a large language model to obtain the target model.

[0138] In some embodiments, the corpus data includes user behavior characteristics, and the training of the large language model to obtain the target model further includes: using a clustering analysis algorithm to obtain multiple user groups based on the user behavior characteristics; using a deep learning algorithm to analyze the multiple user groups to obtain analysis results; and fine-tuning the large language model based on the analysis results to obtain the target model.

[0139] In some embodiments, the target model includes a causal analysis mechanism, which is expressed as:

[0140]

[0141] Where CA represents the causal analysis function, which is used to identify and emphasize the causal paths directly related to node α, and W (l) and layer-specific weights, b (l) is the layer-specific bias, H (l) is the hidden state of layer l, is the set of neighbor nodes of node α, σ represents the activation function, which is used to introduce nonlinear characteristics in the neural network, and v represents a neighbor node of node α in the set.

[0142] In some embodiments, the target model includes a knowledge retrieval optimization strategy, and the knowledge retrieval optimization strategy is expressed as:

[0143]

[0144] Among them, E u represents the query word set expanded based on the user's query intent, s j represents the weight of the extended word j, c j represents the document coverage of the extended word j, I u represents the set of documents containing the query word u, ω i represents the weight of word u in document i, d i Represents the length of document i.

[0145] In some embodiments, a decorator function is used to decorate the plurality of program functions, and an action manager is used to schedule and manage the plurality of decorated program functions.

[0146] The present application provides a method, device, electronic device and storage medium for generating feedback information. The method includes: obtaining query information input by a user, wherein the query information is multimodal information; obtaining a prediction result based on the query information and historical information, wherein the prediction result includes a topic related to the query information; obtaining a program function corresponding to the prediction result based on the prediction result and a function mapping table; executing the program function to convert the prediction result into input data of a target model; and generating feedback information corresponding to the query information based on the input data and the target model. By predicting the query information and executing the program function corresponding to the prediction result, more accurate input data of the target model is obtained, and the feedback information generated based on the input data better meets the needs of the user.

[0147] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0148] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0149] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements method 900 as described in any of the above embodiments when executing the program.

[0150] Fig.10 A schematic diagram of an exemplary electronic device according to an embodiment of the present application is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other in communication within the device through the bus 1050.

[0151] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0152] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0153] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0154] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0155] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0156] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0157] The electronic device of the above embodiment is used to implement the corresponding method 900 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0158] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute method 900 described in any of the above embodiments.

[0159] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0160] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute method 900 as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0161] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0162] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0163] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0164] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for generating feedback information, comprising: Acquire query information input by a user, wherein the query information is multimodal information; Obtaining a prediction result according to the query information and the historical information, wherein the prediction result includes a topic related to the query information; According to the prediction result and the function mapping table, a program function corresponding to the prediction result is obtained; Executing the program function to convert the prediction result into input data of the target model; Based on the input data and the target model, feedback information corresponding to the query information is generated.

2. The method of claim 1, wherein: The obtaining of prediction results according to the query information and the historical information further comprises: Obtain common topics related to the query information in a small database; The prediction result is obtained according to the query information, the historical information and the common topics.

3. The method of claim 1, wherein: The target model is trained by the following method: Acquire corpus data, and extract first sample data from the corpus data according to a certain ratio; According to the first sample data, second sample data is obtained, where the second sample data is similar to the corpus data; The first sample data and the second sample data are used to train a large language model to obtain the target model.

4. The method of claim 3, wherein: The corpus data includes user behavior features, and the training of the large language model to obtain the target model further includes: Using a cluster analysis algorithm, a plurality of user groups are obtained according to the user behavior characteristics; Analyze the multiple user groups using a deep learning algorithm to obtain analysis results; The large language model is fine-tuned according to the analysis result to obtain the target model.

5. The method of claim 1, wherein: The method further comprises: The decorator function is used to decorate the plurality of program functions, and the action manager is used to schedule and manage the plurality of decorated program functions.

6. The method of claim 1, wherein: The target model includes a causal analysis mechanism, which is expressed as: Where CA represents the causal analysis function, which is used to identify and emphasize the causal paths directly related to node α, and W (l) and layer-specific weights, b (l) is the layer-specific bias, H (l) is the hidden state of layer l, is the set of neighbor nodes of node α, σ represents the activation function, which is used to introduce nonlinear characteristics in the neural network, and v represents a neighbor node of node α in the set.

7. The method of claim 1, wherein: The target model includes a knowledge retrieval optimization strategy, which is expressed as: Among them, E u represents the query word set expanded based on the user's query intent, s j represents the weight of the extended word j, c j represents the document coverage of the extended word j, I u represents the set of documents containing the query word u, ω i represents the weight of word u in document i, d i Represents the length of document i.

8. A device for generating feedback information, comprising: A multimodal input processing module, configured to obtain query information input by a user, wherein the query information is multimodal information; a dynamic topic prediction module, configured to obtain a prediction result according to the query information and the historical information, wherein the prediction result includes a topic related to the query information; A program function mapping module, configured to obtain a program function corresponding to the prediction result according to the prediction result and a function mapping table; An execution module, configured to execute the program function to convert the prediction result into input data of a target model; The reasoning module is configured to generate feedback information corresponding to the query information based on the input data and the target model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images