Machine language large model construction method and system

Abstract

The invention provides a machine language large model construction method and system, and the method comprises the steps: receiving user input data, defining an intelligent semantic association network, and extracting the semantic association between a text and visual or auditory data; a cross-modal attention mechanism and an adversarial generative network are utilized to enhance the multi-modal data relationship in the association, and a multi-modal perception mechanism is formed; based on the mechanism, a multi-level feedback system is constructed, feedback signals from different levels are processed, and an optimized prediction path is generated; key parameters are dynamically adjusted in combination with a task adaptive module and the optimized prediction path, and an adaptive learning rate adjustment strategy is generated; based on an incremental context extension module, an intelligent semantic association network, a multi-modal sensing mechanism, a multi-level feedback system and a task adaptive module, a machine language large model is constructed. According to the method, the understanding and processing capability of the model on complex multi-modal data is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the fields of machine learning and natural language technology, and in particular, to a method and system for constructing a large machine language model. Background Art

[0002] In today's artificial intelligence field, the ability to process multimodal data is crucial to achieve understanding and interaction of complex tasks. Current technical solutions are widely used in many fields such as smart assistants, autonomous driving, and medical diagnosis. These systems usually use independent or simple fusion methods to process data of different modalities. Specifically, existing methods first process the data of each modality separately, and then fuse the results in some way at the later stage, or preliminarily fuse the data of different modalities in the feature extraction stage. Although these methods can achieve certain results in specific application scenarios, they fail to fully explore the deep semantic associations between modalities, limiting the comprehensive utilization of information.

[0003] Existing multimodal processing solutions usually use independent or simple fusion methods to process data of different modalities. Some methods process the data of each modality separately, and then fuse the results in some way at the later stage. Although this method is simple and direct, it fails to fully utilize the associations between the modalities. Another common method is to perform a preliminary fusion of data of different modalities in the feature extraction stage, but this shallow fusion lacks the mining of deep semantic associations and cannot effectively capture the complex relationships between modalities. In addition, some models use fixed architectures and parameter settings. Although they can perform well on specific tasks, they have poor adaptability when facing new tasks or environmental changes. Although these existing solutions have made progress in some aspects, they have failed to fully address the challenges in multimodal data processing, especially in terms of dynamic adaptability and deep semantic associations.

[0004] Existing solutions have obvious limitations when processing multimodal data, especially in terms of deep semantic associations and dynamic adaptability. First, most existing methods fail to fully explore the deep semantic associations between text and related visual or auditory data, resulting in low information utilization and affecting the overall performance of the model. Second, many models rely on fixed parameter configurations and are difficult to dynamically adjust according to task changes, which limits the generalization ability and adaptability of the model. In addition, most existing systems lack an effective multi-level feedback mechanism and cannot optimize the prediction path in real time, which limits the performance improvement of the model in complex environments. Summary of the invention

[0005] The embodiments of the present application provide a method and system for constructing a large machine language model to solve the problem of low understanding and processing capabilities of complex multimodal data in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for constructing a large machine language model, comprising: Accepting user input data and defining an intelligent semantic association network, obtaining semantic associations based on the intelligent semantic association network combined with the user input data; using a cross-modal attention mechanism and a generative adversarial network to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and determining a multimodal perception mechanism; Based on the multimodal perception mechanism, a multi-level feedback system is constructed, and the multi-level feedback system is used to process feedback signals from different levels to generate an optimized prediction path; Based on the multi-level feedback system, a task adaptation module is constructed using a meta-learning framework and a reinforcement learning algorithm, and based on the task adaptation module and the optimized prediction path, key parameters in the task adaptation module are adjusted to generate an adaptive learning rate adjustment strategy; Based on the optimized prediction path and the adaptive learning rate adjustment strategy, an incremental context extension module is constructed using the structured representation learning technology based on graph neural networks. Based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system and the task adaptation module, a large machine language model is constructed.

[0007] Optionally, based on the multi-level feedback system, a task adaptation module is constructed using a meta-learning framework and a reinforcement learning algorithm, and based on the task adaptation module and the optimized prediction path, key parameters in the task adaptation module are adjusted to generate an adaptive learning rate adjustment strategy, including: Using a meta-learning framework and a reinforcement learning algorithm, an initial task adaptation module is constructed based on the multi-level feedback system, and the conversion efficiency of the initial task adaptation module between different tasks is optimized by a Q-learning algorithm and a policy gradient method; Based on the optimized prediction path, a variational autoencoder is used to analyze the change trend from the data distribution of feedback signals at different levels in the multi-level feedback system, and the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate a target task adaptation module; By using the context-aware mechanism, the target task adaptive module adjusts the learning strategy according to the background information of different tasks to generate an adaptive learning rate adjustment strategy.

[0008] Optionally, the context-aware mechanism is used to enable the target task adaptive module to adjust the learning strategy according to the background information of different tasks to generate an adaptive learning rate adjustment strategy, including: Utilizing a context-aware mechanism, combined with a graph neural network and an attention mechanism, key features are analyzed from the background information of different tasks in the multi-level feedback system, wherein the key features include task type, data distribution characteristics, and historical performance; Based on the key features, in combination with Bayesian optimization and Gaussian process regression, the learning rate in the target task adaptive module is dynamically adjusted to generate a preliminary adaptive learning rate adjustment strategy; Using a time series prediction model, optimizing the preliminary adaptive learning rate adjustment strategy through reinforcement learning to generate an optimized learning rate adjustment strategy; Combining a genetic algorithm and a differential evolution algorithm, searching for an optimal learning rate adjustment strategy from the optimized learning rate adjustment strategy; An active learning mechanism is introduced to select target data from the optimal learning rate adjustment strategy to generate an adaptive learning rate adjustment strategy.

[0009] Optionally, the context-aware mechanism is used in combination with a graph neural network and an attention mechanism to analyze key features from background information of different tasks in the multi-level feedback system, including: Using data mining technology, background information of different tasks in the multi-level feedback system is collected and preprocessed to obtain task background data of the multi-level feedback system; Introducing a context-aware mechanism, combined with principal component analysis and factor analysis, the task background data is preliminarily screened and dimensionally reduced to obtain the target task background information; Applying graph neural networks and complex network analysis, the complex relationships between tasks are analyzed from the background information of the target tasks to construct a task relationship graph; The attention mechanism is applied in combination with deep reinforcement learning to highlight the important features in the task relationship graph and obtain the key feature weight distribution result; The task relationship diagram and the key feature weight distribution result are analyzed to extract key features.

[0010] Optionally, based on the optimized prediction path, a variational autoencoder is used to analyze the change trend from the data distribution of feedback signals at different levels in the multi-level feedback system, and key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate a target task adaptation module, including: According to the optimized prediction path, using variational autoencoders and statistical analysis tools, modeling the data distribution of feedback signals at different levels in the multi-level feedback system, and analyzing potential patterns and change rules from the feedback information to generate data distribution change trends; Based on the data distribution change trend, the key parameters in the initial task adaptive module are dynamically adjusted through Bayesian optimization, Gaussian process regression and gradient boosting tree to generate a target task adaptive module.

[0011] Optionally, the utilizing a cross-modal attention mechanism and a generative adversarial network to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and determining a multimodal perception mechanism, includes: Using a hierarchical graph neural network, an initial intelligent semantic association network is defined, and an initial semantic association is analyzed from the intrinsic connection between the text content of the initial intelligent semantic association network and the visual or auditory data associated with the text content; Based on the initial semantic association, a cross-modal attention mechanism is introduced, and a bidirectional long short-term memory network is combined to jointly encode the text in the initial semantic association and the corresponding visual or auditory data to obtain a multimodal information representation; Constructing a generative adversarial network based on the multimodal information representation, and analyzing key data from an adversarial training process between a generator and a discriminator in the generative adversarial network; Based on the key data and in combination with a variational autoencoder, the initial intelligent semantic association network is optimized to obtain a target intelligent semantic association network, and based on the target intelligent semantic association network, a multimodal perception mechanism is determined.

[0012] Optionally, the constructing of an incremental context expansion module based on the optimized prediction path and the adaptive learning rate adjustment strategy using a structured representation learning technology based on a graph neural network includes: Determine an incremental update rule by using the optimized prediction path and the adaptive learning rate adjustment strategy in combination with time series analysis and Bayesian optimization; Based on the incremental update rule, the multimodal data in the multimodal perception mechanism is modeled and processed by using a structured representation learning technology based on a graph neural network in combination with a hierarchical clustering algorithm to obtain a structured representation of the multimodal data; According to the multimodal data structured representation, in combination with a dynamic graph convolutional network and a variational autoencoder, the data obtained from the multi-level feedback system is encoded, and the data is input into the multimodal data structured representation to output an updated context representation; Applying an attention mechanism in combination with deep reinforcement learning to highlight key features in the updated context representation to obtain an optimized context representation; Combining reinforcement learning and genetic algorithms, optimizing and adjusting the parameters in the incremental update process of the task adaptive module to generate an optimized parameter configuration; An incremental context extension module is generated based on the optimized context representation and the optimized parameter configuration.

[0013] In a second aspect, an embodiment of the present application provides a system for constructing a large machine language model, including: A receiving module, used for receiving user input data, defining an intelligent semantic association network, and obtaining semantic association based on the intelligent semantic association network combined with the user input data; An enhancement module, for enhancing the relationship between the text in the semantic association and the visual or auditory data related to the text by using a cross-modal attention mechanism and a generative adversarial network, and determining a multimodal perception mechanism; A construction module, used to construct a multi-level feedback system based on the multimodal perception mechanism, and use the multi-level feedback system to process feedback signals from different levels to generate an optimized prediction path; An adjustment module is used to construct a task adaptation module based on the multi-level feedback system using a meta-learning framework and a reinforcement learning algorithm, and to adjust key parameters in the task adaptation module based on the task adaptation module and the optimized prediction path to generate an adaptive learning rate adjustment strategy; A module is used to construct an incremental context extension module based on the optimized prediction path and the adaptive learning rate adjustment strategy using a structured representation learning technology based on a graph neural network, and a large machine language model is constructed based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system, and the task adaptation module. In a third aspect, an embodiment of the present application provides a computing device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute a large machine language model construction method described in any one of the first aspects.

[0014] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement a method for constructing a large machine language model as described in any one of the first aspects.

[0015] In an embodiment of the present application, user input data is received, and an intelligent semantic association network is defined; semantic associations are obtained based on the intelligent semantic association network combined with the user input data; a cross-modal attention mechanism and a generative adversarial network are used to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and a multimodal perception mechanism is determined; based on the multimodal perception mechanism, a multi-level feedback system is constructed, and the multi-level feedback system is used to process feedback signals from different levels to generate an optimized prediction path; based on the multi-level feedback system, a task adaptation module is constructed using a meta-learning framework and a reinforcement learning algorithm, and based on the task adaptation module and the optimized prediction path, key parameters in the task adaptation module are adjusted to generate an adaptive learning rate adjustment strategy; based on the optimized prediction path and the adaptive learning rate adjustment strategy, an incremental context extension module is constructed using a structured representation learning technology based on a graph neural network, and a large machine language model is constructed based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system, and the task adaptation module.

[0016] The technical solution of this application has the following beneficial effects: The intelligent semantic association network of this application effectively captures the deep semantic connection between text and related visual or auditory data, and improves the model's ability to understand multimodal data. Through the cross-modal attention mechanism and adversarial generative network, the association between text and other modal data is enhanced, enabling the model to more comprehensively understand complex multimodal inputs. The multi-level feedback system processes feedback signals at different levels in real time, generates optimized prediction paths, and ensures that the model can maintain high performance in different tasks. The task adaptation module combines the optimized prediction path, dynamically adjusts key parameters, and generates an adaptive learning rate adjustment strategy, so that the model can quickly adapt and continuously optimize in a changing environment. Based on the incremental context extension module, the model can gradually introduce new information, maintain existing performance while continuously improving, and ensure long-term effectiveness and flexibility. The overall architecture integrates the intelligent semantic association network, multimodal perception mechanism, multi-level feedback system, and task adaptation module to build a powerful large machine language model with the ability to efficiently handle complex tasks.

[0017] Furthermore, the embodiment of the present application also constructs a task adaptation module through a multi-level feedback system combined with a meta-learning framework and a reinforcement learning algorithm, and adjusts key parameters based on this module and the optimized prediction path to generate an adaptive learning rate adjustment strategy. Specifically, first, the meta-learning framework and the reinforcement learning algorithm are used to construct an initial task adaptation module based on the multi-level feedback system, and the conversion efficiency of the module between different tasks is optimized by the Q learning algorithm and the policy gradient method; then, based on the optimized prediction path, a variational autoencoder is used to analyze the data distribution change trend of feedback signals at different levels in the multi-level feedback system, and the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate the final target task adaptation module; finally, the context-aware mechanism is used to enable the target task adaptation module to adjust the learning strategy according to different task background information, thereby generating an adaptive learning rate adjustment strategy.

[0018] Through the above method, not only the task adaptation ability and learning efficiency of the model are significantly improved, but also the model can be continuously optimized when facing new tasks and environmental changes. The task adaptation module constructed by the meta-learning framework and reinforcement learning algorithm can efficiently switch between different tasks, improving the flexibility and generalization ability of the model. The application of variational autoencoder combined with Bayesian optimization enables the model to extract valuable change trends from the complex data distribution of multi-level feedback systems, dynamically adjust key parameters, and further enhance the adaptability of the model to different task backgrounds. In addition, the application of context-aware mechanism enables the model to flexibly adjust the learning strategy according to the specific task background information and generate the optimal learning rate adjustment strategy, thereby ensuring the high performance and rapid convergence of the model in various application scenarios. In summary, this method effectively solves the problems of insufficient deep semantic association, static parameter configuration, and imperfect feedback mechanism in the existing scheme, and provides a more intelligent and efficient multimodal processing solution.

[0019] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of a method for constructing a large machine language model provided in an embodiment of the present application; Figure 2A schematic diagram of the structure of a machine language large model building system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0023] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0025] Figure 1 A flowchart of a method for constructing a large machine language model is provided for an embodiment of the present application, such as Figure 1 As shown, the method includes: Step 101: receiving user input data, defining an intelligent semantic association network, and obtaining semantic association based on the intelligent semantic association network combined with the user input data; In this step, multimodal input data is received from the user, which can be in the form of text, images, or audio. The intelligent semantic association network is a complex neural network architecture designed to capture the deep semantic relationships between multiple modal data such as text, images, and audio. Through multi-level feature extraction and association analysis, the network can identify implicit connections between different modal data, thereby providing rich semantic information for subsequent processing. For example, in natural language processing, it can identify keywords in sentences and match them with key elements in related images or audio clips to enhance the understanding of multimodal data.

[0026] In actual operation, after receiving user input, the system performs preliminary processing on multimodal data through the preprocessing module, such as text segmentation, image feature extraction, and audio transcription. Then, an intelligent semantic association network is defined, and multi-level feature extraction and association analysis are used to identify implicit connections between different modal data and generate specific semantic associations. These associations include not only direct mapping relationships, but also indirect semantic connections, providing a solid foundation for subsequent steps.

[0027] For example, in an intelligent assistant application, users may interact with the system through voice commands, text input, or uploading pictures. For example, a user uploads a photo with the tag "beach" and asks about nearby restaurants. The system first receives the photo and tag information, and after preprocessing, uses an intelligent semantic association network to identify visual elements such as the beach and waves in the photo, and associates them with the "beach" tag to generate initial semantic associations, providing a basis for the next step of multimodal perception.

[0028] Step 102: using a cross-modal attention mechanism and a generative adversarial network to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and determining a multimodal perception mechanism; In this step, the cross-modal attention mechanism dynamically adjusts the importance of different modal data in the fusion process by introducing attention weights, so that the model can focus on the most relevant parts. The adversarial generative network is used to generate high-quality synthetic data to enhance the model's understanding of the relationship between multimodal data. This method can not only improve the model's ability to process existing data, but also enrich training samples by generating new data and improve the model's generalization ability.

[0029] In practice, using the cross-modal attention mechanism, the model calculates the attention scores between the text and other modal data, which reflect the correlation between different modal data. Then, the adversarial generative network is used to generate new multimodal data pairs, which are trained to further enhance the model's understanding of multimodal relationships. Finally, the multimodal perception mechanism is determined in this way, making the model more sensitive and accurate when processing multimodal data.

[0030] For example, continuing with the example of the smart assistant above, when a user asks about nearby restaurants, the system uses a cross-modal attention mechanism to identify key elements in the photo (such as the beach, waves) and establishes a stronger association with the text label "beach". At the same time, the adversarial generative network generates more similar multimodal data pairs, making the model more robust and accurate when facing similar queries. This not only improves the system's response speed, but also improves user satisfaction.

[0031] Step 103: Based on the multimodal perception mechanism, a multi-level feedback system is constructed, and the multi-level feedback system is used to process feedback signals from different levels to generate an optimized prediction path; In this step, the multi-level feedback system is a hierarchical structure, with each layer responsible for processing a specific type of feedback signal, from low-level raw data to high-level task results. Through this hierarchical processing, the system is able to gradually refine and optimize its prediction path, ensuring that each decision is based on comprehensive and accurate information. The multi-level feedback system can not only adjust the behavior of the model in real time, but also record historical feedback to provide a basis for future improvements.

[0032] In actual operation, based on the multimodal perception mechanism, a multi-level feedback system is constructed. The system is divided into multiple levels, and each level processes feedback signals at different levels. For example, the lowest level processes the original multimodal data, the middle level processes the preliminary feature extraction results, and the highest level processes the final task output. Through layer-by-layer processing and feedback, the system can continuously optimize its prediction path to ensure that each prediction is optimal. In addition, the system will adjust parameters based on historical feedback to further improve the accuracy of the prediction.

[0033] For example, in a smart assistant app, the multi-level feedback system first receives photos and voice commands uploaded by users as raw data at the bottom layer. Next, the system processes this data in the middle layer and extracts key features, such as the names of dishes on the menu and the word "restaurant" in the voice. Finally, at the highest level, the system combines this information to generate an optimized prediction path, which is a list of nearby restaurants. If the user is not satisfied with the recommendation, the system will collect feedback and make corresponding adjustments at the next query, gradually optimizing the recommendation results and providing a better user experience.

[0034] Step 104: Based on the multi-level feedback system, a task adaptation module is constructed using a meta-learning framework and a reinforcement learning algorithm, and based on the task adaptation module and the optimized prediction path, key parameters in the task adaptation module are adjusted to generate an adaptive learning rate adjustment strategy; In this step, the task adaptation module enables the model to quickly switch between different tasks and efficiently adapt to new tasks through the meta-learning framework and reinforcement learning algorithm. The meta-learning framework allows the model to learn from past tasks, while the reinforcement learning algorithm optimizes the model's behavior through a reward mechanism. By dynamically adjusting key parameters, the task adaptation module can generate an adaptive learning rate adjustment strategy to ensure that the model maintains optimal performance in different tasks and environments.

[0035] In actual operation, based on the optimized prediction path provided by the multi-level feedback system, the task adaptation module is constructed using the meta-learning framework and reinforcement learning algorithm. This module optimizes the conversion efficiency between different tasks through the Q-learning algorithm and the policy gradient method to ensure that the model can quickly adapt to new tasks. At the same time, the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate the final target task adaptation module. This process allows the model to flexibly adjust the learning strategy according to the current task context and generate the optimal learning rate adjustment strategy.

[0036] For example, in smart assistant applications, the task adaptation module can quickly adjust its own parameter configuration to adapt to different query types based on the user's historical interaction records and current query content. For example, when users frequently query restaurant information, the module will optimize restaurant-related parameters to improve query efficiency. When users start asking about tourist attractions, the module can quickly switch to parameter configurations that adapt to tourist scenarios. In this way, the system can not only make accurate recommendations in a short period of time, but also continuously optimize itself as user behavior changes to provide more personalized services.

[0037] Step 105: Based on the optimized prediction path and the adaptive learning rate adjustment strategy, an incremental context extension module is constructed using the structured representation learning technology based on graph neural networks, and a large machine language model is constructed based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system and the task adaptation module.

[0038] In this step, the incremental context expansion module implements structured representation learning through graph neural networks, allowing the model to gradually introduce new information without affecting existing performance. This module can dynamically expand the context range based on the optimized prediction path and adaptive learning rate adjustment strategy to ensure that the model remains efficient when processing long sequences or multi-step reasoning tasks. Finally, a complete large machine language model was constructed by combining intelligent semantic association networks, multimodal perception mechanisms, multi-level feedback systems, and task adaptation modules.

[0039] In actual operation, based on the optimized prediction path and adaptive learning rate adjustment strategy, the structured representation learning technology of graph neural network is used to build an incremental context expansion module. This module gradually introduces new information to expand the context range and ensure that the model is still efficient when processing complex tasks. Combining intelligent semantic association network, multimodal perception mechanism, multi-level feedback system and task adaptation module, a powerful machine language model is finally built. The model can perform well in a variety of application scenarios and has efficient adaptive capabilities and incremental learning characteristics.

[0040] For example, in smart assistant applications, the incremental context expansion module can gradually expand the scope of context when the user asks multiple questions in succession, ensuring that each answer is based on the latest information. For example, the user first asks about nearby restaurants, then asks about the specific dishes of a restaurant, and finally asks about the opening hours of the restaurant. The module will gradually introduce this new information, expand the scope of context, and ensure that each answer is accurate. At the same time, other components such as intelligent semantic association networks and multimodal perception mechanisms work together to improve the overall performance of the system and provide users with a seamless interactive experience.

[0041] Through the above five steps, this application builds a highly intelligent machine language model, which significantly improves the model's ability to understand and process multimodal data. The intelligent semantic association network and cross-modal attention mechanism ensure the mining of deep semantic associations, the multi-level feedback system and task adaptation module achieve efficient dynamic adjustment and optimization, and the incremental context expansion module ensures the flexibility and high performance of the model when processing complex tasks. In the end, the model not only performs well in initial performance, but also continues to improve in the face of new tasks and environmental changes. It is suitable for a wide range of natural language processing and other multimodal application fields, greatly improving user experience and system performance.

[0042] In order to solve the problem that the task adaptation module adjustment is not flexible enough and the learning rate adjustment strategy lacks dynamism in the existing method, in one or more of the above embodiments, the step 104 is based on the multi-level feedback system, using a meta-learning framework and a reinforcement learning algorithm to build a task adaptation module, and based on the task adaptation module and the optimized prediction path, adjusting the key parameters in the task adaptation module to generate an adaptive learning rate adjustment strategy, including: The meta-learning framework and reinforcement learning algorithm are used to construct an initial task adaptation module based on the multi-level feedback system, and the conversion efficiency of the initial task adaptation module between different tasks is optimized by the Q-learning algorithm and the policy gradient method; based on the optimized prediction path, the variational autoencoder is used to analyze the change trend from the data distribution of feedback signals at different levels in the multi-level feedback system, and the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate a target task adaptation module; the context-aware mechanism is used to enable the target task adaptation module to adjust the learning strategy according to the background information of different tasks, and generate an adaptive learning rate adjustment strategy.

[0043] In this embodiment, the construction of the task adaptation module not only relies on the traditional static parameter configuration, but also introduces a meta-learning framework and a reinforcement learning algorithm. The meta-learning framework allows the model to quickly learn experience from previous tasks and apply it to new tasks, thereby improving the efficiency of task conversion. The reinforcement learning algorithm optimizes the behavior of the model through a reward mechanism to ensure that it can switch efficiently between different tasks. In addition, the variational autoencoder is used to analyze the data distribution change trend of the multi-level feedback system, while the Bayesian optimization dynamically adjusts the key parameters of the initial task adaptation module, so that the model can adjust the learning strategy according to the specific task background information, and finally generate an adaptive learning rate adjustment strategy.

[0044] In the embodiment of the present application, the meta-learning framework and reinforcement learning algorithm are first used to construct an initial task adaptation module based on a multi-level feedback system. This module optimizes the conversion efficiency between different tasks through the Q learning algorithm and the policy gradient method to ensure that the model can quickly adapt to new tasks. Then, based on the optimized prediction path, the variational autoencoder is used to analyze the data distribution change trend of feedback signals at different levels in the multi-level feedback system, and the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate the final target task adaptation module. Finally, the context-aware mechanism is used to enable the target task adaptation module to adjust the learning strategy according to the background information of different tasks, and generate the optimal learning rate adjustment strategy. This process allows the model to flexibly adjust the learning strategy according to the current task background to ensure that efficient performance is maintained when facing complex environments.

[0045] Here is a specific example: In the application scenario of an intelligent customer service system, users may ask many types of queries, such as product information, order status, and after-sales service. In order to improve the response speed and accuracy, the task adaptation module first uses the meta-learning framework and reinforcement learning algorithm to build an initial module based on a multi-level feedback system, and optimizes the conversion efficiency between different tasks through the Q learning algorithm and the policy gradient method. For example, when a user asks for product information for the first time, the system quickly identifies the user's intention and provides an accurate answer. Then, based on the optimized prediction path, the system uses a variational autoencoder to analyze the data distribution change trend of feedback signals at different levels, and dynamically adjusts key parameters through Bayesian optimization to generate the final target task adaptation module. For example, when multiple attributes of the same product are asked continuously, the system captures the association between these queries and optimizes the parameter configuration. Finally, the context-aware mechanism is used to adjust the learning strategy according to different task backgrounds to generate the optimal learning rate adjustment strategy. For example, when a user asks about after-sales service, the system quickly adjusts the parameters related to after-sales service to improve query efficiency. In this way, the system can not only give accurate responses in a short time, but also continuously optimize itself to provide more personalized services.

[0046] In order to solve the problem that the task adaptive module adjustment in the existing method is not flexible enough and the learning rate adjustment strategy lacks dynamism, in the above multiple embodiments, the context-aware mechanism is used in step 104 to enable the target task adaptive module to adjust the learning strategy according to the background information of different tasks to generate an adaptive learning rate adjustment strategy, and further includes: Utilizing the context-aware mechanism, combined with graph neural networks and attention mechanisms, key features are analyzed from the background information of different tasks in the multi-level feedback system, and the key features include task type, data distribution characteristics and historical performance; based on the key features, combined with Bayesian optimization and Gaussian process regression, the learning rate in the target task adaptive module is dynamically adjusted to generate a preliminary adaptive learning rate adjustment strategy; utilizing a time series prediction model, the preliminary adaptive learning rate adjustment strategy is optimized through reinforcement learning to generate an optimized learning rate adjustment strategy; combining genetic algorithms and differential evolution algorithms, the optimal learning rate adjustment strategy is searched from the optimized learning rate adjustment strategy; introducing an active learning mechanism, selecting target data from the optimal learning rate adjustment strategy to generate an adaptive learning rate adjustment strategy. Optionally, the context-aware mechanism is used in combination with a graph neural network and an attention mechanism to analyze key features from the background information of different tasks in the multi-level feedback system, including: using data mining technology to collect and preprocess the background information of different tasks in the multi-level feedback system to obtain task background data of the multi-level feedback system; introducing a context-aware mechanism, combined with principal component analysis and factor analysis, to perform preliminary screening and dimensionality reduction on the task background data to obtain target task background information; applying graph neural networks and complex network analysis to analyze the complex relationships between tasks from the target task background information to construct a task relationship graph; applying an attention mechanism in combination with deep reinforcement learning to highlight important features in the task relationship graph to obtain key feature weight allocation results; analyzing the task relationship graph and the key feature weight allocation results to extract key features.

[0047] In order to solve the problem that the analysis of data distribution change trend in the existing method is not precise enough and the adjustment of key parameters lacks flexibility, in one or more of the above embodiments, the step 104, based on the optimized prediction path, uses a variational autoencoder to analyze the change trend from the data distribution of feedback signals at different levels in the multi-level feedback system, and dynamically adjusts the key parameters in the initial task adaptation module through Bayesian optimization to generate a target task adaptation module, and also includes: According to the optimized prediction path, the data distribution of feedback signals at different levels in the multi-level feedback system is modeled using variational autoencoders and statistical analysis tools, and potential patterns and change rules are analyzed from the feedback information to generate data distribution change trends; based on the data distribution change trends, the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization, Gaussian process regression and gradient boosting tree to generate a target task adaptation module.

[0048] In this embodiment, the optimized prediction path is used to guide the variational autoencoder and statistical analysis tools to model the data distribution of feedback signals at different levels in the multi-level feedback system. These feedback signals include real-time feedback from user interactions, historical task results, and intermediate representations within the model. Through modeling, the system can capture potential patterns and changing patterns and generate accurate data distribution change trends. Subsequently, based on these changing trends, advanced algorithms such as Bayesian optimization, Gaussian process regression, and gradient boosting trees are used to dynamically adjust the key parameters in the initial task adaptation module, ultimately generating a more flexible and efficient target task adaptation module.

[0049] In the embodiment of the present application, firstly, according to the optimized prediction path, the data distribution of feedback signals at different levels in the multi-level feedback system is modeled using variational autoencoders and statistical analysis tools, and the potential patterns and change rules are analyzed to generate data distribution change trends. Then, based on these change trends, the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization, Gaussian process regression and gradient boosting tree to ensure that each parameter is optimally configured according to the latest data distribution changes. This process not only improves the model's adaptability to new tasks, but also enhances its stability in complex environments.

[0050] Here is a specific example: In the application scenario of an intelligent customer service system, users may raise various types of queries, such as product information, order status, and after-sales service. In order to improve the response speed and accuracy, the system first uses the variational autoencoder and statistical analysis tools to model the data distribution of feedback signals at different levels in the multi-level feedback system based on the optimized prediction path. For example, when a user continuously asks about multiple attributes of the same product, the system captures the association between these queries through the variational autoencoder, and identifies the potential patterns and change rules through statistical analysis tools to generate data distribution change trends.

[0051] Next, based on these changing trends, the system dynamically adjusts the key parameters in the initial task adaptation module through Bayesian optimization, Gaussian process regression, and gradient boosting tree. For example, when users begin to frequently ask about after-sales service, the system quickly adjusts the parameters related to after-sales service to optimize query efficiency. In this way, the system can not only give accurate responses in a short time, but also continuously optimize itself as user behavior changes, providing more personalized services.

[0052] In order to solve the problem that the multimodal data fusion in the existing method is not deep enough and the semantic association analysis is not accurate enough, in one or more of the above embodiments, the use of the cross-modal attention mechanism and the adversarial generative network in step 102 to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and determine the multimodal perception mechanism, includes: An initial intelligent semantic association network is defined by using a hierarchical graph neural network, and the initial semantic association is analyzed from the intrinsic connection between the text content of the initial intelligent semantic association network and the visual or auditory data associated with the text content; based on the initial semantic association, a cross-modal attention mechanism is introduced, and combined with a bidirectional long short-term memory network, the text in the initial semantic association and the corresponding visual or auditory data are jointly encoded to obtain a multimodal information representation; based on the multimodal information representation, an adversarial generative network is constructed, and key data is analyzed from the adversarial training process between the generator and the discriminator in the adversarial generative network; based on the key data, the initial intelligent semantic association network is optimized in combination with a variational autoencoder to obtain a target intelligent semantic association network, and based on the target intelligent semantic association network, a multimodal perception mechanism is determined.

[0053] In this embodiment, a hierarchical graph neural network (HGNN) is used to define an initial intelligent semantic association network, which can analyze the initial semantic association from the intrinsic connection between the text content and its associated visual or auditory data. The initial semantic association refers to the preliminary correspondence between different modal data. Next, a cross-modal attention mechanism is introduced in combination with a bidirectional long short-term memory network (Bidirectional LongShort-Term Memory, BiLSTM) to jointly encode these initial semantic associations and generate a multimodal information representation. This representation not only includes a direct mapping of text and visual / auditory data, but also covers deeper semantic connections. Finally, by constructing an adversarial generative network, key data is extracted from the adversarial training process between the generator and the discriminator, and the initial intelligent semantic association network is optimized in combination with a variational autoencoder to obtain a target intelligent semantic association network, thereby determining a multimodal perception mechanism.

[0054] In an embodiment of the present application, a hierarchical graph neural network is first used to define an initial intelligent semantic association network, analyze the intrinsic connection between the text content and the associated visual or auditory data, and generate initial semantic associations. Then, based on these initial semantic associations, a cross-modal attention mechanism is introduced in combination with BiLSTM to perform joint encoding processing on the text and the corresponding visual or auditory data to obtain a multimodal information representation. Then, according to the multimodal information representation, an adversarial generative network is constructed, and key data is analyzed through the adversarial training process between the generator and the discriminator. Finally, the initial intelligent semantic association network is optimized in combination with a variational autoencoder to obtain a target intelligent semantic association network, and finally a multimodal perception mechanism is determined. This process ensures the depth of multimodal data fusion and the accuracy of semantic association analysis.

[0055] Here is a specific example: In a multimedia content recommendation system, users may upload photos or video clips with text descriptions. In order to improve the accuracy and personalization of recommendations, the system first uses a hierarchical graph neural network to define an initial intelligent semantic association network, analyze the intrinsic connection between the text description of the photo or video and its visual content, and generate initial semantic associations. For example, when a user uploads a photo with the label "beach", the system identifies visual elements such as the beach and waves, and establishes a preliminary association. Then, based on these initial semantic associations, the system introduces a cross-modal attention mechanism combined with BiLSTM to jointly encode text and visual data to generate a multimodal information representation, which not only recognizes labels but also understands specific visual elements. Then, based on the multimodal information representation, an adversarial generative network is constructed to extract key data through adversarial training of the generator and the discriminator. Finally, the initial intelligent semantic association network is optimized in combination with a variational autoencoder to obtain a target intelligent semantic association network, thereby determining a multimodal perception mechanism, more accurately understanding the content uploaded by users, and providing personalized recommendation services.

[0056] In order to solve the problem that the incremental update of the context extension module in the existing method is not flexible enough and the multimodal data modeling is not fine enough, in one or more of the above embodiments, the incremental context extension module is constructed based on the optimized prediction path and the adaptive learning rate adjustment strategy in step 105 using the structured representation learning technology based on the graph neural network, including: The optimized prediction path and the adaptive learning rate adjustment strategy are used in combination with time series analysis and Bayesian optimization to determine incremental update rules; based on the incremental update rules, the multimodal data in the multimodal perception mechanism is modeled and processed using a structured representation learning technology based on a graph neural network in combination with a hierarchical clustering algorithm to obtain a structured representation of the multimodal data; according to the structured representation of the multimodal data, in combination with a dynamic graph convolutional network and a variational autoencoder, the data obtained from the multi-level feedback system is encoded, and the data is input into the structured representation of the multimodal data to output an updated context representation; an attention mechanism is applied in combination with deep reinforcement learning to highlight key features in the updated context representation to obtain an optimized context representation; reinforcement learning and a genetic algorithm are combined to optimize and adjust the parameters in the incremental update process in the task adaptation module to generate an optimized parameter configuration; an incremental context extension module is generated based on the optimized context representation and the optimized parameter configuration.

[0057] In this embodiment, the optimized prediction path and adaptive learning rate adjustment strategy are used to combine time series analysis and Bayesian optimization to determine the incremental update rules. These rules guide the system on how to gradually introduce new information without affecting existing performance. Based on the incremental update rules, the multimodal data in the multimodal perception mechanism is modeled and processed using the structured representation learning technology based on the graph neural network and the hierarchical clustering algorithm to obtain a structured representation of the multimodal data. These representations not only capture the association between different modal data, but also reflect their inherent structure. Subsequently, the data obtained from the multi-level feedback system is encoded in combination with the dynamic graph convolutional network and the variational autoencoder, and the data is input into the structured representation of the multimodal data, and the updated context representation is output. In order to further optimize, the attention mechanism is applied in combination with deep reinforcement learning to highlight the key features and generate an optimized context representation. Finally, the parameters in the incremental update process in the task adaptation module are optimized and adjusted in combination with reinforcement learning and genetic algorithm to generate an optimized parameter configuration, thereby constructing an incremental context extension module.

[0058] In the embodiment of the present application, the prediction path after optimization and the adaptive learning rate adjustment strategy are first used, combined with time series analysis and Bayesian optimization, to determine the incremental update rules. Then, based on these rules, the multimodal data in the multimodal perception mechanism is modeled and processed using the structured representation learning technology based on the graph neural network and the hierarchical clustering algorithm to generate a multimodal data structured representation. Then, the data in the multi-level feedback system is encoded in combination with the dynamic graph convolutional network and the variational autoencoder, and the updated context representation is output. On this basis, the attention mechanism is applied to highlight the key features in combination with deep reinforcement learning, and the optimized context representation is generated. Finally, the incremental update parameters in the task adaptation module are optimized and adjusted by reinforcement learning and genetic algorithm, and the optimized parameter configuration is generated, and the incremental context extension module is finally constructed. This process ensures the flexibility of the context extension module and the precision of multimodal data modeling.

[0059] Here is a specific example: In an intelligent assistant application scenario, users may ask multiple related questions in succession, such as asking about restaurant information, menu content, and business hours. In order to improve the response speed and accuracy, the system first uses the optimized prediction path and adaptive learning rate adjustment strategy, combined with time series analysis and Bayesian optimization, to determine the incremental update rules to guide the gradual introduction of new information without affecting the existing performance. Based on these rules, the system uses graph neural networks and hierarchical clustering algorithms to model multimodal data and generate structured representations. For example, when a user uploads a photo containing "menu" and asks about nearby restaurants, the system identifies the names of dishes on the menu and associates them with the queried restaurant. Then, the dynamic graph convolutional network and variational autoencoder are combined to encode the data in the feedback system and output the updated context representation. The attention mechanism and deep reinforcement learning are applied to highlight the key features and generate the optimized context representation. Finally, the parameter configuration in the task adaptation module is optimized through reinforcement learning and genetic algorithm, and the incremental context expansion module is constructed to ensure that the system is still efficient when facing complex needs and provide more accurate and personalized answers.

[0060] This application takes into account that in complex multimodal task processing, traditional static learning rates and fixed parameter configurations are difficult to adapt to the ever-changing task context information. In order to improve the model's adaptability and generalization performance, a method is needed that can dynamically adjust the learning strategy according to the task context. Especially when faced with a variety of task types, data distribution characteristics and historical performance, the model must have the ability to flexibly adjust the learning rate to ensure efficient conversion and continuous optimization between different tasks. In addition, existing methods often ignore the impact of task background information on learning strategies, resulting in poor performance of the model when dealing with new tasks or environmental changes. Therefore, a new optional solution is proposed, which includes: The use of the context-aware mechanism enables the target task adaptive module to adjust the learning strategy according to the background information of different tasks and generate an adaptive learning rate adjustment strategy, including: Utilizing a context-aware mechanism, in combination with a graph neural network and an attention mechanism, key features are analyzed from background information of different tasks provided by the multi-level feedback system, wherein the key features are obtained by calculating attention weights; The calculation formula of attention weight is as follows: ; in, Indicates The unnormalized attention scores of features or data points, and are the query vector and the key vector respectively, and is the weight matrix, is the bias term, is the adjustment coefficient, is the additional weight matrix, is the context feature vector, is the amplitude coefficient, is the angular frequency, Indicates the time step, indicating the current time or sequence position, is the phase angle, represents the corresponding attention weight, Indicates The unnormalized attention scores of features or data points, Represents an index variable, which is used to traverse all features or data points; The key features are used to adjust the learning strategy in the target task adaptive module to generate an adaptive learning rate adjustment strategy, wherein the adaptive learning rate adjustment strategy is determined by calculating an adaptive learning rate; The adaptive learning rate adjustment formula is as follows: ; in, Indicates time The learning rate at represents the initial learning rate, represents the attenuation coefficient, represents the amplitude, represents the angular frequency, represents the phase angle, and are the asymptotic growth coefficient and rate, respectively. and is the additional Gaussian attenuation coefficient.

[0061] For formula ; The following is a detailed explanation of each parameter: Refers to The unnormalized attention score of a feature or data point. It is used to measure the importance of the feature in the current task.

[0062] and They are the query vector and the key vector, representing the association between data of different modalities. They are usually extracted by the encoder.

[0063] and Refers to the weight matrix, which is used to linearly transform the query vector and the key vector to enhance the expressiveness of the model. These matrices are usually learned during the training process.

[0064] Refers to the bias term, which is used to adjust the baseline level of the overall score to prevent the output value from being too small or too large. It is usually a constant, initially set to 0 or a small value, and adjusted during training.

[0065] Refers to the adjustment coefficient, which controls the influence of context features. It is determined through experiments or hyperparameter tuning.

[0066] Refers to the additional weight matrix used to perform a linear transformation on the context feature vector. It is also learned through the training process.

[0067] Refers to the context feature vector, which represents the background information related to the current task. It can be extracted from a multi-level feedback system.

[0068] Refers to the amplitude coefficient, which controls the amplitude of the sine function. It can be determined through experiments or hyperparameter tuning.

[0069] Refers to angular frequency, which indicates the speed of periodic changes. It is set according to the characteristics of the task, for example, it may be set according to the time interval in a time series task.

[0070] Refers to the time step, which indicates the current time or sequence position. It is obtained directly from the input data.

[0071] Refers to the phase angle, indicating the starting position of the periodic change. It is set according to the characteristics of the task, for example, it may be set according to the initial state in a time series task.

[0072] Refers to the corresponding attention weight, which is used to normalize the importance of each feature and ensure that the sum of all weights is 1. This allows the model to allocate computing resources more efficiently.

[0073] The following is an introduction to the design reasons of each sub-item: Query-Key Similarity Term The purpose of its design is to measure the similarity between data of different modalities and help the model identify which features are most relevant.

[0074] The bias term (b) is designed to prevent the output value from being too small or too large, providing additional flexibility.

[0075] Contextual Feature Enhancements The purpose of its design is to introduce contextual features and enhance the model's understanding of background information.

[0076] The reason for adding it is to serve as an additional item to enhance the model's sensitivity to complex background information.

[0077] Periodic Variation The purpose of the design is to introduce the time factor so that the model can dynamically adjust the attention weights, which is especially important when processing time series data or sequence tasks.

[0078] The reason for adding it is to serve as a dynamic adjustment item to ensure that the model can adapt to the periodically changing task environment.

[0079] For formula ; The following is a detailed explanation of each parameter: Refers to time The learning rate is used to dynamically adjust the learning strategy of the model.

[0080] Refers to the initial learning rate, which is usually determined by experience or experiments and is the learning rate when the model starts training.

[0081] Refers to the decay coefficient, which controls the exponential decay speed of the learning rate over time. It is determined by experiments or hyperparameter tuning.

[0082] Refers to the amplitude, controls the amplitude of the sine wave, and introduces periodic changes. Determined by experiments or hyperparameter tuning.

[0083] Refers to angular frequency, which indicates the speed of periodic changes. Set according to the characteristics of the task.

[0084] Refers to the phase angle, indicating the starting position of the periodic change. Set according to the characteristics of the task.

[0085] Refers to the asymptotic growth coefficient, which controls the growth rate of the logarithmic term. It is determined through experiments or hyperparameter tuning.

[0086] Refers to the rate that affects the growth rate of the logarithmic term. It can be determined through experiments or hyperparameter tuning.

[0087] Refers to the Gaussian decay coefficient, which controls the magnitude of the Gaussian decay term. It is determined by experiments or hyperparameter tuning. It mainly affects the initial level and maximum value of the learning rate.

[0088] Refers to the Gaussian decay coefficient, which controls the width of the Gaussian decay term. It is determined by experiments or hyperparameter tuning. Its main function is to control how fast the learning rate changes over time.

[0089] The following is an introduction to the design reasons of each sub-item: Exponential decay term The design purpose is to simulate the phenomenon that the learning rate gradually decreases over time, which helps the model to be more stable in the later stages of training.

[0090] Sine wave term The purpose of design is to introduce periodic changes so that the learning rate increases in certain time periods, which helps the model explore new solution spaces.

[0091] The reason for adding it is to act as a fluctuation term and provide short-term learning rate adjustment.

[0092] Logarithmic Growth Term The design purpose is to introduce a progressive growth feature, so that the learning rate increases rapidly in the early stage of training and increases slowly in the later stage, which helps to optimize the model at different stages.

[0093] The reason for adding it is to act as a gradual term to provide mid-term learning rate adjustment.

[0094] Gaussian attenuation term The purpose of the design is to introduce Gaussian decay characteristics, so that the learning rate is large at the beginning of training and then decreases rapidly, which helps the model converge quickly in the early stage.

[0095] The overall design aims to build a flexible and adaptive learning rate adjustment mechanism, which enables the model to dynamically adjust its learning strategy in different task contexts. By combining multiple mathematical functions, the formula not only takes into account long-term trends in time (such as exponential decay), but also introduces short-term fluctuations (such as sine waves) and mid-term adjustments (such as logarithmic growth), as well as the need for rapid initial convergence (such as Gaussian decay). This multi-dimensional design ensures that the model maintains efficient performance in the face of complex and changing task environments, ultimately improving the system's response speed and accuracy.

[0096] Here is a specific example: In the application scenario of an intelligent customer service system, users may raise various types of queries, such as product information, order status, and after-sales service. In order to improve the response speed and accuracy, the system needs to dynamically adjust the learning strategy and learning rate according to different task contexts.

[0097] Assume that the system is currently processing a query about product information. The task type is "product information query", the data distribution characteristics are "high-frequency words are concentrated in product feature descriptions", and the historical performance is "similar queries have a high frequency in the past month".

[0098] Based on the above task background information, the system uses graph neural networks to extract relevant features and calculates the importance of each feature through the attention weight formula. For example: and They are query vector and key vector respectively, and their values ​​are and . and is the weight matrix, and its values ​​are and .

[0099] is the bias term, and its value is 0.5. is the adjustment coefficient, and its value is 0.1. is an additional weight matrix, whose value is . is the context feature vector, and its value is . is the amplitude coefficient, and its value is 0.2. is the angular frequency, and its value is 0.5. is the time step, and its value is 1. is the phase angle, and its value is 0.3.

[0100] Calculation results: ; After normalization, we get the attention weight .

[0101] According to the calculated attention weights, the learning strategy in the target task adaptation module is adjusted. For example, for key features with high attention weights (such as product feature descriptions), the system will assign higher weights, thereby focusing more on learning these features.

[0102] According to the adjusted learning strategy, the learning rate is calculated using the adaptive learning rate adjustment formula. Assumptions: Initial learning rate ; Attenuation coefficient ;amplitude ; Angular frequency ; Phase angle ; Progressive growth coefficient ;rate ; Gaussian attenuation coefficient .

[0103] Calculation results: ; From the calculation results, we can see that the learning rate obtained by the system is 0.01193. Assuming that the learning rate is set in the range of [0, 1] according to the requirements, it means that when processing the current task (product information query), the system uses a poor learning rate in order to adapt to the new task background information more quickly. As time and task complexity change, the learning rate will gradually adjust to ensure that the system maintains efficient conversion and continuous optimization between different tasks. This method not only improves the response speed and accuracy of the system, but also enhances its adaptability in complex and changing task environments, ultimately improving user experience and satisfaction.

[0104] Figure 2 A structural diagram of a machine language large model construction system is provided for an embodiment of the present application, such as Figure 2 As shown, the system includes: The receiving module 21 is used to receive user input data, define an intelligent semantic association network, and obtain semantic association based on the intelligent semantic association network combined with the user input data; An enhancement module 22, configured to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text by using a cross-modal attention mechanism and a generative adversarial network, and determine a multimodal perception mechanism; A construction module 23 is used to construct a multi-level feedback system based on the multimodal perception mechanism, and use the multi-level feedback system to process feedback signals from different levels to generate an optimized prediction path; An adjustment module 24 is used to construct a task adaptation module based on the multi-level feedback system using a meta-learning framework and a reinforcement learning algorithm, and adjust key parameters in the task adaptation module based on the task adaptation module and the optimized prediction path to generate an adaptive learning rate adjustment strategy; Module 25 is used to construct an incremental context extension module based on the optimized prediction path and the adaptive learning rate adjustment strategy using the structured representation learning technology based on the graph neural network, and to construct a large machine language model based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system and the task adaptation module.

[0105] Figure 2 The machine language large model building system can execute Figure 1 The implementation principle and technical effect of the method for constructing a large machine language model described in the embodiment are not described in detail. The specific way in which each module and unit performs operations in the large machine language model construction system in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.

[0106] In one possible design, Figure 2 A machine language large model building system of the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0107] The processing component 32 is used to: receive user input data and define an intelligent semantic association network, and obtain semantic association based on the intelligent semantic association network combined with the user input data; use a cross-modal attention mechanism and a generative adversarial network to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and determine a multimodal perception mechanism; based on the multimodal perception mechanism, build a multi-level feedback system, and use the multi-level feedback system to process feedback signals from different levels to generate an optimized prediction path; based on the multi-level feedback system, use a meta-learning framework and a reinforcement learning algorithm to build a task adaptation module, and based on the task adaptation module and the optimized prediction path, adjust the key parameters in the task adaptation module to generate an adaptive learning rate adjustment strategy; based on the optimized prediction path and the adaptive learning rate adjustment strategy, use a structured representation learning technology based on a graph neural network to build an incremental context extension module, and build a large machine language model based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system and the task adaptation module.

[0108] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0109] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0110] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0111] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0112] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0113] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0114] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 A method for constructing a large machine language model in the illustrated embodiment.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing a large machine language model, characterized in that: include: Receive user input data, and define an intelligent semantic association network, and obtain semantic association based on the intelligent semantic association network combined with the user input data; Using a cross-modal attention mechanism and a generative adversarial network, the relationship between the text in the semantic association and the visual or auditory data related to the text is enhanced to determine a multimodal perception mechanism; Based on the multimodal perception mechanism, a multi-level feedback system is constructed, and the multi-level feedback system is used to process feedback signals from different levels to generate an optimized prediction path; Based on the multi-level feedback system, a task adaptation module is constructed using a meta-learning framework and a reinforcement learning algorithm, and based on the task adaptation module and the optimized prediction path, key parameters in the task adaptation module are adjusted to generate an adaptive learning rate adjustment strategy; Based on the optimized prediction path and the adaptive learning rate adjustment strategy, an incremental context extension module is constructed using the structured representation learning technology based on graph neural networks. Based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system and the task adaptation module, a large machine language model is constructed.

2. The method according to claim 1, characterized in that The method of constructing a task adaptation module based on the multi-level feedback system using a meta-learning framework and a reinforcement learning algorithm, and adjusting key parameters in the task adaptation module based on the task adaptation module and the optimized prediction path to generate an adaptive learning rate adjustment strategy includes: Using a meta-learning framework and a reinforcement learning algorithm, an initial task adaptation module is constructed based on the multi-level feedback system, and the conversion efficiency of the initial task adaptation module between different tasks is optimized by a Q-learning algorithm and a policy gradient method; Based on the optimized prediction path, a variational autoencoder is used to analyze the change trend from the data distribution of feedback signals at different levels in the multi-level feedback system, and the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate a target task adaptation module; By using the context-aware mechanism, the target task adaptive module adjusts the learning strategy according to the background information of different tasks to generate an adaptive learning rate adjustment strategy.

3. The method according to claim 2, characterized in that The use of the context-aware mechanism enables the target task adaptive module to adjust the learning strategy according to the background information of different tasks and generate an adaptive learning rate adjustment strategy, including: Utilizing a context-aware mechanism, combined with a graph neural network and an attention mechanism, key features are analyzed from the background information of different tasks in the multi-level feedback system, wherein the key features include task type, data distribution characteristics, and historical performance; Based on the key features, in combination with Bayesian optimization and Gaussian process regression, the learning rate in the target task adaptive module is dynamically adjusted to generate a preliminary adaptive learning rate adjustment strategy; Using a time series prediction model, optimizing the preliminary adaptive learning rate adjustment strategy through reinforcement learning to generate an optimized learning rate adjustment strategy; Combining a genetic algorithm and a differential evolution algorithm, searching for an optimal learning rate adjustment strategy from the optimized learning rate adjustment strategy; An active learning mechanism is introduced to select target data from the optimal learning rate adjustment strategy to generate an adaptive learning rate adjustment strategy.

4. The method according to claim 3, characterized in that The context-aware mechanism is used in combination with the graph neural network and the attention mechanism to analyze key features from the background information of different tasks in the multi-level feedback system, including: Using data mining technology, background information of different tasks in the multi-level feedback system is collected and preprocessed to obtain task background data of the multi-level feedback system; Introducing a context-aware mechanism, combined with principal component analysis and factor analysis, the task background data is preliminarily screened and dimensionally reduced to obtain the target task background information; Applying graph neural networks and complex network analysis, the complex relationships between tasks are analyzed from the background information of the target tasks to construct a task relationship graph; The attention mechanism is applied in combination with deep reinforcement learning to highlight the important features in the task relationship graph and obtain the key feature weight distribution result; The task relationship diagram and the key feature weight distribution result are analyzed to extract key features.

5. The method according to claim 2, characterized in that: Based on the optimized prediction path, the variational autoencoder is used to analyze the change trend from the data distribution of feedback signals at different levels in the multi-level feedback system, and the key parameters in the initial task adaptation module are dynamically adjusted through Bayesian optimization to generate a target task adaptation module, including: According to the optimized prediction path, using variational autoencoders and statistical analysis tools, modeling the data distribution of feedback signals at different levels in the multi-level feedback system, and analyzing potential patterns and change rules from the feedback information to generate data distribution change trends; Based on the data distribution change trend, the key parameters in the initial task adaptive module are dynamically adjusted through Bayesian optimization, Gaussian process regression and gradient boosting tree to generate a target task adaptive module.

6. The method according to claim 1, characterized in that The method utilizes a cross-modal attention mechanism and a generative adversarial network to enhance the relationship between the text in the semantic association and the visual or auditory data related to the text, and determines a multimodal perception mechanism, including: Using a hierarchical graph neural network, an initial intelligent semantic association network is defined, and an initial semantic association is analyzed from the intrinsic connection between the text content of the initial intelligent semantic association network and the visual or auditory data associated with the text content; Based on the initial semantic association, a cross-modal attention mechanism is introduced, and a bidirectional long short-term memory network is combined to jointly encode the text in the initial semantic association and the corresponding visual or auditory data to obtain a multimodal information representation; Constructing a generative adversarial network based on the multimodal information representation, and analyzing key data from an adversarial training process between a generator and a discriminator in the generative adversarial network; Based on the key data and in combination with a variational autoencoder, the initial intelligent semantic association network is optimized to obtain a target intelligent semantic association network, and based on the target intelligent semantic association network, a multimodal perception mechanism is determined.

7. The method according to claim 1, characterized in that The method of constructing an incremental context expansion module based on the optimized prediction path and the adaptive learning rate adjustment strategy by using a structured representation learning technology based on a graph neural network includes: Determine an incremental update rule by using the optimized prediction path and the adaptive learning rate adjustment strategy in combination with time series analysis and Bayesian optimization; Based on the incremental update rule, the multimodal data in the multimodal perception mechanism is modeled and processed by using a structured representation learning technology based on a graph neural network in combination with a hierarchical clustering algorithm to obtain a structured representation of the multimodal data; According to the multimodal data structured representation, in combination with a dynamic graph convolutional network and a variational autoencoder, the data obtained from the multi-level feedback system is encoded, and the data is input into the multimodal data structured representation to output an updated context representation; Applying an attention mechanism in combination with deep reinforcement learning to highlight key features in the updated context representation to obtain an optimized context representation; Combining reinforcement learning and genetic algorithms, optimizing and adjusting the parameters in the incremental update process of the task adaptive module to generate an optimized parameter configuration; An incremental context extension module is generated based on the optimized context representation and the optimized parameter configuration.

8. A machine language large model construction system, characterized in that: include: A receiving module, used for receiving user input data, defining an intelligent semantic association network, and obtaining semantic association based on the intelligent semantic association network combined with the user input data; An enhancement module, for enhancing the relationship between the text in the semantic association and the visual or auditory data related to the text by using a cross-modal attention mechanism and a generative adversarial network, and determining a multimodal perception mechanism; A construction module, used to construct a multi-level feedback system based on the multimodal perception mechanism, and use the multi-level feedback system to process feedback signals from different levels to generate an optimized prediction path; An adjustment module is used to construct a task adaptation module based on the multi-level feedback system using a meta-learning framework and a reinforcement learning algorithm, and to adjust key parameters in the task adaptation module based on the task adaptation module and the optimized prediction path to generate an adaptive learning rate adjustment strategy; A module is used to construct an incremental context extension module based on the optimized prediction path and the adaptive learning rate adjustment strategy using a structured representation learning technology based on a graph neural network, and a large machine language model is constructed based on the incremental context extension module, the intelligent semantic association network, the multimodal perception mechanism, the multi-level feedback system and the task adaptation module.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a machine language large model construction method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a method for constructing a large machine language model as described in any one of claims 1 to 7 is implemented.

Images