Multimodal data processing method and device based on large language model and reinforcement learning

Abstract

The present application provides a multimodal data processing method and device based on a large language model and reinforcement learning, which belongs to the field of computer model and video data processing. The method includes: receiving user input information in a data generation and configuration interface, determining the configuration information for data generation based on the user input information and importing the original multimodal data, and dynamically determining the data generation path based on the configuration information to generate target multimodal data; annotating the original multimodal data and evaluating the data processing quality by combining a large language model and a reinforcement learning algorithm, and iteratively optimizing the generated target multimodal data based on the data processing quality; calculating the priority of each mode based on the interdependence of the multimodal data traffic, adaptively adjusting the multimodal data transmission bandwidth based on the priority, and automatically adjusting the traffic allocation based on the data transmission bandwidth to achieve data transmission management of multimodal data streams. The present application enhances the automation capability of multimodal data processing.

Description

Technical Field

[0001] The present application belongs to the field of computer model and video data processing, and in particular, relates to a multimodal data processing method and device based on a large language model and reinforcement learning. Background Art

[0002] Multimodal video processing involves the processing technology of multiple modal data (such as video, text, audio, and images, etc.), aiming to extract features, identify patterns, and build knowledge graphs by fusing these information sources, so as to better understand and process video content. In multimodal video processing systems, as the amount and complexity of data increase, traditional centralized or single-module systems are difficult to efficiently handle the generation, preprocessing, and training requirements of multimodal data.

[0003] First, multimodal data processing usually requires manual configuration of the data generation process, which is complex and time-consuming. Traditional video processing solutions are difficult to provide a flexible and easy configuration interface to automatically generate multimodal data, especially when generating interactive data in a reinforcement learning (RL) environment.

[0004] Secondly, traditional video data processing systems rely on manual labeling and manual decision-making, lack the intelligent enhancement of large-scale language models (LLMs) and reinforcement learning, and have insufficient automated data processing and labeling capabilities, resulting in high system operation and maintenance costs.

[0005] In addition, traditional video processing solutions lack effective real-time data flow management and monitoring capabilities and are unable to respond quickly when system bottlenecks occur, resulting in degraded system performance. The fusion of multimodal data also lacks precise fusion support. Summary of the invention

[0006] The purpose of this application is to provide a multimodal data processing method and device based on a large language model and reinforcement learning, aiming to enhance the automation capability of multimodal data processing through a large language model and reinforcement learning.

[0007] According to a first aspect of the present application, a multimodal data processing method based on a large language model and reinforcement learning is provided, comprising:

[0008] Receiving user input information in a data generation and configuration interface, determining configuration information for data generation according to the user input information and importing original multimodal data, and dynamically determining a data generation path based on the configuration information to generate target multimodal data;

[0009] By combining a large-scale language model and a reinforcement learning algorithm, the original multimodal data is annotated and the data processing quality is evaluated, and the generated target multimodal data is iteratively optimized according to the data processing quality;

[0010] The priority of each mode is calculated according to the interdependence of multimodal data traffic, the multimodal data transmission bandwidth is adaptively adjusted according to the priority, and the traffic allocation is automatically adjusted based on the data transmission bandwidth to realize data transmission management of multimodal data streams.

[0011] In an optional implementation, the dynamically determining a data generation path based on the configuration information further includes:

[0012] The rule engine uses the following formula to dynamically determine the path for data generation:

[0013] ;

[0014] Among them, β j is the balance coefficient of mode j; k represents the number of modal levels or tasks, n is the number of modes, Q is the normalization factor, is the total time, Status and configuration The data generation weights are: For modal The adaptive generation rate of is the multi-layer data quality control parameter, For configuration Dynamic weight optimization term, A build balancing function used to balance build rate and resource allocation during the build process.

[0015] In an optional embodiment, the data generates weights It is expressed as:

[0016] ;

[0017] in, For modal The priority weight of Representing modality In Status and configuration The complexity of data generation is as follows.

[0018] In an optional embodiment, the generating balance function Used to balance spawn rate and resource allocation during spawning:

[0019] ;

[0020] in, is the balance coefficient, indicating the modal The balance requirement for rate and generation resources; c is the current configuration, which indicates the actual state of the system at a certain moment; c′ is the changing configuration, which indicates the configuration that may change during time evolution or system adjustment, and is used for dynamic optimization and real-time adjustment.

[0021] In an optional embodiment, the evaluating the data processing quality further includes using the following formula to evaluate the quality Q(s, a) of the data processing:

[0022] ;

[0023] in is the multi-level reward mechanism function, is the strategy balance factor, For multi-layer nested Q-value estimation, For modal The adaptive weight value of Indicates the current status; Indicates the current action; Representing hierarchies or modalities in multimodal or multitask environments; Indicates the number of modal levels or tasks; Indicates the number of modes; Indicates the next state; Indicates the next action. represents the weighting factor; Represents the discount factor.

[0024] In an optional embodiment, the multi-level reward mechanism function It is expressed as:

[0025] ;

[0026] in Used to weigh short-term gains, is the local modal weight, is the first The reward function of the modality; M is the number of modalities in each layer; R global ( s,a ) is the global reward, through the global factor Adjustment;

[0027] The adaptive weight value Use an entropy-based weight adjustment mechanism:

[0028] ;

[0029] in, For modal The importance weight of For the The entropy value of the modality, which represents the information density, is used to dynamically adjust the impact of the modality on decision making.

[0030] In an optional embodiment, the multi-layer nested Q value estimation Through dynamic adjustment of the attention mechanism, important features in each modality are captured:

[0031] ;

[0032] in, is the attention score, is the weight matrix extracted from modal features, For the Tier Q-value estimation of the mode.

[0033] In an optional implementation manner, the adaptively adjusting the multimodal data transmission bandwidth according to the priority further includes:

[0034] Real-time analysis of system throughput (t):

[0035] ;

[0036] in is the data flow priority adjustment parameter for mode j, is the bandwidth weighting term, is the modal weight distribution, is the entropy increase optimization term, It is a real-time load balancing item; Indicates the number of modal levels or tasks; is the total time; Indicates time At the moment in the submodule The modal data traffic.

[0037] In an optional implementation, the data stream priority adjustment parameter of the mode j is for:

[0038] ;

[0039] in, is the dependence coefficient between modes, modal data flow of the modal; For the modal data flow of the modal; Indicates the total time; Indicates the total number of modes. Indicates the index of the other modality for comparison; represents the time variable in the time integral and is between 0 and T.

[0040] In an optional implementation, the entropy increase optimization term Based on the optimization mechanism of information entropy, the entropy increase of the modal data stream is adaptively controlled:

[0041] ;

[0042] in, represents the real-time information entropy of the modal data stream, is the maximum information entropy; Indicates time At the moment in the submodule The modal Data traffic volume;

[0043] The real-time load balancing item It is used to balance the load of each modal data stream and smooth the data stream intensity through the convolution filter:

[0044] ;

[0045] in, is the convolution kernel function, is the window length.

[0046] According to a second aspect of the present application, a multimodal data processing device based on a large language model and reinforcement learning is provided, comprising:

[0047] A data generation unit, configured to receive user input information in a data generation and configuration interface, determine configuration information for data generation according to the user input information and import the original multimodal data, and dynamically determine a data generation path based on the configuration information to generate target multimodal data;

[0048] A data iteration optimization unit, used to annotate the original multimodal data and evaluate the data processing quality by combining a large-scale language model and a reinforcement learning algorithm, and iteratively optimize the generated target multimodal data according to the data processing quality;

[0049] A data transmission unit is used to calculate the priority of each mode according to the interdependence of multimodal data traffic, adaptively adjust the multimodal data transmission bandwidth according to the priority, and automatically adjust the traffic allocation based on the data transmission bandwidth to achieve data transmission management of multimodal data streams.

[0050] A third aspect of the present application provides an electronic device, including a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute the method of the first aspect.

[0051] A fourth aspect of the present application provides a computer-readable storage medium, which stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method of the first aspect.

[0052] Compared with the related art, the technical solution of this application has the following advantages:

[0053] Through the modular data generation and configuration interface, the data generation process is simplified, the need for manual configuration and manual labeling is reduced, and processing efficiency is improved; by integrating large language models and reinforcement learning, the system achieves automation and intelligence in data flow optimization and resource allocation, improving system response speed; the data proxy module effectively optimizes the transmission and scheduling of multimodal data, ensuring the stable operation of the system in complex environments while improving data transmission efficiency.

[0054] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained through the structures and processes indicated in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. It is obvious that the drawings described below are certain embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a flowchart of a multimodal data processing method based on a large language model and reinforcement learning according to the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The method provided in the present application can be implemented in the following terminal environment, and the terminal may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method described in the following embodiments.

[0059] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory.

[0060] The memory may include random access memory (RAM) or read-only memory (ROM). The memory may be used to store instructions, programs, codes, code sets or instructions.

[0061] The display screen is used to display the user interface of each application.

[0062] In addition, those skilled in the art can understand that the structure of the above terminal does not constitute a limitation on the terminal, and the terminal may include more or fewer components, or combine certain components, or arrange the components differently. For example, the terminal also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, and a power supply, which will not be described in detail here.

[0063] Based on the above analysis, the present application provides a multimodal data processing method and device based on a large language model and reinforcement learning. The large language model (LLM) and reinforcement learning (RL) are used to implement data cleaning, quality assessment and dynamic scheduling of data streams through a rule engine, a data agent, and a data generation and configuration interface according to the provided data and the model tasks set by the user. In the modular data generation and configuration interface, functions such as adaptive generation parameter adjustment, dynamic weight optimization, and multi-layer quality control are provided to make the generation and configuration of multimodal data more flexible and efficient, reducing manual intervention and configuration time. The large language model and reinforcement learning are integrated, and the multi-level reward mechanism and modal weighted optimization of reinforcement learning are used to enhance the automation capability of multimodal data processing and reduce dependence on manual decision-making. Real-time load balancing, adaptive bandwidth allocation, and entropy increase optimization mechanisms are used to effectively manage multimodal data streams and ensure data transmission efficiency and system stability.

[0064] See also Figure 1 The multimodal data processing method based on a large language model and reinforcement learning provided in this application includes:

[0065] Step 101: receiving user input information in a data generation and configuration interface, determining configuration information for data generation according to the user input information and importing original multimodal data, and dynamically determining a data generation path based on the configuration information to generate target multimodal data.

[0066] Specifically, by providing a modular data generation and configuration interface, users are allowed to set the data generation process through simple graphical drag and drop or code configuration. Import and preprocess multiple modal data (such as video, text, audio, and images). For complex configurations in the data generation process, a flexible and automatic data generation and configuration mechanism is provided to reduce the complexity of manual configuration and improve data generation efficiency. This application uses a rule engine that uses the following formula to dynamically determine the path of data generation:

[0067] ;

[0068] Among them, β j is the balance coefficient of mode j; k represents the number of modal levels or tasks, n is the number of modes, is the total time, Status and configuration The data generation weights below are used to dynamically adjust the data generation priority, combining the modal characteristics and generation complexity to ensure that the generation weights of different modal data are reasonably distributed:

[0069] ;

[0070] in, For modal The priority weight of Representing modality In Status and configuration The data generation complexity is , where n is the number of modalities.

[0071] For modal Adaptive generation rate of , by controlling this rate, to ensure efficient resource allocation:

[0072] ;

[0073] in, For modal In Status and time The amount of data generated.

[0074] Specifically, It can be the current state of the system, representing the task context of the data generation process, involving a series of environmental information and operating conditions, such as the system's operating status, configuration conditions, resource allocation, data generation purpose, system status, data volume, time requirements, generation environment characteristics, etc.

[0075] The i-th stage multi-layer data quality control parameters, quality control at different data generation stages is used to ensure consistency. The stages include, for example, data collection, preprocessing, enhancement and other different stages.

[0076] ;

[0077] in, Indicates the configuration of stage i and status Lower mode The quality control requirements are used to automatically optimize the resource allocation at different stages. n represents the number of modalities. k represents the total number of data generation stages.

[0078] For configuration The dynamic weight optimization term is used to automatically adjust the weight of each mode based on the generation complexity:

[0079] ;

[0080] in, represents the weight optimization coefficient, Representing modality In the The complexity of data generation under item configuration, The maximum complexity of the system.

[0081] To generate a balance function, it is used to balance the generation rate and resource allocation during the generation process:

[0082] ;

[0083] in, is the balance coefficient, indicating the modal The balance requirement between rate and generation resources is used to adjust the generation rate in real time during the generation process; c is the current configuration, which indicates the actual state of the system at a certain moment; c′ is the changing configuration, which indicates the configuration that may change during time evolution or system adjustment, and is used for dynamic optimization and real-time adjustment.

[0084] Step 102: By combining a large-scale language model and a reinforcement learning algorithm, the original multimodal data is annotated and the data processing quality is evaluated, and the generated target multimodal data is iteratively optimized according to the data processing quality.

[0085] Combining large-scale language models and reinforcement learning, LLM is used for preliminary data processing and annotation, and reinforcement learning algorithms are used for data iteration optimization. In each data iteration, the system uses the following formula to evaluate the quality of data processing Q(s,a), using reinforcement learning and large-scale language models to improve the intelligence of data processing and reduce dependence on manual annotation and decision-making. Multimodal weights, dynamic adjustments, global and local reward mechanisms are comprehensively considered to adapt to the needs of multimodal data and multi-task environments.

[0086] ;

[0087] in:

[0088] Represents the current state. This is the input in reinforcement learning, usually a representation or feature from the environment, which may be part of multimodal data.

[0089] Indicates the current action. This is in the state Under this condition, the agent takes actions that aim to optimize long-term returns.

[0090] Represents a level or modality in a multimodal or multitask environment. This index can be used to distinguish different modalities or tasks, each of which may have different reward functions or features.

[0091] Indicates the number of modal levels or tasks. For example, This may correspond to the number of tasks, or the number of layers of different modalities (e.g. vision, language, audio, etc.).

[0092] Indicates the number of modes. In the reward function, this value represents the number of all modes and is used to calculate the impact of global rewards or state transitions.

[0093] Indicates the next state, that is, executing the action After that, the environment is transferred to the new state.

[0094] Indicates the next action, that is, in state Next, the action taken by the agent.

[0095] represents the weighting factor. It controls the current The balance between the current reward and the update calculated based on the current reward and future rewards. The larger the value, the current The larger the proportion during the update.

[0096] Represents the discount factor, which is used to calculate the discounted value of future rewards. It controls the importance of future rewards relative to current rewards, and its value is usually between In the range, smaller values ​​mean that the agent focuses more on short-term rewards.

[0097] in It is a multi-level reward mechanism function that combines local and global objectives to improve the capture of complex features:

[0098] ;

[0099] in is a local reward used to weigh short-term benefits. is the local modal weight, is the first The reward function of the modality; M is the number of modalities in each layer; R global ( s,a ) is the global reward, through the global factor Adjustments ensure long-term optimization, defined as incremental changes in the effects of environmental data iterations.

[0100] For modal The adaptive weight value uses an entropy-based weight adjustment mechanism to ensure that the model runs stably under noisy or uneven data conditions:

[0101] ;

[0102] in, For modal The importance weight of For the The entropy value of the modality, which represents the information density, is used to dynamically adjust the impact of the modality on decision making.

[0103] For multi-layer nested Q-value estimation, dynamic adjustment is made through the attention mechanism to capture important features in each modality:

[0104] ;

[0105] in, is the attention score, is the weight matrix extracted from modal features, For the Tier Q-value estimation of the mode. Representative Level, Modal in state The feature extraction function under is used to The corresponding modal data is processed to extract features that are valuable for decision-making, thereby dynamically adjusting the model's attention to different modalities.

[0106] is the policy balance factor, and Bayesian policy optimization is used to optimize the balance between modalities for a multi-task environment:

[0107] ;

[0108] in, is the balance parameter, k is the number of tasks, For the The priority of tasks is dynamically adjusted according to their importance, ensuring that the model can perform multimodal tasks more effectively.

[0109] The specific process of LLM in data processing and annotation is as follows:

[0110] 1. Data cleaning

[0111] 1.1 Automated Data Cleaning Strategy

[0112] Text data cleaning: Use LLM to perform semantic understanding and context analysis, extract text information related to the target task from multimodal data, and filter out noise data.

[0113] Example: Use the prompt "Extract key sentences describing scene changes in a video" to extract scene descriptions.

[0114] Visual data cleaning: LLM is combined with image / video description models to automatically mark and remove low-quality image frames (such as blur, poor lighting, etc.) through natural language prompts, and supplement missing frames based on contextual logic.

[0115] Cross-modal association verification: For multimodal data such as video, text, and audio, the semantic alignment capability of LLM is used to analyze the consistency between modalities and eliminate data that does not conform to logic or task objectives.

[0116] 1.2 Dynamic Rules Engine

[0117] LLM is used to generate and optimize dynamic rule sets to guide the data cleaning process.

[0118] Example: Generate rules based on user descriptions, such as “exclude all clips containing irrelevant sounds”, and dynamically adjust rule parameters using reinforcement learning.

[0119] 2. Data Annotation

[0120] 2.1 Automatic annotation generation

[0121] LLM-based natural language generation (NLG) capability: Generate high-quality tags and metadata descriptions for video data.

[0122] Example: Generate a description based on the video content: "The scene takes place outdoors at night, and two people are talking."

[0123] Scene-based hierarchical annotation: LLM uses hierarchical semantic understanding to perform hierarchical annotation of complex scenes (such as "character action > dialogue content > background information"), and interacts with the image model to generate accurate annotations on the timeline.

[0124] 2.2 Self-supervised annotation optimization

[0125] On the existing labeled data, LLM is used to generate enhanced prompt words, perform reasoning annotation on the unlabeled data, and optimize the annotation accuracy based on modal consistency check.

[0126] Example: For a multi-voice video with subtitles, optimize the annotation results using translation prompts generated from the subtitles.

[0127] 2.3 Interactive Annotation Verification

[0128] Provides LLM-based annotation error correction function, allowing users to enter prompt words to quickly verify or revise annotation content.

[0129] Example: The user enters "Fix all incorrect subtitle timelines", and the system automatically adjusts the subtitle annotations to synchronize with the video.

[0130] 2.4 Data entropy increase optimization

[0131] LLM is used to identify information redundancy and entropy increase points in multimodal data, and to improve data transmission efficiency through semantic clustering and optimization of audio, video, and text content.

[0132] Example: Analyze recurring background noise or subtitles and automatically compress the relevant data.

[0133] Step 103: Calculate the priority of each mode according to the interdependence of multimodal data traffic, adaptively adjust the multimodal data transmission bandwidth according to the priority, and automatically adjust the traffic allocation based on the data transmission bandwidth to achieve data transmission management of multimodal data streams.

[0134] Specifically, the data proxy module is used to manage, dynamically route and monitor multimodal data flows. It analyzes the system's throughput (t) and load in real time to ensure smooth data flow. Through real-time load balancing, adaptive bandwidth allocation and entropy increase optimization mechanisms, it efficiently manages multimodal data flows and ensures data transmission stability.

[0135] ;

[0136] The bandwidth weighting item dynamically adjusts the data bandwidth weight based on the modality priority:

[0137] ;

[0138] in, is the adjustment parameter, n is the number of modes, Indicates Prioritize the mode to automatically adjust traffic distribution based on the current bandwidth load; Indicates the number of modal levels or tasks; is the total time; Indicates time At the moment in the submodule The modal data traffic.

[0139] For modal The data flow priority adjustment parameters are used to calculate the priority according to the dependency of the modal data flow and to dynamically adjust the weights between the modalities:

[0140] ;

[0141] in, is the dependence coefficient between modes, reflecting other modes The strength of the reliance on the current modality; For the modal data flow of the modal; For the modal data flow of the modal; Indicates the total time. Indicates the total number of modes. Indicates the index of the other modality to be compared. All modes In time The maximum flow within is used as a normalization term to balance the priority. represents the time variable in the time integral and is between 0 and T.

[0142] is the modal weight distribution. Multimodal weights are used to adjust the data flow strength of different submodules to ensure balanced resource allocation:

[0143] ;

[0144] in, For submodules In time For modal The flow demand, k represents the number of sub-modules, and n represents the number of modes.

[0145] It is the entropy increase optimization item. Based on the optimization mechanism of information entropy, it adaptively controls the entropy increase of the modal data flow to ensure that each modal data maintains fluidity and reduces unnecessary noise flow. It is defined as:

[0146] ;

[0147] in, represents the real-time information entropy of the modal data stream, is the maximum information entropy, which is used to control the purity of each modal data stream. Indicates time At the moment in the submodule The modal The data flow reflects the sub-module In a specific mode The amount of real-time data transmission on the submodule is used to describe With modal The data exchange intensity between them is the core traffic measurement indicator in the entire bandwidth allocation and entropy increase optimization process.

[0148] It is a real-time load balancing item used to balance the load of each modal data stream. It smoothes the data stream intensity through the convolution filter to keep the load stable. It is defined as:

[0149] ;

[0150] in, is the convolution kernel function, is the window length. This item is used to smooth the output when the data flow changes suddenly, ensuring load balancing.

[0151] The sidecar proxy module captures and forwards all multimodal data flows passing through the node. The control plane aggregates the modal data flow statistics uploaded by all sidecars and calculates the global , evaluate the transmission performance of each modal data of the system. Calculation results, combined with the priority of each mode and dependencies , dynamically generate traffic control strategies, including priority lists and bandwidth allocation weights and routing path adjustment rules. Determine the traffic distribution weight of each mode. When the mode priority increases or the dependency increases, the sidecar proxy adjusts the route to a better path.

[0152] Ensure that each sidecar receives real-time updated control parameters. Adjust the priority of data streams in real time according to the global policy issued by the control plane ( )、Bandwidth Allocation( ) and routing, dynamically adjusting the traffic distribution on the current transmission path. The sidecar calculates the information entropy of the modal flow locally , combined with the entropy increase optimization result of sidecar feedback , take flow or speed limiting measures for modes with severe noise traffic, impose restrictions on low entropy (high noise) traffic, and ensure traffic purity. Locally monitor data flow status and collect throughput-related indicators (such as actual traffic , packet loss rate, latency) and feeds back key indicators to the control plane for further optimization. At the same time, the sidecar performs local optimization measures based on the current traffic status, such as through convolutional smoothing mechanism ( ) to alleviate short-term load fluctuations.

[0153] For example, when the control plane detects that the throughput of a certain mode is far below the expected value, it triggers routing reconfiguration to dynamically adjust the transmission path of the mode or increase its priority. You can adjust the throughput formula parameters (such as ), optimize system performance.

[0154] It can be seen that the multimodal data processing method based on large language model and reinforcement learning provided by this application has the following advantages compared with related technologies:

[0155] Through the modular data generation and configuration interface, the data generation process is simplified, the need for manual configuration and manual labeling is reduced, and processing efficiency is improved; by integrating large language models and reinforcement learning, the system achieves automation and intelligence in data flow optimization and resource allocation, improving system response speed; the data proxy module effectively optimizes the transmission and scheduling of multimodal data, ensuring the stable operation of the system in complex environments while improving data transmission efficiency.

[0156] Accordingly, the present application provides, in a second aspect, a multimodal data processing device based on a large language model and reinforcement learning, comprising:

[0157] A data generation unit, configured to receive user input information in a data generation and configuration interface, determine configuration information for data generation according to the user input information and import the original multimodal data, and dynamically determine a data generation path based on the configuration information to generate target multimodal data;

[0158] A data iteration optimization unit, used to annotate the original multimodal data and evaluate the data processing quality by combining a large-scale language model and a reinforcement learning algorithm, and iteratively optimize the generated target multimodal data according to the data processing quality;

[0159] A data transmission unit is used to calculate the priority of each mode according to the interdependence of multimodal data traffic, adaptively adjust the multimodal data transmission bandwidth according to the priority, and automatically adjust the traffic allocation based on the data transmission bandwidth to achieve data transmission management of multimodal data streams.

[0160] The above-mentioned device can be implemented by the multimodal data processing method based on a large language model and reinforcement learning provided by the embodiment of the first aspect above. The specific implementation method can be found in the description in the embodiment of the first aspect, and will not be repeated here.

[0161] It is understood that the structures, names and parameters described in the above embodiments are only examples. Those skilled in the art can also easily think of combinations and adjustments to the structural features of the above embodiments according to the use requirements, and should not limit the concept of the present application to the specific details of the above examples.

[0162] The present application also provides an electronic device, including a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is used to read the instructions and execute any one of the methods in the aforementioned first aspect. The processor and the memory may be connected via a bus or other means, taking the bus connection as an example. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned chips.

[0163] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0164] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, wherein the plurality of instructions can be read by a processor and execute any one of the methods in the aforementioned first aspect. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0166] Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that he or she may still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multimodal data processing method based on a large language model and reinforcement learning, characterized in that: include: Receiving user input information in a data generation and configuration interface, determining configuration information for data generation according to the user input information and importing original multimodal data, and dynamically determining a data generation path based on the configuration information to generate target multimodal data; By combining a large-scale language model and a reinforcement learning algorithm, the original multimodal data is annotated and the data processing quality is evaluated, and the generated target multimodal data is iteratively optimized according to the data processing quality; Calculate the priority of each mode according to the interdependence of multimodal data traffic, adaptively adjust the multimodal data transmission bandwidth according to the priority, and automatically adjust the traffic allocation based on the data transmission bandwidth to achieve data transmission management of multimodal data streams; The dynamically determining the data generation path based on the configuration information further comprises: Adopt the rule engine and use the following formula to dynamically determine the path of data generation D(s,c) : ； in, is the balance coefficient of mode j; k represents the number of modal levels or tasks, n is the number of modes, Q is the normalization factor, is the total time, Status and configuration The data generation weights are: For modal The adaptive generation rate of is the multi-layer data quality control parameter, For configuration Dynamic weight optimization term, A generation balance function for balancing generation rate and resource allocation during generation; The status is the current state of the system, representing the task context of the data generation process, including environmental information and operating conditions, wherein the environmental information and operating conditions are the system's operating state, configuration conditions, resource allocation, data generation purpose, system state, data volume, time requirements, and generation environment characteristics; ； Indicates the configuration of stage i and status Lower mode The quality control requirements are used to automatically optimize the resource allocation at different stages; n represents the number of modalities; k represents the total number of data generation stages; The data generates weights It is expressed as: ； in, For modal The priority weight of Representing modality In Status and configuration The complexity of data generation under The generated balance function It is expressed as: ； in, is the balance coefficient, indicating the modal Balancing needs for rate and generation resources; is the current configuration, which indicates the actual state of the system at a certain moment; c′ is the changing configuration, which indicates the configuration that may change during time evolution or system adjustment, and is used for dynamic optimization and real-time adjustment.

2. The multimodal data processing method based on large language model and reinforcement learning according to claim 1, characterized in that: The evaluating the data processing quality further includes using the following formula to evaluate the quality Q(s,a) of the data processing: ； in is the multi-level reward mechanism function, is the strategy balance factor, For multi-layer nested Q-value estimation, For modal The adaptive weight value of Indicates the current status; Indicates the current action; Representing hierarchies or modalities in multimodal or multitask environments; Indicates the number of modal levels or tasks; Indicates the number of modes; Indicates the next state; Indicates the next action. represents the weighting factor; Represents the discount factor.

3. The multimodal data processing method based on large language model and reinforcement learning according to claim 2, characterized in that: The multi-level reward mechanism function It is expressed as: ； in Used to weigh short-term gains, is the local modal weight, is the first The reward function of the modality; M is the number of modes in each level, R global ( s,a ) is the global reward, through the global factor Adjustment; The adaptive weight value Use an entropy-based weight adjustment mechanism: ； in, For modal The importance weight of For the The entropy value of the modality, which represents the information density, is used to dynamically adjust the impact of the modality on decision making.

4. The multimodal data processing method based on large language model and reinforcement learning according to claim 3, characterized in that: The multi-layer nested Q value estimation Through dynamic adjustment of the attention mechanism, important features in each modality are captured: ； in, is the attention score, is the weight matrix extracted from modal features, For the Tier Q-value estimation of the mode.

5. The multimodal data processing method based on large language model and reinforcement learning according to claim 1, characterized in that: The step of adaptively adjusting the multimodal data transmission bandwidth according to the priority further includes: Real-time analysis of system throughput (t): ； in is the data flow priority adjustment parameter for mode j, is the bandwidth weighting term, is the modal weight distribution, is the entropy increase optimization term, It is a real-time load balancing item; Indicates the number of modal levels or tasks; is the total time; express At the moment in the submodule Modal data traffic.

6. The multimodal data processing method based on large language model and reinforcement learning according to claim 5, characterized in that: The data stream priority adjustment parameter of mode j for: ； in, is the dependence coefficient between modes, For the modal data flow of the modal; For the modal data flow of the modal; Indicates the total time; Indicates the total number of modes; Indicates the index of the other modality for comparison; represents the time variable in the time integral and is between 0 and T.

7. The multimodal data processing method based on large language model and reinforcement learning according to claim 6, characterized in that: The entropy increase optimization term Based on the optimization mechanism of information entropy, the entropy increase of the modal data stream is adaptively controlled: ； in, represents the real-time information entropy of the modal data stream, is the maximum information entropy; Indicates time At the moment in the submodule The modal Data traffic volume; The real-time load balancing item It is used to balance the load of each modal data stream and smooth the data stream intensity through the convolution filter: ； in, is the convolution kernel function, is the window length.

8. A multimodal data processing device based on a large language model and reinforcement learning, characterized in that: include: A data generation unit, configured to receive user input information in a data generation and configuration interface, determine configuration information for data generation according to the user input information and import the original multimodal data, and dynamically determine a data generation path based on the configuration information to generate target multimodal data; A data iteration optimization unit, used to annotate the original multimodal data and evaluate the data processing quality by combining a large-scale language model and a reinforcement learning algorithm, and iteratively optimize the generated target multimodal data according to the data processing quality; A data transmission unit, configured to calculate the priority of each mode according to the interdependence of multimodal data traffic, adaptively adjust the multimodal data transmission bandwidth according to the priority, and automatically adjust the traffic allocation based on the data transmission bandwidth to achieve data transmission management of multimodal data streams; The dynamically determining the data generation path based on the configuration information further comprises: Adopt the rule engine and use the following formula to dynamically determine the path of data generation D(s,c) : ； in, is the balance coefficient of mode j; k represents the number of modal levels or tasks, n is the number of modes, Q is the normalization factor, is the total time, Status and configuration The data generation weights are: For modal The adaptive generation rate of is the multi-layer data quality control parameter, For configuration Dynamic weight optimization term, A generation balance function for balancing generation rate and resource allocation during generation; The status is the current state of the system, representing the task context of the data generation process, including environmental information and operating conditions, wherein the environmental information and operating conditions are the system's operating state, configuration conditions, resource allocation, data generation purpose, system state, data volume, time requirements, and generation environment characteristics; ； Indicates the configuration of stage i and status Lower mode The quality control requirements are used to automatically optimize the resource allocation at different stages; n represents the number of modalities; k represents the total number of data generation stages; The data generates weights It is expressed as: ； in, For modal The priority weight of Representing modality In Status and configuration The complexity of data generation under The generated balance function It is expressed as: ； in, is the balance coefficient, indicating the modal The balance requirement for rate and generation resources; c is the current configuration, which indicates the actual state of the system at a certain moment; c′ is the changing configuration, which indicates the configuration that may change during time evolution or system adjustment, and is used for dynamic optimization and real-time adjustment.

9. An electronic device, characterized in that: It includes a processor and a memory, the memory stores multiple instructions, and the processor is used to read the instructions and execute the multimodal data processing method based on a large language model and reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor and executed by the multimodal data processing method based on a large language model and reinforcement learning as described in any one of claims 1 to 7.

Images