Multi-modal large model adaptive iterative reasoning method and system for long video

Abstract

The invention discloses a multi-modal large model adaptive iterative reasoning method and system for a long video, and belongs to the technical field of multi-modal artificial intelligence and reinforcement learning training. According to the method, a self-adaptive multi-round iterative reasoning framework is adopted, a model autonomously selects a direct answer according to a current reasoning state, and fine-grained time positioning or representation optimization of self-extraction visual clues is executed. Wherein in the fine-grained time positioning, a key time slice related to a task is dynamically selected for an output target interval through discrete marking; according to representation optimization, visual clues are extracted and injected into visual representation, and the model is supported to integrate dispersed multi-modal information in a long time range. Meanwhile, a reinforcement learning framework based on an entropy guided branch exploration mechanism and a composite reward mechanism is adopted, and an adaptive decision strategy of the model is enhanced. The method does not depend on a specific model structure, has good universality and expansibility, and can significantly improve reasoning integrity, stability and overall efficiency in a long video understanding task.

Description

Technical Field

[0001] This invention relates to the fields of multimodal large language models (MLLMs) and reinforcement learning, and particularly to an adaptive iterative inference method and system for multimodal large models for long videos. Background Technology

[0002] With the rapid development of artificial intelligence technology, multimodal large models have been widely used in video understanding, intelligent question answering, and human-computer interaction. In practical applications, the duration of video data is constantly increasing, and long videos are gradually becoming the mainstream data format, such as teaching videos, meeting minutes, documentaries, film and television content analysis, and multi-scene interactive videos. These long videos are usually long in duration, have complex content structures, contain multiple semantic stages, and frequent scene changes, which places higher demands on the model's temporal modeling ability, visual understanding ability, and cross-modal reasoning ability. Compared with static image or short video tasks, long video understanding faces more severe technical challenges. On the one hand, key information related to the user's question in long videos is usually scattered at different time locations, and a large amount of video content is irrelevant to the target task. The model needs to accurately locate relevant segments in a long time span and with highly redundant information. On the other hand, the scenes within long videos change frequently and the semantic span is large. A single visual encoding often makes it difficult to form a stable and question-related multimodal representation, thus affecting the accuracy of subsequent reasoning and answer generation.

[0003] Most existing long-video understanding methods employ a single-pass processing paradigm, where key information retrieval, visual representation, and question answering are completed in a single forward inference process. These methods typically rely on fixed video sampling strategies or predefined temporal segmentation methods, determining the video content to be understood before inference begins. However, in long-video scenarios, this approach struggles to guarantee that the initially selected video segments contain all key information, easily overlooking important temporally dispersed evidence, leading to incomplete or erroneous inference results.

[0004] To improve long-form video understanding, some studies have attempted to introduce temporal localization mechanisms to locate relevant segments within the video. However, existing temporal localization methods largely rely on timestamp regression or fixed window matching, which not only involves complex training processes but also has limited generalization capabilities, making it difficult to handle complex scenarios where multiple key events are distributed across different time periods. Regarding the construction of visual representations, existing methods typically encode the video content holistically in the absence of problem constraints, generating relatively general visual features for subsequent reasoning. However, this one-off, problem-agnostic encoding approach fails to fully extract discriminative information from the visual content that is highly relevant to the specific reasoning problem. For long videos or complex scenarios, different problems often focus on different spatiotemporal regions and semantic cues, and static, uniform visual representations struggle to model key visual evidence specifically based on problem semantics, thus limiting the model's expressive power and reasoning accuracy in complex visual reasoning tasks. Furthermore, some studies have attempted to introduce multi-turn reasoning or external tool invocation mechanisms, but these methods mostly rely on manually set fixed inference rounds or heuristic rules, lacking the ability to dynamically adjust the inference depth based on problem complexity. In practical applications, this fixed strategy can easily lead to two extreme situations: insufficient reasoning for complex problems, while causing unnecessary computational overhead for simple problems, thus affecting the overall efficiency of the system.

[0005] In recent years, reinforcement learning has been gradually applied to the inference optimization of multimodal models to guide the model to learn better strategies in multi-step decision-making processes. However, existing reinforcement learning methods still have shortcomings in long video understanding scenarios, such as a lack of fine-grained feedback for different inference actions, difficulty in balancing inference performance and computational cost, and the tendency to encounter instability or ineffective exploration during the inference process.

[0006] In summary, how to achieve precise localization of key time segments and adaptive enhancement of visual representations in long video understanding tasks, while effectively controlling inference costs while ensuring inference accuracy, remains a crucial technical problem that urgently needs to be solved in the field of multimodal large models. Summary of the Invention

[0007] This invention addresses the shortcomings of existing technologies by proposing a multimodal large-scale adaptive iterative inference method and system for long-video understanding. To address the problems in existing long-video understanding methods, such as scattered evidence fragments, strong cross-time-period dependencies, and the difficulty in extracting key semantics related to the problem from visual encoding, this invention employs a multi-round iterative inference framework to dynamically optimize model input and visual representations through multiple rounds of inference interaction. Furthermore, it introduces two types of controllable "intrinsic skills" during the inference process—fine-grained temporal localization and visual representation optimization that self-extracts visual cues. Simultaneously, in reinforcement learning training, it integrates entropy-guided reinforcement learning training and a composite reward mechanism based on a process reward model, forming a reinforcement learning training method that enables the model to adaptively decide whether to continue localization and whether to trigger visual representations based on uncertainty, thereby improving inference robustness and computational efficiency while maintaining accuracy.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] In a first aspect, this invention proposes an adaptive iterative inference method for multimodal large models oriented towards long videos, wherein the multimodal large model includes at least a visual encoder and a large language model, and the method includes:

[0010] For a given long video and text question, multiple rounds of iterative reasoning are performed. In each round of reasoning, the multimodal large model adaptively selects and executes one of three actions based on the current reasoning state: directly generating the final answer, performing fine-grained temporal localization, or performing visual representation optimization by extracting visual cues. The current reasoning state includes the video input, the text question, and decision information from previous rounds.

[0011] When the model chooses to directly generate the final answer, it outputs the final reasoning result;

[0012] When the model selects to perform fine-grained temporal localization, it outputs one or more time intervals represented by discrete time marker pairs; and based on the time intervals, it extracts the corresponding video segments from the video input of the current round and updates the video input.

[0013] When the model chooses to perform visual representation optimization actions that extract visual cues, it generates visual cues that are semantically related to the current text question and injects the visual cues into the high-level encoding process of the visual encoder in the next round of inference.

[0014] We employ a reinforcement learning framework based on entropy-guided branching and a compound reward mechanism to optimize the model's adaptive action selection strategy.

[0015] Furthermore, the decision information for each round refers to the action type and its parameters selected by the model in that round.

[0016] Furthermore, the output of one or more time intervals represented by discrete-time marker pairs includes:

[0017] Map the timeline of the input video to a normalized percentage range;

[0018] The percentage interval is discretized into a hierarchical label set, which includes a first-layer label for representing coarse-grained intervals and a second-layer label for representing fine-grained offsets. A first-layer label and a second-layer label are combined to form a label pair. The first-layer label determines the coarse-grained interval, and the second-layer label specifies the offset within the interval, together representing a specific time position point.

[0019] The time interval is represented by a start position marker pair and an end position marker pair, and all time intervals serve as action parameters for fine-grained time positioning actions.

[0020] Furthermore, the process of generating the visual cues includes:

[0021] When the model determines that visual cues need to be generated, the large language model decodes and generates a token limited to the start and end labels as an action parameter; the hidden layer vector corresponding to the action parameter is extracted from the hidden layer of the large language model, and this set of vectors constitutes the visual cue.

[0022] Furthermore, the visual cues are injected into the high-level encoding process of the visual encoder in the next round of reasoning, including:

[0023] The visual cues are concatenated with the original input features of the i-th Transformer module in the visual encoder, where L-N+1≤i≤L, L is the number of Transformer modules in the visual encoder, and N is the number of Transformer modules into which the visual cues are injected.

[0024] The concatenated features are input into the i-th Transformer module for processing. The visual representation part corresponding to the original input feature position of the i-th Transformer module is extracted from the output features and used as the original input feature of the next Transformer module.

[0025] The visual representation output by the Lth Transformer module is used as the output feature of the visual encoder.

[0026] Furthermore, the entropy-guided branching exploration mechanism includes:

[0027] Using the entropy value of the model's output distribution at the initial stage of inference as a benchmark, monitor the change in the entropy value of the output action parameter distribution when the model performs fine-grained temporal localization or visual representation optimization actions that extract visual cues.

[0028] When the entropy value changes beyond a preset threshold, the model is determined to be in a state of high uncertainty. The current inference trajectory is copied and expanded into two independent branches for parallel exploration, forming multiple candidate inference paths.

[0029] Furthermore, the composite reward mechanism includes a correct answer reward, a temporal location quality reward based on the degree of overlap between the predicted time interval and the actual time interval, a visual representation quality reward based on the semantic relevance between visual cues and textual questions, and a format reward based on whether the model output follows a predetermined format.

[0030] Secondly, this invention proposes a multimodal large model adaptive iterative inference system for long videos, which is used to implement the above-mentioned multimodal large model adaptive iterative inference method.

[0031] Thirdly, the present invention proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor, when executing the computer program, implements the above-mentioned multimodal large model adaptive iterative inference method for long videos.

[0032] Fourthly, the present invention proposes a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the above-mentioned multimodal large model adaptive iterative inference method for long videos.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] This invention introduces a fine-grained temporal localization mechanism and a visual representation optimization mechanism that extracts visual cues in multimodal large-scale adaptive iterative reasoning for long videos. The fine-grained and interpretable temporal localization mechanism adapts to the temporal localization of videos of arbitrary length, effectively improving the coverage of key evidence in long videos and reducing interference from irrelevant frames. The visual representation optimization mechanism enables the model to strengthen high-level semantic information related to the current reasoning objective, improving understanding performance in complex reasoning tasks. Furthermore, a reinforcement learning training framework based on entropy-guided and composite reward mechanisms is proposed. This allows the model to adaptively adjust the number of reasoning rounds according to the complexity of the question and the characteristics of the video content, avoiding ineffective or redundant reasoning behavior, and training the model to invoke various skills in a timely manner, improving the overall system efficiency while ensuring reasoning effectiveness. This invention effectively reduces reasoning errors caused by insufficient initial evidence selection, improving the accuracy of responses and the stability of reasoning in long video understanding tasks. Attached Figure Description

[0035] Figure 1 This is a flowchart of the multimodal large model adaptive iterative inference framework of the present invention;

[0036] Figure 2 This is a schematic diagram of the multimodal large model adaptive iterative inference of the present invention;

[0037] Figure 3 This is a schematic diagram of the fine-grained time positioning of the present invention;

[0038] Figure 4 This is a schematic diagram illustrating the visual representation optimization of the self-extracted visual cues according to the present invention.

[0039] Figure 5 This is a schematic diagram of the entropy-guided branch exploration control mechanism of the present invention;

[0040] Figure 6 This is a schematic diagram of the composite reward calculation mechanism based on the process reward model of the present invention. Detailed Implementation

[0041] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0042] This invention proposes an adaptive iterative inference method for long-form videos using a multimodal large-scale model. The multimodal large-scale model includes a visual encoder for processing raw video frames, a projection layer for aligning visual features with textual features, and a large language model responsible for semantic understanding and inference decisions. These components can all be implemented using existing, mature structures and models in the field, and their specific working principles are not detailed here. This invention, based on the multimodal large-scale model architecture, enables the model to dynamically adjust its inference focus and representation method through multiple rounds of iteration and feedback.

[0043] The specific steps are as follows:

[0044] (1) Adaptive multi-stage inference optimization based on inference feedback

[0045] like Figure 1 and Figure 2 The framework flow and reasoning diagram shown are illustrated in this reasoning framework, which is based on rounds. It operates in an iterative manner, where in each round of inference, the input information received by the model comes from the result of the previous round of optimization. When the original input information is received, one of the three decision actions—direct response, time positioning, and visual representation optimization—is selected to be executed, thereby achieving progressive optimization of the input and reasoning states. This process specifically includes the following two sub-steps.

[0046] (1-1) Adaptive policy decision

[0047] The inference state received by the model at the start of round t of inference. (The input information for the current round) consists of the following three parts:

[0048] Video input for the current round Its content may be the original long video, or it may be a collection of key segments obtained after previous rounds of screening or extraction.

[0049] Text input problem : Text questions submitted by users that require answers.

[0050] Decision information from historical cycles Records all previous rounds (round 1 to round 2). The decision information for the wheel includes the selected action and its action parameters.

[0051] Based on the above reasoning state The multimodal large model will generate the inference representation for the current round. This is used to guide subsequent strategy selection and answer generation, and at the same time, This information will be appended to the history record and used to update the decision information for the next round of input, i.e., the [number]th [round]. The historical decision information of the wheel is updated to .

[0052] (1-2) Decision execution and input update

[0053] Inference representation of the current round It includes the action selected by the model in the current round. And its action parameters, the action value can only be one of the following three decisions, each decision corresponds to a different execution logic and update method for the next round of input:

[0054] (1-2-1) Answer directly

[0055] The model will select this action when it determines that the current input contains sufficient and consistent visual information to support problem reasoning. Upon execution, the model first outputs... <answer>The label indicates the action to be selected. Then output the final answer directly, and in< / answer> The label indicates the end of the answer. This terminates the multi-round reasoning process, ending further iterations. The model is currently in the [stage / phase]. <answer> and< / answer> The generated content within the tags will serve as the final answer to the user's question.

[0056] (1-2-2) Time positioning

[0057] This action is selected when the model determines that the current video input still contains a large amount of irrelevant content, or that key evidence is scattered over time and has not yet been effectively focused. During execution, the model outputs... <segstart>The label indicates the action to be selected. Then, time interval representations of one or more video segments are generated, as detailed in step (2).

[0058] In this case, the time localization module uses the time interval representation output by the model to locate the input video of the current round t. Edit the video, extract the key video clips, and combine these clips into a new video input. Input the updated video As the first Video input for round-robin reasoning.

[0059] (1-2-3) Visual Representation Optimization

[0060] This action is selected when the model determines that the currently focused video segment is semantically insufficient to support accurate inference. During execution, the model outputs... <represtart>The label indicates the action to be selected. Then, visual clues related to the semantics of the question are generated, as detailed in step (3).

[0061] In this case, the visual cues output by the model It will be in the This serves as crucial supplementary information in round-robin reasoning. Specifically, it is used in the analysis of video input. Visual cues during the encoding process It will be injected into the subsequent high-level coding layers of the visual encoder, thereby obtaining a more semantically prominent visual representation.

[0062] (2) Fine-grained time positioning

[0063] like Figure 3 As shown, the fine-grained temporal localization module is used to accurately locate key segments in the video that are relevant to the current problem from the long time sequence during the multimodal reasoning process of long video, and to provide focused video input for subsequent reasoning.

[0064] The specific steps are as follows:

[0065] (2-1) Normalized Time Axis and Discrete Interval Encoding

[0066] To avoid the instability and generalization difficulties caused by directly regressing absolute timestamps under different video durations, this embodiment normalizes the timeline of the input video and further performs discrete interval encoding.

[0067] Specifically, assuming the total duration of the input video V is T, this embodiment maps the video timeline to a uniform percentage space. This represents the percentage of a relative position within the entire video at any given time. Furthermore, this percentage space is discretized into several predefined timestamps to describe candidate time segments using discrete symbols rather than second-level timestamps.

[0068] This embodiment uses hierarchical discrete tags for encoding, including:

[0069] The first layer is coarse-grained interval labeling. Its set is { <t0> , <t1> ,…, <t9>Each marker corresponds to a tenth of the total video duration, representing a portion of the video's length. Interval.

[0070] Second layer: Fine-grained offset markers. Its set is { <u0> , <u1> ,…, <u9>} represents a further tenths shift within the current coarse-grained range.

[0071] Any time location can be represented by a pair of time markers. <ti> <uj>The only certainty is that its corresponding video time percentage is:

[0072]

[0073] Where i and j correspond to the indices of coarse and fine-grained tags, respectively, for example, tag pairs <t1> <u3>The corresponding percentage position p = 10 × 1 + 3 = 13, which is 13% of the video.

[0074] Using the discrete interval representation method described above, any segment in a video of any duration can be uniformly represented using discrete time markers of "start position - end position" without relying on specific second-level timestamps.

[0075] (2-2) Adaptive time interval prediction

[0076] In this invention, the generation of time intervals is not a fixed preprocessing step, but is modeled as an optional decision-making behavior in the multimodal reasoning process, which is autonomously decided by the model during the reasoning process.

[0077] (2-2-1) Location Trigger Judgment

[0078] During the t-th round of inference, the model is based on the inference state. Make a decision. When the model deems it necessary to obtain further key evidence fragments from the video, it generates a specific starting label. <segstart>This indicates that the subsequent generated results are used for time interval positioning.

[0079] (2-2-2) Generation of time intervals

[0080] When a localization action is triggered, the model outputs one or more time intervals. Each time interval is generated using explicit start and end markers, with a basic format such as:

[0081]

[0082] in, Indicates the starting position of the interval. Indicates the end position of the interval.

[0083] When dealing with long videos where evidence is scattered, the model allows for the generation of multiple (K) time intervals in a single inference:

[0084]

[0085] To cover multiple discontinuous key segments. Located at the start label. <segstart>and closing tags <segend>The content between these parameters represents the motion parameters for the time-based positioning action.

[0086] By transforming time-based localization into a discrete marker sequence generation problem, this invention makes the prediction process of time intervals naturally compatible with the autoregressive generation mechanism of language models, thereby improving the stability and interpretability of long video localization.

[0087] (2-3) Time-based segment extraction and input construction

[0088] After the model generates one or more time intervals in the t-th round of inference, the system will crop segments of the original video according to the time intervals to construct the refined video input required for the next round of inference.

[0089] Specifically, for the k-th time interval generated by the model, the corresponding video time range is:

[0090]

[0091] in, and These are the percentage values ​​corresponding to the start and end positions of the interval, respectively.

[0092] Convert the above percentages to timestamps, and then trim the corresponding segments from the original video to obtain a set of refined video clips:

[0093]

[0094] To support the next round of reasoning, the system reconstructs the extracted video segments, constructs a refined video input, replaces the original long video, and uses it together with the original question text and decision information from previous rounds as the next round's input. Input of the wheel.

[0095] (3) Optimization of visual representation based on self-extracted visual cues

[0096] The multimodal reasoning model autonomously generates visual cues related to the problem during multiple rounds of reasoning, and uses these cues to perform targeted optimization of video visual encoding in subsequent rounds, thereby obtaining an enhanced visual representation that is closer to the current reasoning goal.

[0097] (3-1) Visual cue generation

[0098] In this embodiment, the generation of visual cues is automatically completed by the multimodal reasoning model during the reasoning process, without the need for additional manual annotation or external rule setting.

[0099] (3-1-1) Clue generation trigger

[0100] In the During rounds of reasoning, the model is based on the reasoning state. Make a decision. When the model determines that further key evidence segments from the video are needed, it generates a specific start marker. <represtart>This indicates that the subsequent generated results serve as visual cues.

[0101] (3-1-2) Generation of visual cue vectors

[0102] The model is based on a predefined format in <represtart>and <repreend>The tags generate action parameters for the current action, serving as the basis for visual cues. Specifically, the hidden state vectors corresponding to the parameter tokens between the start and end tags are extracted from the hidden layers of the large language model. These vectors form visual cues to guide the updating of visual representations, represented as follows:

[0103]

[0104] in, Let M represent the m-th vector in the visual cue, where M is the number of vectors in the visual cue, which is fixed at 4 in this embodiment. <represtart>and <repreend>The number of tokens between tags is 4.

[0105] The visual cues are used to characterize the key visual semantic directions that the model focuses on under the current reasoning objective, so as to provide guiding information for subsequent visual recoding.

[0106] (3-2) Visual cue preservation

[0107] After the model completes the reasoning of the current round (round t) and outputs the visual cue, the visual cue C is saved and used as the external guiding signal input for the visual encoder in the next round of reasoning.

[0108] (3-3) Visual Representation Optimization Based on Visual Cues

[0109] like Figure 4 As shown, in the first In this embodiment, a visual encoder recoding method based on cue injection is used to obtain enhanced visual encoding.

[0110] Specifically, the visual encoder consists of multiple cascaded Transformer modules: the input to the first Transformer module is the sampled video frame, and the input to each subsequent Transformer module is the output of the previous module. Generally, the earlier modules focus on extracting low-level visual detail features, while the later modules focus on extracting abstract high-level visual semantic information.

[0111] Let the visual encoder be composed of A series of Transformer modules are connected together, the first... The inputs and outputs of each Transformer module are represented as follows:

[0112]

[0113] in, This represents the output sequence of the i-th Transformer module. This represents the input sequence of the i-th Transformer module, which is also the... The output sequence of each Transformer module, This represents the transformation operation of the i-th Transformer module.

[0114] To enhance high-level semantic alignment while preserving underlying details, this embodiment injects visual cues C into the last four Transformer modules of the visual encoder.

[0115] Specifically, in the last four Transformer modules: for the i-th module, the i-th module will... The output sequence of each module is the same as that of the previous round ( The visual cues output and saved from the first module are pieced together and used as input for the i-th module. :

[0116]

[0117] Subsequently, Transformation operations through the i-th Transformer module Obtain the output sequence It is generally believed that input Corresponding position It is the visual encoding actually obtained by the i-th module, and corresponds to the position of the visual cue C. It can be discarded directly.

[0118]

[0119] Therefore, Extract the processing result of the i-th module and use it for the i-th module. The inputs of each module are concatenated. In the output of the last module, the vector at the corresponding position is extracted according to the same rules as the final visual code of the video, that is, the enhanced visual code that is more relevant to the question text.

[0120] (4) Reinforcement learning training framework based on entropy-guided and compound reward mechanism

[0121] This invention provides a reinforcement learning training framework for multi-turn reasoning in long videos. Based on the GRPO algorithm, it expands candidate reasoning trajectories at key uncertain decision points through entropy-guided uncertainty monitoring and branch exploration control; and optimizes reasoning quality through a composite reward function, thereby improving reasoning accuracy and evidence localization capabilities while controlling overall reasoning cost.

[0122] (4-1) Entropy-guided branch exploration control mechanism

[0123] Entropy refers to "Shannon entropy" in information theory, which is used to quantify the uncertainty of a model's predictive distribution. When the model is very certain about the next token to be generated, its output predictive distribution is "sharp," and the entropy value is low; conversely, when the model is very uncertain about the next token, the predictive distribution is "flat," and the entropy value is high.

[0124] For each token generated in each step, the corresponding entropy value can be calculated using the following formula:

[0125]

[0126] in, It is the entropy of the token at position t. This represents the probability that the model predicts the v-th word at position t, where V is the vocabulary size.

[0127] (4-1-1) Calculate the initial entropy value

[0128] like Figure 5 As shown, the initial inference path is first copied into n / 2 branches, and each branch performs autoregressive decoding independently. The average entropy of the first 10 tokens generated at the start of inference in each branch of the model is used as the baseline entropy. And this data is kept unchanged during the training process as a comparison.

[0129] (4-1-2) Calculate the change in entropy when a skill is invoked.

[0130] Subsequently, each branch continues to decode independently. During the model's multi-round inference process, whenever "temporal localization" or "visual representation optimization" is triggered, the corresponding trigger label (e.g., ...) is output from the model. <segstart>or <represtart>The current entropy value is calculated in the same way in the token following the given information. Therefore, the change in entropy can be calculated:

[0131]

[0132] Among them, in calculation At that time, it will be divided by This is done by normalizing it to the range of 0-1. In this embodiment, if <segstart>or <represtart>If the number of tokens following a tag exceeds 10, only the first 10 tokens will be used.

[0133] If you encounter <answer>The identifier indicates that the branch will decode until generation stops.

[0134] (4-1-3) Entropy-guided branching exploration mechanism

[0135] When the entropy change ,in A preset threshold is used to determine that the model is currently in a state of high uncertainty. In this state, in order to cover more potential reasonable inference trajectories, the present invention will copy the current inference trajectory branch, expand it into two parallel branches, and allow each branch to independently continue decoding and generating subsequent content to form multiple candidate inference trajectories.

[0136] This process checks whether the total number of branches has reached the preset upper limit n. If it has not, new branches continue to participate in subsequent decoding and judgment loops; if it has, it stops replicating, meaning the branch exploration process terminates after reaching the preset maximum number of branches. Through this branch exploration mechanism, more possible reasoning and evidence retrieval paths can be covered at key uncertain nodes.

[0137] (4-2) Composite reward mechanism based on process reward model

[0138] like Figure 6 As shown, in order to guide the model to make reasonable use of time positioning and visual representation optimization in multi-round inference, this invention adopts a composite reward mechanism based on the Process Reward Model (PRM) to evaluate each candidate inference trajectory generated by the model.

[0139] (4-2-1) Reward for correct answers

[0140] For each complete inference trajectory generated by the model A reward is given based on the consistency between the final output answer and the labeled answer. This reward is defined as follows:

[0141]

[0142] (4-2-2) Timing Positioning Quality Bonus

[0143] When the model outputs video time intervals during inference to locate key segments, this invention evaluates the localization quality based on the degree of overlap between the predicted and actual time intervals. Let the set of predicted time intervals be P, and the set of actual time intervals be G. Then, the localization reward can be defined as a function based on the intersection-over-union (IoU) ratio:

[0144]

[0145] This reward encourages the model to output more accurate time intervals, thereby improving its ability to locate key information in long videos.

[0146] (4-2-3) Visual Representation Quality Award

[0147] To ensure that the visual cues generated by the model when triggering visual representation optimization operations are highly relevant to the task semantics, this invention introduces a visual representation quality reward. Specifically, the user's question text and the visual cues generated by the model are mapped to the same semantic space, and their similarity is calculated as a reward:

[0148]

[0149] in, and Represents a vectorized mapping function. This represents the question text provided by the user, and C represents the visual cue output by the model in the current round. This represents the function for calculating cosine similarity.

[0150] (4-2-4) Format Reward

[0151] For the model's output in round t, the following rewards are given based on whether the answer follows a fixed format rule:

[0152]

[0153] The standard format varies depending on the type of decision made by the model:

[0154] If the model's decision in the current round is a direct answer (detecting that the output contains...) <answer>The standard format requires that the answer content output by the model be included in the... <answer> and< / answer> In the tag.

[0155] If the decision of the current round model is time positioning (detecting that the output contains...) <segstart>The standard format requires that the action parameters corresponding to the time intervals output by the model be included in the standard format. <segstart>and <segend>Within a tag, and between two tags, the parameter token can only be drawn from the set { <t0> , <t1> ,…, <t9>}、{ <u0> , <u1> ,…, <u9>Choose from}

[0156] If the decision of the model in the current round is visual representation optimization (detecting that the output contains...) <represtart>The standard format requires that the action parameters corresponding to the visual cues output by the model be included in the model's output. <represtart>and <repreend>In the tag.

[0157] (4-2-5) Total Reward

[0158] The total reward for each answer is the weighted sum of all rewards received, that is:

[0159]

[0160] In this example, If a certain action type is executed multiple times, then the sum of the rewards for those multiple executions of that action type is taken.

[0161] (4-2-6) Estimation of relative advantage and calculation of loss function

[0162] This training framework is based on the GRPO algorithm, which optimizes the sequence generation mechanism to an entropy-controlled sequence generation mechanism and optimizes the reward function to the composite reward mechanism based on the process reward model proposed in this invention.

[0163] Apart from the improvements mentioned above, this invention maintains consistency with the original GRPO algorithm in terms of advantage estimation method, policy gradient calculation method, loss function construction form, and parameter update process.

[0164] Based on the same inventive concept, this embodiment also provides a multimodal large-model adaptive iterative inference system for long videos, the system comprising:

[0165] The adaptive inference decision module is used to perform multi-round iterative inference on a given long video and text question. In each round of inference, the multimodal large model adaptively selects and executes one of three actions based on the current inference state: directly generating the final answer, performing fine-grained temporal localization, or performing visual representation optimization by extracting visual cues. The current inference state includes the video input, the text question, and decision information from previous rounds. When the model chooses to directly generate the final answer, it outputs the final inference result.

[0166] The fine-grained temporal localization module is used to output one or more time intervals represented by discrete time marker pairs when the model selects to perform fine-grained temporal localization actions; and based on the time intervals, extract the corresponding video segments from the video input of the current round and update the video input;

[0167] The visual representation optimization module is used to generate visual cues related to the semantics of the current text problem when the model selects to perform the visual representation optimization action of extracting visual cues, and to inject the visual cues into the high-level encoding process of the visual encoder in the next round of inference.

[0168] The reinforcement learning training module is used to optimize the model's adaptive action selection strategy using a reinforcement learning framework based on entropy-guided branch exploration and a compound reward mechanism.

[0169] For the system embodiments, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments; the implementation methods of the remaining modules will not be repeated here. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0170] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0171] It should also be noted that the multimodal large model adaptive iterative inference method for long videos in the above embodiments can essentially be executed by a computer program. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the method provided in the above embodiments, which includes a memory and a processor;

[0172] The memory is used to store computer programs;

[0173] The processor is configured to implement the multimodal large model adaptive iterative inference method for long videos in the above embodiments when executing the computer program.

[0174] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium.

[0175] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the method provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the multimodal large model adaptive iterative inference method for long videos in the above embodiments.

[0176] It is understood that the computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0177] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.< / repreend> < / represtart> < / represtart> < / u1> < / u0> < / t9> < / t1> < / t0> < / segend> < / segstart> < / segstart> < / answer> < / answer> < / represtart> < / segstart> < / represtart> < / segstart> < / repreend> < / represtart> < / repreend> < / represtart> < / represtart> < / segend> < / segstart> < / segstart> < / t1> < / uj> < / ti> < / u1> < / u0> < / t1> < / t0> < / represtart> < / segstart>

Claims

1. A multimodal large-scale adaptive iterative inference method for long videos, characterized in that, The multimodal large model includes at least a visual encoder and a large language model, and the method includes: For a given long video and text question, multiple rounds of iterative reasoning are performed. In each round of reasoning, the multimodal large model adaptively selects and executes one of three actions based on the current reasoning state: directly generating the final answer, performing fine-grained temporal localization, or performing visual representation optimization by extracting visual cues. The current reasoning state includes the video input, the text question, and decision information from previous rounds. When the model chooses to directly generate the final answer, it outputs the final reasoning result; When the model selects to perform fine-grained temporal localization, it outputs one or more time intervals represented by discrete time marker pairs; and based on the time intervals, it extracts the corresponding video segments from the video input of the current round and updates the video input. When the model chooses to perform visual representation optimization actions that extract visual cues, it generates visual cues that are semantically related to the current text question and injects the visual cues into the high-level encoding process of the visual encoder in the next round of inference. We employ a reinforcement learning framework based on entropy-guided branching and a compound reward mechanism to optimize the model's adaptive action selection strategy.

2. The multimodal large-model adaptive iterative inference method for long videos according to claim 1, characterized in that, The decision information for each round refers to the action type and its parameters selected by the model in that round.

3. The multimodal large-model adaptive iterative inference method for long videos according to claim 1, characterized in that, The output of one or more time intervals represented by discrete-time marker pairs includes: Map the timeline of the input video to a normalized percentage range; The percentage interval is discretized into a hierarchical label set, which includes a first-layer label for representing coarse-grained intervals and a second-layer label for representing fine-grained offsets. A first-layer label and a second-layer label are combined to form a label pair. The first-layer label determines the coarse-grained interval, and the second-layer label specifies the offset within the interval, together representing a specific time position point. The time interval is represented by a start position marker pair and an end position marker pair, and all time intervals serve as action parameters for fine-grained time positioning actions.

4. The multimodal large-model adaptive iterative inference method for long videos according to claim 1, characterized in that, The process of generating the visual cues includes: When the model determines that visual cues need to be generated, the large language model decodes and generates a token limited to the start and end labels as an action parameter; the hidden layer vector corresponding to the action parameter is extracted from the hidden layer of the large language model, and this set of vectors constitutes the visual cue.

5. The multimodal large-model adaptive iterative inference method for long videos according to claim 1 or 4, characterized in that, The visual cues are injected into the high-level encoding process of the visual encoder in the next round of reasoning, including: The visual cues are concatenated with the original input features of the i-th Transformer module in the visual encoder, where L-N+1≤i≤L, L is the number of Transformer modules in the visual encoder, and N is the number of Transformer modules into which the visual cues are injected. The concatenated features are input into the i-th Transformer module for processing. The visual representation part corresponding to the original input feature position of the i-th Transformer module is extracted from the output features and used as the original input feature of the next Transformer module. The visual representation output by the Lth Transformer module is used as the output feature of the visual encoder.

6. The multimodal large-model adaptive iterative inference method for long videos according to claim 1, characterized in that, The entropy-guided branch exploration mechanism includes: Using the entropy value of the model's output distribution at the initial stage of inference as a benchmark, monitor the change in the entropy value of the output action parameter distribution when the model performs fine-grained temporal localization or visual representation optimization actions that extract visual cues. When the entropy value changes beyond a preset threshold, the model is determined to be in a state of high uncertainty. The current inference trajectory is copied and expanded into two independent branches for parallel exploration, forming multiple candidate inference paths.

7. The multimodal large-model adaptive iterative inference method for long videos according to claim 1, characterized in that, The composite reward mechanism includes a reward for correct answer, a temporal location quality reward based on the degree of overlap between the predicted time interval and the actual time interval, a visual representation quality reward based on the semantic relevance between visual cues and textual questions, and a format reward based on whether the model output follows a predetermined format.

8. A multimodal large-model adaptive iterative inference system for long videos, used to implement the multimodal large-model adaptive iterative inference method of claim 1, characterized in that, The system includes: The adaptive inference decision module is used to perform multi-round iterative inference on a given long video and text question. In each round of inference, the multimodal large model adaptively selects and executes one of three actions based on the current inference state: directly generating the final answer, performing fine-grained temporal localization, or performing visual representation optimization by extracting visual cues. The current inference state includes the video input, the text question, and decision information from previous rounds. When the model chooses to directly generate the final answer, it outputs the final inference result. The fine-grained temporal localization module is used to output one or more time intervals represented by discrete time marker pairs when the model selects to perform fine-grained temporal localization actions; and based on the time intervals, extract the corresponding video segments from the video input of the current round and update the video input; The visual representation optimization module is used to generate visual cues related to the semantics of the current text problem when the model selects to perform the visual representation optimization action of extracting visual cues, and to inject the visual cues into the high-level encoding process of the visual encoder in the next round of inference. The reinforcement learning training module is used to optimize the model's adaptive action selection strategy using a reinforcement learning framework based on entropy-guided branch exploration and a compound reward mechanism.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the multimodal large model adaptive iterative inference method for long videos as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal large model adaptive iterative inference method for long videos as described in any one of claims 1 to 7.

Images