End-to-end data dynamic collaboration system and method based on multi-modal data chain

Abstract

The invention discloses an end-to-end data dynamic collaboration system and method based on a multi-modal data link, particularly relates to the technical field of data dynamic collaboration, and aims to accurately reveal the error consistency risk in the system through a confidence coefficient resonance out-of-control coefficient and combine with a multi-modal collaboration path oscillation coefficient constructed in each round of feedback optimization process so as to improve the robustness of the system. The convergence state and the parameter disturbance trend of the system in an end-to-end feedback closed loop are quantified, and the convergence state and the parameter disturbance trend and the confidence coefficient resonance out-of-control coefficient jointly form an advertising strategy cooperation / feedback double oscillation situation analysis model. According to the method, accurate quantitative evaluation and risk judgment of the operation state of a complex dynamic system are realized, a modal confidence gating mechanism and an advertisement strategy feedback damping mechanism are constructed based on an advertisement strategy cooperation / feedback dual oscillation situation analysis index, and dynamic reconstruction is performed on the system state from two key levels of a cooperation signal inlet and a feedback path.

Description

Technical Field

[0001] This invention relates to the field of dynamic data collaboration technology, and more specifically, to an end-to-end dynamic data collaboration system and method based on multimodal data chains. Background Technology

[0002] With the continuous evolution of intelligent advertising delivery technology, the core direction of digital advertising system optimization strategies has become how to achieve the fusion analysis and accurate response of multimodal information across multiple dimensions such as user behavior modeling, content understanding, and scene perception. Currently, mainstream advertising platforms widely integrate multimodal data sources from users' image interests, voice interactions, text browsing, geolocation, click behavior, and social relationships, forming a multimodal data chain spanning the entire user lifecycle. An end-to-end dynamic collaborative optimization model built upon this data chain can achieve coordinated control of key strategies such as ad content selection, display timing, and user response prediction, significantly improving the relevance and conversion efficiency of ad delivery.

[0003] However, existing multimodal collaborative methods still face many structural challenges in the actual operation of advertising strategy optimization, especially in terms of uneven modal quality, insufficient feedback adjustment strategies, and weak system dynamic convergence capabilities. Among these, one of the most representative technical challenges is the coupling problem between confidence resonance runaway (collaborative drift) and multimodal feedback path oscillation. Specifically, in the multimodal collaboration process, low-quality modes are easily pulled by high-confidence modes, resulting in misjudgment drift and superficially consistent but actually erroneous judgment results. Once such "erroneous consistency" is accepted by the feedback mechanism, it will further induce the system to repeatedly reinforce the weights of erroneous modes, leading to the overall system entering a closed loop of feedback misadjustment – ​​drift aggravation – convergence runaway, forming a dual oscillation state of collaborative reasoning and feedback optimization.

[0004] Such phenomena not only disrupt the original modal confidence system, but also cause high-quality modal information to be marginalized, shielded, or downweighted in the feedback, thereby causing cognitive bias and response errors in the system's perception of the true state, ultimately leading to irreversible collaborative imbalance and state drift risks. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an end-to-end dynamic data collaboration system and method based on a multimodal data chain to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The end-to-end dynamic data collaboration method based on multimodal data chains includes the following steps: Step S1: Obtain confidence resonance runaway information of each modality in the delivery response prediction process, and obtain the confidence resonance runaway coefficient based on the confidence resonance runaway information; Step S2: Obtain the multimodal collaborative path oscillation information in each round of recommendation feedback update of the advertising strategy engine, and obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information; Step S3: Construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift in the current collaborative system. Step S4: Based on the risk of irreversible collaborative imbalance and state drift in the collaborative system, construct an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism to reconstruct the system state.

[0007] In a preferred embodiment, by acquiring the confidence resonance runaway information of each modality in the delivery response prediction process, the cooperative drift of the low confidence modality under the dynamic guidance of the high confidence modality is analyzed, and the confidence resonance runaway coefficient is obtained to measure the degree of cooperative drift of the low confidence modality under the dynamic guidance of the high confidence modality. The logic for obtaining the confidence resonance runaway coefficient is as follows: Get each modality At time t, the modal state includes a semantic embedding vector. and confidence output Mark the modal state as ; Calculate the mode in time interval Confidence perturbation response value ; Define modal perturbation diagram All modes constitute the node set of the modal perturbation graph. ,in Let be the number of modes, if the modes Modal Simultaneously satisfy: Then a directed edge is established. ; A perturbation flow matrix is ​​constructed on the modal perturbation graph to indicate whether there is a perturbation "led by a high-confidence mode to a low-confidence mode". The elements of the perturbation flow matrix are: ,in The disturbance state value. This is a preset confidence threshold. For low confidence modes Obtain the maximum length of its disturbed chain. ; Construct a perturbation propagation factor matrix based on the confidence level perturbation response values, where the elements of the perturbation propagation factor matrix are: ,in As a disturbance propagation factor, For modality The confidence level perturbation response value, The path hop count represents the number of perturbations from mode. Propagation to modality Path hop count in the modal perturbation graph; Identify the set of resonant closed-loop paths present in the modal perturbation graph. Calculate the resonance intensity : ; Calculate the confidence level resonance runaway coefficient : ,in For each low-confidence mode of the resonant closed-loop path Average path length of the disturbed chain: , denoted as the mode number of the resonant closed-loop path.

[0008] In a preferred embodiment, by obtaining the multimodal collaborative path oscillation information in each round of recommendation feedback update of the advertising strategy engine, the risk of continuous oscillation and non-convergence of modal collaborative parameters in the closed loop due to unstable loss function or excessive feedback adjustment in the end-to-end feedback adjustment is analyzed, and the multimodal collaborative path oscillation coefficient is obtained to measure the degree of risk of continuous oscillation and non-convergence of modal collaborative parameters in the closed loop. The logic for obtaining the oscillation coefficient of the multimodal cooperative path is as follows: For mode Obtain the collaborative parameter trajectory in consecutive feedback rounds , indicating the mode after the nth feedback. The collaborative fusion parameter vector; Calculate the difference vector of continuous disturbance response : , , The total number of feedback rounds recorded; Calculate the feedback response to the mode for each feedback based on the continuous disturbance response difference vector. rate of change of disturbance intensity of collaborative fusion parameters ; For modes Continuous disturbance response sequence Perform Fourier transform: Calculate the frequency domain perturbation energy spectrum : ,in For high-frequency sub-intervals; The non-convergent trend and "periodic oscillation instability" of the modes in the feedback are analyzed to obtain the local oscillation index of each mode. : For any mode pair ( , Record the oscillation synchronization index between modes : ; The oscillation coefficient of the multimodal cooperative path is calculated based on the local oscillation index and the oscillation synchronization index. : ,in The modal number.

[0009] In a preferred embodiment, an advertising strategy synergy / feedback dual oscillation trend analysis model is constructed based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient to obtain the advertising strategy synergy / feedback dual oscillation trend analysis index. The advertising strategy synergy / feedback dual oscillation trend analysis model is based on the following formula: In the formula An index analyzing the dual oscillation trend of advertising strategy coordination and feedback. The confidence level resonance runaway coefficient is the coefficient. The multimodal cooperative path oscillation coefficient, represent the preset proportional coefficients of the confidence resonance runaway coefficient and the multimodal cooperative path oscillation coefficient, respectively, and All are greater than 0.

[0010] In a preferred embodiment, the advertising strategy synergy / feedback dual oscillation situation analysis index is compared with a preset advertising strategy synergy / feedback dual oscillation situation analysis index threshold to assess the risk of irreversible synergy imbalance and state drift in the current synergy system, as follows: If the advertising strategy synergy / feedback dual oscillation trend analysis index is greater than the advertising strategy synergy / feedback dual oscillation trend analysis index threshold, then a high risk of synergy imbalance and state drift will be generated. If the advertising strategy synergy / feedback dual oscillation trend analysis index is less than or equal to the advertising strategy synergy / feedback dual oscillation trend analysis index threshold, then a low synergy imbalance and state drift risk will be generated.

[0011] In a preferred embodiment, when a high risk of collaborative imbalance and state drift is generated, the system state is reconstructed by constructing an advertising modal confidence gating mechanism and an advertising strategy feedback damping mechanism based on the current confidence resonance runaway coefficient and multimodal collaborative path oscillation coefficient. The advertising modality confidence gating mechanism is as follows: ,in For modality The confidence output, The output is the confidence score after reconstruction; The feedback damping mechanism of the advertising strategy is as follows: ,in For modality The Nth round of collaborative fusion parameter vector, This is the reconstructed N+1 round collaborative fusion parameter vector.

[0012] In a preferred embodiment, the end-to-end data dynamic collaboration system based on a multimodal data chain includes a resonance runaway module, a collaboration path oscillation module, a risk comprehensive analysis module, and a state reconstruction module. The resonance runaway module is used to obtain the confidence resonance runaway information of each mode in the delivery response prediction process, and to obtain the confidence resonance runaway coefficient based on the confidence resonance runaway information; The collaborative path oscillation module is used to obtain collaborative path oscillation information between multiple modalities in each round of recommendation feedback update of the advertising strategy engine, and to obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information. The risk comprehensive analysis module is used to construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift of the current collaborative system. The state reconstruction module is used to reconstruct the system state by constructing an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism based on the risk of irreversible collaborative imbalance and state drift in the collaborative system.

[0013] The technical effects and advantages of this invention are as follows: 1. This invention constructs a confidence resonance runaway coefficient to dynamically capture the phenomenon of "irrational collaborative drift" caused by low-confidence modes being pulled by high-confidence modes in system collaborative reasoning, thus accurately revealing the risk of "error consistency" within the system. Compared to traditional static modal fusion methods, this method, based on joint modeling of modal confidence gradients and dynamic guidance effects, possesses stronger real-time perception capabilities and modal shift sensitivity. It can identify potential signs of collaborative imbalance before the system exhibits global misjudgment. Secondly, by combining the multimodal collaborative path oscillation coefficients constructed in each round of feedback optimization, it quantifies the convergence state and parameter perturbation trend of the system in the end-to-end feedback loop, promptly detecting feedback oscillation behavior caused by misjudged feedback, unstable loss functions, or uncontrolled optimization step size. Together with the confidence resonance uncontrolled coefficient, it constitutes a dual oscillation situation analysis model for advertising strategy collaboration / feedback. Through the real-time output of the dual oscillation situation analysis index for advertising strategy collaboration / feedback, it achieves accurate quantitative assessment and risk judgment of the operating state of complex dynamic systems. Based on the dual oscillation situation analysis index for collaboration / feedback, it further constructs a modal confidence gating mechanism and a parameter feedback damping mechanism to dynamically reconstruct the system state from two key levels: collaborative signal entry and feedback path. Attached Figure Description

[0014] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the system in Embodiment 2 of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1: Figure 1 The present invention provides an end-to-end dynamic data collaboration method based on a multimodal data link, comprising the following steps: Step S1: Obtain confidence resonance runaway information of each modality in the delivery response prediction process, and obtain the confidence resonance runaway coefficient based on the confidence resonance runaway information; Step S2: Obtain the multimodal collaborative path oscillation information in each round of recommendation feedback update of the advertising strategy engine, and obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information; Step S3: Construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift in the current collaborative system. Step S4: Based on the risk of irreversible collaborative imbalance and state drift in the collaborative system, construct an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism to reconstruct the system state; In step S1, firstly, multimodal advertising data sources such as user behavior logs, content semantic features, environmental context, and social interactions are collected, and confidence resonance runaway information of each modality in the delivery response prediction process is obtained, and confidence resonance runaway coefficient is obtained based on the confidence resonance runaway information. In this invention, the confidence resonance runaway coefficient is used to quantify the degree of collaborative drift caused by the dynamic guidance of high-confidence modes to low-confidence modes during the operation of a multimodal system. It is a core indicator for measuring the credibility and stability of multimodal collaboration. This coefficient comprehensively considers the changes in confidence gradients between modes, the deviation of the current mode state, and the dynamic characteristics of the system's collaborative weight distribution. It can accurately reflect whether low-confidence modes are forced to collaborate into an incorrect judgment path without sufficient semantic support, thus forming a "drift consensus." When the confidence resonance runaway coefficient is large, it indicates that there is a significant imbalance in modal guidance in the current system. That is, some high-confidence modes, due to weight dominance or historical feedback accumulation, cause weak modes to undergo passive resonance drift. This abnormal collaboration is often characterized by semantic consensus, but in fact, it masks the semantic conflicts and cognitive differences between modal judgments. Once the erroneous modal "consensus" becomes the input basis for subsequent inference and feedback optimization, it will not only amplify the risk of system misjudgment but also significantly interfere with the convergence path of model parameters, ultimately inducing oscillating optimization or even irreversible shift in the model.

[0017] Conversely, when the confidence resonance runaway coefficient is small, it indicates that the current modal cooperative structure of the system is stable, the weak modes are not irrationally guided, and cooperative decisions rely more on the intrinsic semantic consistency of the modes. The cooperative mechanism is manifested as genuine complementarity and information fusion, rather than drift-based unification. This low resonance state reflects the system's strong ability to suppress erroneous information sources in dynamic cooperation, effectively maintaining the operation of each mode within its confidence boundary, and improving the overall robustness and anti-interference ability of the system.

[0018] The dynamic risk assessment mechanism based on the confidence resonance runaway coefficient can serve as a system-level collaborative stability indicator to predict and intervene in the risk of irreversible collaborative imbalance and state drift during the collaborative process. Therefore, by acquiring confidence resonance runaway information among multiple modes, we can analyze the cooperative drift of low-confidence modes under the dynamic guidance of high-confidence modes, and obtain the confidence resonance runaway coefficient to measure the degree of cooperative drift of low-confidence modes under the dynamic guidance of high-confidence modes. The logic for obtaining the confidence resonance runaway coefficient is as follows: Get each modality At time t, the modal state includes a semantic embedding vector. and confidence output Mark the modal state as ; It should be noted that the semantic embedding vector can be obtained by performing feature extraction and high-dimensional semantic mapping on the original modal data through a modal encoder. The confidence output refers to the credibility or probability score corresponding to the prediction output of the modality for the current collaborative decision-making task, which can be extracted through the classifier or decision-maker head structure. The above-mentioned semantic embedding extraction and confidence output acquisition methods are mature technologies widely used in existing multimodal systems, and will not be elaborated here. Calculate the mode in time interval Confidence perturbation response value : ; Define modal perturbation diagram All modes constitute the node set of the modal perturbation graph. ,in Let be the number of modes, if the modes Modal Simultaneously satisfy: Then a directed edge is established. , indicating potential resonance conduction; It should be noted that in the above formula For modality Modal The cosine similarity of the semantic embedding vectors. The preset cosine similarity threshold, The preset confidence level perturbation response difference threshold; A perturbation flow matrix is ​​constructed on the modal perturbation graph to indicate whether there is a perturbation "led by a high-confidence mode to a low-confidence mode". The elements of the perturbation flow matrix are: ,in The disturbance state value. This is a preset confidence threshold. For low confidence modes ( ), obtain the maximum length of its disturbed chain. : ,in Let be the set of perturbation paths, representing the path that starts from any mode and ends at the mode. The set of all perturbation paths to the destination. Let each edge in the perturbation path represent a path. From the modal To mode One-step disturbance propagation; The perturbation propagation factor matrix is ​​constructed based on the confidence level perturbation response values, where the elements of the perturbation propagation factor matrix are: ,in For the perturbation propagation factor, For modality The confidence level perturbation response value, The path hop count represents the number of perturbations from mode. Propagation to modality Path hop count in the modal perturbation graph; Identify the set of resonant closed-loop paths present in the modal perturbation graph. Calculate the resonance intensity : ; It should be noted that the resonant closed-loop path is constructed only when the perturbation chain loop path exists (i.e., ): ; In one optional example, the resonance intensity is calculated as follows: Assuming the current number of modes is 4, at time t, the output values ​​of the confidence perturbation response for the corresponding modes are as follows: , , , ; There are perturbations between its modes, namely: , , ; And there exists a resonant closed-loop path ( ); Calculate the perturbation propagation factor: , , ; The resonance intensity is then: ; Calculate the confidence level resonance runaway coefficient : ,in For each low-confidence mode of the resonant closed-loop path Average path length of the disturbed chain: , The number of modes in the resonant closed-loop path; It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. Step S2: Obtain multimodal collaborative path oscillation information in each round of recommendation feedback (such as CTR response, click behavior, dwell time, etc.) update of the advertising strategy engine, and obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information; The multimodal cooperative path oscillation coefficient in this invention is an important metric for quantitatively evaluating the dynamic stability of a system in end-to-end feedback control. Its core purpose is to reflect whether, during multiple rounds of feedback learning, the multimodal cooperative inference mechanism experiences continuous oscillations in the cooperative parameters within the closed-loop feedback path due to factors such as drastic changes in the loss function, flawed feedback path design, or imperfect modal control mechanisms. This oscillations can ultimately prevent the system from converging and may even lead to an irrecoverable cooperative imbalance state characterized by "oscillation-amplification-drift." This coefficient not only dynamically monitors the steady-state behavior of the system during the training or inference phases but also provides early warnings of potential dynamic runaway risks, offering significant guidance for the stable operation and accuracy of multimodal intelligent systems. A large multimodal cooperative path oscillation coefficient typically indicates the accumulation of several system risk characteristics: First, the parameters in the system feedback loop, after multiple adjustments, exhibit high-frequency positive and negative oscillations with increasing amplitude, indicating that the current feedback mechanism fails to effectively balance and adjust error information. Second, the coupling structure between modes is in a state of continuous perturbation, failing to achieve convergent parameter synchronization. Third, during optimization, the loss function exhibits significant non-monotonicity and instability, with misleading information in the feedback path accumulating round by round, ultimately leading the model into a "blind tuning" state, unable to achieve a stable optimal solution. Conversely, a smaller multimodal cooperative path oscillation coefficient indicates that the system possesses good self-stabilizing capabilities during dynamic feedback. Specifically, this manifests as: model parameters stabilizing after a finite number of feedback adjustments, no longer exhibiting large-scale repeated adjustments; the cooperative coupling between modes gradually solidifying into a stable set of information paths and weight structures; and the loss function exhibiting good monotonic decreasing characteristics during training, with the feedback mechanism not generating interference amplification. In this case, the system demonstrates strong steady-state cooperative capabilities, is not easily affected by sudden modal drift or deviations in individual modal information, and can effectively maintain the overall correctness and robustness of the task. Therefore, a significant reduction in the oscillation coefficient often means that the feedback path of the multimodal system has gradually converged to a robust optimal solution set, the feedback control strategy is highly effective, and the overall system operation tends to be rational and stable.

[0019] The greatest benefit of the multimodal cooperative path oscillation coefficient assessment system for assessing cooperative imbalance and state drift risk lies in providing a dynamic, perceptible, parameter-traceable, and decision-intervention-enabled stability index system for the entire end-to-end multimodal system. It can reveal a deeper issue: whether the model's internal operating logic is stable.

[0020] Therefore, by acquiring the oscillation information of the cooperative path between multiple modes, we can analyze the risk of continuous oscillation and non-convergence of the modal cooperative parameters in the closed loop due to the instability of the loss function or excessive feedback control in the end-to-end feedback adjustment, and obtain the multimodal cooperative path oscillation coefficient to measure the degree of risk of continuous oscillation and non-convergence of the modal cooperative parameters in the closed loop. The logic for obtaining the oscillation coefficient of the multimodal cooperative path is as follows: For mode Obtain the collaborative parameter trajectory in consecutive feedback rounds , indicating the mode after the nth feedback. The collaborative fusion parameter vector (which can be fusion weights, attention vectors, gating factors, etc.); Calculate the difference vector of continuous disturbance response : , , The total number of feedback rounds recorded; Calculate the feedback response to the mode for each feedback based on the continuous disturbance response difference vector. rate of change of disturbance intensity of collaborative fusion parameters : ,in Let L2 norm be the difference vector of continuous disturbance responses. This is to prevent division by zero by a very small constant (generally taken as...). ); For modes Continuous disturbance response sequence Perform Fourier transform: ,in This is the frequency domain representation of a continuous perturbation response sequence. Calculate the frequency domain perturbation energy spectrum using Fourier transform. : ,in This is a high-frequency sub-interval (e.g., more than one-third of the frequency band above the sampling frequency) used to identify the intensity of high-frequency oscillations; The non-convergent trend and "periodic oscillation instability" of the modes in the feedback are analyzed to obtain the local oscillation index of each mode. : : For any mode pair ( , Record the oscillation synchronization index between modes : ,in For modality The continuous disturbance response difference vector; The oscillation coefficient of the multimodal cooperative path is calculated based on the local oscillation index and the oscillation synchronization index. : ,in The modal number; It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. Step S3: Construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift in the current collaborative system. Based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, an advertising strategy synergy / feedback dual oscillation trend analysis model is constructed to obtain the advertising strategy synergy / feedback dual oscillation trend analysis index. The formula used in the advertising strategy synergy / feedback dual oscillation trend analysis model is as follows: In the formula An index analyzing the dual oscillation trend of advertising strategy coordination and feedback. The confidence level resonance runaway coefficient is the coefficient. The multimodal cooperative path oscillation coefficient, represent the preset proportional coefficients of the confidence resonance runaway coefficient and the multimodal cooperative path oscillation coefficient, respectively, and All are greater than 0; It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. The settings should be tailored to the specific circumstances. For example, an expert-empowered approach could be adopted, where experts in relevant fields are invited to determine the pre-defined proportions for each indicator through professional opinion surveys and comprehensive evaluations. It can be 0.5 or 0.5; As can be seen from the above calculation expressions, the larger the confidence resonance runaway coefficient and the larger the multimodal collaborative path oscillation coefficient, the larger the advertising strategy collaborative / feedback dual oscillation situation analysis index. This indicates that the current collaborative state of the multimodal system is more unstable and is on the verge of higher risk collaborative imbalance and state drift. It may trigger the system to generate collaborative misleading patterns such as non-convergence, erroneous reinforcement, or even "illusory consistency" during training or inference. Conversely, the smaller the confidence resonance runaway coefficient and the smaller the multimodal collaborative path oscillation coefficient, the smaller the advertising strategy collaborative / feedback dual oscillation situation analysis index. This indicates that the system is currently in a low-risk and stable collaborative state, which means that the collaborative system has better convergence, coordination, and robustness in the end-to-end dynamic operation process. The advertising strategy synergy / feedback dual oscillation trend analysis index is compared with the preset advertising strategy synergy / feedback dual oscillation trend analysis index threshold to assess the risk of irreversible synergy imbalance and state drift in the current synergy system, as follows: If the advertising strategy synergy / feedback dual oscillation trend analysis index is greater than the advertising strategy synergy / feedback dual oscillation trend analysis index threshold, it indicates that the current system has both a high confidence resonance runaway risk and a synergy path feedback oscillation risk. The two risk factors have worked together and amplified each other, which may lead the system into an irreversible synergy drift and non-convergence state, generating a high synergy imbalance and state drift risk. If the advertising strategy synergy / feedback dual oscillation trend analysis index is less than or equal to the advertising strategy synergy / feedback dual oscillation trend analysis index threshold, it indicates that the system as a whole is in the synergy stable range, the current intermodal cooperation is normal, the feedback adjustment mechanism is stable, the model convergence is good, the synergy system still has the ability to self-regulate and absorb anomalies, generating low synergy imbalance and state drift risk. It should be noted that the threshold for the advertising strategy synergy / feedback dual oscillation trend analysis index can be preset according to specific application needs, for example, in a static setting method: The maximum steady-state exponent can be set based on the offline simulation training process; Dynamic adaptive method: During online operation, the average and fluctuation amplitude of the situation index over the past N rounds are calculated through a sliding window, and the threshold is updated in real time; it is suitable for scenarios with drastic environmental changes or variable input distribution (such as cross-modal video stream perception systems). Expert empowerment + experience calibration method: Multimodal modeling and deployment experts were invited to set initial values ​​based on task importance and system tolerance, and then iteratively revised based on the trial operation results. Step S4: Based on the risk of irreversible collaborative imbalance and state drift in the collaborative system, construct an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism to reconstruct the system state; When high collaborative imbalance and state drift risks are generated, the system state is reconstructed by constructing an advertising modal confidence gating mechanism and an advertising strategy feedback damping mechanism based on the current confidence resonance runaway coefficient and multimodal collaborative path oscillation coefficient. The advertising modality confidence gating mechanism is as follows: ,in For modality The confidence output, The output is the confidence score after reconstruction; The feedback damping mechanism of the advertising strategy is as follows: ,in For modality The Nth round of collaborative fusion parameter vector, This is the reconstructed N+1 round collaborative fusion parameter vector; It should be noted that the above formulas are all dimensionless calculations. Commonly used methods for removing dimensions include Min-Max normalization and Z-Score standardization, which will not be elaborated here. This invention constructs a confidence resonance runaway coefficient to dynamically capture the phenomenon of "irrational collaborative drift" caused by low-confidence modes being pulled by high-confidence modes in system collaborative reasoning, thus accurately revealing the risk of "error consistency" within the system. Compared to traditional static modal fusion methods, this method, based on joint modeling of modal confidence gradients and dynamic guidance effects, possesses stronger real-time perception capabilities and modal shift sensitivity. It can identify potential signs of collaborative imbalance before the system exhibits global misjudgment. Secondly, by combining the multimodal collaborative path oscillation coefficients constructed in each round of feedback optimization, it quantifies the convergence state and parameter perturbation trend of the system in the end-to-end feedback loop, promptly detecting feedback oscillation behavior caused by misjudged feedback, unstable loss functions, or uncontrolled optimization step size. Together with the confidence resonance uncontrolled coefficient, it constitutes a dual oscillation situation analysis model for advertising strategy collaboration / feedback. Through the real-time output of the dual oscillation situation analysis index for advertising strategy collaboration / feedback, it achieves accurate quantitative assessment and risk judgment of the operating state of complex dynamic systems. Based on the dual oscillation situation analysis index for collaboration / feedback, it further constructs a modal confidence gating mechanism and a parameter feedback damping mechanism to dynamically reconstruct the system state from two key levels: collaborative signal entry and feedback path.

[0021] Example 2: This example introduces an end-to-end dynamic data collaboration system based on a multimodal data link, such as... Figure 2 As shown, it includes a resonance runaway module, a cooperative path oscillation module, a risk comprehensive analysis module, and a state reconstruction module; The resonance runaway module is used to obtain the confidence resonance runaway information of each mode in the delivery response prediction process, and to obtain the confidence resonance runaway coefficient based on the confidence resonance runaway information; The collaborative path oscillation module is used to obtain collaborative path oscillation information between multiple modalities in each round of recommendation feedback update of the advertising strategy engine, and to obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information. The risk comprehensive analysis module is used to construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift of the current collaborative system. The state reconstruction module is used to reconstruct the system state by constructing an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism based on the risk of irreversible collaborative imbalance and state drift in the collaborative system. The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0022] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0023] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0024] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and method described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0025] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways.

[0026] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An end-to-end dynamic data collaboration method based on multimodal data links, characterized in that: Includes the following steps: Step S1: Obtain confidence resonance runaway information of each modality in the delivery response prediction process, and obtain the confidence resonance runaway coefficient based on the confidence resonance runaway information; Step S2: Obtain the multimodal collaborative path oscillation information in each round of recommendation feedback update of the advertising strategy engine, and obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information; Step S3: Construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift in the current collaborative system. Step S4: Based on the risk of irreversible collaborative imbalance and state drift in the collaborative system, construct an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism to reconstruct the system state.

2. The end-to-end dynamic data collaboration method based on a multimodal data link according to claim 1, characterized in that: By acquiring confidence resonance runaway information of each modality in the delivery response prediction process, we analyze the collaborative drift of the low-confidence modality under the dynamic guidance of the high-confidence modality and obtain the confidence resonance runaway coefficient to measure the degree of collaborative drift of the low-confidence modality under the dynamic guidance of the high-confidence modality. The logic for obtaining the confidence resonance runaway coefficient is as follows: Get each modality At time t, the modal state includes a semantic embedding vector. and confidence output Mark the modal state as ; Calculate the mode in time interval Confidence perturbation response value ; Define modal perturbation diagram All modes constitute the node set of the modal perturbation graph. ,in Let be the number of modes, if the modes Modal Simultaneously satisfy: Then a directed edge is established. ; A perturbation flow matrix is ​​constructed on the modal perturbation graph to indicate whether there is a perturbation "led by a high-confidence mode to a low-confidence mode". The elements of the perturbation flow matrix are: ,in The disturbance state value. This is a preset confidence threshold. For low confidence modes Obtain the maximum length of its disturbed chain. ; Construct a perturbation propagation factor matrix based on the confidence level perturbation response values, where the elements of the perturbation propagation factor matrix are: ,in As a disturbance propagation factor, For modality The confidence level perturbation response value, The path hop count represents the number of perturbations from mode. Propagation to modality Path hop count in the modal perturbation graph; Identify the set of resonant closed-loop paths present in the modal perturbation graph. Calculate the resonance intensity : ; Calculate the confidence level resonance runaway coefficient : ,in For each low-confidence mode of the resonant closed-loop path Average path length of the disturbed chain: , denoted as the mode number of the resonant closed-loop path.

3. The end-to-end dynamic data collaboration method based on a multimodal data link according to claim 1, characterized in that: By acquiring multimodal collaborative path oscillation information in each round of recommendation feedback update of the advertising strategy engine, we analyze the risk of modal collaborative parameters continuously oscillating and failing to converge in the closed loop due to unstable loss function or excessive feedback adjustment during end-to-end feedback adjustment. We also obtain the multimodal collaborative path oscillation coefficient to measure the degree of risk of modal collaborative parameters continuously oscillating and failing to converge in the closed loop. The logic for obtaining the oscillation coefficient of the multimodal cooperative path is as follows: For mode Obtain the collaborative parameter trajectory in consecutive feedback rounds , indicating the mode after the nth feedback. The collaborative fusion parameter vector; Calculate the difference vector of continuous disturbance response : , , The total number of feedback rounds recorded; Calculate the feedback response to the mode for each feedback based on the continuous disturbance response difference vector. rate of change of disturbance intensity of collaborative fusion parameters ; For modes Continuous disturbance response sequence Perform Fourier transform: Calculate the frequency domain perturbation energy spectrum : ,in For high-frequency sub-intervals; The non-convergent trend and "periodic oscillation instability" of the modes in the feedback are analyzed to obtain the local oscillation index of each mode. : For any mode pair ( , Record the oscillation synchronization index between modes : ; The oscillation coefficient of the multimodal cooperative path is calculated based on the local oscillation index and the oscillation synchronization index. : ,in The modal number.

4. The end-to-end dynamic data collaboration method based on a multimodal data link according to claim 1, characterized in that: Based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, an advertising strategy synergy / feedback dual oscillation trend analysis model is constructed to obtain the advertising strategy synergy / feedback dual oscillation trend analysis index. The formula used in the advertising strategy synergy / feedback dual oscillation trend analysis model is as follows: In the formula An index analyzing the dual oscillation trend of advertising strategy coordination and feedback. The confidence level resonance runaway coefficient is the coefficient. The multimodal cooperative path oscillation coefficient, represent the preset proportional coefficients of the confidence resonance runaway coefficient and the multimodal cooperative path oscillation coefficient, respectively, and All are greater than 0.

5. The end-to-end dynamic data collaboration method based on a multimodal data link according to claim 4, characterized in that: The advertising strategy synergy / feedback dual oscillation trend analysis index is compared with the preset advertising strategy synergy / feedback dual oscillation trend analysis index threshold to assess the risk of irreversible synergy imbalance and state drift in the current synergy system, as follows: If the advertising strategy synergy / feedback dual oscillation trend analysis index is greater than the advertising strategy synergy / feedback dual oscillation trend analysis index threshold, then a high risk of synergy imbalance and state drift will be generated. If the advertising strategy synergy / feedback dual oscillation trend analysis index is less than or equal to the advertising strategy synergy / feedback dual oscillation trend analysis index threshold, then a low synergy imbalance and state drift risk will be generated.

6. The end-to-end dynamic data collaboration method based on a multimodal data link according to claim 5, characterized in that: When high collaborative imbalance and state drift risks are generated, the system state is reconstructed by constructing an advertising modal confidence gating mechanism and an advertising strategy feedback damping mechanism based on the current confidence resonance runaway coefficient and multimodal collaborative path oscillation coefficient. The advertising modality confidence gating mechanism is as follows: ,in For modality The confidence output, The output is the confidence score after reconstruction; The feedback damping mechanism of the advertising strategy is as follows: ,in For modality The Nth round of collaborative fusion parameter vector, This is the reconstructed N+1 round collaborative fusion parameter vector.

7. An end-to-end dynamic data collaboration system based on a multimodal data link, used to implement the end-to-end dynamic data collaboration method based on a multimodal data link as described in any one of claims 1-6, characterized in that: It includes a resonance runaway module, a cooperative path oscillation module, a risk comprehensive analysis module, and a state reconstruction module; The resonance runaway module is used to obtain the confidence resonance runaway information of each modality in the delivery response prediction process, and to obtain the confidence resonance runaway coefficient based on the confidence resonance runaway information; The collaborative path oscillation module is used to obtain collaborative path oscillation information between multiple modalities in each round of recommendation feedback update of the advertising strategy engine, and to obtain the multimodal collaborative path oscillation coefficient based on the collaborative path oscillation information. The risk comprehensive analysis module is used to construct an advertising strategy collaboration / feedback dual oscillation situation analysis model based on the confidence resonance runaway coefficient and the multimodal collaborative path oscillation coefficient, obtain the advertising strategy collaboration / feedback dual oscillation situation analysis index, and assess the risk of irreversible collaboration imbalance and state drift of the current collaborative system. The state reconstruction module is used to reconstruct the system state by constructing an advertising modality confidence gating mechanism and an advertising strategy feedback damping mechanism based on the risk of irreversible collaborative imbalance and state drift in the collaborative system.

Images