Orthogonal low-rank adaptation prediction method and system, equipment and storage medium

Abstract

The invention discloses an orthogonal low-rank adaptation prediction method, system and device and a storage medium, and belongs to the technical field of graph neural networks, and the method comprises the steps: determining a main expert based on to-be-predicted cross-domain graph data and an orthogonal low-rank adaptation expert database; obtaining activation features and basic output of each layer based on the to-be-predicted cross-domain graph data; on the basis of activation features of each layer, an orthogonal low-rank adaptive expert database and a symmetric double-drive router obtain an expert gating data set; acquiring an activated auxiliary expert set based on the expert gating data set and a preset gating threshold, and acquiring auxiliary expert weight distribution based on the activated auxiliary expert set and the expert gating data set; based on the activation features of each layer, obtaining main expert output and a plurality of auxiliary expert low-rank adaptation residual errors, and further obtaining target feature representation; and inputting the target feature representation into an independent task head corresponding to the main expert for forward calculation to obtain a social scene prediction result, so that cross-domain positive migration of social scene knowledge is realized, and the social scene prediction precision is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of graph neural network technology, and in particular to a prediction method, system, device, and storage medium for orthogonal low-rank adaptation. Background Technology

[0002] In practical applications in industry and scientific research, various graph data, such as academic citation networks, co-purchasing networks, social networks, and transportation infrastructure networks, often originate from different data domains and are continuously and sequentially generated over time. These cross-domain sequential scenarios demand continuous learning from graph models. That is, the model needs to maintain its performance in the old domain while constantly absorbing knowledge from new domains, and it also needs to leverage existing knowledge from the old domains to drive the learning process in the new domains. In the typical graph data application scenario of social context prediction, this prediction relies on graph data formed by social networks. It constructs a cross-domain social graph using users, content, and interaction relationships from multiple social platforms as nodes and edges. It needs to accurately determine social contexts such as normal natural discussions, marketing promotions, collaborative manipulation of data, rapid spread of rumors, and the actual outbreak of sudden events. However, graph data from different social platforms and different social regions exhibit significant inter-domain structural differences, and the continuous and sequential generation of social data over time places higher demands on the cross-domain continuous learning capabilities of graph models. Due to significant structural differences in graph domains, such as isomorphism, sparsity, and community structure, existing technologies mainly adopt methods such as direct full-parameter fine-tuning or sequential fine-tuning, continuous learning with replay and regularization, parameter isolation or dynamic expansion of parameters for efficient fine-tuning combined with hybrid expert routing to adapt to such scenarios. However, none of these methods can effectively solve the core problem of sequential adaptation of graph models. This will directly lead to poor cross-domain adaptation of social scenario prediction, low prediction accuracy, and high deployment and maintenance costs, making it difficult to meet the intelligent analysis needs of actual social business.

[0003] Specifically, existing technologies, when adapting to cross-domain sequence graph data for social scenario prediction, typically employ direct full-parameter fine-tuning or sequential fine-tuning. This approach is prone to catastrophic forgetting because parameter updates for subsequent social domain tasks can overwrite previous social domain knowledge. For example, parameters learned for e-commerce social platform fraud scenarios may overwrite parameters learned for natural discussion scenarios on social platforms. Furthermore, a single parameter update system struggles to accommodate the structural differences between different social domain graph data, failing to adapt to the diverse characteristics of multi-source social scenarios. Using replay and regularization-based continuous learning methods for cross-domain sequence tasks in social scenario prediction cannot fundamentally avoid conflicts in model update directions between different social domains. Additionally, the massive nature of social data means that data replay significantly increases storage and training overhead. In situations with strong structural drift in social scenarios, such as sudden public opinion crises... The rapid changes in social graph structures lead to highly unstable predictive performance of models. While methods such as parameter isolation or dynamic scaling can preserve model knowledge from different social domains and maintain predictive performance for corresponding scenarios to some extent, they also cause a continuous increase in model parameters and computational overhead, significantly raising the deployment cost of social scenario prediction models and making it difficult to meet the needs of industrial-grade social business deployments. Using efficient parameter fine-tuning combined with hybrid expert routing to handle cross-domain graph data for social scenario prediction still suffers from significant parameter interference between different social domain tasks when there are no structural constraints on the direction of model updates. Furthermore, if the router training mechanism is unreasonable, it is prone to time bias, such as excessive bias towards recently learned social domain tasks, making it difficult to learn globally consistent cross-social domain transfer combination strategies and effectively reuse knowledge from different social domains to achieve positive transfer in scenario prediction.

[0004] In summary, existing technologies often require additional online fine-tuning for new cross-domain social graph data during the inference stage of social scenario prediction. This not only significantly increases the real-time analysis cost of social business but also results in poor cross-domain transferability of social scenario prediction, making it difficult to guarantee prediction accuracy and stability. The deployment and long-term maintenance costs of the model remain high, making it impossible to achieve efficient, stable, continuous learning and accurate prediction in social scenario prediction scenarios. Consequently, it is difficult to meet the real-time and intelligent needs of actual business operations such as social content governance, public opinion early warning, and risk control review. Summary of the Invention

[0005] This invention provides a prediction method, system, device, and storage medium for orthogonal low-rank adaptation, which solves the technical problems of online fine-tuning during the inference stage, poor cross-domain transfer effect, and high deployment cost in existing graph cross-domain sequence learning. It utilizes expert combination to achieve cross-domain positive transfer of social context knowledge, thereby improving the accuracy and stability of social context prediction.

[0006] This invention provides a prediction method for orthogonal low-rank adaptation, applied to a pre-trained graph prediction model. The graph prediction model includes a frozen backbone network, an orthogonal low-rank adaptation expert database, and a symmetric dual-drive router. The method includes: Obtain cross-domain graph data to be predicted based on pre-acquired multi-source social data streams; The master expert is determined based on the cross-domain graph data to be predicted and the orthogonal low-rank fitting expert database. The cross-domain graph data to be predicted is input into the frozen backbone network for layer-by-layer message passing to obtain the activation features and basic outputs of each layer. Based on the activation features of each layer, the orthogonal low-rank adaptive expert library, and the symmetric dual-drive router, expert benefit calculation is performed to obtain the expert gating data set. Based on the expert gating data set and the preset gating threshold, auxiliary experts are screened to obtain an active auxiliary expert set that meets the preset gating value condition, and the auxiliary expert weight distribution is obtained based on the active auxiliary expert set and the expert gating data set. Based on the activation features of each layer, the low-rank adaptation residual of the master expert corresponding to the master expert is obtained, and the master expert output is obtained based on the low-rank adaptation residual of the master expert and the basic output. Based on the activation features of each layer, obtain several low-rank adaptation residuals of all auxiliary experts in the set of activated auxiliary experts, and superimpose them based on the output of the main expert, the low-rank adaptation residuals of several auxiliary experts and the weight distribution of auxiliary experts to obtain the target feature representation. The target feature representation is input into the independent task head corresponding to the master expert for forward computation to obtain the social context prediction result.

[0007] This invention provides a prediction method for orthogonal low-rank adaptation. First, it determines the primary expert by acquiring the social task graph data to be tested. Then, it freezes the backbone network to obtain activation features and basic outputs. Next, a symmetric dual-drive router calculates expert gating, filters activation auxiliary experts, and assigns weights. Finally, it fuses the low-rank adaptation residuals of the primary and auxiliary experts to obtain the target feature representation, which is then input into the corresponding task header to complete the prediction. Freezing the backbone network ensures the stability of the general representation, avoiding knowledge overwriting caused by full parameter updates. The orthogonal low-rank adaptation expert database enables modular reuse of cross-domain knowledge, and the symmetric dual-drive router adaptively selects effective expert combinations, enabling prediction of cross-domain graph data without online fine-tuning. This prediction method effectively solves the technical problems of existing graph cross-domain sequence learning, such as the need for online fine-tuning during the inference stage, poor cross-domain transfer performance, and high deployment costs. Furthermore, it achieves positive cross-domain transfer based on the selection of expert combinations, improving the accuracy and stability of social scenario prediction. It can accurately capture the contextual association features of different social domains based on cross-domain data characteristics from multiple platforms and scenarios in social scenario prediction, significantly improving the recognition accuracy and response speed for typical social scenarios such as marketing dissemination, collaborative traffic boosting, and rumor propagation. At the same time, the inference mode, which requires no online fine-tuning, perfectly adapts to the real-time analysis needs of social media sentiment analysis, content governance, and other businesses, effectively reducing the deployment and maintenance costs of intelligent analysis systems in the social domain.

[0008] Further, the step of obtaining the low-rank adaptation residual of the master expert corresponding to the master expert based on the activation features of each layer, and obtaining the master expert output based on the low-rank adaptation residual of the master expert and the basic output, includes: Based on the chief expert, obtain the current update direction basis and the current low-rank fit coefficient matrix; Linear projection is performed based on the current layer activation features and the current update direction basis in the activation features of each layer to obtain low-dimensional features; The low-dimensional features are nonlinearly activated, and back projection is performed based on the current low-rank adaptation coefficient matrix to obtain the master expert low-rank adaptation residual.

[0009] The above scheme obtains low-dimensional features by linearly projecting the layer activation features onto the update direction basis corresponding to the master expert. After nonlinear activation, the residuals are obtained by back-projecting the low-rank adaptation coefficient matrix. A fixed update direction basis is used to limit the safe update space for low-rank adaptation. Adaptation is completed by decomposing the low-rank matrix with only a small number of coefficient matrix parameters, achieving efficient parameter updates. At the same time, it reduces memory and computational overhead and ensures the relevance of residual calculation. This allows the master expert output to accurately match the feature requirements of the corresponding task, solving the technical problems of unconstrained update direction, easy interference in parameter updates, and insufficient relevance of adaptation in existing efficient parameter fine-tuning.

[0010] Furthermore, the pre-training process of the graph prediction model includes: Obtain the multi-source social cross-domain graph task sequence, and perform fine-tuning training actions on the graph data of any task sample in the multi-source social cross-domain graph task sequence and the pre-built graph basic model in chronological order, and obtain the current orthogonal low-rank adaptation expert library and the current symmetric dual-drive router; The current orthogonal low-rank adaptation expert library and the current symmetric dual-drive router are used as the initial orthogonal low-rank adaptation expert library and the initial symmetric dual-drive router for the next training task graph data. The fine-tuning training action is repeatedly executed until all task sample graph data in the multi-source social cross-domain graph task sequence complete the fine-tuning training action, thereby obtaining the fine-tuned orthogonal low-rank adaptation expert library and symmetric dual-drive router, and then obtaining a graph prediction model including the frozen backbone network, the orthogonal low-rank adaptation expert library and the symmetric dual-drive router.

[0011] The above scheme trains by successively fine-tuning the multi-source social cross-domain graph task sequence in chronological order, using the expert database and router obtained from the previous task as the initial values ​​for the next task for continuous iteration. This allows the model to gradually absorb new domain social knowledge while retaining historical domain knowledge, achieving cumulative learning of knowledge. It solves the technical problems of subsequent tasks overwriting the knowledge of previous tasks, easily causing catastrophic forgetting, and being unable to continuously absorb new knowledge in the scenario of cross-domain sequence arrival in existing direct full-parameter sequential fine-tuning.

[0012] Furthermore, for any task sample graph data, the fine-tuning training action includes: Based on the graph-based model, a frozen backbone model including several adaptable layers is obtained. The task sample graph data is input into the frozen backbone network for layer-by-layer message passing to obtain the historical layer activation features of the samples. For any current adaptable layer among the aforementioned adaptable layers: The current layer sample update direction basis is obtained based on the historical layer activation features of the samples. Based on the pre-acquired initial orthogonal low-rank fitting expert database and the preset symmetric training principle, any task expert is selected in turn as the sample master expert, the corresponding sample auxiliary expert set is determined, and several hybrid expert combinations are formed based on all sample master experts and the corresponding sample auxiliary expert sets. A joint loss function is constructed based on the pre-acquired initial symmetric dual-drive router, the task sample graph data, several hybrid expert combinations, and the frozen backbone network; Based on the joint optimization loss function and the preset joint optimization objective, the parameters of the initial symmetric dual-drive router, the initial data of the sample task head, and the initial low-rank adaptation coefficient matrix are trained and updated to obtain the current symmetric dual-drive router, the current sample low-rank adaptation coefficient, and the current task head parameters. Based on the current layer sample update direction basis, the current sample low-rank adaptation coefficient and the current task header parameters, a current task expert is constructed, and the current task expert is incorporated into the initial orthogonal low-rank adaptation expert library to form the current orthogonal low-rank adaptation expert library.

[0013] The above scheme uses a frozen graph-based backbone network. Based on this frozen backbone network, it obtains the activation features of sample layers to determine the update direction basis for each adaptable layer. Then, based on the principle of symmetric training, it selects sample master and assistant experts to form a hybrid combination, constructs a joint loss function to complete the parameter update of the router, task head, and low-rank adaptation coefficients, and finally incorporates the current task experts into the expert database. The symmetric training principle allows all experts to take turns participating in training as master experts, reducing the router's time bias and enabling the router to learn a globally consistent expert combination strategy. At the same time, the low-rank adaptation design that separates the update direction basis from the coefficient matrix achieves efficient parameter update by training only the coefficient matrix. This solves the technical problems of time bias in router training, difficulty in learning global transfer strategies, high parameter update overhead, and severe cross-task interference in existing hybrid expert routing.

[0014] Furthermore, the construction of the joint loss function based on the pre-acquired initial symmetric dual-drive router, the task sample graph data, several hybrid expert combinations, and the frozen backbone network includes: The sample gating distribution is obtained based on the pre-acquired initial symmetric dual-drive router and several hybrid expert combinations; The mixed test scenario loss is obtained based on the task sample graph data, several mixed expert combinations, and the frozen backbone network. The main sample scenario loss is obtained based on the task sample graph data, the sample master experts corresponding to several mixed expert combinations, and the frozen backbone network. The task loss is obtained based on the mixed test scenario loss and the main sample scenario loss. Based on the aforementioned mixed-test scenario loss, main sample scenario loss, and sample gating distribution, the utility prediction regression loss is obtained. A positive expert set and a negative expert set are constructed based on a preset gating threshold, the sample gating distribution, and an initial orthogonal low-rank fitting expert database. The contrast loss is calculated based on the sample principal experts, the positive expert set, and the negative expert set, and the contrast loss is calculated by symmetric averaging of the contrast loss of all sample principal experts to obtain the contrast loss. A joint loss function is constructed based on the task loss, utility prediction regression loss, contrast loss, and preset orthogonal constraints.

[0015] The above scheme obtains the sample gating distribution, calculates the task loss, utility prediction regression loss, and contrastive loss, and forms a joint loss function by combining them with preset orthogonal constraints. The difference between the loss predicted by the main and auxiliary experts and that predicted by the main expert alone is used as supervision data through the utility prediction regression loss, enabling the router to accurately learn the payoff patterns of the expert combination. The contrastive loss is used to construct positive and negative expert sets to differentiate expert functions. The orthogonal constraints structurally restrict the update direction. The three factors are combined with the task loss to achieve joint optimization, ensuring the training accuracy of the current task. This achieves effective router training and expert specialization, solving the technical problems of router training without clear supervision signals, redundant and conflicting expert functions, and easy conflicts in cross-task update directions in the existing technology.

[0016] Further, obtaining the current layer sample update direction basis based on the sample history layer activation features includes: Obtain the forward layer sample activation features from the historical layer activation features of the samples; Obtain the forward sample history subspace, and perform orthogonal transformation based on the forward sample history subspace to construct the orthogonal basis matrix of the sample history subspace of the current adaptable layer; Orthogonal complementary projection is performed based on the orthogonal basis matrix of the sample history subspace to obtain the orthogonal complementary projection matrix; the sample forward layer residual activation features are obtained based on the forward layer sample activation features and the orthogonal complementary projection matrix. Singular value decomposition is performed on the forward layer residual activation features of the sample to construct a left singular vector matrix; The current layer sample update direction basis is constructed based on the preset rank, the preset low-rank condition, and the left singular vector matrix; The current layer sample update direction basis is incorporated into the orthogonal basis matrix of the sample history subspace to obtain the current sample subspace of the current adaptable layer. The current sample subspace is then used as the forward sample history subspace of the next adaptable layer.

[0017] The above scheme obtains residual features by projecting the activation features of the feedforward layer onto the orthogonal complement of the historical subspace, extracts the principal direction through singular value decomposition to construct the update direction basis, and then incorporates it into the historical subspace to complete the update. By using orthogonal complement projection, the activation features of the new task are isolated from the historical knowledge subspace, so that the update direction of the new task remains orthogonal to the historical direction. Then, the principal direction is extracted based on singular value decomposition to construct a safe update subspace for the new task, thereby continuously orthogonalizing and updating the complete historical subspace. This ensures that the orthogonal constraints of subsequent tasks are always effective, effectively suppressing parameter interference across tasks. It solves the technical problem in existing playback and regularization-based continuous learning that it is difficult to avoid cross-domain update direction conflicts from the root and that the effect is unstable under strong structure drift.

[0018] Furthermore, for the first task sample image data, before performing the fine-tuning training action, the following steps are included: The parameters of the backbone network of the graph-based model are frozen to form a frozen backbone network including several adaptable layers. Perform router initialization operations to form the initial symmetric dual-drive router for the first task sample graph data; Initialize the initial orthogonal low-rank adaptation expert library of the first task sample graph data to an empty set; Empty state initialization is performed on the historical knowledge subspace states of several adaptable layers to obtain an empty state knowledge subspace, and the empty state knowledge subspace is used as the forward sample historical subspace of the first task sample graph data.

[0019] The above scheme establishes a standardized initial state for training the first cross-domain social task, sets up a frozen backbone network to provide a stable general representation base for all subsequent tasks, and initializes an empty expert library and historical subspace so that the update direction of the first task is not subject to historical constraints and can autonomously construct the initial knowledge subspace.

[0020] This invention provides a prediction system for orthogonal low-rank adaptation, implemented based on the aforementioned prediction method for orthogonal low-rank adaptation. The system includes: a data acquisition module, a graph prediction model module, an activation control module, and a prediction output module. The graph prediction model module includes a pre-trained graph prediction model module, which further includes a frozen backbone network, an orthogonal low-rank adaptation expert database, and a symmetric dual-drive router. The data acquisition module is used to acquire cross-domain graph data to be predicted based on the pre-acquired multi-source social data stream; The graph prediction model module is used to determine the master expert based on the cross-domain graph data to be predicted and the orthogonal low-rank fitting expert library; The graph prediction model module is also used to input the cross-domain graph data to be predicted into the frozen backbone network for layer-by-layer message passing, and to obtain the activation features and basic outputs of each layer. The graph prediction model module is also used to calculate expert benefits based on the activation features of each layer, the orthogonal low-rank adaptive expert library and the symmetric dual-drive router, and to obtain the expert gating data set. The activation control module is used to screen auxiliary experts based on the expert gating data set and the preset gating threshold, obtain an activated auxiliary expert set that meets the preset gating value condition, and obtain the auxiliary expert weight distribution based on the activated auxiliary expert set and the expert gating data set. The prediction output module is used to obtain the low-rank adaptation residual of the master expert corresponding to the master expert based on the activation features of each layer, and to obtain the master expert output based on the low-rank adaptation residual of the master expert and the basic output; to obtain several low-rank adaptation residuals of auxiliary experts corresponding to all auxiliary experts in the set of activated auxiliary experts based on the activation features of each layer, and to superimpose the master expert output, several low-rank adaptation residuals of auxiliary experts and auxiliary expert weight distribution to obtain the target feature representation; to input the target feature representation into the independent task head corresponding to the master expert for forward computation to obtain the social scenario prediction result.

[0021] The present invention provides an orthogonal low-rank adaptation prediction system. Through modular architecture design, core technologies such as orthogonal low-rank adaptation, symmetric dual-drive routing, and expert combination fusion are encapsulated into corresponding sub-modules, which greatly improves the maintainability and scalability of the system. The collaborative operation of each module ensures the process-oriented and efficient nature of cross-domain social graph prediction.

[0022] The present invention also provides a device, which is a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a prediction method for orthogonal low-rank adaptation provided by the present invention.

[0023] The present invention also provides a storage medium, which is a computer-readable storage medium, comprising: a stored computer program, wherein, when the computer program is executed, the device where the storage medium is located executes a prediction method for orthogonal low-rank adaptation provided by the present invention.

[0024] The implementation of this invention yields the following beneficial effects: Based on specific social scenario prediction, this invention involves the continuous fine-tuning of graph-based models in scenarios where graph data (from different sources and with different structures) arrives sequentially across multiple graph domains. Addressing the problems of catastrophic forgetting, unstable cross-domain transfer, and mutual interference in parameter updates that arise in existing methods during cross-domain sequential learning, this invention proposes a continuous fine-tuning framework based on orthogonal subspace constraints and low-rank adaptation expert combination routing. This framework restricts the low-rank adaptation update direction of each new domain / task to the orthogonal complement space of the historical knowledge subspace, thus suppressing inter-task interference from a mechanistic perspective. Furthermore, it constructs an expert library from the low-rank adaptation modules of each task, and automatically selects / combines experts from the input graph through a router to achieve cross-domain positive transfer. This effectively improves performance while ensuring parameter efficiency (updating only a small number of low-rank adaptation parameters). The model's averaging and stability through cross-domain continuous learning reduce forgetting and enable expert combination predictions without additional online fine-tuning during the inference phase. Particularly noteworthy is its ability to quickly adapt to the contextual recognition needs of different social domains, given the real-time influx of data from multiple platforms and rapidly changing scenario characteristics in social context prediction. This significantly improves the efficiency and accuracy of identifying typical social scenarios such as rumor spread, coordinated traffic manipulation, and sudden public opinion crises. It effectively reduces the real-time analysis costs of social business, achieving superior cross-domain migration effects in social context prediction. While ensuring the accuracy and stability of social context predictions, it also reduces the deployment and long-term maintenance costs of the model, thus enabling efficient, stable, and accurate continuous learning and prediction in social context prediction scenarios. This meets the real-time and intelligent needs of practical businesses such as social content governance, public opinion early warning, and risk control auditing.

[0025] Specifically: Through the core mechanism of orthogonal low-rank adaptation, the update of new tasks is restricted to the orthogonal complement space of the historical knowledge subspace, significantly reducing cross-task parameter interference structurally and effectively mitigating the catastrophic forgetting problem. This prevents the model training of new social domains from overwriting the recognition knowledge of existing social scenarios, ensuring stable prediction capabilities for historical social scenarios. Leveraging the symmetric training, utility prediction supervision, and expert differentiation constraints of the symmetric dual-drive router, the router learns globally consistent expert combination strategies, achieving positive transfer of cross-domain knowledge. This effectively improves the long-term stability and prediction accuracy of cross-domain learning, thereby accurately uncovering knowledge connections between different social domains. This allows the scenario recognition experience of mature social domains to provide effective support for new domains, realizing the reuse of cross-domain capabilities for social scenario prediction. The parameter design of low-rank adaptation only updates a small number of coefficient matrix parameters. Combined with the strategy of freezing the backbone network, this significantly reduces the GPU memory, storage, and time costs during training, eliminating the need for online fine-tuning during the inference phase. The expert-led prediction system further reduces deployment and maintenance costs, perfectly adapting to the real-time analysis needs of social media sentiment early warning and content governance. It requires no additional online fine-tuning computing power and can quickly respond to inference requests from social data streams. Simultaneously, the overall solution achieves technological implementation across multiple dimensions, including methodology, model, system, terminal deployment, and storage media. This ensures the accuracy and stability of predictions in multi-source social cross-domain graph data scenarios while enhancing the engineering practicality, scalability, and portability of the method. It effectively improves the long-term availability and reliability of cross-industry graph intelligence systems in scenarios such as e-commerce, financial risk control, and social networks. The constructed and trained graph prediction model can continuously absorb new domain knowledge in scenarios with incoming cross-domain sequences while stably retaining the prediction capabilities of the old domain. This provides efficient and stable technical support for practical businesses such as intelligent risk control, sentiment monitoring, and content management in the social media field, helping social platforms achieve more accurate scenario-based governance and decision-making. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of a prediction method for orthogonal low-rank adaptation provided in this embodiment; Figure 2 This is a schematic diagram of a prediction system for orthogonal low-rank adaptation provided in this embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0030] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0033] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0034] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0035] Example 1: This embodiment provides a prediction method for orthogonal low-rank adaptation, applied to a pre-trained graph prediction model. The graph prediction model includes a frozen backbone network, an orthogonal low-rank adaptation expert database, and a symmetric dual-drive router, such as... Figure 1 As shown, the method includes: S1. Obtain cross-domain graph data to be predicted based on pre-acquired multi-source social data streams; S2. Determine the master expert based on the cross-domain graph data to be predicted and the orthogonal low-rank fitting expert database; S3. Input the cross-domain graph data to be predicted into the frozen backbone network for layer-by-layer message passing, and obtain the activation features and basic outputs of each layer. S4. Based on the activation features of each layer, the orthogonal low-rank adaptation expert library and the symmetric dual-drive router, calculate the expert benefit and obtain the expert gating data set. S5. Based on the expert gating data set and the preset gating threshold, perform auxiliary expert screening to obtain an active auxiliary expert set that meets the preset gating value conditions, and obtain the auxiliary expert weight distribution based on the active auxiliary expert set and the expert gating data set. S6. Based on the activation features of each layer, obtain the low-rank adaptation residual of the master expert corresponding to the master expert, and obtain the master expert output based on the low-rank adaptation residual of the master expert and the basic output. S7. Based on the activation features of each layer, obtain several low-rank adaptation residuals of all auxiliary experts in the set of activated auxiliary experts, and superimpose them based on the output of the main expert, the low-rank adaptation residuals of several auxiliary experts and the weight distribution of auxiliary experts to obtain the target feature representation. S8. Input the target feature representation into the independent task head corresponding to the master expert for forward computation to obtain the social scenario prediction result.

[0036] In specific implementation, this embodiment takes continuous scene recognition of multi-source social media as a concrete application example. The system continuously accesses social data streams from different social platforms or different community domains (i.e., multi-source / multi-domain), thus obtaining pre-acquired multi-source social data streams. For the same topic or event, it aggregates its dissemination and interaction information from various data sources to construct a task graph data, thereby obtaining the cross-domain graph data to be predicted. This cross-domain graph data to be predicted includes: a set of nodes, where nodes can be content entities such as user accounts, posts / comments, etc.; a set of edges, where edges represent the interaction relationships between users and content (posting, commenting, forwarding, liking, @mentioning, etc.) and the relationships between users (following, mutual following, reply chains, etc., determined according to the actual available data); and a node feature matrix (or graph-level feature matrix) composed of the attributes associated with nodes and edges, such as user activity frequency, account duration, follower / follower statistics, device or geographic distribution characteristics, content theme / similarity characteristics, external link ratio, keyword hit rate, and dissemination structure and interaction pattern characteristics (such as dissemination depth, aggregation degree, repetitive behavior patterns, etc.). In the inference phase of the graph prediction model, the orthogonal low-rank adaptation prediction method provided in this embodiment, after prediction, will output a clear downstream business result for the task graph data, namely the social scenario prediction result. This social scenario prediction result includes: providing a judgment result and its confidence / risk level among predefined multiple scenario categories, such as "normal natural discussion," "marketing promotion diffusion," "collaborative traffic manipulation," "rapid spread of rumors," and "real outbreak of sudden events." For example, when the input cross-domain graph data to be predicted exhibits characteristics such as a large number of accounts intensively forwarding highly similar content within a short period and the interaction structure showing multi-cluster synchronous startup, the final output social scenario prediction result can be: judged as a "collaborative traffic manipulation" scenario, and given a corresponding confidence level or risk level. This output can be directly used to trigger content governance, public opinion warning, or risk control review processes, providing a decision-making basis for intelligent operation and management in the social field.

[0037] In the specific implementation process, when acquiring the cross-domain graph data to be predicted based on the pre-acquired multi-source social data stream, the pre-acquired multi-source social data stream includes several cross-domain graph data to be predicted in chronological order.

[0038] Optionally, step S6 includes: Based on the chief expert, obtain the current update direction basis and the current low-rank fit coefficient matrix; Linear projection is performed based on the current layer activation features and the current update direction basis in the activation features of each layer to obtain low-dimensional features; The low-dimensional features are nonlinearly activated, and back projection is performed based on the current low-rank adaptation coefficient matrix to obtain the master expert low-rank adaptation residual.

[0039] In practical implementation, when the graph prediction model performs inference prediction, it merges the expert gating datasets output by the symmetric dual-drive router to form an activation auxiliary expert set. And the gating is normalized to obtain the auxiliary expert weight distribution. .

[0040] The final representation is obtained by combining the output of the primary expert with the residual weighted combination of the secondary expert. The specific process is shown in the following formula: ; In the formula, This indicates that the basic output of the frozen backbone network for predicting cross-domain graph data is... This represents the low-rank fitting residual of the principal expert, determined by the principal expert's fitting parameters. (based on The low-rank adaptation matrix is ​​calculated and then the task-specific information corresponding to the current master expert is injected. This represents the weighted sum of the low-rank adaptation residuals of several auxiliary experts, and the summation iterates through all sets of activated auxiliary experts. The expert in charge.

[0041] Finally Input task t, the independent task header corresponding to the main expert. Perform forward computation and output the predicted social context prediction results; use the respective retained task heads when reasoning in historical tasks.

[0042] The aforementioned prediction process (i.e., the inference phase of the graph prediction model) requires no additional online fine-tuning. The symmetric dual-drive router first outputs an expert gating data set based on the cross-domain graph data to be predicted, the orthogonal low-rank adaptation expert library, and the main expert selection rules. It then filters the data using a preset threshold to obtain an active auxiliary expert set and normalizes the auxiliary expert gating values ​​within this set to generate corresponding combined weights. This allows for adaptive selection of transferable historical experts based on the input, improving the model's generalization ability and prediction stability. The final feature representation of the graph prediction model is obtained by superimposing the main expert output with the residuals of multiple auxiliary experts and weighting them according to the combined weights. Inputting this final feature representation into the corresponding task header completes the prediction. This method enables cross-domain positive transfer without online fine-tuning, effectively reducing model deployment costs. For inference of historical tasks, the corresponding independent task header is directly called, and the router executes the same expert combination process as described above. This effectively maintains the prediction performance of the historical domain without replaying historical data or performing full parameter updates, improving the stability of the historical domain during long-term model operation.

[0043] Optionally, the pre-training process of the graph prediction model includes: Obtain the multi-source social cross-domain graph task sequence, and perform fine-tuning training actions on the graph data of any task sample in the multi-source social cross-domain graph task sequence and the pre-built graph basic model in chronological order, and obtain the current orthogonal low-rank adaptation expert library and the current symmetric dual-drive router; The current orthogonal low-rank adaptation expert library and the current symmetric dual-drive router are used as the initial orthogonal low-rank adaptation expert library and the initial symmetric dual-drive router for the next training task graph data. The fine-tuning training action is repeatedly executed until all task sample graph data in the multi-source social cross-domain graph task sequence complete the fine-tuning training action, thereby obtaining the fine-tuned orthogonal low-rank adaptation expert library and symmetric dual-drive router, and then obtaining a graph prediction model including the frozen backbone network, the orthogonal low-rank adaptation expert library and the symmetric dual-drive router.

[0044] In the specific implementation process, training sets, test sets, and validation sets are constructed based on historical social data streams from data sources, business systems, public datasets, and different social platforms or community domains, forming the multi-source social cross-domain graph task sequence, represented as follows: The total number of tasks is T, and these tasks arrive sequentially across domains, meaning adjacent tasks may come from different distributions to reflect cross-domain variations. These tasks are arranged in chronological order to form a task sequence, enabling continuous cross-domain fine-tuning during subsequent training. The task sample image data corresponding to the t-th task is represented as follows: ,in Represents a set of nodes (with a number of nodes). , ), Denotes the set of edges. Represents the node feature matrix, where R represents the set of real numbers, and d represents the feature dimension.

[0045] Optionally, a supervision signal as a label vector is added to the multi-source social cross-domain graph task sequence. This is used for fine-tuning training. The supervision signal is a label manually labeled by domain experts or the review team in real business scenarios, or a label formed from the platform's historical handling conclusions (such as violation judgments and manual review results); or a label and partition given in advance in a public benchmark dataset. During training, these labels and partitions are directly read from the data file as supervision signals (for example, public datasets such as OGB / ogbn provide node category labels and standard training / validation / test partitions with the data release).

[0046] Optionally, for any task sample graph data, the fine-tuning training action includes: Based on the graph-based model, a frozen backbone model including several adaptable layers is obtained. The task sample graph data is input into the frozen backbone network for layer-by-layer message passing to obtain the historical layer activation features of the samples. For any current adaptable layer among the aforementioned adaptable layers: The current layer sample update direction basis is obtained based on the historical layer activation features of the samples. Based on the pre-acquired initial orthogonal low-rank fitting expert database and the preset symmetric training principle, any task expert is selected in turn as the sample master expert, the corresponding sample auxiliary expert set is determined, and several hybrid expert combinations are formed based on all sample master experts and the corresponding sample auxiliary expert sets. A joint loss function is constructed based on the pre-acquired initial symmetric dual-drive router, the task sample graph data, several hybrid expert combinations, and the frozen backbone network; Based on the joint optimization loss function and the preset joint optimization objective, the parameters of the initial symmetric dual-drive router, the initial data of the sample task head, and the initial low-rank adaptation coefficient matrix are trained and updated to obtain the current symmetric dual-drive router, the current sample low-rank adaptation coefficient, and the current task head parameters. Based on the current layer sample update direction basis, the current sample low-rank adaptation coefficient and the current task header parameters, a current task expert is constructed, and the current task expert is incorporated into the initial orthogonal low-rank adaptation expert library to form the current orthogonal low-rank adaptation expert library.

[0047] In the specific implementation process, the process parameters of the backbone network of the pre-trained graph-based model are frozen, serving as the frozen backbone network. The pre-trained backbone network is loaded, and its parameters are kept frozen throughout the entire sequence learning process (i.e., the fine-tuning training process) to avoid full parameter drift leading to the overwriting of old domain capabilities. This frozen backbone network is denoted as... ,in The entire sequence learning process is kept frozen and not updated. This frozen backbone network only performs forward computation to provide a general representation and a stable base, so that the updates of continuous learning are focused on the low-rank adaptation expert library and the symmetric dual-drive router, reducing the risk of forgetting and resource overhead.

[0048] Optionally, obtaining the current layer sample update direction basis based on the sample history layer activation features includes: Obtain the forward layer sample activation features from the historical layer activation features of the samples; Obtain the forward sample history subspace, and perform orthogonal transformation based on the forward sample history subspace to construct the orthogonal basis matrix of the sample history subspace of the current adaptable layer; Orthogonal complementary projection is performed based on the orthogonal basis matrix of the sample history subspace to obtain the orthogonal complementary projection matrix; the sample forward layer residual activation features are obtained based on the forward layer sample activation features and the orthogonal complementary projection matrix. Singular value decomposition is performed on the forward layer residual activation features of the sample to construct a left singular vector matrix; The current layer sample update direction basis is constructed based on the preset rank, the preset low-rank condition, and the left singular vector matrix; The current layer sample update direction basis is incorporated into the orthogonal basis matrix of the sample history subspace to obtain the current sample subspace of the current adaptable layer. The current sample subspace is then used as the forward sample history subspace of the next adaptable layer.

[0049] In this embodiment, the frozen backbone network is a graph neural network including L adaptable layers, and the layer index is set to... ;No. The layer input / output representation dimensions are respectively , ..., When obtaining the current layer activation features of a sample by performing layer-by-layer message passing / aggregation on the task sample graph data based on the frozen backbone network, the current layer activation features of the sample in the l-th layer (i.e., the currently adaptable layer) are represented as follows: , , σ(⋅) represents the activation features of the forward layer samples of the l-th layer, which is the activation features of the samples of the (l-1)-th layer in the current layer; σ(⋅) represents the non-linear activation function (such as ReLU, GELU, Sigmoid, etc.), which introduces non-linear transformations into the network, allowing the model to fit complex data patterns; This represents the learnable weight matrix of the l-th layer, with dimension 1. Used to transfer input features from Dimension mapping to dimension.

[0050] Specifically, this embodiment constructs an orthogonal low-rank adaptation expert through the following process: For each task sample graph data t, a low-rank adaptation update mechanism is introduced at each adaptable layer, as shown in the following equation: The dimension constraint of the low-rank matrix is: And satisfy the low-rank condition Where r represents the preset rank, This represents the current layer sample update direction basis of the l-th layer. This represents the low-rank adaptation coefficient of the current sample in layer l. Specifically, a low-rank adaptation module is introduced into each adaptable layer of the frozen backbone network, so that the output of the layer is determined by the frozen backbone output and the low-rank adaptation residual; where A is used as the update direction basis (set to be fixed and untrainable), and B is used as the coefficient (set to be trainable).

[0051] For each layer l, the update direction of the historical task at that layer is based on A (history). The historical knowledge subspace is summarized and formed, which is the forward sample history subspace of the current adaptable layer; at the same time, its orthogonal normalized basis matrix is ​​maintained to facilitate subsequent orthogonal complementary projection. Specifically, for the current adaptable layer l, the forward sample history subspace can be represented as follows: Obtain the orthogonal normalized basis matrix of the current sample subspace as follows: Orthogonal complementary projection matrix is ​​obtained by performing orthogonal complementary projection based on the orthogonal basis matrix of the sample history subspace. Next, activate the projection (to isolate new information), and... Projecting onto the orthogonal complement of the historical subspace, we obtain the activation features of the sample forward layer residuals. Specifically, as shown in the following formula: In this embodiment, I is used as the identity matrix, which is used to construct a projection operator that removes historical features and retains only new features. The sample forward layer residual activation feature represents the part newly added in the current task and linearly independent of the historical direction.

[0052] Next, subspace extraction (constructing safe directions) is performed. First, singular value decomposition is performed on the projected activation covariance to obtain the left singular vector matrix U, as shown in the following equation: In the formula, This represents singular value decomposition. express The transpose of the matrix has the following dimensions: , Represents a singular value diagonal matrix with dimension . , Denotes a right singular vector matrix with dimension . , The covariance matrix representing the residual activation features has dimensions of . Since the activation features of the input residuals of singular value decomposition are symmetric positive semi-definite matrices (covariance matrices), they satisfy the following condition: =U. Next, subspace extraction is performed based on a preset rank r and a preset low-rank condition. The preset low-rank condition is used to construct the current layer sample update direction basis from the extracted r principal directions. Then we get This process isolates new information added by new tasks from the mechanism, avoiding interference and forgetting caused by the overlap between the learning direction and the historical direction. It also limits the allowed update directions to a safe space orthogonal to the historical direction, structurally reducing cross-task conflicts.

[0053] Then, the historical basis is updated based on the current layer sample update direction basis. The current layer sample update direction basis of the current training task is incorporated into the forward sample historical subspace and orthogonalized based on QR matrix decomposition to obtain the current sample subspace of the current adaptable layer. The specific formula is as follows: The process involves concatenating the orthogonal basis matrix of the historical sample subspace with the current layer sample update direction basis column-wise to obtain a new matrix. This concatenated matrix is ​​then subjected to QR decomposition, decomposing it into an orthogonal matrix Q and an upper triangular matrix R. The resulting orthogonal matrix Q is the updated orthogonal basis matrix of the current sample subspace, with its column vectors being pairwise orthogonal and of unit length, ensuring the orthogonality between the historical knowledge subspace and the new task update direction. This process aims to merge the old and new direction bases and re-orthogonalize them, ensuring that the historical knowledge subspace always maintains a clean and orthogonal basis. This structurally eliminates cross-task parameter interference and allows the historical knowledge subspace to expand with task accumulation, guaranteeing that subsequent tasks can still construct new directions on its orthogonal pad.

[0054] Based on this, the process by which the current adaptable layer obtains the activation features of the current layer of the sample can be represented as follows: ; in, This represents the basic forward propagation function of the l-th layer of the frozen backbone network. This represents task sample graph data; This represents the fixed parameter set of the l-th layer of the frozen backbone network, which remains unchanged during training. It is used to extract general basic features from graph data, ensuring the stability of basic representations across tasks. This embodiment sets... To fix the untrainable, only train. (and the corresponding independent task header) makes updates strictly limited to safe orthogonal directions.

[0055] In this embodiment, the low-rank adaptation parameter set of the t-th task sample graph data is defined as follows: Let the cross-layer low-rank adaptation modules (A and B of each layer) of task t be used as the low-rank adaptation experts of task t (i.e., the current task experts). These experts are used to represent the incremental knowledge of the domain / task. Then, the current orthogonal low-rank adaptation expert library formed after training on task t should be: Furthermore, in this embodiment, an independent task header is reserved for each task. This is used to process the label space of different tasks, and the task supervision loss set in this embodiment is as follows: ; in, This represents the set of training nodes for the t-th task, which is the subset of nodes used for training the model in the current task. This indicates that the prediction for node v is the true label. The probability value, where the set of low-rank parameters that can be trained for the current task is . - This represents the cross-entropy loss term for a single node. The closer the model's predicted probability value is to 1, the smaller this term becomes, indicating a more accurate prediction; conversely, the larger the value becomes, indicating a greater prediction error.

[0056] This training process employs standard supervised learning loss (e.g., cross-entropy for node classification). During the current task training phase, A remains fixed and is not updated; only B (and the current task header parameters) are updated, ensuring the graph-based model can only adapt along safe orthogonal directions. After training, the (A, B) set across layers of the task is solidified as new experts and added to the expert database. By using fixed directions and trainable amplitudes, updates are restricted to a safe space, improving stability and reducing forgetting. Specifically, the logarithm and negative of the probability corresponding to the true class in the predicted probability of the training node are taken, and then averaged across the training nodes. This loss is used to optimize B and the task header parameters for the current task, driving performance improvement without violating the safe directional constraints of A.

[0057] In the specific implementation process, after task t is fully trained, the following results are obtained: and A current task expert is formed, which can be stored and reused to modularly encapsulate the incremental knowledge of each task, facilitating subsequent combination and transfer. After training up to task t, all current task experts up to task t are accumulated to form the current orthogonal low-rank adaptation expert library, providing a set of composable knowledge components for cross-domain transfer. Simultaneously, each task / domain retains an independent task header to adapt to potential differences in label spaces between different tasks. During training, the task header is updated together with the parameters B of the current task. The independent task header parameters of the current task are determined based on the supervision signal of task t, the current task identifier, etc., which avoids negative transfer caused by inconsistencies in label spaces between different tasks and ensures that the corresponding task header is called during historical task inference.

[0058] Optionally, the construction of the joint loss function based on the pre-acquired initial symmetric dual-drive router, the task sample graph data, several hybrid expert combinations, and the frozen backbone network includes: The sample gating distribution is obtained based on the pre-acquired initial symmetric dual-drive router and several hybrid expert combinations; The mixed test scenario loss is obtained based on the task sample graph data, several mixed expert combinations, and the frozen backbone network. The main sample scenario loss is obtained based on the task sample graph data, the sample master experts corresponding to several mixed expert combinations, and the frozen backbone network. The task loss is obtained based on the mixed test scenario loss and the main sample scenario loss. Based on the aforementioned mixed-test scenario loss, main sample scenario loss, and sample gating distribution, the utility prediction regression loss is obtained. A positive expert set and a negative expert set are constructed based on a preset gating threshold, the sample gating distribution, and an initial orthogonal low-rank fitting expert database. The contrast loss is calculated based on the sample principal experts, the positive expert set, and the negative expert set, and the contrast loss is calculated by symmetric averaging of the contrast loss of all sample principal experts to obtain the contrast loss. A joint loss function is constructed based on the task loss, utility prediction regression loss, contrast loss, and preset orthogonal constraints.

[0059] In its implementation, this embodiment uses a symmetric dual-drive router to achieve utility prediction and comparative expert specialization. Specifically: The symmetric training principle is set up so that, when training task t, each existing expert k∈{1,…,t} in the pre-acquired initial orthogonal low-rank fitting expert library takes turns as the sample master expert to learn the combination strategy from multiple perspectives, reduce the time bias, and form several hybrid expert combinations.

[0060] When the primary expert is k and a set of secondary experts includes secondary expert j, the gated data corresponding to secondary expert j is output based on the initial symmetric dual-drive router. , representing the expected benefit of auxiliary expert j collaborating with the current sample master expert k. Gating data corresponding to all hybrid expert combinations are obtained to obtain the sample gating distribution. Specifically, during training task t, the router is not only trained from the perspective of the current task expert, but each existing expert in the expert database takes turns as the sample master expert, learning its combination strategy with other experts on the same task data. This weakens the time bias that later tasks receive more biased attention, learns globally consistent transfer combination rules, and improves the long-term stability of the graph prediction model. The initial symmetric dual-drive router outputs a gating value (0 to 1) for each candidate auxiliary expert based on the task sample graph data and the master expert index (i.e., the identification information / identity code of the sample master expert), indicating whether the collaboration between the auxiliary expert and the master expert is likely to bring benefits. The process begins by using a frozen backbone network (with the adaptive increments of the primary expert samples overlaid) to perform a forward encoding on the input task sample graph data, obtaining the global context features of the task sample graph data (which can be obtained by pooling / aggregating node representations). Then, these global context features, along with the identity codes of the primary expert and candidate auxiliary experts from the set of auxiliary experts, are input into an initial symmetric dual-drive router. This initial symmetric dual-drive router outputs a gating value for each candidate auxiliary expert through a small neural network segment (such as a multilayer perceptron), indicating whether collaboration between the auxiliary expert and the primary expert is likely to bring benefits, thus obtaining the sample gating distribution for all hybrid expert combinations. Finally, auxiliary experts to be activated are selected based on a threshold, and the gating values ​​are normalized to obtain the combined weights used for subsequent expert fusion. This process enables the final graph prediction model to adaptively select transferable experts based on the input. This process is identical to the inference process of the graph prediction model.

[0061] In this embodiment, the experiential synergy utility is defined as follows: ; in, This indicates that only the graph prediction model of the master expert k is enabled. This indicates a graph prediction model jointly used by the lead expert and assistant experts. This represents the loss in the main sample scenario. This represents the loss in the mixed testing scenario. The difference in task loss is used as an empirical measure of collaborative benefit. If the task loss decreases after combination, it indicates that the auxiliary expert has made a positive contribution to the current input, providing explicit supervision for router gating learning, rather than relying solely on heuristic rules. The optimal expert combination is obtained through the calculation of this empirical collaborative utility, and the corresponding task loss is obtained. .

[0062] Next, we define the utility prediction regression loss. The specific formula is as follows: ; Align the router's gating predictions with the collaborative revenue signals, and train the initial symmetric dual-drive router with a regressive loss (e.g., mean squared error) to make its output closer to the real revenue, so that the router learns which expert combinations are more effective under which graph inputs.

[0063] This embodiment also adds comparison expert specialization, using a gate threshold. A positive and negative set is formed, and experts in the positive set (experts worth collaborating with) and experts in the negative set (experts not worth collaborating with) are selected based on the gate value and threshold, as shown in the following formula: ; In the formula: Indicates a collection of positive experts. Let represent the set of negative experts with the k-th expert as the principal expert, including experts who have no significant positive transfer to the current principal expert or even produce negative transfer. This indicates the preset gate threshold.

[0064] Furthermore, a contrastive learning-based differentiation constraint is introduced: a contrastive loss (in the form of InfoNCE) is used to bring the positive set closer and push away the negative set, that is, to bring closer the similarity of positive set experts in the router's internal representation space and push away the similarity of negative set experts; a consistency loss is obtained, which is symmetrically averaged over all master expert perspectives (master expert k) to promote clear expert function boundaries, reduce redundant conflicts, and improve combinatorial stability and interpretability, thus yielding the contrastive loss. The specific process is shown in the following formula: ; In the formula, This represents the single-expert comparison loss term when the k-th expert is the principal expert, based on the positive expert set constructed earlier. With negative experts The calculated feature representations are used to guide the model to learn feature representations that are more similar to positive expert combinations and more distant from negative expert combinations, and are used to suppress negative transfer and strengthen positive transfer.

[0065] In summary, this embodiment defines the set of parameters to be trained as follows: Where φ represents the initial symmetric dual-drive router parameters. The initial low-rank adaptation coefficient matrix for all layers of the t-th task. The parameter set is used to learn the adaptation weights of the current task in the safe orthogonal direction. This is the initial data for the sample task header. In this embodiment, the joint loss function is constructed as follows: In the formula and These are pre-set weighting coefficients, which must also satisfy pre-set orthogonal constraints, specifically... This constraint ensures that the new task update subspace is orthogonal to the historical subspace, thus structurally suppressing cross-task interference and forgetting.

[0066] Based on this, the preset joint optimization objective set in this embodiment is: , .

[0067] In practical applications, when performing joint optimization and constraints based on a preset joint optimization objective, task loss ensures the accuracy of the current task, utility regression loss ensures the reliability of gating predictions, and differentiation loss ensures expert specialization. It also consistently satisfies the structural constraint that the new task A is orthogonal to the historical subspace, jointly addressing... The parameters are updated to obtain the current symmetric dual-drive router, the current sample low-rank adaptation coefficient, and the current task header parameters. Based on this, the final graph prediction model takes into account task performance, transferability, and long-term stability, and avoids forgetting and time bias.

[0068] Optionally, for the first task sample image data, before performing the fine-tuning training action, the following steps are included: The parameters of the backbone network of the graph-based model are frozen to form a frozen backbone network including several adaptable layers. Perform router initialization operations to form the initial symmetric dual-drive router for the first task sample graph data; Initialize the initial orthogonal low-rank adaptation expert library of the first task sample graph data to an empty set; Empty state initialization is performed on the historical knowledge subspace states of several adaptable layers to obtain an empty state knowledge subspace, and the empty state knowledge subspace is used as the forward sample historical subspace of the first task sample graph data.

[0069] In practical applications, this embodiment employs a symmetric training routing mechanism. A symmetric protocol is used during router training, allowing different experts to take turns acting as the primary expert for combined learning. This effectively reduces time bias during training and improves the transferability of knowledge across tasks. Simultaneously, the difference in task loss between the primary expert's individual prediction and the combined prediction of the primary expert and auxiliary experts is used as a collaborative utility signal to supervise the router, enabling it to accurately learn which expert combinations can bring positive transfer benefits to the current task. Furthermore, an expert differentiation constraint is introduced as a regularization term to promote the specialization of expert functions, further improving the prediction effect and stability of expert combinations. In addition, the dimension size of the low-rank adaptation can be flexibly selected according to actual computing power and performance requirements, achieving an optimal trade-off between model prediction accuracy and computational cost.

[0070] Based on the above implementation process, this embodiment also has several equivalent alternatives: the subspace extraction method can use principal component analysis, orthogonal decomposition, randomized decomposition and other methods to construct an approximately orthogonal update subspace, as long as the constraint that the new task update direction is orthogonal or approximately orthogonal to the historical direction is met; the router structure can use multilayer perceptron, attention network, graph neural network, dual tower structure and other methods to realize expert weight output, as long as it can generate composable expert weights and use synergistic utility or differentiation constraints to complete training; the expert granularity can be divided according to task, domain cluster, structure type or hierarchical module to adapt to longer task sequences and finer-grained specialization needs.

[0071] This embodiment provides a prediction method based on orthogonal low-rank fitting. Through the construction of low-rank fittings constrained by orthogonal subspaces, the update direction of the new task remains structurally orthogonal or approximately orthogonal to the historical update direction, thereby reducing cross-task parameter interference and effectively mitigating the catastrophic forgetting problem in continuous learning. The "expert database + dynamic routing combination" architecture design enables efficient reuse of fitting capabilities across different graph domains, significantly improving cross-domain positive transfer performance and long-term model stability. Simultaneously, the introduction of expert differentiation constraints encourages the specialization of different low-rank fitting experts, reducing redundancy and conflict among experts and further enhancing the prediction performance and interpretability of expert combinations. In terms of economic efficiency, this invention trains only a small number of low-rank fitting parameters and router parameters, significantly reducing memory usage, storage overhead, and time costs during training compared to full-parameter fine-tuning. During the inference phase, dynamic expert combination and prediction can be completed with a single forward propagation, eliminating the need for online fine-tuning for new inputs and effectively reducing the cost of model deployment and long-term maintenance. In terms of social impact, this invention can improve the long-term availability and reliability of cross-industry graph intelligence systems, which is conducive to continuous iterative upgrades in scenarios such as e-commerce, financial risk control, transportation and infrastructure governance, and scientific knowledge graphs, and provides stable and efficient technical support for intelligent analysis and prediction of multi-source cross-domain graph data.

[0072] In summary, this embodiment proposes a comprehensive technical framework for continuous fine-tuning of graph cross-domain sequences. For each sequentially arriving new graph task, a corresponding low-rank fitting expert is trained within a predefined orthogonal subspace. Through orthogonal knowledge isolation, efficient parameter updates are achieved while minimizing interference from new task training on existing knowledge, stably preserving the model's prior processing capabilities for historical tasks. This embodiment innovatively introduces a symmetric dual-drive router. On one hand, this router accurately evaluates the contribution of each expert to the current input and task through utility prediction; on the other hand, it uses a comparative expert specialization strategy to define the functional boundaries of different experts, thereby learning a globally applicable expert combination strategy. During the inference phase, the router can dynamically select and fuse multiple low-rank fitting experts based on input features, achieving effective cross-domain knowledge transfer and significant mitigation of catastrophic forgetting. Furthermore, each graph domain retains its own independent task head during fitting training. During inference, the corresponding dedicated task head is invoked for different domains, and the selected low-rank fitting experts are dynamically routed and fused to generate the final feature representation and complete the prediction output. The core innovations of this embodiment are reflected in three aspects: First, an orthogonal low-rank adaptation update mechanism for continuous fine-tuning of graph cross-domain sequences is designed, which strictly restricts the update direction of new tasks to the orthogonal complement space of the historical knowledge subspace, thereby structurally reducing cross-task parameter interference and effectively alleviating the forgetting problem in continuous learning. Second, a routing supervision method based on collaborative utility signals is proposed, using the difference in task loss between the prediction of the master expert alone and the prediction of the master-assistant expert combination as the collaborative utility signal, to explicitly supervise the router's learning of the cross-domain migration law that brings positive transfer benefits to the expert combination. Third, a constraint mechanism to promote expert differentiation is constructed, which improves the functional specialization of each low-rank adaptation expert through comparative learning-style differentiation regularization constraints, reduces functional redundancy and conflict among experts, and further improves the prediction effect and stability of the expert combination.

[0073] Example 2: This embodiment provides a prediction system for orthogonal low-rank fitting, implemented based on the prediction method for orthogonal low-rank fitting described above, such as... Figure 2 As shown, the system includes: a data acquisition module, a graph prediction model module, an activation control module, and a prediction output module. The graph prediction model module includes a pre-trained graph prediction model module, which further includes a frozen backbone network, an orthogonal low-rank adaptation expert database, and a symmetric dual-drive router. The data acquisition module is used to acquire cross-domain graph data to be predicted based on the pre-acquired multi-source social data stream; The graph prediction model module is used to determine the master expert based on the cross-domain graph data to be predicted and the orthogonal low-rank fitting expert library; The graph prediction model module is also used to input the cross-domain graph data to be predicted into the frozen backbone network for layer-by-layer message passing, and to obtain the activation features and basic outputs of each layer. The graph prediction model module is also used to calculate expert benefits based on the activation features of each layer, the orthogonal low-rank adaptive expert library and the symmetric dual-drive router, and to obtain the expert gating data set. The activation control module is used to screen auxiliary experts based on the expert gating data set and the preset gating threshold, obtain an activated auxiliary expert set that meets the preset gating value condition, and obtain the auxiliary expert weight distribution based on the activated auxiliary expert set and the expert gating data set. The prediction output module is used to obtain the low-rank adaptation residual of the master expert corresponding to the master expert based on the activation features of each layer, and to obtain the master expert output based on the low-rank adaptation residual of the master expert and the basic output; to obtain several low-rank adaptation residuals of auxiliary experts corresponding to all auxiliary experts in the set of activated auxiliary experts based on the activation features of each layer, and to superimpose the master expert output, several low-rank adaptation residuals of auxiliary experts and auxiliary expert weight distribution to obtain the target feature representation; to input the target feature representation into the independent task head corresponding to the master expert for forward computation to obtain the social scenario prediction result.

[0074] Example 3: This embodiment addresses the scenario of multi-source cross-domain graph node classification, specifically for scenarios where graph tasks from different sources / structures, such as citation graphs, e-commerce co-purchasing graphs, and social graphs, arrive sequentially. It enables continuous learning and inference for node classification tasks, maintaining the performance of historical tasks and reusing transferable knowledge while learning new tasks. Specifically, this embodiment provides a continuous fine-tuning system for cross-domain node classification. This system includes: a data receiving and preprocessing module, a frozen backbone inference module, a hierarchical activation extraction module, a layered historical subspace state management module, a new task safety direction construction module (implementing orthogonal complementary projection and main direction extraction), a low-rank adaptation training module (training only the coefficient matrix and corresponding task headers), an expert database management module (implementing expert solidification and retrieval), a router training and inference module (implementing gating output and expert combination), and a task header output module. The expert database stores the low-rank adaptation parameter set and its independent task header for each task. The historical subspace state is a layered orthogonal basis state used for constructing safety directions for subsequent tasks. The specific implementation steps of this system are as follows: The system initializes by inputting the pre-trained graph base model, task sequence data source, and training hyperparameters (including low-rank size, gating threshold, loss weight, etc.), loading the pre-trained backbone network and freezing its parameters, initializing the parameters of the symmetric dual-drive router, setting the orthogonal low-rank adaptation expert library and the historical subspace state of each layer to empty, initializing an independent task head for the first task, and outputting the frozen backbone model, empty expert library, empty historical subspace state, router initial parameters, and task head initial parameters, providing a stable general representation base and necessary state management for continuous learning; The first task involves training and expert database entry. Input the graph data (including node structure, features, and supervision labels) and frozen backbone model for Task 1. Extract activation features from each layer through the backbone forward pass to construct a safe update direction basis for each layer of Task 1 and fix it as untrainable parameters. Only train the low-rank adaptation coefficient matrix and the task head of Task 1 to complete node classification prediction. Solidify the cross-layer low-rank adaptation parameters and task head of Task 1 into Expert 1 and store it in the expert database. Update the historical subspace state of each layer to record the update direction basis of Task 1. Output Expert 1, Task 1 task head, and the updated expert database and historical subspace state to form the first reusable expert and establish a historical direction benchmark. For the t-th task (t≥2), continuous fine-tuning and expert database entry are performed. Input task t graph data, frozen backbone model, historical subspace state, and expert database. Activation features of each layer are extracted via the backbone forward pass. For each layer, the activation of task t is projected onto the historical subspace orthogonal complement to obtain residual activation features isolated from historical knowledge. The principal direction is extracted from the residuals to construct the current layer's safe update direction basis and is fixed. Training is completed only with the low-rank adaptation coefficient matrix and task t task head. The cross-layer low-rank adaptation parameters of task t and the task head are solidified into expert t and stored in the expert database. The historical subspace state of each layer is updated through QR orthogonalization. The output is expert t, task t task head, and the updated expert database and historical subspace state. This structurally alleviates cross-task forgetting and forms reusable independent experts. The router undergoes symmetric training. Inputs include task t data, an expert database (1, ..., t), router parameters, and threshold and weight settings. A symmetric training protocol is used to allow each expert to take turns as the primary expert. The router outputs gating scores to candidate secondary experts. The difference between the task loss predicted by the primary expert alone and the combined prediction of the primary and secondary experts is used as the collaborative utility signal. Combined with expert differentiation constraints, this process jointly supervises router training, learns transferable expert combination strategies, and outputs updated router parameters. This approach ensures performance for the current task while reducing time bias and improving cross-task transferability. In the inference phase, dynamic expert combination is used. The input includes the graph to be predicted, the expert library, the router, and the corresponding task header of the target task (or the system determines the master expert and task header according to the domain identifier / preset rules). After the master expert is determined, the router outputs the gating value, filters and activates the set of auxiliary experts, and normalizes them to obtain the combination weight. The output of the master expert and the residuals of the auxiliary experts are fused according to the weight to obtain the final feature representation. The corresponding task header is input to output the node classification prediction result. Cross-domain positive migration can be achieved without online fine-tuning, controlling deployment costs. For inference of historical tasks, the corresponding task header is directly called and the same expert combination process is executed, maintaining the historical domain performance without replaying data or updating all parameters.

[0075] Example 4: This embodiment addresses resource-constrained online graph inference scenarios, specifically the issue of an ever-expanding expert database due to increasing task numbers. While meeting resource constraints such as GPU memory, latency, and storage, it achieves efficient prediction and stable output of cross-domain graph data, enabling expert selection and inference in resource-constrained online deployments. The system components provided in this embodiment include: a hierarchical expert database storage module (dividing experts into hot / cold categories), an on-demand loading module, a Top-K activation control module, a routing and rollback strategy module, and an online inference service module (integrating backbone inference, expert combination, and task head prediction). The specific implementation steps of this system are as follows: The online service is initialized by inputting the expert library, router and resource constraint parameters (including the upper limit of video memory, the maximum number of active experts K, the rollback threshold, etc.), loading and freezing the backbone network and router parameters, setting the frequently used experts as hot experts to reside in video memory, and building an index storage for the remaining experts as cold experts, setting the activation upper limit and rollback rules for the Top-K experts, and outputting a runnable online inference instance and resource control strategy to keep the inference resource overhead within the deployable range. Online inference request processing takes the input graph to be predicted, expert database index, router, and corresponding task header as input. First, the primary expert is determined. The router performs utility scoring on all auxiliary experts and selects the Top-K active experts. If the selected auxiliary expert is a cold expert, on-demand loading is triggered. Based on the set of active experts and normalized weights, the output of the primary expert and the residuals of the auxiliary experts are fused. The prediction result is obtained by inputting the task header. When resources exceed the limit or the gating confidence is lower than the threshold, a fallback strategy is triggered, which enables only the primary expert or reduces the number of active experts. The node classification prediction result is output (which may include logs such as the selected expert set, inference time, and memory usage). Under limited resources, the expert combination strategy is reused to maintain stable output and control deployment costs.

[0076] Example 5: Based on the above-described embodiment of the prediction method for orthogonal low-rank adaptation, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the prediction method for orthogonal low-rank adaptation according to any embodiment of the present invention.

[0077] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0078] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0079] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0080] Based on the above-described method embodiments, another embodiment of the present invention provides a storage medium including a stored computer program. The storage medium is a computer-readable storage medium, wherein, when the computer program is running, it controls the device where the storage medium is located to execute the prediction method of orthogonal low-rank adaptation as described in any of the above-described method embodiments of the present invention.

[0081] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0082] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A prediction method for orthogonal low-rank fitting, characterized in that, An application to a pre-trained graph prediction model, the graph prediction model comprising a frozen backbone network, an orthogonal low-rank adaptive expert database, and a symmetric dual-drive router, the method comprising: Obtain cross-domain graph data to be predicted based on pre-acquired multi-source social data streams; The master expert is determined based on the cross-domain graph data to be predicted and the orthogonal low-rank fitting expert database. The cross-domain graph data to be predicted is input into the frozen backbone network for layer-by-layer message passing to obtain the activation features and basic outputs of each layer. Based on the activation features of each layer, the orthogonal low-rank adaptive expert library, and the symmetric dual-drive router, expert benefit calculation is performed to obtain the expert gating data set. Based on the expert gating data set and the preset gating threshold, auxiliary experts are screened to obtain an active auxiliary expert set that meets the preset gating value condition, and the auxiliary expert weight distribution is obtained based on the active auxiliary expert set and the expert gating data set. Based on the activation features of each layer, the low-rank adaptation residual of the master expert corresponding to the master expert is obtained, and the master expert output is obtained based on the low-rank adaptation residual of the master expert and the basic output. Based on the activation features of each layer, obtain several low-rank adaptation residuals of all auxiliary experts in the set of activated auxiliary experts, and superimpose them based on the output of the main expert, the low-rank adaptation residuals of several auxiliary experts and the weight distribution of auxiliary experts to obtain the target feature representation. The target feature representation is input into the independent task head corresponding to the master expert for forward computation to obtain the social context prediction result.

2. The prediction method for orthogonal low-rank fitting as described in claim 1, characterized in that, The step of obtaining the low-rank adaptation residual of the master expert corresponding to the master expert based on the activation features of each layer, and obtaining the master expert output based on the low-rank adaptation residual of the master expert and the basic output, includes: Based on the chief expert, obtain the current update direction basis and the current low-rank fit coefficient matrix; Linear projection is performed based on the current layer activation features and the current update direction basis in the activation features of each layer to obtain low-dimensional features; The low-dimensional features are nonlinearly activated, and back projection is performed based on the current low-rank adaptation coefficient matrix to obtain the master expert low-rank adaptation residual.

3. The prediction method for orthogonal low-rank fitting as described in claim 1, characterized in that, The pre-training process of the graph prediction model includes: Obtain the multi-source social cross-domain graph task sequence, and perform fine-tuning training actions on the graph data of any task sample in the multi-source social cross-domain graph task sequence and the pre-built graph basic model in chronological order, and obtain the current orthogonal low-rank adaptation expert library and the current symmetric dual-drive router; The current orthogonal low-rank adaptation expert library and the current symmetric dual-drive router are used as the initial orthogonal low-rank adaptation expert library and the initial symmetric dual-drive router for the next training task graph data. The fine-tuning training action is repeatedly executed until all task sample graph data in the multi-source social cross-domain graph task sequence complete the fine-tuning training action, thereby obtaining the fine-tuned orthogonal low-rank adaptation expert library and symmetric dual-drive router, and then obtaining a graph prediction model including the frozen backbone network, the orthogonal low-rank adaptation expert library and the symmetric dual-drive router.

4. The prediction method for orthogonal low-rank fitting as described in claim 3, characterized in that, For any task sample graph data, the fine-tuning training action includes: Based on the graph-based model, a frozen backbone model including several adaptable layers is obtained. The task sample graph data is input into the frozen backbone network for layer-by-layer message passing to obtain the historical layer activation features of the samples. For any current adaptable layer among the aforementioned adaptable layers: The current layer sample update direction basis is obtained based on the historical layer activation features of the samples. Based on the pre-acquired initial orthogonal low-rank fitting expert database and the preset symmetric training principle, any task expert is selected in turn as the sample master expert, the corresponding sample auxiliary expert set is determined, and several hybrid expert combinations are formed based on all sample master experts and the corresponding sample auxiliary expert sets. A joint loss function is constructed based on the pre-acquired initial symmetric dual-drive router, the task sample graph data, several hybrid expert combinations, and the frozen backbone network; Based on the joint optimization loss function and the preset joint optimization objective, the parameters of the initial symmetric dual-drive router, the initial data of the sample task head, and the initial low-rank adaptation coefficient matrix are trained and updated to obtain the current symmetric dual-drive router, the current sample low-rank adaptation coefficient, and the current task head parameters. Based on the current layer sample update direction basis, the current sample low-rank adaptation coefficient and the current task header parameters, a current task expert is constructed, and the current task expert is incorporated into the initial orthogonal low-rank adaptation expert library to form the current orthogonal low-rank adaptation expert library.

5. The prediction method for orthogonal low-rank fitting as described in claim 4, characterized in that, The joint loss function constructed based on the pre-acquired initial symmetric dual-drive router, the task sample graph data, several hybrid expert combinations, and the frozen backbone network includes: The sample gating distribution is obtained based on the pre-acquired initial symmetric dual-drive router and several hybrid expert combinations; The mixed test scenario loss is obtained based on the task sample graph data, several mixed expert combinations, and the frozen backbone network. The main sample scenario loss is obtained based on the task sample graph data, the sample master experts corresponding to several mixed expert combinations, and the frozen backbone network. The task loss is obtained based on the mixed test scenario loss and the main sample scenario loss. Based on the aforementioned mixed-test scenario loss, main sample scenario loss, and sample gating distribution, the utility prediction regression loss is obtained. A positive expert set and a negative expert set are constructed based on a preset gating threshold, the sample gating distribution, and an initial orthogonal low-rank fitting expert database. The contrast loss is calculated based on the sample principal experts, the positive expert set, and the negative expert set, and the contrast loss is calculated by symmetric averaging of the contrast loss of all sample principal experts to obtain the contrast loss. A joint loss function is constructed based on the task loss, utility prediction regression loss, contrast loss, and preset orthogonal constraints.

6. The prediction method for orthogonal low-rank fitting as described in claim 4, characterized in that, The step of obtaining the current layer sample update direction basis based on the historical layer activation features of the samples includes: Obtain the forward layer sample activation features from the historical layer activation features of the samples; Obtain the forward sample history subspace, and perform orthogonal transformation based on the forward sample history subspace to construct the orthogonal basis matrix of the sample history subspace of the current adaptable layer; Orthogonal complementary projection is performed based on the orthogonal basis matrix of the sample history subspace to obtain the orthogonal complementary projection matrix; the sample forward layer residual activation features are obtained based on the forward layer sample activation features and the orthogonal complementary projection matrix. Singular value decomposition is performed on the forward layer residual activation features of the sample to construct a left singular vector matrix; The current layer sample update direction basis is constructed based on the preset rank, the preset low-rank condition, and the left singular vector matrix; The current layer sample update direction basis is incorporated into the orthogonal basis matrix of the sample history subspace to obtain the current sample subspace of the current adaptable layer. The current sample subspace is then used as the forward sample history subspace of the next adaptable layer.

7. The prediction method for orthogonal low-rank fitting as described in claim 6, characterized in that, For the first task sample image data, before performing the fine-tuning training action, the following steps are included: The parameters of the backbone network of the graph-based model are frozen to form a frozen backbone network including several adaptable layers. Perform router initialization operations to form the initial symmetric dual-drive router for the first task sample graph data; Initialize the initial orthogonal low-rank adaptation expert library of the first task sample graph data to an empty set; Empty state initialization is performed on the historical knowledge subspace states of several adaptable layers to obtain an empty state knowledge subspace, and the empty state knowledge subspace is used as the forward sample historical subspace of the first task sample graph data.

8. A prediction system for orthogonal low-rank fitting, characterized in that, Based on the prediction method for orthogonal low-rank adaptation as described in any one of claims 1-7, the system includes: a data acquisition module, a graph prediction model module, an activation control module, and a prediction output module. The graph prediction model module includes a pre-trained graph prediction model module, which further includes a frozen backbone network, an orthogonal low-rank adaptation expert database, and a symmetric dual-drive router. The data acquisition module is used to acquire cross-domain graph data to be predicted based on the pre-acquired multi-source social data stream; The graph prediction model module is used to determine the master expert based on the cross-domain graph data to be predicted and the orthogonal low-rank fitting expert library; The graph prediction model module is also used to input the cross-domain graph data to be predicted into the frozen backbone network for layer-by-layer message passing, and to obtain the activation features and basic outputs of each layer. The graph prediction model module is also used to calculate expert benefits based on the activation features of each layer, the orthogonal low-rank adaptive expert library and the symmetric dual-drive router, and to obtain the expert gating data set. The activation control module is used to screen auxiliary experts based on the expert gating data set and the preset gating threshold, obtain an activated auxiliary expert set that meets the preset gating value condition, and obtain the auxiliary expert weight distribution based on the activated auxiliary expert set and the expert gating data set. The prediction output module is used to obtain the low-rank adaptation residual of the master expert corresponding to the master expert based on the activation features of each layer, and to obtain the master expert output based on the low-rank adaptation residual of the master expert and the basic output; to obtain several low-rank adaptation residuals of auxiliary experts corresponding to all auxiliary experts in the set of activated auxiliary experts based on the activation features of each layer, and to superimpose the master expert output, several low-rank adaptation residuals of auxiliary experts and auxiliary expert weight distribution to obtain the target feature representation; to input the target feature representation into the independent task head corresponding to the master expert for forward computation to obtain the social scenario prediction result.

9. A device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a prediction method for orthogonal low-rank adaptation as described in any one of claims 1-7.

10. A storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, the device containing the storage medium is controlled to perform a prediction method for orthogonal low-rank adaptation as described in any one of claims 1-7.

Images