A method and system for generating stereoscopic images based on multimodal fusion

Abstract

This invention relates to a portrait generation method, and more particularly to a three-dimensional portrait generation method and system based on multimodal fusion, aiming to solve the problems of single data dimension, static and one-sided evaluation, and lack of dynamic optimization in traditional cadre evaluation methods. This invention significantly improves the comprehensiveness, accuracy, dynamism, and intelligence of cadre portraits by deeply integrating multimodal data, introducing a dynamic evaluation model, and employing a dual-loop optimization mechanism, providing strong support for scientific decision-making and precise cadre management.

Description

Technical Field

[0001] This invention relates to a portrait generation method, specifically a method and system for generating stereoscopic portraits based on multimodal fusion. Background Technology

[0002] Cadres are the core resource for organizational development, and the scientific evaluation and accurate appointment of cadres are crucial to the achievement of organizational strategic goals. Traditional cadre evaluation and management methods typically rely on limited and static data sources such as organizational personnel files, annual performance evaluations, and democratic appraisals. Evaluation methods often rely primarily on manual file review and periodic assessments, which have the following limitations: First, the evaluation dimensions are one-sided, making it difficult to form a comprehensive understanding. Traditional methods rely heavily on structured files and summary materials, lacking effective means of observation and quantification for implicit traits such as cadres' daily behavioral characteristics, informal team influence, and decision-making logic when handling complex problems. This results in evaluation results that may be one-sided and lack a comprehensive understanding.

[0003] Secondly, the evaluation results are static and lack timeliness and foresight. A cadre's abilities and performance are dynamically evolving, while traditional evaluations are often "snapshot" assessments conducted annually or semi-annually, failing to reflect in real-time changes in a cadre's abilities and performance fluctuations in responding to emergencies and advancing major projects. Furthermore, historical achievements or mistakes are typically assessed using fixed weights or simple subjective judgments, failing to scientifically model the dynamic decay of their influence over time and rank, thus affecting the fairness and accuracy of the evaluation.

[0004] Finally, the evaluation models are rigid, lacking adaptability and intelligence. Once established, existing evaluation systems often have fixed evaluation indicators and model weights, making it difficult to adapt to shifts in organizational strategic focus (e.g., from a business expansion phase to an innovation-driven phase). The evaluation process mainly focuses on summarizing historical data, lacking the ability to predict future development potential, extrapolate the causal effects of personnel decisions (such as job rotation and training), and simulate the synergistic effectiveness of leadership team configuration.

[0005] Therefore, there is an urgent need for a cadre evaluation method that can break through the limitations of traditional methods. This method should be able to integrate multi-source heterogeneous data to construct a dynamic and comprehensive cadre profile. Furthermore, the profile model itself should have the ability to adaptively optimize and intelligently evolve in order to adapt to the complex and ever-changing organizational environment and the development trajectory of individual cadres.

[0006] This application is made to solve the aforementioned technical problems. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for generating stereoscopic images based on multimodal fusion, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A method for generating stereoscopic images based on multimodal fusion includes the following steps: Acquire pre-defined multimodal cadre data covering organizational personnel, business performance, and behavioral characteristics; Multi-level feature modeling is performed on the multimodal cadre data to obtain multiple feature vectors representing basic qualities, leadership dynamics, and integrity risks. Based on a pre-defined cross-modal fusion framework, the multiple feature vectors are dynamically weighted and fused to generate an initial 3D portrait of the cadre; and Based on a dual-loop optimization framework, the cadre 3D portrait is dynamically optimized and adaptively enhanced to generate the final cadre 3D portrait; wherein, the dual-loop optimization framework includes: an inner loop that fine-tunes the portrait feature dimensions based on reinforcement learning, and an outer loop that optimizes the model hyperparameters based on swarm intelligence algorithms.

[0009] As a preferred embodiment of the present invention, after the step of acquiring multimodal cadre data, the method further includes: Historical data is managed using a spatiotemporal decay factor, which is based on a learnable time decay parameter and an index term that quantifies the value of cross-rank experience, to dynamically model the influence of historical events on cadres.

[0010] As a preferred embodiment of the present invention, the multi-level feature modeling step, including the modeling of leadership dynamics, comprises: By employing the L-Transformer architecture, which integrates temporal coding and graph attention networks, we can simultaneously capture the temporal dependence of leadership evolution and the team network influence, and generate dynamic leadership evaluation indicators.

[0011] As a preferred embodiment of the present invention, the modeling of integrity risks in the multi-level feature modeling step includes: A generative adversarial network anti-fraud model is constructed, in which the generator is used to simulate the feature distribution of compliance cadres, and the discriminator is used to identify abnormal patterns through comparative learning to output an integrity risk score.

[0012] As a preferred embodiment of the present invention, the dynamic weighted fusion step in the cross-modal fusion framework includes: Pre-trained language models, 3D convolutional neural networks, and graph aggregation networks were used to encode data in text, video, and graph modalities, respectively. By using a gating attention mechanism, the fusion weights are dynamically allocated based on the contribution of each modality of data to the current profile generation task.

[0013] As a preferred embodiment of the present invention, in the steps of the dual-loop optimization framework, the outer loop optimization includes: A mutation probability associated with the organization's strategic orientation is introduced, wherein the mutation probability is inversely proportional to the semantic similarity between the model parameters and the current strategic keywords, in order to guide the optimization direction to align with the organization's strategy.

[0014] As a preferred embodiment of the present invention, after generating the final three-dimensional portrait of the cadre, the method further includes: Based on the KL divergence calculation, the difference between the current image feature distribution and the baseline distribution is calculated. When the difference exceeds the dynamic warning threshold, the image drift warning and correction mechanism is triggered.

[0015] As a preferred embodiment of the present invention, it further includes: Based on the final three-dimensional profile of cadres, a structured causal model is constructed. Through counterfactual reasoning, the causal effect of specific intervention measures, such as job rotation or special training, on the future performance of cadres is quantified.

[0016] As a preferred embodiment of the present invention, it further includes: The final three-dimensional portrait of cadres is applied to a multi-agent simulation system for optimizing the configuration of leadership teams. The simulation system simulates and predicts the collaborative effectiveness of different team combinations through dynamic role evolution equations.

[0017] On the other hand, the present invention also provides a stereoscopic image generation system based on multimodal fusion, comprising: The data acquisition module is used to acquire pre-defined multimodal cadre data covering organizational personnel, business performance, and behavioral characteristics; The feature modeling module is used to perform multi-level feature modeling on the multimodal cadre data to obtain multiple feature vectors representing basic qualities, leadership dynamics, and integrity risks. The fusion generation module is used to dynamically weight and fuse the multiple feature vectors based on a preset cross-modal fusion framework to generate an initial 3D portrait of the cadre; and The dynamic optimization module is used to dynamically optimize and adaptively enhance the cadre 3D portrait based on a dual-loop optimization framework to generate the final cadre 3D portrait; wherein, the dual-loop optimization framework includes: an inner loop mechanism for fine-tuning the portrait feature dimensions based on reinforcement learning, and an outer loop mechanism for optimizing the model hyperparameters based on swarm intelligence algorithms.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This application aims to overcome the technical challenges of traditional cadre evaluation methods, such as low efficiency in manual data processing, one-sided information due to a single evaluation dimension, delayed updates to evaluation results, and rigid evaluation models that are unable to adapt to dynamic data flows and organizational strategic changes. It provides a method and system capable of generating a comprehensive, dynamic, and accurate three-dimensional profile that reflects the overall situation of cadres and can continuously evolve.

[0019] By acquiring multimodal data covering organizational personnel, business performance, and behavioral characteristics, and performing multi-level feature modeling, this application overcomes the problem of one-sided evaluation caused by the reliance on a single data source in traditional methods, forming a multi-dimensional profile. In particular, through dynamic modeling of key characteristics such as leadership, the profile can reflect the evolution of cadres' abilities over time, rather than static conclusions, significantly improving the depth and timeliness of the evaluation.

[0020] This application innovatively introduces a dual-loop optimization framework. This framework combines the rapid fine-tuning capability of reinforcement learning (inner loop) with the global optimization capability of swarm intelligence (outer loop), enabling the cadre profiling model to move beyond a static state and continuously and intelligently iterate and optimize itself based on new data inputs and organizational strategic guidance. This fundamentally solves the core problem of rigid evaluation models in existing technologies, which are unable to adapt to dynamic environments, ensuring the long-term effectiveness and accuracy of the profiling.

[0021] Therefore, this application, through the combination of multimodal fusion and dual-cycle dynamic optimization, not only enhances the breadth and depth of cadre evaluation, but also endows it with the characteristic of "living," enabling it to evolve intelligently and adaptively, thereby providing organizations with more scientific, forward-looking, and reliable support for key decisions such as cadre selection, training, and appointment. Attached Figure Description

[0022] Figure 1 This is a diagram of the overall system architecture of the present invention.

[0023] The attached diagrams are labeled as follows: 1. Data acquisition module; 2. Feature modeling module; 3. Fusion generation module; 4. Dynamic optimization module. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0025] This application provides a stereoscopic image generation system based on multimodal fusion, whose architecture may include: a data acquisition module 1, a feature modeling module 2, a fusion generation module 3, and a dynamic optimization module 4. Correspondingly, this application also provides a stereoscopic image generation method implemented based on this system.

[0026] In a specific embodiment, the detailed process of this method is as follows: Step 1: Obtain pre-defined multimodal cadre data covering organizational personnel, business performance, and behavioral characteristics.

[0027] This step corresponds to the system's data acquisition module 1. To construct a comprehensive cadre profile, the system integrates cadre data from various sources and types. These data sources are not limited to traditional organizational and personnel data (such as work experience, annual performance evaluation results, democratic appraisals, and reports of significant personal matters), but also innovatively integrate: Business performance data, such as the achievement rate of key performance indicators in the area of ​​responsibility, contribution to major projects, number of innovative patents, etc., are used to quantify the actual output of cadres.

[0028] Behavioral characteristic data: such as analyzing approval logs of office automation systems to uncover their decision-making style and efficiency, using natural language processing technology to perform emotion recognition on meeting speeches, or recording their response timeliness in emergency events, etc.

[0029] In addition, it can also include public opinion monitoring data, anti-corruption risk data, psychological trait data, and growth trajectory data, which together constitute a multimodal, comprehensive data foundation.

[0030] In a preferred embodiment, a data governance step is included after data acquisition. To scientifically assess the impact of historical data, this application introduces a spatiotemporal decay factor to govern the historical data. This factor is implemented through a dynamic decay model. To achieve this. Among them, It is the time span since the event occurred. This represents the difference between the job level at the time the event occurred and the current job level. Unlike traditional fixed weights, this uses a time decay parameter. And cross-job level impact parameters It is learnable; the model can automatically learn the optimal decay rate of historical event influence based on data, and simultaneously, through the exponential term... The potential value and transferability of cross-level work experience are scientifically quantified, making data governance more accurate and fair.

[0031] Step 2: Perform multi-level feature modeling on the multimodal cadre data to obtain multiple feature vectors representing basic qualities, leadership dynamics, and integrity risks.

[0032] This step corresponds to feature modeling module 2 of the system. The module receives the data after governance and performs in-depth analysis and modeling of cadres from different levels and dimensions.

[0033] Basic competency layer modeling aims to quantify cadres' fundamental competencies, such as political literacy and professional skills. For example, political literacy is quantified by analyzing the semantic density of keywords in job performance reports; professional matching is assessed by constructing knowledge graphs in professional fields and calculating the similarity between the graph embedding vectors of cadres' personal experience and job requirements.

[0034] Dynamic Leadership Assessment: To capture the dynamic evolution of leadership, this application employs an innovative L-Transformer architecture. This architecture integrates temporal coding (such as Time2Vec) and graph attention networks (GATv2). Temporal coding captures the inherent patterns and dependencies in the development of leadership over time, while graph attention networks analyze its influence within the team's collaborative network. The combination of these two approaches enables simultaneous assessment from both "time" and "space" dimensions, generating dynamic leadership assessment indicators that incorporate time dependence and network influence, providing a more profound understanding than static scoring.

[0035] Integrity Risk Prediction: To accurately identify potential integrity risks, this application constructs a generative adversarial network (GAN) anti-fraud model. The generator learns from a large amount of feature data on compliance officials to simulate a "standard" distribution of compliance official characteristics. The discriminator, acting as a highly sensitive "identifier," compares the characteristics of the official to be evaluated with the "standard" characteristics generated by the generator through comparative learning, thereby identifying subtle abnormal patterns (e.g., a sudden increase in the number of related businesses, significant deviations between personal consumption records and income, etc.), and ultimately outputs a quantitative integrity risk score.

[0036] Step 3: Based on the preset cross-modal fusion framework, the multiple feature vectors are dynamically weighted and fused to generate an initial three-dimensional portrait of the cadre.

[0037] This step corresponds to the system's fusion generation module 3. After obtaining the feature vectors at each level, it is necessary to effectively fuse this information from different modalities and dimensions to form a unified profile.

[0038] In a preferred embodiment, this application employs a cross-modal fusion framework called CROSS-Fusion. Specifically: Modality-specific encoding: First, the most suitable model is used for deep feature extraction for different modalities of data. For example, for text modalities such as performance reports, a pre-trained language model (such as BERT-wwm) is used to extract deep semantics; for video modalities such as meeting speeches, a three-dimensional convolutional neural network (3D-CNN) is used to analyze micro-expressions and body language; for graph modalities such as team relationships, a graph aggregation network (such as GraphSAGE) is used to aggregate the features of neighbor nodes.

[0039] Dynamic weighted fusion: After obtaining the feature representations of each modality, instead of simply concatenating or averaging them, a gated attention mechanism is used. This mechanism dynamically and learnably assigns weights to the contribution of each modality based on the specific needs of the current profile generation task (e.g., assessing innovation or crisis management capabilities). If the current task relies more on text analysis, the weight of the text modality will automatically increase. This approach makes the fusion process more intelligent and targeted.

[0040] Step 4: Based on the dual-loop optimization framework, the cadre 3D portrait is dynamically optimized and adaptively enhanced to generate the final cadre 3D portrait.

[0041] This step is one of the core innovations of this application, corresponding to the dynamic optimization module 4 of the system. The generated initial image is not the final result, but needs to be "brought to life" through a continuous optimization process.

[0042] The dual-loop optimization framework designed in this application includes: Inner Loop: Feature dimension fine-tuning based on reinforcement learning. This is a high-frequency optimization loop (e.g., updates every minute or hour). The system treats minor adjustments to the profile as "actions" and the resulting improvement in prediction accuracy or fit to new data as "rewards." Through reinforcement learning, the model can quickly refine the various feature dimensions of the profile to adapt to recent changes in the cadre's status.

[0043] Outer Loop: Model hyperparameter optimization based on swarm intelligence algorithms (such as genetic algorithms). This is a low-frequency macro-optimization loop (e.g., quarterly or semi-annual updates). It mainly focuses on global optimization of the hyperparameters of the entire profile generation model (such as the learning rate of each network layer, the parameters of the decay factor, etc.). In a particularly preferred embodiment, to ensure that the optimization direction is consistent with organizational goals, this application introduces a mutation probability associated with the organizational strategic orientation into the outer loop. Specifically, the probability Pmutate of the mutation operation in the genetic algorithm is dynamically adjusted based on the semantic similarity between the model parameters and the current organizational strategic keywords (such as "reform and innovation" and "risk prevention and control"). If a set of parameters is more in line with the current strategic orientation, its probability of being "mutated" will decrease, thus being retained. This design cleverly "encodes" the macro-level organizational strategy into the model's evolution process, guiding the profile model to evolve in a direction that meets the organization's needs.

[0044] In addition to the core processes described above, this application also includes some advanced implementation methods: Profile Drift Detection and Correction: To ensure the long-term effectiveness of profiles, the system also deploys an early warning mechanism. This mechanism periodically (e.g., every 24 hours) calculates the difference between the overall feature distribution of the current profile and a set baseline distribution based on KL divergence. KL divergence measures the degree of difference between two probability distributions. When the calculated difference exceeds a dynamically set early warning threshold (which can be appropriately relaxed as the profile generation time increases), the system triggers a profile drift warning, indicating to the manager that the cadre's profile may have changed significantly and needs to be corrected or reassessed.

[0045] Counterfactual Reasoning Based on Personnel Profiles: The generated final personnel profiles are not only used for evaluation but also for decision-making. This application constructs a structured causal model and utilizes the rich features in the final profiles as variables to perform counterfactual reasoning. For example, the system can answer questions such as: "If this cadre had not received six months of specialized financial training (a counterfactual hypothesis), what would their financial risk management capability score be one year later?" By quantifying the causal effect between intervention and no intervention, organizations can more scientifically assess the actual effectiveness of various training measures and achieve precise training.

[0046] Profile-Based Leadership Team Optimization: The final cadre profiles can be used as input to a multi-agent simulation system for optimizing leadership team configuration. In this system, each cadre is modeled as an "agent," with their behavioral patterns and capability parameters defined by their profile. The system can preset different team roles (such as decision-maker, executor, coordinator, etc.) and simulate the synergistic effectiveness of different cadre combinations through a dynamic role evolution equation. This equation considers the driving force of job requirements on individual roles, as well as the mutual influence among members (modeled through graph attention networks). Through simulation, the integration, decision-making efficiency, and potential conflicts of different team combinations can be predicted before personnel adjustments, thereby reducing trial-and-error costs and achieving a scientific allocation of leadership teams.

[0047] System Implementation Example: This application also provides a stereoscopic image generation system based on multimodal fusion. The system includes: Data acquisition module 1 is configured to perform the data acquisition and processing functions described in step one above.

[0048] Feature modeling module 2 is configured to perform multi-level feature modeling in step two above, including specific implementations of L-Transformer and GAN models.

[0049] The fusion generation module 3 is configured to perform cross-modal fusion in step three above, particularly dynamic weighting based on gated attention.

[0050] Dynamic optimization module 4 is configured to execute the dual-loop optimization framework in step four above. Internally, it implements a reinforcement learning mechanism for the inner loop and a swarm intelligence optimization mechanism related to the strategy for the outer loop.

[0051] These modules can be implemented in the form of software, hardware, or a combination thereof. They work together to complete the entire process of stereoscopic image generation and optimization described in this application.

[0052] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0053] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating stereoscopic images based on multimodal fusion, characterized in that, Includes the following steps: Acquire pre-defined multimodal cadre data covering organizational personnel, business performance, and behavioral characteristics; Multi-level feature modeling is performed on the multimodal cadre data to obtain multiple feature vectors representing basic qualities, leadership dynamics, and integrity risks. Based on a preset cross-modal fusion framework, the multiple feature vectors are dynamically weighted and fused to generate an initial three-dimensional portrait of the cadre. as well as Based on a dual-loop optimization framework, the cadre 3D portrait is dynamically optimized and adaptively enhanced to generate the final cadre 3D portrait; wherein, the dual-loop optimization framework includes: an inner loop that fine-tunes the portrait feature dimensions based on reinforcement learning, and an outer loop that optimizes the model hyperparameters based on swarm intelligence algorithms.

2. The method according to claim 1, characterized in that, Following the step of acquiring multimodal cadre data, the method further includes: Historical data is managed using a spatiotemporal decay factor, which is based on a learnable time decay parameter and an index term that quantifies the value of cross-rank experience, to dynamically model the influence of historical events on cadres.

3. The method according to claim 1, characterized in that, In the steps of the multi-level feature modeling, the modeling of leadership dynamics includes: By employing the L-Transformer architecture, which integrates temporal coding and graph attention networks, we can simultaneously capture the temporal dependence of leadership evolution and the team network influence, and generate dynamic leadership evaluation indicators.

4. The method according to claim 1, characterized in that, The steps of the multi-level feature modeling include modeling corruption risks, which include: A generative adversarial network anti-fraud model is constructed, in which the generator is used to simulate the feature distribution of compliance cadres, and the discriminator is used to identify abnormal patterns through comparative learning to output an integrity risk score.

5. The method according to claim 1, characterized in that, The dynamic weighted fusion process in the cross-modal fusion framework includes: Pre-trained language models, 3D convolutional neural networks, and graph aggregation networks were used to encode data in text, video, and graph modalities, respectively. By using a gating attention mechanism, the fusion weights are dynamically allocated based on the contribution of each modality of data to the current profile generation task.

6. The method according to claim 1, characterized in that, In the steps of the dual-loop optimization framework, the outer loop optimization includes: A mutation probability associated with the organization's strategic orientation is introduced, wherein the mutation probability is inversely proportional to the semantic similarity between the model parameters and the current strategic keywords, in order to guide the optimization direction to align with the organization's strategy.

7. The method according to claim 1, characterized in that, After generating the final 3D portrait of the cadre, the following is also included: Based on the KL divergence calculation, the difference between the current image feature distribution and the baseline distribution is calculated. When the difference exceeds the dynamic warning threshold, the image drift warning and correction mechanism is triggered.

8. The method according to claim 1, characterized in that, Also includes: Based on the final three-dimensional profile of cadres, a structured causal model is constructed. Through counterfactual reasoning, the causal effect of specific intervention measures, such as job rotation or special training, on the future performance of cadres is quantified.

9. The method according to claim 1, characterized in that, Also includes: The final three-dimensional portrait of cadres is applied to a multi-agent simulation system for optimizing the configuration of leadership teams. The simulation system simulates and predicts the collaborative effectiveness of different team combinations through dynamic role evolution equations.

10. A stereoscopic image generation system based on multimodal fusion, characterized in that, include: The data acquisition module (1) is used to acquire preset multimodal cadre data covering organizational personnel, business performance and behavioral characteristics; The feature modeling module (2) is used to perform multi-level feature modeling on the multimodal cadre data to obtain multiple feature vectors representing basic qualities, leadership dynamics and integrity risks respectively; The fusion generation module (3) is used to dynamically weight and fuse the multiple feature vectors based on a preset cross-modal fusion framework to generate an initial three-dimensional portrait of the cadre. as well as The dynamic optimization module (4) is used to dynamically optimize and adaptively enhance the three-dimensional portrait of the cadre based on the dual-loop optimization framework to generate the final three-dimensional portrait of the cadre; wherein, the dual-loop optimization framework includes: an inner loop mechanism for fine-tuning the portrait feature dimension based on reinforcement learning, and an outer loop mechanism for optimizing the model hyperparameters based on swarm intelligence algorithm.

Images