Heterogeneous fusion method, system and equipment for brain-like intelligent driving and medium

Abstract

The invention relates to the technical field of brain-like intelligent driving, in particular to a brain-like intelligent driving heterogeneous fusion method, system and device and a medium, a collaborative computing basis is constructed through a fusion architecture, a brain-like pulse unit and a deep learning unit are integrated, and heterogeneous data interaction is achieved in cooperation with bidirectional signal conversion. A cross-modal distillation algorithm is proposed, knowledge from a source modal to a target modal is migrated through comparative learning, a meta-learning framework is introduced to optimize the migration process, few-sample task adaptation is supported, and multi-modal data support is enhanced; on the basis of a quantum enhancement calculation module, a neural pulse signal is coded by utilizing quantum state characteristics, parameter solution is converted into a Hamiltonian optimization problem in combination with a quantum optimization algorithm, and a classical-quantum hybrid calculation framework is formed; through event-driven calculation, long-distance dependence modeling and a quantum acceleration mechanism, cross-modal knowledge migration, adaptive task optimization and leap-type improvement of calculation efficiency are realized, and artificial intelligence technology upgrading is promoted.

Description

Technical Field

[0001] The present invention relates to the field of brain-inspired intelligent drive technology, and in particular to a heterogeneous fusion method, system, device and medium for brain-inspired intelligent drive. Background Art

[0002] With the rapid development of information technology, artificial intelligence has been widely used in many fields. Artificial intelligence technology with deep learning as its core has made major breakthroughs in image recognition, natural language processing, speech synthesis and other fields, demonstrating efficient resource utilization to achieve parallelization, efficient computing and low-power computing of pulse neural network models. However, traditional deep learning models rely on a large amount of labeled data and are difficult to migrate to new tasks or fields. Traditional architectures usually use simple data splicing or feature fusion methods when processing multimodal data, which makes it difficult to explore deep semantic associations between modalities. In addition, they have poor adaptability when facing new tasks and are difficult to achieve cross-modal knowledge transfer and adaptive learning. To this end, the present invention provides an artificial intelligence architecture based on brain-like intelligence drive. Summary of the Invention

[0003] In view of the above existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides a brain-like intelligence-driven heterogeneous fusion method and system to solve the problem that traditional deep learning models rely on a large amount of labeled data and are difficult to migrate to new tasks or fields. Traditional architectures usually use simple data splicing or feature fusion methods when processing multimodal data, which makes it difficult to explore deep semantic associations between modalities, and have poor adaptability when facing new tasks, making it difficult to achieve cross-modal knowledge transfer and adaptive learning.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a brain-like intelligent driven heterogeneous fusion method, comprising:

[0007] Design a fusion architecture and build a collaborative computing foundation;

[0008] Propose an algorithm to transfer knowledge from the source modality to the target modality through contrastive learning, introduce a learning framework to optimize the migration process, and enrich multimodal data support through enhanced computing;

[0009] Based on enhanced computing, the characteristics are used to encode the pulses and optimize the solution process.

[0010] As a preferred solution of the brain-like intelligence-driven heterogeneous fusion method described in the present invention, the fusion architecture is designed and the collaborative computing foundation is constructed, including:

[0011] Use neuron model to simulate the potential dynamic characteristics of biological neurons;

[0012] Dependency parsing, capturing cross-level feature relationships;

[0013] Build a bidirectional conversion mechanism between pulses and features to achieve collaborative processing.

[0014] As a preferred solution of the brain-like intelligence-driven heterogeneous fusion method described in the present invention, the proposed algorithm transfers the knowledge of the source modality to the target modality through comparative learning, including:

[0015] Extract feature representation through pre-training and use functions for alignment;

[0016] A mapping and conversion mechanism for inter-modal feature representation is established, and parameters are adjusted through gradient optimization to achieve knowledge transfer in the target area.

[0017] As a preferred solution of the brain-like intelligence-driven heterogeneous fusion method described in the present invention, the introduction of a learning framework to optimize the migration process and the enrichment of multimodal data support through enhanced computing include:

[0018] Train the learning framework on the task distribution to generate initialization parameters that can adapt to the feature representation of the new task;

[0019] Expand the coding dimension through quantum state superposition and entanglement mechanisms;

[0020] By providing high-dimensional support through enhanced computing, the learning framework can adapt to new tasks in the cross-modal transfer process.

[0021] As a preferred solution of the brain-like intelligent driven heterogeneous fusion method described in the present invention, based on enhanced computing, pulses are encoded using characteristics to optimize the solution process, including:

[0022] Utilize quantum state characteristics to encode classical pulse signals into superposition state representation;

[0023] By entangled and fused multimodal pulse signals, a joint representation space is constructed to enhance the semantic association between modalities;

[0024] Parameter optimization is converted into quantum state evolution, and the parameters are iteratively solved using a collaborative framework.

[0025] As a preferred solution of the brain-like intelligence-driven heterogeneous fusion method described in the present invention, the extraction of feature representations through pre-training and alignment using functions include:

[0026] A pre-trained multimodal model is used to extract features of the source modality and target modality respectively, and the contrast loss function is used to maximize the similarity of matching images and texts of positive samples and minimize the similarity of mismatching images and texts of negative samples.

[0027] As a preferred solution of the brain-like intelligence-driven heterogeneous fusion method described in the present invention, the generating of initialization parameters that can adapt to the feature representation of the new task includes:

[0028] When faced with a new task, the initialization parameters generated by the meta-learner are used to adjust the model through gradient updates so that the target modality feature representation adapts to the distribution of the new task;

[0029] Among them, the meta-learner includes a meta-training phase, in which the meta-learner is trained on a small number of tasks, learning to adjust the model parameters to adapt to new tasks, and the meta-objective function is defined as minimizing the loss on the new task.

[0030] In a second aspect, the present invention provides a brain-inspired intelligent-driven heterogeneous fusion system, comprising:

[0031] Converged architecture module, designing converged architecture and building collaborative computing foundation;

[0032] The migration optimization module proposes an algorithm to transfer the knowledge of the source modality to the target modality through comparative learning, introduces a learning framework to optimize the migration process, and enriches multimodal data support through enhanced computing;

[0033] The coding enhancement module encodes the pulses based on enhanced calculation and optimizes the solution process.

[0034] In a third aspect, the present invention provides an electronic device, comprising:

[0035] memory and processor;

[0036] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of a brain-like intelligence-driven heterogeneous fusion method are implemented.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the heterogeneous fusion method driven by brain-like intelligence.

[0038] Compared with the existing technology, the beneficial effects of the present invention are as follows: the present invention simulates the dynamic characteristics of biological neuron membrane potential through the SNN layer, the deep learning layer of the Transformer architecture captures long-distance dependencies, and cooperates with the signal conversion module to realize bidirectional conversion between pulse signals and continuous feature maps, laying the foundation for efficient operation of the system. The inter-modal self-attention distillation algorithm introduces contrastive learning and meta-learning frameworks to realize cross-modal knowledge transfer and zero-sample and small-sample task adaptation, so that the model can quickly adapt to new tasks and enhance cross-domain self-adaptation capabilities. The quantum enhanced computing module deeply mines the superposition and entanglement characteristics of quantum states, encodes neuron pulses into quantum bits, constructs a quantum pulse neural network layer, and combines it with the quantum approximate optimization algorithm (QAOA) to convert parameter optimization problems into quantum Hamiltonian solutions, forming a classical-quantum hybrid computing framework, which greatly improves computing efficiency and optimization capabilities. The architecture integrates cutting-edge technologies such as pulse neural networks, deep learning, multimodal learning and quantum computing to achieve multi-technology collaborative innovation and performance leapfrog improvement, and accelerate the transformation and upgrading of artificial intelligence industries in various fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is a schematic diagram of the overall process of a brain-like intelligence-driven heterogeneous fusion method described in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0042] Example 1, with reference to Figure 1 , as one embodiment of the present invention, provides a brain-like intelligent driven heterogeneous fusion method, comprising:

[0043] S1: Design a converged architecture and build a collaborative computing foundation;

[0044] S2: Propose an algorithm to transfer knowledge from the source modality to the target modality through contrastive learning, introduce a learning framework to optimize the transfer process, and enrich multimodal data support through enhanced computing;

[0045] S3: Based on enhanced computing, the characteristics are used to encode the pulses and optimize the solution process.

[0046] It should be noted that current artificial intelligence systems have inherent defects in multimodal data processing, such as semantic association fragmentation, insufficient cross-task adaptability, and inefficient parameter optimization. Traditional methods find it difficult to balance the computational efficiency of biological inspiration with the generalization requirements of complex tasks.

[0047] Therefore, in response to the above-mentioned problems of weak modal collaboration foundation, rigid knowledge transfer mechanism, and single computing optimization path, through steps S1-S3, the fusion architecture constructed by S1 breaks through the heterogeneous barriers of pulse and continuous signals, laying the foundation for unified multimodal processing; the algorithm mechanism proposed by S2 achieves cross-modal semantic alignment through contrastive learning, and combines the adaptive framework to eliminate task migration obstacles; the enhanced computing implemented by S3 deeply integrates quantum state characteristics, reconstructs the pulse coding and parameter optimization paradigm; the three steps work together to form a technical closed loop of infrastructure innovation-migration mechanism upgrade-computing efficiency leap, and systematically solve the industrial-level problems of weak multimodal correlation, poor task generalization, and slow optimization speed.

[0048] Example 2, reference Figure 1 , is an embodiment of the present invention. Based on the above embodiments, a brain-like intelligent driven heterogeneous fusion method is provided.

[0049] In the implementation mode of the present application, a fusion architecture is designed in step S1 to build a collaborative computing foundation. By designing a heterogeneous fusion architecture of pulse neural networks and deep learning models, the pulse-deep collaborative neural network serves as the basic architecture, providing event-driven computing and hierarchical feature extraction capabilities, providing a basis for the efficient operation of the entire system, and making subsequent transfer learning and quantum computing enhancements possible.

[0050] In an optional implementation, the design of the fusion architecture in step S1 and the construction of the collaborative computing foundation can also be achieved by introducing a hierarchical processing mechanism of spatiotemporal features; for the industrial equipment anomaly detection scenario, a micro-pulse neural network array is deployed to process high-frequency sensor pulse streams and capture millisecond-level current fluctuation events; at the same time, a lightweight Transformer unit is configured to parse the temporal semantics of the equipment operation log, and the pulse event features are aligned with the text maintenance record features through an asynchronous signal conversion interface to form a cross-modal collaborative representation of the equipment health status; this mechanism utilizes the event-driven characteristics of the pulse network to reduce edge power consumption, and at the same time models long-term fault dependencies through deep learning units, providing a low-latency, highly interpretable industrial knowledge representation foundation for subsequent transfer learning modules.

[0051] In another optional implementation, the design of the fusion architecture in step S1 and the establishment of a collaborative computing foundation can also be achieved by constructing a biosignal-environmental context coupling architecture. For intelligent prosthetic control scenarios, a layered spiking neuron network is used to decode the millisecond-level burst discharge pattern of electromyographic signals to identify the user's movement intention in real time. A convolutional attention module is simultaneously deployed to analyze the spatial distribution of obstacles in the environmental visual stream, and a dynamic weighted signal conversion gateway is used to fuse bio-pulse features with environmental semantic features to generate a motion decision vector. This architecture ensures real-time physiological signal processing through the biocompatibility of the spiking network, and utilizes deep learning units to understand complex environmental contexts, providing the quantum enhancement module with a multimodal control instruction stream with precise spatial and temporal alignment.

[0052] In the embodiment of the present application, the design of the fusion architecture and the establishment of the collaborative computing foundation in step S1 also include:

[0053] SNN layer, deep learning layer and signal conversion module. The SNN layer adopts the LIF neuron model to simulate the dynamic characteristics of the membrane potential of biological neurons. The deep learning layer selects the Transformer architecture and uses the self-attention mechanism to capture long-distance dependencies. The signal conversion module is inserted between the SNN layer and the Transformer layer to achieve bidirectional conversion between pulse signals and continuous feature maps.

[0054] The dynamic equation of neuronal membrane potential of the LIF neuron model is expressed as:

[0055]

[0056] Where τ is the membrane time constant, V(t) is the membrane potential, and I(t) is the input current. When V(t)≥V th (threshold), the neuron fires a pulse and resets to V reset ;

[0057] Signal conversion uses rate coding to convert the pulse sequence of SNN into a continuous feature map, counts the number of pulses emitted by each neuron in the time window T, and generates a feature map FSNN = n / T.

[0058] In the implementation manner of the present application, an algorithm is proposed in step S2, which transfers the knowledge of the source modality to the target modality through contrastive learning, introduces a learning framework to optimize the migration process, and enriches multimodal data support through enhanced computing. By proposing an inter-modal self-attention distillation algorithm, the knowledge of the source modality is transferred to the target modality through contrastive learning to achieve zero-sample or small-sample task adaptation, and introduces a meta-learning framework to optimize the migration process, so that the model can quickly adjust the feature representation when encountering new tasks, thereby improving cross-domain adaptability. The multimodal dynamic knowledge transfer mechanism realizes cross-modal adaptive learning based on the pre-trained model, which not only enhances the versatility of the system, but also provides rich multimodal data support for the quantum enhanced computing module, which helps to improve the efficiency and accuracy of quantum coding.

[0059] In an optional implementation, an algorithm is proposed in step S2 to transfer the knowledge of the source modality to the target modality through comparative learning. A learning framework is introduced to optimize the migration process. Multimodal data support is enriched through enhanced computing, and cross-modal adapters can also be dynamically generated through meta-learning. For industrial equipment fault prediction scenarios, the pre-training phase only requires a small amount of normal vibration sensor signals (source modality) and operation and maintenance log text (target modality). The meta-learning framework automatically constructs potential association rules between signals and text. When a new unknown fault type is added, the system generates a lightweight adapter module in real time based on the pulse characteristics of quantum coding, dynamically mapping the peak characteristics of the vibration spectrum to natural language descriptions (such as "bearing high-frequency resonance"), achieving zero-sample fault attribution analysis.

[0060] In another optional embodiment, an algorithm is proposed in step S2, which transfers the knowledge of the source modality to the target modality through contrastive learning, introduces a learning framework to optimize the migration process, and enriches multimodal data support through enhanced computing. It can also be achieved through a dynamic incremental learning mechanism for industrial defect detection: based on the product surface image (source modality) and the sensor timing signal (target modality), a multi-scale defect feature distillation pipeline is designed: first, the image defect texture and the sensor waveform peak feature are aligned through contrastive learning, and then a meta-learning framework is introduced to construct an incremental task sequence (such as adding new defect categories such as scratches and rust). Only 1-3 samples are input each time to dynamically update the cross-modal adapter parameters to quickly adapt to new defect types in the production line; at the same time, the fused cross-modal features are generated through the quantum coding layer to generate an entangled state pulse sequence, thereby enhancing the quantum state recognition sensitivity of micro-defects.

[0061] In the embodiment of the present application, the algorithm proposed in step S2 transfers the knowledge of the source modality to the target modality through comparative learning, introduces a learning framework to optimize the migration process, and enriches the multimodal data support through enhanced computing, further comprising:

[0062] Use a pre-trained multimodal model (such as CLIP) to extract features of the source modality (image) and the target modality (text), respectively. Image features: Fimg = VisionEncoder (I), text features: Ftext = TextEncoder (T), and maximize the similarity of positive sample pairs (matching image-text) and minimize the similarity of negative sample pairs (mismatching image-text) through a contrast loss function (such as Info NCE), expressed as:

[0063]

[0064] Where s(·,·) is the cosine similarity and τ is the temperature hyperparameter;

[0065] During the meta-training phase, the meta-learner is trained on a small number of tasks to learn to adjust the model parameters to adapt to the new task. The meta-objective function is defined as minimizing the loss on the new task as follows:

[0066]

[0067] Among them, θ is the model parameter, α is the learning rate, and T is the task distribution;

[0068] When faced with a new task, the initialization parameter θ* generated by the meta-learner is used to quickly adjust the model through a small number of gradient update steps, so that the target modality feature representation adapts to the distribution of the new task, combining the natural language description with the generator of the target modality.

[0069] In the implementation mode of the present application, in step S3, based on enhanced computing, the pulses are encoded using the characteristics, and the solution process is optimized by adding a quantum enhanced computing module. The superposition and entanglement characteristics of the quantum state are used to encode the neuron pulses into quantum bits, and a quantum pulse neural network layer is designed. The quantum enhanced computing module further improves the computing efficiency and optimization capability of the system through quantum pulse coding and QAOA optimization algorithm, and adds a quantum enhanced computing module.

[0070] In the embodiment of the present application, step S3 is based on enhanced calculation, using characteristics to encode pulses and optimize the solution process, and further includes:

[0071] Encode classical pulse signals into quantum bits, use quantum superposition states to simultaneously represent multiple pulse modes, fuse multimodal pulse signals through entangled states, and construct a joint quantum representation space to enhance semantic associations between modalities;

[0072] Combining the quantum approximate optimization algorithm (QAOA), the spiking neural network parameter optimization problem is transformed into a quantum Hamiltonian problem. QAOA is used to iteratively solve the optimal parameters on a quantum processor, accelerating convergence through a classical-quantum hybrid computing framework.

[0073] The calculation formula of the plurality of pulse modes is represented as:

[0074] |psi> = sigma0|0> + beta|1>, |alpha| 2 + |beta| 2 = 1;

[0075] Wherein, |0> represents no pulse, and |1> represents pulse;

[0076] The quantum-enhanced computing module includes a quantum pulse coding layer, and uses superposition and entanglement characteristics of a quantum state to code a neuron pulse into a quantum bit, a quantum pulse neural network (QS-SNN) layer is designed, and a quantum approximate optimization algorithm (QAOA) is used to accelerate parameter optimization of the SNN.

[0077] The calculation formula of the classical-quantum hybrid computing framework is represented as:

[0078]

[0079] Wherein, psi(gamma, beta)|HC|psi(gamma, beta)> is an expected value of a quantum state psi(gamma, beta) under a problem Hamiltonian HC, and represents an optimization target corresponding to current parameters gamma and beta.

[0080] In summary, the present application is based on a brain-like intelligent driven artificial intelligence architecture, and a pulse-depth collaborative neural network is used as a core to construct a heterogeneous fusion system. The SNN layer simulates the dynamic characteristics of the biological neuron membrane potential, the deep learning layer of the Transformer architecture captures the long-distance dependency relationship, and the signal conversion module realizes the bidirectional conversion of the pulse signal and the continuous feature map, thereby laying a foundation for efficient operation of the system. The inter-modal self-attention distillation algorithm introduces a contrastive learning and meta-learning framework, realizes cross-modal knowledge transfer and zero-shot and small-sample task adaptation, enables the model to quickly adapt to new tasks, and enhances the cross-domain adaptive ability. The quantum-enhanced computing module deeply excavates the superposition and entanglement characteristics of the quantum state, encodes the neuron pulse into a quantum bit, constructs a quantum pulse neural network layer, and combines the quantum approximate optimization algorithm (QAOA) to convert the parameter optimization problem into a quantum Hamiltonian solution, forms a classical-quantum hybrid computing framework, greatly improves the computing efficiency and optimization ability, integrates the pulse neural network, deep learning, multi-modal learning and quantum computing and other frontier technologies, realizes multi-technology collaborative innovation and performance leap, and accelerates the transformation and upgrading of the artificial intelligence industry in various fields.

[0081] Embodiment 3 is an embodiment of the present application, and provides a brain-like intelligent driven heterogeneous fusion method. In order to verify the beneficial effects of the present application, economic benefit calculation and simulation experiments are used for scientific demonstration.

[0082] By comparing the loss function (Info NCE) and the meta-learning framework of the present invention, the model's ability to capture cross-modal semantics in zero-shot scenarios is improved by 8.7% (image-text matching) and 7.7% (classification), and the convergence speed of the meta-training phase is increased by 40%. This proves that the meta-learning framework can quickly optimize model parameters and quickly adapt to new task distributions. The data is shown in Table 1:

[0083] Table 1 Cross-modal transfer task (image-text zero-shot learning)

[0084]

[0085]

[0086] Comparing traditional SGD with the quantum approximate optimization algorithm (QAOA) of our invention, QAOA reduces the number of iterations for spiking neural network parameter optimization by 85% and the error by 62.2%, demonstrating the excellent acceleration effect of quantum state superposition and entanglement on optimization problems. QAOA shortens the computation time by 73% compared to traditional SGD, demonstrating the significant advantages of quantum Hamiltonian solution in large-scale parameter optimization. The data are shown in Table 2:

[0087] Table 2 Quantum enhanced computing module performance (parameter optimization task)

[0088]

[0089] By using quantum entangled states to fuse multimodal pulse signals, the feature fusion time was reduced by 46.6%, demonstrating the efficient processing capability of quantum representation space for multimodal data. The cross-modal retrieval accuracy was improved by 16.5%, verifying the effectiveness of the joint quantum representation space in mining deep semantic associations between modalities. The data is shown in Table 3:

[0090] Table 3 Multimodal data processing efficiency (MNIST+Audio dataset)

[0091] Model architecture Feature fusion time consumption (ms) Cross-modal retrieval accuracy (%) Traditional feature concatenation 28.5 76.3 Quantum pulse encoding fusion 15.2 88.9

[0092] It can be seen that the technical advantages of the theoretical framework and algorithm design mentioned in this invention are verified:

[0093] Cross-modal transfer capability: The inter-modal self-attention distillation algorithm and meta-learning framework significantly enhance the adaptability of zero-sample or small-sample tasks; Quantum computing gain: The quantum enhanced computing module greatly accelerates the parameter optimization process through the QAOA algorithm and reduces the solution error; Multimodal processing efficiency: Quantum pulse coding technology greatly optimizes the fusion and retrieval performance of multimodal data.

[0094] Example 4. The above is a schematic scheme of a brain-like intelligence-driven heterogeneous fusion method. It should be noted that the technical solution of this brain-like intelligence-driven heterogeneous fusion system and the technical solution of the above-mentioned brain-like intelligence-driven heterogeneous fusion method belong to the same concept. For details not described in detail in the technical solution of the brain-like intelligence-driven heterogeneous fusion system in this embodiment, please refer to the description of the technical solution of the above-mentioned brain-like intelligence-driven heterogeneous fusion method.

[0095] This embodiment also provides a brain-inspired intelligent-driven heterogeneous fusion system, including:

[0096] Converged architecture module, designing converged architecture and building collaborative computing foundation;

[0097] The migration optimization module proposes an algorithm to transfer the knowledge of the source modality to the target modality through comparative learning, introduces a learning framework to optimize the migration process, and enriches multimodal data support through enhanced computing;

[0098] The coding enhancement module encodes the pulses based on enhanced calculation and optimizes the solution process.

[0099] This embodiment also provides an electronic device suitable for a brain-like intelligent driven heterogeneous fusion scenario, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions, thereby implementing a brain-like intelligent driven heterogeneous fusion method proposed in the above embodiment.

[0100] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, a heterogeneous fusion method for brain-like intelligent driving is implemented as proposed in the above embodiment.

[0101] The storage medium proposed in this embodiment and the heterogeneous fusion method for realizing a brain-like intelligent drive proposed in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0102] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general hardware, and of course can also be implemented by hardware. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0103] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A brain-like intelligence-driven heterogeneous fusion method, characterized in that: include: Design a fusion architecture and build a collaborative computing foundation; Propose an algorithm to transfer knowledge from the source modality to the target modality through contrastive learning, introduce a learning framework to optimize the migration process, and enrich multimodal data support through enhanced computing; Based on enhanced computing, the characteristics are used to encode the pulses and optimize the solution process.

2. The brain-like intelligent driven heterogeneous fusion method according to claim 1, characterized in that: Design a converged architecture and build a collaborative computing foundation, including: Use neuron model to simulate the potential dynamic characteristics of biological neurons; Dependency parsing, capturing cross-level feature relationships; Build a bidirectional conversion mechanism between pulses and features to achieve collaborative processing.

3. The brain-like intelligent driven heterogeneous fusion method according to claim 2, characterized in that: The proposed algorithm transfers the knowledge of the source modality to the target modality through contrastive learning, including: Extract feature representation through pre-training and use functions for alignment; A mapping and conversion mechanism for inter-modal feature representation is established, and parameters are adjusted through gradient optimization to achieve knowledge transfer in the target area.

4. The brain-like intelligent driven heterogeneous fusion method according to claim 3, characterized in that: The introduction of the learning framework optimizes the migration process and enriches multimodal data support through enhanced computing, including: Train the learning framework on the task distribution to generate initialization parameters that can adapt to the feature representation of the new task; Expand the coding dimension through quantum state superposition and entanglement mechanisms; By providing high-dimensional support through enhanced computing, the learning framework can adapt to new tasks in the cross-modal transfer process.

5. The brain-like intelligent driven heterogeneous fusion method according to claim 4, characterized in that: Based on enhanced computing, the pulses are encoded using characteristics to optimize the solution process, including: Utilize quantum state characteristics to encode classical pulse signals into superposition state representation; By entangled and fused multimodal pulse signals, a joint representation space is constructed to enhance the semantic association between modalities; Parameter optimization is converted into quantum state evolution, and the parameters are iteratively solved using a collaborative framework.

6. The brain-like intelligent driven heterogeneous fusion method according to claim 5, characterized in that: The feature representation is extracted through pre-training and aligned using a function, including: A pre-trained multimodal model is used to extract features of the source modality and target modality respectively, and the contrast loss function is used to maximize the similarity of matching images and texts of positive samples and minimize the similarity of mismatching images and texts of negative samples.

7. The brain-like intelligent driven heterogeneous fusion method according to claim 6, characterized in that: The generating of initialization parameters that are adaptable to the feature representation of the new task includes: When faced with a new task, the initialization parameters generated by the meta-learner are used to adjust the model through gradient updates so that the target modality feature representation adapts to the distribution of the new task; Among them, the meta-learner includes a meta-training phase, in which the meta-learner is trained on a small number of tasks, learning to adjust the model parameters to adapt to new tasks, and the meta-objective function is defined as minimizing the loss on the new task.

8. A brain-like intelligence-driven heterogeneous fusion system, applying the method according to any one of claims 1 to 7, characterized in that: include: Converged architecture module, designing converged architecture and building collaborative computing foundation; The migration optimization module proposes an algorithm to transfer the knowledge of the source modality to the target modality through comparative learning, introduces a learning framework to optimize the migration process, and enriches multimodal data support through enhanced computing; The coding enhancement module encodes the pulses based on enhanced calculation and optimizes the solution process.

9. An electronic device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the brain-like intelligence-driven heterogeneous fusion method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of a brain-like intelligence-driven heterogeneous fusion method as described in any one of claims 1 to 7.

Images