An AI large model-based multi-modal data search method and system

Abstract

The application discloses a kind of multi-modal data search method and system based on AI big model, it is related to multi-modal data processing technical field, including, multi-modal data input quantum state propagation engine, it is mapped to quantum entanglement state using quantum neural radiation field model, generates quantum entanglement state encoding feature tensor;Quantum state propagation path of quantum entanglement state encoding feature tensor is dynamically planned based on nonlinear soliton wave equation, constructs quantum topological index network and knowledge graph;User search instruction is compiled as quantum measurement operator, executes quantum state propagation path navigation in quantum topological index network, generates cross-modal matching candidate result set through path interference effect;Quantum decoherence driven probability reordering is carried out to candidate result set, and high confidence result is screened;The application solves the problem of high response delay caused by calculation efficiency bottleneck through amplitude damping channel and volatility weight calculation.

Description

Technical Field

[0001] This invention relates to the field of multimodal data processing technology, and in particular to a multimodal data search method and system based on a large AI model. Background Technology

[0002] Multimodal data search technology has made progress in the field of artificial intelligence in recent years. Its core goal is to achieve efficient cross-modal retrieval and matching by integrating information from multiple modalities such as text, images, audio, and video. Current mainstream methods mainly rely on deep learning frameworks to achieve joint representation of different modalities by constructing a multimodal embedding space. For example, multimodal models based on the Transformer architecture map text and images into a shared semantic space through large-scale pre-training, enabling cross-modal retrieval. Feature fusion techniques are widely used to enhance the relevance of multimodal information to improve retrieval accuracy.

[0003] Despite significant breakthroughs in multimodal retrieval, existing methods still suffer from two shortcomings. Firstly, their ability to handle cross-modal semantics is limited. Due to the inherent differences between data from different modalities, the linear embedding space mapping relied upon by current methods struggles to capture deep-seated nonlinear associations, thus limiting the accuracy of cross-modal matching. Secondly, multimodal data processing within classical computing architectures suffers from computational efficiency bottlenecks. Facing the real-time retrieval demands of massive heterogeneous data, existing methods rely on computationally complex algorithms for feature extraction, fusion, and matching, resulting in high response latency and making them unsuitable for large-scale application scenarios. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a multimodal data search method based on large AI models to solve the problems of limited accuracy and high response latency in cross-modal matching.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] Firstly, the present invention provides a multimodal data search method based on a large AI model, comprising,

[0008] Multimodal data is obtained from multiple heterogeneous data sources, including structured data, semi-structured data, and unstructured data;

[0009] Multimodal data is input into the quantum state propagation engine and mapped to the quantum entangled state using the quantum neural radiation field model to generate a quantum entangled state encoded feature tensor.

[0010] Based on the dynamic programming of quantum state propagation paths of quantum entangled state encoded feature tensors according to nonlinear soliton wave equations, a quantum topological index network and knowledge graph are constructed.

[0011] The user search instructions are compiled into quantum measurement operators, and quantum state propagation path navigation is performed in the quantum topology indexing network to generate a candidate result set for cross-modal matching through path interference effects;

[0012] The candidate result set is reordered by quantum decoherence-driven probability to filter high-confidence results;

[0013] Based on high-confidence results and knowledge graphs, a generative AI large model is used to generate search results.

[0014] As a preferred embodiment of the multimodal data search method based on a large AI model described in this invention, the specific steps for constructing a quantum topology index network based on the quantum state propagation path of the quantum entangled state-encoded feature tensor dynamically planned according to the nonlinear soliton wave equation are as follows:

[0015] Set an upper limit for the propagation time and an energy convergence threshold for the nonlinear soliton wave equation;

[0016] Divide the time into blocks based on the upper limit of propagation time, assign a set of parameterized rotating gates to each time block, insert CNOT gates between adjacent time blocks, and assemble all time blocks, parameterized rotating gates and CNOT gates to form a quantum state propagation simulation circuit;

[0017] Based on the quantum state propagation simulation circuit, the parameters of the parameterized rotating gate are optimized, the optimization termination is determined according to the energy convergence threshold, and the ground state probability amplitude of all qubits is recorded to obtain the quantum state amplitude distribution dataset.

[0018] A critical threshold is set based on the quantum state amplitude distribution dataset, and key nodes are selected. The quantum amplitude correlation coefficient between each pair of key nodes is calculated. A connection threshold is set based on the quantum amplitude correlation coefficient, and key node pairs are selected.

[0019] To create bidirectional connection edges for the selected key node pairs, the quantum amplitude correlation coefficient is used as the edge weight;

[0020] The path propagation entropy value is obtained by calculating the Shannon entropy of each key node;

[0021] A quantum topological index network is constructed by combining all key nodes, bidirectional connection edges, and path propagation entropy values.

[0022] A knowledge graph is constructed based on key nodes, edge weights, and quantum amplitude correlation coefficients.

[0023] As a preferred embodiment of the multimodal data search method based on AI large model described in this invention, the step of obtaining multimodal data from multiple heterogeneous data sources refers to establishing a data access interface and connecting it with multiple heterogeneous data sources, classifying and allocating the data collection tasks, extracting metadata from each heterogeneous data source and cleaning and verifying it to obtain multimodal data.

[0024] As a preferred embodiment of the multimodal data search method based on a large AI model described in this invention, the steps of inputting multimodal data into a quantum state propagation engine, mapping it to a quantum entangled state using a quantum neural radiation field model, and generating a quantum entangled state encoded feature tensor are as follows:

[0025] Multimodal data is segmented into discrete word units, and normalized word vectors are generated by performing a rotation gate operation through a quantum word embedding layer. Quantum amplitude encoding is used to map the vectors to a superposition of ground state quantum states.

[0026] Multimodal data is decoded into an RGB pixel matrix, and a quantum filter bank is used to perform a convolution operation on the RGB pixel matrix to generate quantum frequency domain states through quantum Fourier transform.

[0027] The ground state superposition quantum state and the quantum frequency domain state are loaded into the quantum neural radiation field model, and cross-modal entangled quantum states are generated through the Hadamard gate and the CNOT gate.

[0028] Based on cross-modal entangled quantum states, quantum entangled state encoded feature tensors are generated through quantum principal component analysis.

[0029] As a preferred embodiment of the multimodal data search method based on a large AI model described in this invention, the steps of compiling user search instructions into quantum measurement operators, performing quantum state propagation path navigation in a quantum topology indexing network, and generating a candidate result set for cross-modal matching through path interference effects are as follows:

[0030] The user's search command is parsed into entity words and relation words. Entity words are generated into complex field quantum states through a quantum word embedding layer, and relation words are compiled into Pauli rotation gate parameter sequences.

[0031] Generate the unitary matrix of the quantum measurement operator based on the Pauli rotation gate parameter sequence;

[0032] Based on the unitary matrix of quantum measurement operators and quantum topological indexing networks, the path interference effect is triggered by the control bit-target bit gate operation to obtain the path superposition quantum state;

[0033] Based on the quantum state in the complex field, the quantum state at the end of the path and the quantum state of the target entity are defined. The inner product of the quantum state at the end of the path and the quantum state of the target entity is calculated to obtain the cross-modal matching degree.

[0034] Candidate result sets are filtered based on cross-modal matching degree.

[0035] As a preferred embodiment of the multimodal data search method based on a large AI model described in this invention, the specific steps for performing quantum decoherence-driven probability reordering on the candidate result set and filtering high-confidence results are as follows:

[0036] The path superposition quantum state copy is obtained by retrieving the candidate result set and the quantum topology index network;

[0037] Calculate the damping coefficient of the amplitude damping channel based on the path propagation entropy value of the quantum state replica in the path superposition.

[0038] Perform a quantum dephase disturbance operation on the copy of the path superposition quantum state and perform normalization to generate a dephase disturbance quantum state;

[0039] The terminal node of the path propagating along the quantum topology index network is defined as the endpoint critical node. The ground state probability amplitude of the path superimposed quantum state copy and the dephase-disturbed dynamic quantum state at the endpoint critical node is measured, and the volatility is calculated based on the ground state probability amplitude.

[0040] Calculate the volatility weight and entropy weight of the candidate result set, and generate the reordering factor;

[0041] The candidate result set is reordered based on the reordering factor to select high-confidence results.

[0042] As a preferred embodiment of the multimodal data search method based on AI large model described in this invention, the step of generating search results using a generative AI large model based on high-confidence results and knowledge graphs refers to generating an input sequence based on high-confidence results and knowledge graphs, inputting the input sequence into the generative AI large model to generate output text by label, formatting the output text, and adding a search result summary to obtain the search results.

[0043] Secondly, this invention provides a multimodal data search system based on a large AI model, including:

[0044] The acquisition module acquires multimodal data from multiple heterogeneous data sources, including structured data, semi-structured data, and unstructured data.

[0045] The generation module inputs multimodal data into the quantum state propagation engine, maps it to the quantum entangled state using the quantum neural radiation field model, and generates a quantum entangled state encoded feature tensor.

[0046] The module constructs a quantum topology index network and knowledge graph based on the dynamic programming of quantum entangled state encoded feature tensors using nonlinear soliton wave equations.

[0047] The matching module compiles user search commands into quantum measurement operators, performs quantum state propagation path navigation in the quantum topology indexing network, and generates a candidate result set for cross-modal matching through path interference effects;

[0048] The filtering module performs quantum decoherence-driven probability reordering on the candidate result set to filter high-confidence results;

[0049] The enhancement module uses a generative AI model to generate search results based on high-confidence results and knowledge graphs.

[0050] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the multimodal data search method based on a large AI model as described in the first aspect of the present invention.

[0051] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the multimodal data search method based on a large AI model as described in the first aspect of the present invention.

[0052] The beneficial effects of this invention are as follows: by dynamically planning the propagation path of the quantum entangled state encoded feature tensor through the nonlinear soliton wave equation, the nonlinear correlation of multimodal data in the quantum state evolution process can be captured, improving the accuracy of cross-modal matching and reducing redundant calculations; by calculating the amplitude damping channel and volatility weight, the high computational complexity of existing methods is avoided, the response delay of multimodal data processing is reduced, and the problem of high response delay caused by computational efficiency bottleneck is solved. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of a multimodal data search method based on a large AI model.

[0055] Figure 2 This is a schematic diagram of a multimodal data search system based on a large AI model.

[0056] Figure 3 This is a schematic diagram of the processing procedure for the quantum neural radiation field model.

[0057] Figure 4This is a schematic diagram of the quantum state propagation path navigation process. Detailed Implementation

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0059] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0060] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0061] Reference Figures 1-4 As an embodiment of the present invention, a multimodal data search method based on a large AI model is provided, comprising the following steps:

[0062] S1: Obtain multimodal data from multiple heterogeneous data sources, including structured data, semi-structured data, and unstructured data.

[0063] The specific steps are as follows:

[0064] Establish connections between the data access interface and multiple heterogeneous data sources: Configure the protocol interface, input the authentication information of each heterogeneous data source, including IP address, port, username and password, and complete the handshake protocol to verify the validity of the connection.

[0065] The tasks to be collected are classified and allocated: the tasks to be collected are divided into task queues according to heterogeneous data types, and the collection priority is allocated from large to small according to the ratio of update frequency to data volume. Heterogeneous data types include structured, semi-structured and unstructured data.

[0066] Extract metadata from heterogeneous data sources: For structured data, execute table structure query commands to extract field names, data types, and primary and foreign key constraints; for semi-structured data, parse key-value pairs or tag levels to obtain data patterns; and for unstructured data, read file header information to extract basic attributes.

[0067] Remove illegal UTF-8 characters from text fields in structured data, fill numeric fields with column mean, and standardize date field format to ISO8601 standard; fix formatting errors and remove invalid tags in semi-structured data; correct metadata errors and filter corrupted files in unstructured data.

[0068] Validate the integrity of foreign key relationships for structured data, such as whether the user ID exists in the related table and check the validity of the numerical range; truncate field-level errors, such as truncating age 150 to 120; export record-level errors to the isolation area.

[0069] For semi-structured data, validate the consistency of the data structure, for example, check if the JSON key list matches the structure description file, and detect data type conflicts; force data type conversion for field-level errors, and mark the difference keys for record-level errors;

[0070] Verify the integrity of unstructured data files, for example, by comparing the MD5 hash values ​​of unstructured data before and after collection to check attribute compliance; standardize the processing of field-level errors, and record the path of file-level errors and notify the management personnel.

[0071] Structured data is converted into key-value text by column name-value; semi-structured data is recursively expanded into nested structures as key-value text with path tags; dates / numerical values ​​are uniformly formatted as ISO strings; audio data in unstructured data is converted into Mel spectrograms; frequency data is sampled into image sequences according to frame rate.

[0072] The collected structured data is output as a Parquet format file, the semi-structured data is output as an Avro format file, and the unstructured data retains its original binary format and is attached with metadata tags, including source, type and basic attributes, to form multimodal data.

[0073] It should also be noted that by establishing data access interfaces and classifying and allocating tasks, combined with metadata cleaning and verification mechanisms, the collection needs of structured, semi-structured and unstructured data can be fully covered, ensuring the consistency of data quality. At the same time, standardized output formats can improve the efficiency of subsequent processing and lay the foundation for multimodal data fusion.

[0074] S2: Input multimodal data into the quantum state propagation engine, map it to the quantum entangled state using the quantum neural radiation field model, and generate the quantum entangled state encoded feature tensor.

[0075] The specific steps are as follows:

[0076] The semi-structured data and the text data in the structured data are segmented into discrete word units. A rotation gate operation is performed on the discrete word units through a quantum word embedding layer to obtain real-valued word vectors. The real-valued word vectors are normalized using the L2 norm, and the normalized real-valued word vectors are mapped to the ground state superposition quantum state through a quantum amplitude encoding operation.

[0077] The image data in unstructured data is decoded into an RGB pixel matrix using the imdecode() function of the OpenCV library. A quantum filter bank is used to perform a convolution operation on the RGB pixel matrix to obtain spatial structure feature data. The spatial structure feature data is then mapped to the frequency domain through quantum Fourier transform, and the output is a quantum frequency domain state.

[0078] The ground state superposition quantum state is loaded into the first quantum register, and the quantum frequency domain state is loaded into the second quantum register. A Hadamard gate operation is applied to each qubit of the first quantum register and to each qubit of the second quantum register to create a bimodal uniform superposition state. The Nth qubit of the first quantum register is used as the control bit, and the Nth qubit of the second quantum register is used as the target bit. A CNOT gate operation is performed to achieve bit-level entanglement. N is the index of the bit sequence number of the first and second quantum registers. The ground state superposition quantum state and the quantum frequency domain state with the smallest bit sequence number in the first and second quantum registers are selected to generate a cross-modal entangled quantum state for text-image association.

[0079] Constructing a quantum circuit containing a parameterized rotating gate sequence: The quantum circuit is constructed in a stacked structure. The first layer is initialized with Hadamard gates, the second layer deploys a parameterized rotating gate sequence (including Ry / Rz gates), and the third layer inserts fixed CNOT gates to form an entangled architecture, which is then combined into a quantum circuit of the quantum variational autoencoder.

[0080] The quantum variational autoencoder quantum circuit applies the McLachlan variational principle to optimize variational parameters, drives the evolution of cross-modal entangled quantum states along the direction of maximum variance in Hilbert space, projects the quantum state onto the quantum characteristic axis, locks the optimal variational parameters of the quantum variational autoencoder quantum circuit, freezes the quantum circuit operation, and outputs the projected quantum characteristic axis quantum state.

[0081] Perform Pauli-Z basis projection measurement on each qubit of the quantum characteristic axis quantum state to record the probability amplitude of the qubit in the ground state; sort the probability amplitudes of the ground state of all qubits of the quantum characteristic axis quantum state according to the qubit index to generate a quantum entangled state encoded feature tensor composed of real components.

[0082] It should also be noted that by using quantum word embedding layers and quantum filter banks to process text and image data respectively, and by constructing cross-modal entangled quantum states through Hadamard gates and CNOT gates, it is possible to achieve quantum fusion of multimodal features, improve the representation accuracy of text-image association, and provide high-dimensional feature support for subsequent quantum state propagation path planning.

[0083] S3: Based on the nonlinear soliton wave equation, dynamically program the quantum state propagation path of the quantum entangled state encoded feature tensor to construct a quantum topological index network and knowledge graph.

[0084] The specific steps are as follows:

[0085] The quantum entangled state encoded feature tensor is loaded into a quantum register as the initial quantum state input for the quantum topology index network; the parameters of the nonlinear soliton wave equation are input into the quantum simulator, the upper limit of the propagation time is set to the dimension value of the quantum entangled state encoded feature tensor, and the energy convergence threshold is twice the standard deviation of the energy fluctuation.

[0086] Constructing a quantum state propagation simulation circuit: Based on the upper limit of propagation time, the total propagation time is divided into equal-length intervals, each interval being defined as an independent time block, generating a time block sequence; a set of parameterized rotating gates is assigned to each time block, including Ry and Rz rotating gates, with the rotation angles of the Ry and Rz rotating gates set as time-dependent parameter variables; CNOT gates are inserted between adjacent time blocks to form a chain structure, with the last qubit of the previous time block serving as the control bit and the first qubit of the next time block serving as the target bit, performing CNOT gate operations; all time blocks, parameterized rotating gates, and CNOT gates are assembled in time block order to form the quantum state propagation simulation circuit.

[0087] The variational quantum characteristic solver algorithm is executed in a quantum simulator: randomly initialized rotation angle parameters are loaded into the Ry / Rz rotation gate of the quantum state propagation simulation circuit; the quantum state propagation simulation circuit is run on the quantum simulator, with the initial quantum state evolution as input and the propagation state as output; the expectation value measurement of the Pauli-Z operator is performed at the endpoint of each time block to collect quantum state probability amplitude distribution data; the variance of the quantum state probability amplitude distribution is calculated as the energy concentration; the partial derivative of the energy concentration with respect to the rotation angle parameters is calculated using the parameter shift rule, and the gradient direction is adjusted according to the quantum Fisher information matrix to generate the quantum natural gradient; the adjustment amount of the rotation angle parameters is scaled proportionally according to the direction of the quantum natural gradient, and the scaled rotation angle parameters are loaded into the Ry / Rz rotation gate of the quantum state propagation simulation circuit.

[0088] When the energy concentration improvement is lower than the energy convergence threshold after three consecutive iterations, the rotation angle parameter update is immediately stopped, and the current configuration state of the quantum state propagation simulation circuit is locked. At the end of each time block, complete basis vector measurement is performed to record the ground state probability amplitude of all qubits and generate a discretized quantum state amplitude distribution dataset. Based on the quantum state amplitude distribution dataset, a critical threshold is set using the percentile method. For example, the 75th percentile ground state probability amplitude is used as the critical threshold. Quantum nodes in the quantum state amplitude distribution dataset whose ground state probability amplitude exceeds the critical threshold are selected and marked as key nodes of the quantum topology index network.

[0089] Calculate the quantum amplitude correlation coefficient between each pair of key nodes. Based on the quantum amplitude correlation coefficient, set the connection threshold using the percentile method. For example, take the 90th percentile of the quantum amplitude correlation coefficient as the connection threshold. Select key node pairs whose quantum amplitude correlation coefficient exceeds the connection threshold. Create bidirectional connection edges for the selected key node pairs and record the quantum amplitude correlation coefficient as the edge weight. Calculate the Shannon entropy of the probability amplitude distribution dataset of each key node to obtain the path propagation entropy value. Combine all key nodes and bidirectional connection edges and record the path propagation entropy value to form a quantum topology index network. Transform bidirectional connection edges whose edge weight exceeds the median edge weight into a controlled NOT gate sequence of control bit-target bit. Transform bidirectional connection edges whose edge weight does not exceed the median edge weight into a controlled Z-gate sequence of control bit-target bit. Map the key node connection relationship of the quantum topology index network into a hierarchical structure of the quantum state propagation simulation circuit to generate a quantum controlled gate sequence containing a list of control bit and target bit definitions.

[0090] The key nodes of the quantum topology index network are mapped to knowledge graph entities. Based on the quantum amplitude correlation coefficient, the percentile method is used to set the weight threshold interval. For example, the 80th percentile of the quantum amplitude correlation coefficient is taken as the lower limit of the weight threshold interval and the 90th percentile is taken as the upper limit of the weight threshold interval. For each bidirectional connection edge in the quantum topology index network, if the edge weight is within the weight threshold interval, the relationship type is labeled as "correlated". If the edge weight is not less than the upper limit of the weight threshold interval, the relationship type is labeled as "strongly correlated".

[0091] In the quantum topological index network, the key node at the starting end of the connecting edge is taken as the head entity, and the key node at the ending end of the connecting edge is taken as the tail entity. The head entity, relation type, and tail entity are integrated into knowledge graph triples, and all knowledge graph triples are stored in the graph database to form a knowledge graph.

[0092] It should be noted that the expression for calculating the quantum amplitude correlation coefficient between each pair of key nodes is as follows:

[0093]

[0094] Where, ρ AB Let ψ be the quantum amplitude correlation coefficient between key node A and key node B. A Let ψ be the ground state probability amplitude of the critical node A. B Let E be the ground state probability amplitude of the critical node B, and E be the expectation operator, typically taken as the arithmetic mean of the quantum state amplitude distribution dataset. A μ is the arithmetic mean of the ground state probability amplitudes of all critical node A. B σ is the arithmetic mean of the ground state probability amplitudes of all critical node B. A Let σ be the standard deviation of the ground state probability amplitude of the critical node A. B denoted as the standard deviation of the ground state probability amplitude of critical node B.

[0095] It should also be noted that by optimizing the quantum state propagation path through the nonlinear soliton wave equation and combining the quantum amplitude correlation coefficient and Shannon entropy to screen key nodes, the nonlinear correlation of multimodal data in quantum evolution can be captured. The constructed quantum topology index network not only improves the accuracy of cross-modal matching, but also reduces resource consumption by reducing redundant computation.

[0096] S4: Compiles user search instructions into quantum measurement operators, performs quantum state propagation path navigation in the quantum topology indexing network, and generates a candidate result set for cross-modal matching through path interference effects.

[0097] The specific steps are as follows:

[0098] Stop words and punctuation marks are removed from the user's search command text, while entity nouns and relational verbs are retained. Part-of-speech tags are matched based on the semantic mapping rule base to identify and extract core entity words and relational words. The core entity words are segmented into discrete entity word units, and the discrete entity word units are processed through the parameterized rotation gate of the quantum word embedding layer to obtain normalized word vectors. Quantum amplitude encoding is performed on the normalized word vectors to generate complex field quantum states.

[0099] The dependency parsing tree is used to parse the grammatical dependency structure of relation words, resulting in grammatical dependency triples. Each grammatical dependency triple contains the relation word, the dependency relation type, and the dependent word. The difference in word position index between the relation word and the dependent word is defined as the dependency distance. According to the semantic mapping rule base, the grammatical dependency triples are mapped to a Pauli rotation door type sequence. The reciprocal of the dependency distance is converted into a rotation angle value to generate a Pauli rotation door parameter sequence.

[0100] A blank quantum circuit is created, with the number of qubits in the quantum circuit matching the number of qubits in the physical quantum state. The Pauli rotation gate type and rotation angle values ​​are read sequentially from the Pauli rotation gate parameter sequence, and Pauli rotation gates are deployed at the corresponding qubit positions in the blank quantum circuit according to the reading order. The physical quantum state is input as the initial state into the head of the quantum circuit. A corresponding single-qubit unitary matrix is ​​generated for each rotation gate according to the Pauli rotation gate parameter sequence. The single-qubit unitary matrix is ​​expanded to the full qubit space of the quantum circuit, and the non-active qubit positions are filled with identity matrices. Left-multiplying operations are performed on the expanded unitary matrix according to the rotation gate deployment order, and the complex value matrix of the accumulated left-multiplication results is calculated to obtain the global unitary transformation operator of the quantum circuit. Vector normalization is used to normalize each column vector of the global unitary transformation operator of the quantum circuit to generate the unitary matrix representation of the quantum measurement operator.

[0101] Read the quantum controlled gate sequence in the quantum topology index network, including the control bit address, target bit address, and gate type identifier; according to the control bit address and target bit address of the quantum controlled gate sequence, write control bit-target bit controlled NOT gate instructions or control bit-target bit controlled Z gate instructions one by one to the corresponding bit positions of the quantum processor, reset all qubits of the quantum processor to the ground state, apply Hadamard gate operation to the qubits corresponding to the starting key node to generate the initial superposition quantum state; execute the written control bit-target bit controlled NOT gate instructions or control bit-target bit controlled Z gate instructions, apply them to the initial superposition quantum state to obtain the path superposition quantum state propagating along the quantum topology index network, and measure the probability amplitude distribution of the path superposition quantum state.

[0102] From the probability amplitude distribution of the superimposed quantum states along the path, the complex domain quantum states of the key nodes corresponding to the endpoints of the paths propagating along the quantum topology index network are located and taken as the path endpoint quantum states. Among the complex domain quantum states generated by the user search command, the complex domain quantum states with the same key node address are taken as the target entity quantum states. The inner product of the path endpoint quantum states and the target entity quantum states is calculated to generate cross-modal matching degree values. Based on the cross-modal matching degree values, a minimum confidence threshold is set using the percentile method. For example, the 30th percentile value of the cross-modal matching degree value distribution is taken as the minimum confidence threshold. Path endpoint data items (structured / semi-structured / unstructured data units that match the propagation path endpoints in the quantum topology index network) with cross-modal matching degrees exceeding the minimum confidence threshold are screened and sorted in descending order of cross-modal matching degree to obtain a candidate result set containing path endpoint data items and cross-modal matching degrees.

[0103] It should also be noted that: through the path interference effect of quantum measurement operators and quantum topological indexing networks, entity words and relation words in user search instructions can be dynamically matched to generate a candidate result set for cross-modal matching. The cross-modal matching degree screening mechanism based on inner product calculation improves the relevance of search results and supports the parsing ability of complex semantic relationships.

[0104] S5: Perform quantum decoherence-driven probability reordering on the candidate result set to filter high-confidence results.

[0105] The specific steps are as follows:

[0106] For each path endpoint data item in the candidate result set, extract the key node index value from the metadata tag; query the key node metadata database of the quantum topology index network based on the key node index value to obtain the coordinate encoding of the corresponding key node in Hilbert space; retrieve the path superposition quantum state copy associated with the key node index from the amplitude distribution storage area of ​​the quantum topology index network and load it as a probability amplitude distribution dataset.

[0107] Based on the path propagation entropy of the path superposition quantum state replica, the damping coefficient of the amplitude damping channel is calculated using a linear inverse proportional mapping table. The mapping relationship is that the damping coefficient decreases proportionally as the path propagation entropy increases. For each qubit of the path superposition quantum state replica, the following is performed: the probability amplitude of the excited state decreases proportionally to the damping coefficient, the probability amplitude of the ground state increases accordingly, and the sum of squared moduli remains unchanged. The decay probability amplitude is synchronously introduced with a uniformly distributed random phase shift.

[0108] Perform correlated phase noise injection on adjacent qubits: take the qubit with the largest path propagation entropy as the control bit, apply additional random phase rotation to the qubits adjacent to the control bit, and the rotation angle is proportional to the entropy value of the control bit; calculate the sum of squares of the probability amplitude of all ground state components of the path superposition quantum state replica, adjust the real and imaginary parts of the probability amplitude according to the normalization factor so that the total probability is equal to 1, generate the dephase interference dynamic quantum state, and mark the entropy value-damping coefficient mapping record.

[0109] The terminal node of the path propagating along the quantum topology index network is defined as the endpoint critical node. The qubit corresponding to the endpoint critical node in the path superimposed quantum state replica is located, and Pauli-Z basis projection measurement is performed to record the ground state probability amplitude of the path superimposed quantum state replica as the original amplitude. The same endpoint critical node qubit of the dephased disturbance quantum state replica is located, and Pauli-Z basis projection measurement is performed to record the ground state probability amplitude of the dephased disturbance quantum state replica as the perturbation amplitude. The ratio of the absolute difference between the original amplitude and the perturbation amplitude to the original amplitude is taken as the volatility.

[0110] Calculate volatility weight, entropy weight, and reordering factor for each candidate outcome item;

[0111] The volatility weight is calculated using the following expression:

[0112] w = e ―10b ;

[0113] Where w is the volatility weight and b is the volatility;

[0114] Using the arithmetic square root of the path propagation entropy as the entropy weight, the reordering factor is calculated as follows:

[0115] R = M × w × s;

[0116] Where R is the reordering factor, M is the cross-modal matching degree value, and s is the entropy weight;

[0117] Sort the candidate result set in descending order according to the reordering factor value; generate a reordered list of candidate results.

[0118] The entropy threshold is set using the percentile method based on the path propagation entropy value. For example, the 70th percentile of the path propagation entropy value of the candidate result set is taken as the entropy threshold. The volatility threshold is set using the percentile method based on the volatility value. For example, the 30th percentile of the volatility value of the candidate result set is taken as the volatility threshold. Path endpoint data items with path propagation entropy values ​​greater than the entropy threshold and volatility values ​​less than the volatility threshold are selected to obtain high-confidence results.

[0119] It should also be noted that by calculating the amplitude damping channel and volatility weight, and combining the path propagation entropy value to screen high-confidence results, noise interference can be suppressed in the quantum decoherence process, avoiding high computational complexity sorting algorithms, reducing response latency, and solving the problem of insufficient real-time performance caused by efficiency bottlenecks in existing methods.

[0120] S6: Based on high-confidence results and knowledge graphs, use a generative AI large model to generate search results.

[0121] The specific steps are as follows:

[0122] For each path endpoint data item in the high-confidence results, multimodal semantic feature extraction is performed: Parquet file field names and values ​​are parsed using the Apache Parquet-MR library and converted into key-value pair text fragments; Avro key-value pair sets are decoded using the Apache Avro library to generate hierarchical list text in Markdown format; and unstructured image data is converted into text description statements using the CLIP visual encoder.

[0123] Identify knowledge graph entities corresponding to high-confidence results: For structured data, extract the primary key field value of the structured data, match the primary key field value in the knowledge graph entity name, if the primary key match fails, extract all text field values ​​in the structured data, perform edit distance calculation in the knowledge graph entity name, and select the knowledge graph entity with the smallest edit distance as the matching result;

[0124] For semi-structured data, the unique identifier in the semi-structured data is parsed and matched with the unique identifier in the entity name of the knowledge graph. If the unique identifier is missing, the hierarchical path of the semi-structured data is extracted and compared with the attribute path of the knowledge graph entity segment by segment. The entity with the highest path overlap is selected as the matching result.

[0125] For unstructured data, noun phrases are extracted from the text description, and the extracted noun phrases are compared with the names of knowledge graph entities. Case-insensitive inclusion matching is performed. If no knowledge graph entity is matched, the knowledge graph entity with the smallest edit distance is selected as the matching result.

[0126] The system retrieves all direct attributes of knowledge graph entities using a graph database query language, extracts attribute key-value pairs from the knowledge graph entities, retrieves first-degree neighbor entities of the knowledge graph entities, and converts the relation types and neighbor entities into attribute key-value pairs. For each attribute key-value pair, it generates a [attribute key: attribute value] fragment, arranges all [attribute key: attribute value] fragments in alphabetical order, connects the [attribute key: attribute value] fragments with semicolons, and appends them to the knowledge graph entity name to obtain the knowledge graph text.

[0127] Sort each path endpoint data item in the high-confidence results in descending order by reordering factor; extract the semantic feature text of each path endpoint data item, insert separator strings between the semantic features of adjacent data items, and concatenate all semantic feature texts in sorting order to generate continuous text blocks; extract the knowledge graph text corresponding to each high-confidence result, insert semicolons as separators between adjacent knowledge graph texts, and connect all knowledge graph texts to form continuous description blocks; add separator markers at the end of continuous text blocks, and append the corresponding continuous description blocks to the separator markers to form the input sequence;

[0128] The input sequence is processed using the built-in word segmenter of the generative AI large model: each character or word in the input sequence is mapped to an integer tag ID, generating a tag sequence represented by an integer sequence.

[0129] The labeled sequence is input into a generative AI model. Forward propagation is performed on the last labeled sequence to obtain the probability distribution of the next labeled sequence. The K labeled sequences with the highest probabilities in the probability distribution are retained as candidate labeled sequences (K is the bundle width). For each candidate labeled sequence, a new sequence branch is expanded. The cumulative log probability of all branch sequences is calculated. At each step, only the K branch sequences with the highest cumulative probabilities are retained as the candidate sequence set. When a candidate sequence has an end label or reaches its maximum length, the candidate sequence with the highest cumulative probability is selected as the optimal sequence from the candidate sequence set. The integer labeled sequence of the optimal sequence is restored to a string using a word segmenter to obtain the output text. Special control characters in the output text are removed, and consecutive spaces are compressed into a single space character to obtain the natural language enhanced text.

[0130] Scan the control characters in the natural language augmented text and remove all control characters; detect consecutive whitespace characters and replace consecutive whitespace characters with single whitespace characters; traverse the word sequence in the natural language augmented text, and if a word completely matches the semantic feature text of the high-confidence result, locate the position of the matching word; insert square brackets to mark the number after the matching word, and append a fixed-format search result summary string to the end of the natural language augmented text to obtain the search results in natural language format.

[0131] It should also be noted that by inputting semantic feature text concatenation and relation word retrieval templates into the generative AI model, it can generate structured natural language summaries that meet user needs, enhancing the interpretability and usability of search results. At the same time, by annotating metadata identifiers and merging duplicate descriptions, it ensures the integrity and conciseness of information integration.

[0132] This embodiment also provides a multimodal data search system based on a large AI model, including:

[0133] The acquisition module acquires multimodal data from multiple heterogeneous data sources, including structured data, semi-structured data, and unstructured data.

[0134] The generation module inputs multimodal data into the quantum state propagation engine, maps it to the quantum entangled state using the quantum neural radiation field model, and generates a quantum entangled state encoded feature tensor.

[0135] The module constructs a quantum topology index network by dynamically planning the quantum state propagation path of the quantum entangled state-encoded feature tensor based on the nonlinear soliton wave equation.

[0136] The matching module compiles user search commands into quantum measurement operators, performs quantum state propagation path navigation in the quantum topology indexing network, and generates a candidate result set for cross-modal matching through path interference effects;

[0137] The filtering module performs quantum decoherence-driven probability reordering on the candidate result set to filter high-confidence results;

[0138] The enhancement module uses a generative AI model to generate search results based on high-confidence results and knowledge graphs.

[0139] This embodiment also provides a computer device applicable to the multimodal data search method based on a large AI model, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the multimodal data search method based on a large AI model as proposed in the above embodiment.

[0140] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0141] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the multimodal data search method based on a large AI model as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0142] In summary, this invention achieves improved accuracy in cross-modal matching and reduced redundant computation by dynamically planning the propagation path of the quantum entangled state-encoded feature tensor using the nonlinear soliton wave equation. Furthermore, by employing amplitude damping channels and volatility weight calculations, it avoids the high computational complexity of existing methods, reduces the response latency in multimodal data processing, and solves the problem of high response latency caused by computational efficiency bottlenecks.

[0143] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multimodal data search method based on a large AI model, characterized in that: include, Multimodal data is obtained from multiple heterogeneous data sources, including structured data, semi-structured data, and unstructured data; Multimodal data is input into the quantum state propagation engine and mapped to the quantum entangled state using the quantum neural radiation field model to generate a quantum entangled state encoded feature tensor. Based on the dynamic programming of quantum state propagation paths of quantum entangled state encoded feature tensors according to nonlinear soliton wave equations, a quantum topological index network and knowledge graph are constructed. The user search instructions are compiled into quantum measurement operators, and quantum state propagation path navigation is performed in the quantum topology indexing network to generate a candidate result set for cross-modal matching through path interference effects; The candidate result set is reordered by quantum decoherence-driven probability to filter high-confidence results; Based on high-confidence results and knowledge graphs, a generative AI large model is used to generate search results.

2. The multimodal data search method based on large AI models as described in claim 1, characterized in that: The specific steps for constructing a quantum topological index network and knowledge graph based on the quantum state propagation path of the quantum entangled state encoded feature tensor using dynamic programming of the nonlinear soliton wave equation are as follows: Set an upper limit for the propagation time and an energy convergence threshold for the nonlinear soliton wave equation; Divide the time into blocks based on the upper limit of propagation time, assign a set of parameterized rotating gates to each time block, insert CNOT gates between adjacent time blocks, and assemble all time blocks, parameterized rotating gates and CNOT gates to form a quantum state propagation simulation circuit; Based on the quantum state propagation simulation circuit, the parameters of the parameterized rotating gate are optimized, the optimization termination is determined according to the energy convergence threshold, and the ground state probability amplitude of all qubits is recorded to obtain the quantum state amplitude distribution dataset. A critical threshold is set based on the quantum state amplitude distribution dataset, key nodes are selected, and the quantum amplitude correlation coefficient between each pair of key nodes is calculated. A connection threshold is set based on the quantum amplitude correlation coefficient, and key node pairs are selected based on the connection threshold. To create bidirectional connection edges for the selected key node pairs, the quantum amplitude correlation coefficient is used as the edge weight; The path propagation entropy value is obtained by calculating the Shannon entropy of each key node; A quantum topological index network is constructed by combining all key nodes, bidirectional connection edges, and path propagation entropy values. A knowledge graph is constructed based on key nodes, edge weights, and quantum amplitude correlation coefficients.

3. The multimodal data search method based on large AI models as described in claim 1, characterized in that: The process of obtaining multimodal data from multiple heterogeneous data sources refers to establishing a data access interface and connecting it to multiple heterogeneous data sources, classifying and allocating the data collection tasks, extracting metadata from each heterogeneous data source and cleaning and verifying it to obtain multimodal data.

4. The multimodal data search method based on large AI models as described in claim 3, characterized in that: The specific steps for inputting multimodal data into the quantum state propagation engine, mapping it to the quantum entangled state using the quantum neural radiation field model, and generating the quantum entangled state encoded feature tensor are as follows: Multimodal data is segmented into discrete word units, and normalized word vectors are generated by performing a rotation gate operation through a quantum word embedding layer. Quantum amplitude encoding is used to map the vectors to a superposition of ground state quantum states. Multimodal data is decoded into an RGB pixel matrix, and a quantum filter bank is used to perform a convolution operation on the RGB pixel matrix to generate quantum frequency domain states through quantum Fourier transform. The ground state superposition quantum state and the quantum frequency domain state are loaded into the quantum neural radiation field model, and cross-modal entangled quantum states are generated through the Hadamard gate and the CNOT gate. Based on cross-modal entangled quantum states, quantum entangled state encoded feature tensors are generated through quantum principal component analysis.

5. The multimodal data search method based on a large AI model as described in claim 2, characterized in that: The steps of compiling user search commands into quantum measurement operators, performing quantum state propagation path navigation in the quantum topology indexing network, and generating a candidate result set for cross-modal matching through path interference effects are as follows: The user's search command is parsed into entity words and relation words. Entity words are generated into complex field quantum states through a quantum word embedding layer, and relation words are compiled into Pauli rotation gate parameter sequences. Generate the unitary matrix of the quantum measurement operator based on the Pauli rotation gate parameter sequence; Based on the unitary matrix of quantum measurement operators and quantum topological indexing networks, the path interference effect is triggered by the control bit-target bit gate operation to obtain the path superposition quantum state; Based on the quantum state in the complex field, the quantum state at the end of the path and the quantum state of the target entity are defined. The inner product of the quantum state at the end of the path and the quantum state of the target entity is calculated to obtain the cross-modal matching degree. Candidate result sets are filtered based on cross-modal matching degree.

6. The multimodal data search method based on large AI models as described in claim 5, characterized in that: The specific steps for performing quantum decoherence-driven probability reordering on the candidate result set and filtering high-confidence results are as follows: The path superposition quantum state copy is obtained by retrieving the candidate result set and the quantum topology index network; Calculate the damping coefficient of the amplitude damping channel based on the path propagation entropy value of the quantum state replica in the path superposition. Perform a quantum dephase disturbance operation on the copy of the path superposition quantum state and perform normalization to generate a dephase disturbance quantum state; The terminal node of the path propagating along the quantum topology index network is defined as the endpoint critical node. The ground state probability amplitude of the path superimposed quantum state copy and the dephase-disturbed dynamic quantum state at the endpoint critical node is measured, and the volatility is calculated based on the ground state probability amplitude. Calculate the volatility weight and entropy weight of the candidate result set, and generate the reordering factor; The candidate result set is reordered based on the reordering factor to select high-confidence results.

7. The multimodal data search method based on large AI models as described in claim 6, characterized in that: The process of generating search results using a generative AI large model based on high-confidence results and knowledge graphs refers to generating an input sequence based on high-confidence results and knowledge graphs, inputting the input sequence into the generative AI large model to generate output text by labeling, formatting the output text, and adding a search result summary to obtain the search results.

8. A multimodal data search system based on a large AI model, based on the multimodal data search method based on a large AI model as described in any one of claims 1 to 7, characterized in that: include, The acquisition module acquires multimodal data from multiple heterogeneous data sources, including structured data, semi-structured data, and unstructured data. The generation module inputs multimodal data into the quantum state propagation engine, maps it to the quantum entangled state using the quantum neural radiation field model, and generates a quantum entangled state encoded feature tensor. The module constructs a quantum topology index network and knowledge graph based on the dynamic programming of quantum entangled state encoded feature tensors using nonlinear soliton wave equations. The matching module compiles user search commands into quantum measurement operators, performs quantum state propagation path navigation in the quantum topology indexing network, and generates a candidate result set for cross-modal matching through path interference effects; The filtering module performs quantum decoherence-driven probability reordering on the candidate result set to filter high-confidence results; The enhancement module uses a generative AI model to generate search results based on high-confidence results and knowledge graphs.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the multimodal data search method based on AI large model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the multimodal data search method based on AI large model as described in any one of claims 1 to 7.

Images