Multi-modal data search method and system based on AI large model

Abstract

The invention discloses a multi-modal data search method and system based on an AI large model, and relates to the technical field of multi-modal data processing, and the method comprises the steps: inputting multi-modal data into a quantum state propagation engine, mapping the multi-modal data to a quantum entanglement state through a quantum neural radiation field model, and generating a quantum entanglement state coding feature tensor; dynamically planning a quantum state propagation path of the quantum entangled state coding feature tensor based on a nonlinear soliton equation, and constructing a quantum topological index network and a knowledge graph; compiling a user search instruction into a quantum measurement operator, executing quantum state propagation path navigation in the quantum topological index network, and generating a candidate result set of cross-modal matching through a path interference effect; performing quantum de-coherence driven probability reordering on the candidate result set, and screening high-confidence results; through the amplitude damping channel and the fluctuation ratio weight calculation, the problem of high response delay caused by the calculation efficiency bottleneck is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal data processing, and in particular to a multi-modal data search method and system based on an AI large model. BACKGROUND

[0002] Multi-modal data search technology has made progress in the field of artificial intelligence in recent years, and its core goal is to achieve efficient retrieval and matching across modalities by integrating information from text, images, audio, video and other modalities. The current mainstream method mainly relies on a deep learning framework to realize the joint representation of different modal data by constructing a multi-modal embedding space. For example, a multi-modal model based on the Transformer architecture maps text and images into a shared semantic space through large-scale pre-training to achieve cross-modal retrieval. Feature fusion technology is widely used to enhance the relevance of multi-modal information to improve the accuracy of retrieval.

[0003] Although existing methods have made important breakthroughs in multi-modal retrieval, there are still two shortcomings. On the one hand, the existing methods have limited processing capacity for cross-modal semantics. Since data of different modalities have essential differences, the linear embedding space mapping relied on by existing methods is not easy to capture deep nonlinear correlations, resulting in limited accuracy of cross-modal matching. On the other hand, multi-modal data processing under the classical computing architecture has a computational efficiency bottleneck. In the face of real-time retrieval requirements for massive heterogeneous data, existing methods need to rely on high computational complexity algorithms in the feature extraction, fusion and matching process, resulting in high response delay and difficulty in meeting the needs of large-scale application scenarios SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a multi-modal data search method based on an AI large model to solve the problems of limited cross-modal matching accuracy and high response delay.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a multi-modal data search method based on an AI large model, comprising,

[0008] Obtaining multi-modal data from a plurality of heterogeneous data sources, the multi-modal data including structured data, semi-structured data and unstructured data;

[0009] Inputting the multi-modal data into a quantum state propagation engine, mapping to a quantum entangled state using a quantum neural radiation field model, and generating a quantum entangled state encoding feature tensor;

[0010] The quantum state propagation path is based on a nonlinear soliton wave equation dynamic programming quantum entangled state encoding characteristic tensor, a quantum topological index network and a knowledge graph are constructed;

[0011] The user search instruction is compiled into a quantum measurement operator, the quantum state propagation path navigation is performed in the quantum topological index network, and a cross-modal matching candidate result set is generated through path interference effect;

[0012] The candidate result set is subjected to quantum decoherence driven probability reordering, and high confidence results are screened out;

[0013] Based on the high confidence results and the knowledge graph, a generative AI large model is used to generate search results.

[0014] As a preferred scheme of the AI large model-based multi-modal data search method, wherein the quantum state propagation path based on the nonlinear soliton wave equation dynamic programming quantum entangled state encoding characteristic tensor is used to construct a quantum topological index network, and the specific steps are as follows:

[0015] The upper limit of the propagation time of the nonlinear soliton wave equation and the energy convergence threshold are set;

[0016] The time block is divided based on the upper limit of the propagation time, a set of parameterized rotation gates are allocated to each time block, CNOT gates are inserted between adjacent time blocks, and all time blocks, parameterized rotation gates and CNOT gates are assembled to form a quantum state propagation simulation circuit;

[0017] According to the quantum state propagation simulation circuit, the parameters of the parameterized rotation gate are optimized, the optimization termination is determined according to the energy convergence threshold, the ground state probability amplitude of all quantum bits is recorded, and a quantum state amplitude distribution data set is obtained;

[0018] Based on the quantum state amplitude distribution data set, a critical threshold is set, and key nodes are screened out, the quantum amplitude correlation coefficient between each pair of key nodes is calculated, the connection threshold is set based on the quantum amplitude correlation coefficient, and key node pairs are screened out;

[0019] Bidirectional connection edges are created for the screened key node pairs, and the quantum amplitude correlation coefficient is used as the edge weight;

[0020] The Shannon entropy of each key node is calculated to obtain the path propagation entropy value;

[0021] All key nodes, bidirectional connection edges and path propagation entropy values are combined to form a quantum topological index network;

[0022] The knowledge graph is constructed according to the key nodes, edge weights and quantum amplitude correlation coefficients.

[0023] As a preferred scheme of the multi-modal data search method based on the AI large model, the multi-modal data is obtained from a plurality of heterogeneous data sources, that is, a data access interface is connected with the plurality of heterogeneous data sources, a classification and allocation is performed on a to-be-collected task, metadata of each heterogeneous data source is extracted and cleaned and verified, and the multi-modal data is obtained.

[0024] As a preferred scheme of the multi-modal data search method based on the AI large model, the multi-modal data is input into a quantum state propagation engine, is mapped to a quantum entangled state by using a quantum neural radiation field model, and a quantum entangled state encoding feature tensor is generated.

[0025] The multi-modal data is segmented into discrete word units, a rotation gate operation is performed on the multi-modal data by using a quantum word embedding layer to generate a normalized word vector, and the normalized word vector is mapped to a ground state superposition quantum state by using quantum amplitude coding;

[0026] The multi-modal data is decoded into an RGB pixel matrix, a convolution operation is performed on the RGB pixel matrix by using a quantum filter bank, and a quantum frequency domain state is generated by using a quantum Fourier transform;

[0027] The ground state superposition quantum state and the quantum frequency domain state are loaded into the quantum neural radiation field model, and a cross-modal entangled quantum state is generated by using a Hadamard gate and a CNOT gate;

[0028] Based on the cross-modal entangled quantum state, a quantum entangled state encoding feature tensor is generated by using quantum principal component analysis.

[0029] As a preferred scheme of the multi-modal data search method based on the AI large model, the user search instruction is compiled into a quantum measurement operator, a quantum state propagation path navigation is performed in a quantum topological index network, a cross-modal matching candidate result set is generated by using a path interference effect, and specific steps are as follows:

[0030] The user search instruction is parsed into an entity word and a relationship word, the entity word is generated into a complex domain quantum state by using a quantum word embedding layer, and the relationship word is compiled into a Pauli rotation gate parameter sequence;

[0031] A quantum measurement operator unitary matrix is generated according to the Pauli rotation gate parameter sequence;

[0032] Based on the quantum measurement operator unitary matrix and the quantum topological index network, a path interference effect is triggered by using a control bit-target bit gate operation to obtain a path superposition quantum state;

[0033] Based on the complex domain quantum state, a path endpoint quantum state and a target entity quantum state are defined, an inner product of the path endpoint quantum state and the target entity quantum state is calculated, and a cross-modal matching degree is obtained.

[0034] According to the cross-modal matching degree, the candidate result set is screened.

[0035] As a preferred scheme of the multi-modal data search method based on the AI large model, the probability reordering driven by the quantum decoherence of the candidate result set is used to screen high-confidence results, and the specific steps are as follows:

[0036] According to the candidate result set and the quantum topological index network, a path superposition quantum state copy is retrieved;

[0037] Based on the path superposition quantum state copy, the damping coefficient of the amplitude damping channel is calculated by calculating the path propagation entropy value;

[0038] The quantum decoherence disturbance operation is performed on the path superposition quantum state copy, and the normalization processing is performed to generate a decoherence disturbance quantum state;

[0039] The termination position node of the path propagating along the quantum topological index network is defined as the end key node, the ground state probability amplitude of the path superposition quantum state copy and the decoherence disturbance quantum state at the end key node is measured, and the volatility is calculated according to the ground state probability amplitude;

[0040] The volatility weight and the entropy value weight of the candidate result set are calculated, and a reordering factor is generated;

[0041] The candidate result set is reordered based on the reordering factor, and high-confidence results are screened.

[0042] As a preferred scheme of the multi-modal data search method based on the AI large model, the generation of the search result based on the high-confidence result and the knowledge graph using the generative AI large model refers to generating an input sequence based on the high-confidence result and the knowledge graph, inputting the input sequence into the generative AI large model to generate an output text mark by mark, formatting the output text, adding a search result abstract, and obtaining the search result.

[0043] In a second aspect, the present application provides a multi-modal data search system based on an AI large model, comprising,

[0044] The acquisition module acquires multi-modal data from a plurality of heterogeneous data sources, and the multi-modal data includes structured data, semi-structured data and unstructured data;

[0045] The generation module inputs the multi-modal data into the quantum state propagation engine, maps it to the quantum entangled state using the quantum neural radiation field model, and generates the quantum entangled state encoding feature tensor;

[0046] The construction module dynamically programs the quantum state propagation path of the quantum entangled state encoding feature tensor based on the nonlinear solitary wave equation, constructs the quantum topological index network and the knowledge graph;

[0047] The matching module compiles the user search instruction into a quantum measurement operator, performs quantum state propagation path navigation in a quantum topological index network, and generates a cross-modal matching candidate result set through path interference effects;

[0048] The screening module performs quantum decoherence-driven probability reordering on the candidate result set and screens high-confidence results.

[0049] The enhancement module generates search results using a generative AI large model based on high-confidence results and a knowledge graph.

[0050] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the AI large model-based multi-modal data search method according to the first aspect of the present application.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the AI large model-based multi-modal data search method according to the first aspect of the present application.

[0052] The present application has the following advantages: the propagation path of the quantum entangled state encoding feature tensor is dynamically planned through the nonlinear solitary wave equation, which can capture the nonlinear correlation of multi-modal data in the quantum state evolution process, improve the precision of cross-modal matching, and reduce redundant calculations; through amplitude damping channel and volatility weight calculation, the high computational complexity of existing methods is avoided, the response delay of multi-modal data processing is reduced, and the problem of high response delay caused by computational efficiency bottleneck is solved. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0054] Fig. 1 The flowchart of the AI large model-based multi-modal data search method.

[0055] Fig. 2 The schematic diagram of the AI large model-based multi-modal data search system.

[0056] Fig. 3 The schematic diagram of the quantum neural radiation field model processing process.

[0057] Fig. 4A schematic diagram of a quantum state propagation path navigation process. DETAILED DESCRIPTION

[0058] In order to make the above objectives, characteristics and advantages of the present application more apparent, a detailed description of the specific embodiments of the present application will be given below with reference to the accompanying drawings.

[0059] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details given herewith. In other instances, well-known methods have not been described in detail in order to avoid unnecessarily obscuring the present application. Therefore, the specific details set forth hereinafter are merely exemplary.

[0060] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is independent or alternative to other embodiments.

[0061] Reference Signs List Figs. 1-4 For one embodiment of the present application, a multi-modal data search method based on an AI large model is provided, including the following steps:

[0062] S1: Obtain multi-modal data from multiple heterogeneous data sources, the multi-modal data including structured data, semi-structured data and unstructured data.

[0063] The specific steps are as follows,

[0064] Establish a connection between the data access interface and the multiple heterogeneous data sources: configure a protocol interface, input the authentication information of each heterogeneous data source, including IP address, port, username and password, and complete the handshake protocol verification to check the validity of the connection.

[0065] Classify and allocate the to-be-collected tasks: divide the to-be-collected tasks into task queues according to the heterogeneous data types, and allocate the collection priorities from large to small according to the ratio of the update frequency to the data size. The heterogeneous data types include structured, semi-structured and unstructured.

[0066] Extract the metadata of each heterogeneous data source: execute a table structure query command on the structured data to extract the field name, data type and primary foreign key constraint, parse the key-value pairs or label levels of the semi-structured data to obtain the data mode, and read the file header information of the unstructured data to extract the basic attributes.

[0067] For structured data, remove UTF-8 illegal characters in text fields, fill in missing values in numerical fields using column mean, and unify date field format to ISO8601 standard. For semi-structured data, repair format errors and remove invalid tags. For unstructured data, correct metadata errors and filter damaged files.

[0068] For structured data, verify foreign key association integrity, such as whether the user ID exists in the associated table, and check the legality of the numerical range. For field-level errors, use truncation processing, such as truncating age 150 to 120. Record-level errors are exported to the quarantine area.

[0069] For semi-structured data, verify pattern consistency, such as checking whether the JSON key list matches the structure description file and detecting data type conflicts. For field-level errors, perform data type conversion, and for record-level errors, mark the difference keys.

[0070] For unstructured data, verify file integrity, such as comparing the MD5 hash values of unstructured data before and after collection and checking attribute compliance. For field-level errors, perform standardized processing, and for file-level errors, record the path and notify the administrator.

[0071] Convert structured data to key-value pairs in text format by column name-value, recursively expand nested structures of semi-structured data to key-value text with path markers, and uniformly format dates / values to ISO strings. Convert audio data in unstructured data to mel spectrum, and sample frequency data to image sequences at frame rate.

[0072] Output the collected structured data as Parquet format files, semi-structured data as Avro format files, and unstructured data as original binary format with additional metadata tags, including source, type, and basic attributes, forming multi-modal data.

[0073] It should be noted that by establishing data access interfaces and classifying task allocation, combined with metadata cleaning and verification mechanisms, the collection needs of structured, semi-structured, and unstructured data can be fully covered, ensuring the consistency of data quality, and through standardized format output, improving subsequent processing efficiency and laying the foundation for multi-modal data fusion.

[0074] S2: Input multi-modal data into the quantum state propagation engine, map it to quantum entangled states using the quantum neural radiation field model, and generate quantum entangled state encoding feature tensors.

[0075] The specific steps are as follows,

[0076] Splitting the semi-structured data and the text data in the structured data into discrete word units, performing a rotation gate operation on the discrete word units by a quantum word embedding layer to obtain a real number type word vector; adopting L2 norm normalization processing on the real number type word vector, and mapping the normalized real number type word vector to a ground state superposition quantum state through a quantum amplitude coding operation;

[0077] Decoding the image data in the unstructured data into an RGB pixel matrix using the imdecode() function of the OpenCV library, performing a convolution operation on the RGB pixel matrix using a quantum filter bank to obtain spatial structure feature data, and mapping the spatial structure feature data to a frequency domain through a quantum Fourier transform to output a quantum frequency domain state;

[0078] Loading the ground state superposition quantum state to a first quantum register and the quantum frequency domain state to a second quantum register; applying a Hadamard gate operation to each quantum bit of the first quantum register and a Hadamard gate operation to each quantum bit of the second quantum register to create a bimodal uniform superposition state; taking the Nth quantum bit of the first quantum register as a control bit and the Nth quantum bit of the second quantum register as a target bit to perform a CNOT gate operation to realize a bit-level entangled association, N being an index of the bit order number of the first quantum register and the second quantum register, and taking the ground state superposition quantum state and the quantum frequency domain state with the smallest bit order number in the first quantum register and the second quantum register to generate a text-image associated cross-modal entangled quantum state;

[0079] Constructing a quantum variational autoencoder quantum circuit containing a parameterized rotation gate sequence: constructing the quantum variational autoencoder quantum circuit in a layer-by-layer structure, applying a Hadamard gate to initialize the first layer, deploying a parameterized rotation gate sequence (containing Ry / Rz gates) in the second layer, and inserting a fixed CNOT gate in the third layer to form an entanglement architecture, and combining them into a quantum variational autoencoder quantum circuit;

[0080] The quantum variational autoencoder quantum circuit applies the McLachlan variational principle to optimize the variational parameters, drives the cross-modal entangled quantum state to evolve in the Hilbert space along the direction of maximum variance, projects the quantum state to a quantum feature axis, locks the optimal variational parameters of the quantum variational autoencoder quantum circuit, freezes the quantum circuit operation, and outputs the projected quantum feature axis quantum state;

[0081] Performing a Pauli-Z basis vector projection measurement on each quantum bit of the quantum feature axis quantum state, and recording the probability amplitude of the quantum bit being in the ground state; sorting the ground state probability amplitudes of all quantum bits of the quantum feature axis quantum state according to the quantum bit order number to generate a quantum entangled state encoding feature tensor composed of real components.

[0082] It should be noted that: the quantum word embedding layer and the quantum filter set are used to process text and image data respectively, and the Hadamard gate and the CNOT gate are used to construct the cross-modal entangled quantum state, which can realize the quantum fusion of multi-modal features, improve the representation accuracy of text-image association, and provide high-dimensional feature support for subsequent quantum state propagation path planning.

[0083] S3: dynamically planning the quantum state propagation path of the quantum entangled state encoding feature tensor based on the nonlinear solitary wave equation, constructing a quantum topological index network and a knowledge graph.

[0084] The specific steps are as follows,

[0085] Load the quantum entangled state encoding feature tensor into the quantum register as the initial quantum state input of the quantum topological index network; input the nonlinear solitary wave equation parameters into the quantum simulator, set the upper limit of the propagation time to the dimension value of the quantum entangled state encoding feature tensor, and set the energy convergence threshold to 2 times the standard deviation of the energy fluctuation;

[0086] Construct a quantum state propagation simulation circuit: based on the upper limit of the propagation time, divide the total propagation time into equal length intervals, define each interval as an independent time block, and generate a time block sequence; assign a set of parameterized rotation gates to each time block, including Ry rotation gates and Rz rotation gates, and set the rotation angles of the Ry rotation gates and the Rz rotation gates as time-dependent parameter variables; insert CNOT gates between adjacent time blocks to form a chain structure, with the last quantum bit of the previous time block as the control bit and the first quantum bit of the next time block as the target bit, and execute the CNOT gate operation; assemble all time blocks, parameterized rotation gates and CNOT gates in time block order to form a quantum state propagation simulation circuit.

[0087] Execute the variational quantum feature solver algorithm in the quantum simulator: load the randomly initialized rotation angle parameters into the Ry / Rz rotation gates of the quantum state propagation simulation circuit; run the quantum state propagation simulation circuit on the quantum simulator, input the initial quantum state and output the propagation state; perform expectation value measurement of the Pauli-Z operator at the end of each time block to collect quantum state probability amplitude distribution data; calculate the variance value of the quantum state probability amplitude distribution as the energy concentration degree; calculate the partial derivative of the energy concentration degree with respect to the rotation angle parameter using the parameter displacement rule, and generate the quantum natural gradient according to the gradient direction of the quantum Fisher information matrix; adjust the rotation angle parameter according to the quantum natural gradient direction, and load the scaled rotation angle parameter into the Ry / Rz rotation gates of the quantum state propagation simulation circuit;

[0088] When the energy concentration degree promotion amplitude of three consecutive iterations is lower than the energy convergence threshold value, immediately stop the rotation angle parameter update, lock the current configuration state of the quantum state propagation simulation circuit; perform a complete basis vector measurement at the time point at the end of each time block, record the ground state probability amplitude of all quantum bits, and generate a discretized quantum state amplitude distribution data set; set a critical threshold value based on the quantum state amplitude distribution data set using the percentile method, for example, take the 75th percentile ground state probability amplitude as the critical threshold value, and filter out the quantum nodes in the quantum state amplitude distribution data set whose ground state probability amplitude exceeds the critical threshold value, and mark them as key nodes of the quantum topology index network;

[0089] Calculate the quantum amplitude correlation coefficient between each pair of key nodes, set a connection threshold value based on the quantum amplitude correlation coefficient using the percentile method, for example, take the 90th percentile of the quantum amplitude correlation coefficient as the connection threshold value, and select the key node pairs whose quantum amplitude correlation coefficient exceeds the connection threshold value; create a bidirectional connection edge for the selected key node pairs and record the quantum amplitude correlation coefficient as the edge weight; calculate the Shannon entropy of the probability amplitude distribution data set of each key node to obtain the path propagation entropy value; combine all key nodes and bidirectional connection edges and record the path propagation entropy value to form a quantum topology index network, convert the bidirectional connection edges whose edge weights exceed the median value of the edge weights into control bit-target bit CNOT gate sequences, convert the bidirectional connection edges whose edge weights do not exceed the median value of the edge weights into control bit-target bit CNOT gate sequences, map the key node connection relationship of the quantum topology index network to the layer sequence structure of the quantum state propagation simulation circuit, and generate a quantum controlled gate sequence containing a control bit and a target bit definition list.

[0090] Map the key nodes of the quantum topology index network to knowledge graph entities, set a weight threshold value interval based on the quantum amplitude correlation coefficient using the percentile method, for example, take the 80th percentile value of the quantum amplitude correlation coefficient as the lower limit of the weight threshold value interval and the 90th percentile value as the upper limit of the weight threshold value interval, for each bidirectional connection edge in the quantum topology index network, if the edge weight is within the weight threshold value interval, mark the relationship type as "related", and if the edge weight is not less than the upper limit of the weight threshold value interval, mark the relationship type as "strongly related";

[0091] Take the key node at the starting end of the connection edge in the quantum topology index network as the head entity and the key node at the terminating end of the connection edge as the tail entity; integrate the head entity, the relationship type, and the tail entity into a knowledge graph triple, and store all knowledge graph triples into a graph database to form a knowledge graph.

[0092] It should be noted that the quantum amplitude correlation coefficient between each pair of key nodes is calculated, and the expression is:

[0093]

[0094] wherein, ρ AB is the quantum amplitude correlation coefficient between the key node A and the key node B, ψ A is the ground state probability amplitude of the key node A, ψ B is the ground state probability amplitude of the key node B, E is the expectation value operator, and is usually the arithmetic mean of the quantum state amplitude distribution data set, μ A is the arithmetic mean of all ground state probability amplitudes of the key node A, μ B is the arithmetic mean of all ground state probability amplitudes of the key node B, σ A is the standard deviation of the ground state probability amplitude of the key node A, σ B is the standard deviation of the ground state probability amplitude of the key node B.

[0095] It should be noted that: by optimizing the quantum state propagation path through the nonlinear soliton wave equation, and combining the quantum amplitude correlation coefficient and the Shannon entropy to screen the key nodes, the nonlinear correlation of the multi-modal data in quantum evolution can be captured, and the quantum topological index network constructed not only improves the precision of cross-modal matching, but also reduces resource consumption by reducing redundant calculation.

[0096] S4: Compiling the user search instruction into a quantum measurement operator, executing the quantum state propagation path navigation in the quantum topological index network, and generating a candidate result set of cross-modal matching through the path interference effect.

[0097] The specific steps are as follows,

[0098] Remove the stop words and punctuation marks of the user search instruction text, retain the entity nouns and relationship verbs, match the part-of-speech tags based on the semantic mapping rule library, identify and extract the core entity words and relationship words; split the core entity words into discrete entity word units, process the discrete entity word units through the parameterized rotation gate of the quantum word embedding layer, obtain the normalized word vector, perform quantum amplitude encoding operation on the normalized word vector, and generate complex quantum state;

[0099] Parse the syntax dependency structure of the relationship words using the dependency syntax analysis tree to obtain syntax dependency triples, which contain relationship words, dependency relationship types, and subordinate words. The difference between the word position sequence numbers of the relationship words and the subordinate words is defined as the dependency distance. According to the semantic mapping rule library, the syntax dependency triples are mapped to the sequence of Pauli rotation gate types, and the reciprocal of the dependency distance is converted to the rotation angle value to generate the sequence of Pauli rotation gate parameters.

[0100] A blank quantum circuit is created, the number of quantum bits of the quantum circuit is consistent with the number of quantum bits of the entity quantum state, the Pauli rotation gate type and rotation angle value are read from the Pauli rotation gate parameter sequence in turn, and the Pauli rotation gate is deployed at the corresponding quantum bit position of the blank quantum circuit according to the reading order; the entity quantum state is input as an initial state at the first end of the quantum circuit, and the corresponding single quantum bit unitary matrix is generated for each rotation gate according to the Pauli rotation gate parameter sequence; the single quantum bit unitary matrix is extended to the full bit space of the quantum circuit, the unit matrix is filled in the non-acting bit position, the extended unitary matrix is subjected to left multiplication operation according to the rotation gate deployment order, the complex value matrix of the cumulative left multiplication result is calculated, and the overall unitary transformation operator of the quantum circuit is obtained; each column vector of the overall unitary transformation operator of the quantum circuit is normalized by using the vector normalization method, and the unitary matrix representation of the quantum measurement operator is generated.

[0101] The quantum controlled gate sequence in the quantum topology index network is read, including the control bit address, the target bit address and the gate type identifier; according to the control bit address and the target bit address of the quantum controlled gate sequence, the control bit-target bit controlled non gate instruction or the control bit-target bit controlled Z gate instruction is written in the corresponding bit position of the quantum processor, all quantum bits of the quantum processor are reset to the ground state, the Hadamard gate operation is applied to the quantum bits corresponding to the starting key node, and the initial superposition quantum state is generated; the control bit-target bit controlled non gate instruction or the control bit-target bit controlled Z gate instruction written is executed and acts on the initial superposition quantum state, and the path superposition quantum state propagating along the quantum topology index network is obtained, and the probability amplitude distribution of the path superposition quantum state is measured.

[0102] From the probability amplitude distribution of the path superposition quantum state, the key node complex quantum state corresponding to the end point of the path propagating along the quantum topology index network is located as the path end point quantum state, and the complex quantum state of the same key node address in the complex quantum state generated by the user search instruction is taken as the target entity quantum state; the inner product of the path end point quantum state and the target entity quantum state is calculated, and the cross-modal matching degree value is generated; based on the cross-modal matching degree value, the percentile method is used to set the lowest confidence threshold, for example, the 30th percentile value of the cross-modal matching degree value distribution is taken as the lowest confidence threshold, the path end point data item (the structured / semi-structured / unstructured data unit matching the propagation path end point in the quantum topology index network) whose cross-modal matching degree exceeds the lowest confidence threshold is screened out, and the candidate result set containing the path end point data item and the cross-modal matching degree is obtained in descending order of the cross-modal matching degree.

[0103] It should be noted that: through the path interference effect of quantum measurement operator and quantum topological index network, the entity words and relationship words in the user search instruction can be dynamically matched to generate a candidate result set matched across modalities, and the cross-modality matching degree screening mechanism based on inner product calculation improves the relevance of the search results, while supporting the analysis ability of complex semantic relationships.

[0104] S5: Probabilistic reordering of quantum decoherence-driven candidate result set, screening high-confidence results.

[0105] The specific steps are as follows,

[0106] For each path endpoint data item in the candidate result set, extract the key node index value in the metadata tag; query the key node metadata database of the quantum topological index network according to the key node index value to obtain the coordinate encoding of the corresponding key node in the Hilbert space; retrieve the path superposition quantum state copy associated with the key node index from the amplitude distribution storage area of the quantum topological index network and load it as a probability amplitude distribution data set.

[0107] Based on the path propagation entropy value of the path superposition quantum state copy, use a linear inverse mapping table to calculate the damping coefficient of the amplitude damping channel, and the mapping relationship is that the damping coefficient decreases by equal ratio as the path propagation entropy value increases; for each quantum bit of the path superposition quantum state copy: the probability amplitude of the excited state is attenuated by a damping coefficient, the ground state probability amplitude is increased correspondingly, and the modulus square sum is kept unchanged; the decay probability amplitude is introduced with a uniform distribution random phase offset simultaneously;

[0108] Perform correlated phase noise injection on adjacent quantum bits: take the quantum bit with the maximum path propagation entropy value as the control bit, apply an additional random phase rotation to the adjacent quantum bits of the control bit, and the rotation angle is proportional to the entropy value of the control bit; calculate the probability amplitude modulus square sum of all ground state components of the path superposition quantum state copy, adjust the real part and imaginary part of the probability amplitude by the normalization factor, so that the total probability is equal to 1, generate a decoherence disturbance quantum state, and mark the entropy-damping coefficient mapping record.

[0109] Define the termination position node of the path propagating along the quantum topological index network as the endpoint key node, locate the quantum bit corresponding to the endpoint key node in the path superposition quantum state copy, and perform Pauli-Z basis vector projection measurement, record the ground state probability amplitude of the path superposition quantum state copy as the original amplitude; locate the same endpoint key node quantum bit of the decoherence disturbance quantum state copy, and perform Pauli-Z basis vector projection measurement, record the ground state probability amplitude of the decoherence disturbance quantum state copy as the disturbance amplitude; take the ratio of the absolute difference value of the original amplitude and the disturbance amplitude to the original amplitude as the volatility rate;

[0110] Calculate the volatility rate weight, entropy value weight and reordering factor for each candidate result item;

[0111] Calculate the volatility weight, the expression is:

[0112] w=e ―10b ;

[0113] Where w is the volatility weight and b is the volatility;

[0114] The arithmetic square root of the path propagation entropy value is used as the entropy weight to calculate the reordering factor. The expression is:

[0115] R = M × w × s;

[0116] Among them, R is the re-ranking factor, M is the cross-modal matching value, and s is the entropy weight;

[0117] Sort the candidate result set in descending order according to the reordering factor value from large to small; generate a reordered candidate result list;

[0118] The entropy threshold is set using the percentile method based on the path propagation entropy value. For example, the 70th percentile value of the path propagation entropy value of the candidate result set is taken as the entropy threshold. The volatility threshold is set using the percentile method based on the volatility. For example, the 30th percentile value of the volatility of the candidate result set is taken as the volatility threshold. The path endpoint data items whose path propagation entropy value of the candidate result set is greater than the entropy threshold and whose volatility of the candidate result set is less than the volatility threshold are screened to obtain high confidence results.

[0119] It should also be noted that: through the calculation of amplitude damping channels and volatility weights, combined with the path propagation entropy value to screen high-confidence results, it is possible to suppress noise interference during quantum decoherence, avoid high computational complexity sorting algorithms, reduce response delays, and solve the problem of insufficient real-time performance caused by efficiency bottlenecks in existing methods.

[0120] S6: Based on high-confidence results and knowledge graphs, use generative AI big models to generate search results.

[0121] The specific steps are as follows:

[0122] For each path endpoint data item in the high-confidence result, multimodal semantic feature extraction is performed: the ApacheParquet-MR library is used to parse the Parquet file field names and values ​​and convert them into key-value text fragments; the ApacheAvro library is used to decode the Avro key-value pair collection and generate hierarchical list text in Markdown format; and the CLIP visual encoder is used to convert unstructured image data into text description statements.

[0123] Identify the knowledge graph entity corresponding to the high-confidence result: For structured data, extract the primary key field value of the structured data, match the primary key field value in the knowledge graph entity name, if the primary key fails to match, extract all text field values in the structured data, perform edit distance calculation in the knowledge graph entity name, and select the knowledge graph entity with the smallest edit distance as the matching result;

[0124] For semi-structured data, parse the unique identifier in the semi-structured data, match the unique identifier in the knowledge graph entity name, if the unique identifier is missing, extract the hierarchical path of the semi-structured data, compare the hierarchical path with the attribute path of the knowledge graph entity piece by piece, and select the entity with the highest path coincidence degree as the matching result;

[0125] For unstructured data, extract noun phrases from text description sentences, compare the extracted noun phrases with the knowledge graph entity name, perform case-insensitive inclusion matching, and if no knowledge graph entity is matched, select the knowledge graph entity with the smallest edit distance as the matching result.

[0126] Retrieve all direct attributes of the knowledge graph entity through the graph database query language, extract the attribute key-value pairs of the knowledge graph entity, retrieve the first-degree relationship neighbor entities of the knowledge graph entity, and convert the relationship type and neighbor entities into attribute key-value pairs; for each attribute key-value pair, generate a [attribute key: attribute value] segment, arrange all [attribute key: attribute value] segments in alphabetical order, connect the [attribute key: attribute value] segments with semicolons, and append to the knowledge graph entity name to obtain the knowledge graph text;

[0127] Arrange each path endpoint data item in the high-confidence result in descending order according to the reordering factor; extract the semantic feature text of each path endpoint data item, insert a separator string between adjacent data item semantic features, concatenate all semantic feature texts in the order to generate a continuous text block; extract the knowledge graph text corresponding to each high-confidence result, insert a semicolon as a separator between adjacent knowledge graph texts, and connect all knowledge graph texts to form a continuous description block; add a separator mark at the end of the continuous text block, append the corresponding continuous description block to the separator mark, and form an input sequence;

[0128] Use the built-in word segmenter of the generative AI large model to process the input sequence: map each character or word in the input sequence to an integer type token ID, and generate an integer sequence representing the token sequence.

[0129] The marked sequence is input into the generative AI large model, forward propagation calculation is performed on the last token in the marked sequence to obtain a probability distribution of the next token, K tokens with the highest probabilities in the probability distribution are reserved as candidate tokens (K is a beam width), a new sequence branch is extended for each candidate token, the cumulative log probability of all branch sequences is calculated, and only the K branch sequences with the highest cumulative probabilities are reserved as a candidate sequence set at each step; when the candidate sequences appear an end token or reach a maximum length, a candidate sequence with the highest cumulative probability is selected from the candidate sequence set as an optimal sequence, an integer token sequence of the optimal sequence is restored to a string using a tokenizer to obtain an output text; special control characters in the output text are deleted, and consecutive spaces are compressed into a single space character to obtain a natural language enhanced text.

[0130] Control characters in the natural language enhanced text are scanned, and all control characters are removed; consecutive space characters are detected and replaced with a single space character; the sequence of words in the natural language enhanced text is traversed, and if a word is completely matched with a semantic feature text of a high-confidence result, the matched word position is located; a square bracket annotation number is inserted after the matched word, and a search result abstract string in a fixed format is appended at the end of the natural language enhanced text to obtain a search result in a natural language format.

[0131] It should be noted that: by concatenating the semantic feature text and the relationship word retrieval template to input the generative AI large model, a structured natural language abstract that meets the user's needs can be generated, enhancing the explainability and practicality of the search results, and through the metadata identifier annotation and the repeated description merging, the integrity and simplicity of the information integration are ensured.

[0132] The embodiment also provides a multi-modal data search system based on an AI large model, comprising:

[0133] An acquisition module acquires multi-modal data from a plurality of heterogeneous data sources, and the multi-modal data comprises structured data, semi-structured data and unstructured data;

[0134] A generation module inputs the multi-modal data into a quantum state propagation engine, maps the multi-modal data to a quantum entangled state by using a quantum neural radiation field model, and generates a quantum entangled state encoding feature tensor;

[0135] A construction module dynamically plans a quantum state propagation path of the quantum entangled state encoding feature tensor based on a nonlinear soliton wave equation, and constructs a quantum topological index network;

[0136] A matching module compiles a user search instruction into a quantum measurement operator, performs quantum state propagation path navigation in the quantum topological index network, and generates a candidate result set matched across modalities through path interference effects;

[0137] The screening module performs quantum decoherence-driven probabilistic reordering on the candidate result set, and screens high-confidence results;

[0138] The enhancement module generates search results using a generative AI large model based on the high-confidence results and the knowledge graph.

[0139] The embodiment also provides a computer device suitable for the case of the multi-modal data search method based on an AI large model, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the multi-modal data search method based on an AI large model proposed in the above embodiment.

[0140] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. In addition, the input device can also be an external keyboard, touchpad or mouse, etc.

[0141] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to realize the multi-modal data search method based on an AI large model proposed in the above embodiment. The storage medium can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0142] In summary, the application can capture the nonlinear correlation of multi-modal data in the quantum state evolution process by dynamically programming the propagation path of the quantum entangled state encoding characteristic tensor of the nonlinear soliton wave equation, improves the precision of cross-modal matching, and reduces redundant calculation; through amplitude damping channel and volatility weight calculation, the high computational complexity of the existing method is avoided, the response delay of multi-modal data processing is reduced, and the problem of high response delay caused by the computational efficiency bottleneck is solved.

[0143] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A multimodal data search method based on an AI large model, characterized by: include, Acquiring multimodal data from multiple heterogeneous data sources, wherein the multimodal data includes structured data, semi-structured data, and unstructured data; Input multimodal data into the quantum state propagation engine, map it to the quantum entangled state using the quantum neural radiation field model, and generate the quantum entangled state encoding feature tensor; Based on the nonlinear soliton wave equation, the quantum state propagation path of the quantum entangled state encoding characteristic tensor is dynamically planned to construct a quantum topological index network and knowledge graph; Compile user search instructions into quantum measurement operators, perform quantum state propagation path navigation in the quantum topological indexing network, and generate a candidate result set for cross-modal matching through path interference effects; Perform quantum decoherence-driven probability re-ranking on the candidate result set to screen high-confidence results; Based on high-confidence results and knowledge graphs, a generative AI big model is used to generate search results.

2. The multimodal data search method based on the AI ​​large model according to claim 1, characterized in that: The quantum state propagation path of the quantum entangled state encoding characteristic tensor is dynamically planned based on the nonlinear soliton wave equation to construct a quantum topological index network and knowledge graph. The specific steps are as follows: Set the propagation time upper limit and energy convergence threshold of the nonlinear soliton wave equation; The time blocks are divided based on the upper limit of the propagation time. A set of parameterized rotation gates is assigned to each time block, and CNOT gates are inserted between adjacent time blocks. All time blocks, parameterized rotation gates and CNOT gates are assembled to form a quantum state propagation simulation circuit. According to the quantum state propagation simulation circuit, the parameters of the parameterized revolving gate are optimized, the optimization termination is determined according to the energy convergence threshold, and the ground state probability amplitude of all quantum bits is recorded to obtain the quantum state amplitude distribution data set; Based on the quantum state amplitude distribution dataset, a critical threshold is set, key nodes are screened out, and the quantum amplitude correlation coefficient between each pair of key nodes is calculated; The connection threshold is set based on the quantum amplitude correlation coefficient, and key node pairs are screened according to the connection threshold; Create bidirectional connection edges for the selected key node pairs, and use the quantum amplitude correlation coefficient as the edge weight; Calculate the Shannon entropy of each key node to obtain the path propagation entropy value; Combining all key nodes, bidirectional connection edges and path propagation entropy values ​​to form a quantum topological index network; Construct a knowledge graph based on key nodes, edge weights and quantum amplitude correlation coefficients.

3. The multimodal data search method based on the AI ​​large model according to claim 1, characterized in that: Acquiring multimodal data from multiple heterogeneous data sources refers to establishing a connection between a data access interface and multiple heterogeneous data sources, classifying and allocating collection tasks, extracting metadata from each heterogeneous data source, and performing cleaning and verification to obtain multimodal data.

4. The multimodal data search method based on the AI ​​large model according to claim 3, characterized in that: The multimodal data is input into the quantum state propagation engine, mapped to the quantum entangled state using the quantum neural radiation field model, and the quantum entangled state encoding feature tensor is generated. The specific steps are as follows: Multimodal data is segmented into discrete word units, and a revolving gate operation is performed through the quantum word embedding layer to generate normalized word vectors, which are mapped to the ground state superposition quantum state using quantum amplitude coding; Decode the multimodal data into an RGB pixel matrix, perform a convolution operation on the RGB pixel matrix using a quantum filter bank, and generate a quantum frequency domain state through quantum Fourier transform; The ground state superposition quantum state and quantum frequency domain state are loaded into the quantum neural radiation field model, and cross-modal entangled quantum state is generated through Hadamard gate and CNOT gate; Based on cross-modal entangled quantum states, quantum entangled state encoding feature tensors are generated through quantum principal component analysis.

5. The multimodal data search method based on the AI ​​large model according to claim 2, characterized in that: The user search instruction is compiled into a quantum measurement operator, quantum state propagation path navigation is performed in the quantum topological index network, and a candidate result set for cross-modal matching is generated through the path interference effect. The specific steps are as follows: Parse user search commands into entity words and relation words. Entity words are generated into complex domain quantum states through the quantum word embedding layer, and relation words are compiled into Pauli rotation gate parameter sequences. Generate quantum measurement operator unitary matrix according to Pauli rotation gate parameter sequence; Based on the unitary matrix of quantum measurement operators and quantum topological index network, the path interference effect is triggered by controlling the bit-target bit gate operation to obtain the path superposition quantum state; Based on the complex domain quantum state, the path endpoint quantum state and the target entity quantum state are defined, and the inner product of the path endpoint quantum state and the target entity quantum state is calculated to obtain the cross-modal matching degree; Filter candidate result sets based on cross-modal matching.

6. The multimodal data search method based on the AI ​​large model according to claim 5, characterized in that: The quantum decoherence-driven probability reordering of the candidate result set to screen high-confidence results is performed in the following specific steps: Obtain a copy of the path superposition quantum state based on the candidate result set and quantum topological index network retrieval; The damping coefficient of the amplitude damping channel is calculated based on the path propagation entropy value of the path superposition quantum state replica; Performing quantum dephasing interference dynamic operation on the copy of the path superposition quantum state and performing normalization processing to generate a dephasing interference dynamic quantum state; The terminal position node of the path propagating along the quantum topological index network is defined as the terminal key node. The ground state probability amplitude of the superposition quantum state replica and the dephasing interference dynamic quantum state at the terminal key node is measured, and the volatility is calculated based on the ground state probability amplitude. Calculate the volatility weight and entropy weight of the candidate result set and generate the reranking factor; The candidate result set is re-ranked based on the re-ranking factor to filter out high-confidence results.

7. The multimodal data search method based on the AI ​​large model according to claim 6, characterized in that: The method of generating search results based on high-confidence results and knowledge graphs using a generative AI big model refers to generating an input sequence based on high-confidence results and knowledge graphs, inputting the input sequence into the generative AI big model to generate output text one by one, formatting the output text, and adding a search result summary to obtain search results.

8. A multimodal data search system based on an AI large model, based on the multimodal data search method based on an AI large model according to any one of claims 1 to 7, characterized in that: include, An acquisition module, which acquires multimodal data from multiple heterogeneous data sources, wherein the multimodal data includes structured data, semi-structured data, and unstructured data; The generation module inputs multimodal data into the quantum state propagation engine, maps it to the quantum entangled state using the quantum neural radiation field model, and generates the quantum entangled state encoding feature tensor; Building modules, dynamically planning the quantum state propagation path of the quantum entangled state encoding characteristic tensor based on the nonlinear soliton wave equation, and constructing quantum topological index networks and knowledge graphs; The matching module compiles user search instructions into quantum measurement operators, performs quantum state propagation path navigation in the quantum topological indexing network, and generates a candidate result set for cross-modal matching through path interference effects; The screening module performs quantum decoherence-driven probability reordering on the candidate result set to screen high-confidence results; The enhanced module uses a generative AI large model to generate search results based on high-confidence results and knowledge graphs.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multimodal data search method based on the AI ​​large model are implemented in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multimodal data search method based on the AI ​​large model are implemented.

Images