Data resource accurate processing and sharing method based on multi-modal semantic correlation analysis

Abstract

The application provides a data resource accurate processing and sharing method based on multi-modal semantic correlation analysis, relates to the technical field of semantic processing, and comprises the following steps: obtaining fused semantic features by extracting features of multi-modal data and mapping to a unified semantic space, and then constructing a semantic enhancement graph to obtain extended semantic representation through multi-hop reasoning. A hybrid query vector is constructed by combining the two to perform multi-granularity matching, and the results are sorted by confidence weighting through evaluation of cross-modal semantic consistency and other indicators, and finally the graph structure is adaptively adjusted according to user feedback. The application realizes the mining and utilization of deeper semantic correlation of multi-modal data resources, and improves the accuracy, integrity and explainability of the search results.

Description

Technical Field

[0001] This invention relates to semantic processing technology, and more particularly to a method for precise processing and sharing of data resources based on multimodal semantic association analysis. Background Technology

[0002] In existing technologies, data resource retrieval and sharing typically rely on single-modality or simple fusion semantic matching methods. The conventional approach involves extracting features from different modalities such as text, images, and audio separately, calculating similarity using vector space models or shallow fusion networks, and then matching these in a database to return relevant results. While these methods can handle multimodal data to some extent, their core is often limited to a direct, single-hop semantic comparison between the query request and the data resource, lacking effective mining and utilization of deep, complex semantic relationships between data resources. Furthermore, existing methods usually treat the retrieval process as a one-time, static matching, failing to fully consider the dynamic semantic preferences implied by user interaction feedback, making it difficult for the system to continuously optimize its retrieval performance.

[0003] Existing conventional practices have significant drawbacks: due to a lack of in-depth reasoning about the implicit semantic relationships between data resources, the retrieval process is easily constrained by the semantic gap problem. When user query intent is complex or the semantic expression of data resources is incomplete, relying solely on direct matching is insufficient to recall high-value resources that are semantically related but whose surface features do not directly match, resulting in limited recall and accuracy of retrieval results. Existing systems typically employ fixed matching models or weights, failing to adaptively learn and adjust based on actual user behavior and feedback. This makes the retrieval system lack personalization capabilities, making it difficult to track and adapt to the dynamic evolution of user semantic preferences. In the long run, this leads to a gradual increase in the discrepancy between retrieval results and users' actual needs, affecting the efficiency of accurate utilization of shared data resources. Summary of the Invention

[0004] This invention provides a method for precise processing and sharing of data resources based on multimodal semantic association analysis, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a method for precise processing and sharing of data resources based on multimodal semantic association analysis, comprising:

[0006] The query request to be retrieved and its corresponding multimodal input data are obtained. Features are extracted from each modality in the multimodal input data to obtain the feature representation of each modality. Based on the cross-modal semantic mapping relationship, the feature representation of each modality is projected onto a unified semantic space to obtain fused semantic features.

[0007] A semantically enhanced graph structure is constructed, with the fused semantic features as node features and the semantic relationships between data resources as edge connections. Multi-hop semantic reasoning is performed through a graph propagation mechanism to obtain an extended semantic representation.

[0008] A hybrid semantic query vector is constructed based on the fused semantic features and the extended semantic representation, and multi-granular semantic matching is performed in the data resource library using the hybrid semantic query vector to obtain a candidate retrieval result set;

[0009] A semantic confidence evaluation mechanism is established. By calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector, the candidate retrieval result set is weighted and sorted by confidence to generate accurate retrieval results.

[0010] Based on the user's interactive feedback behavior on the precise search results, a semantic preference evolution pattern is extracted, and the edge weights of the semantic enhancement graph structure are adaptively adjusted using the semantic preference evolution pattern.

[0011] A semantically enhanced graph structure is constructed, using the fused semantic features as node features and the semantic relationships between data resources as edge connections. Multi-hop semantic reasoning is then performed through a graph propagation mechanism to obtain an extended semantic representation, including:

[0012] Each data resource and fused semantic feature in the data resource repository is used as a graph node. By calculating the cross-modal semantic coupling degree and semantic structure similarity between nodes, directed edge connections reflecting the semantic dependency strength are established, and edge weights based on semantic transmission credibility are assigned to each edge to construct a semantically enhanced graph structure.

[0013] In the semantic enhancement graph structure, starting from the query node, semantic diffusion is performed layer by layer along the edge weights. In each hop of propagation, the semantic features of the reached node are weighted and fused according to the path dependency. The propagation depth is dynamically truncated and controlled according to the accumulated path weights to obtain a set of multi-hop reachable nodes and their corresponding path aggregation semantic representation.

[0014] The path aggregation semantic representation and the fused semantic features are semantically aligned, and an extended semantic representation is generated through a graph propagation mechanism.

[0015] In the semantically enhanced graph structure, starting from the query node, semantic diffusion is performed layer by layer along the edge weights. In each hop of propagation, the semantic features of the arriving nodes are weighted and fused according to path dependencies. The propagation depth is dynamically truncated based on the accumulated path weights, resulting in a set of multi-hop reachable nodes and their corresponding path-aggregated semantic representations, including:

[0016] The node corresponding to the fused semantic features is set as the query node, and the propagation state and cumulative path weight of the query node are initialized.

[0017] Starting from the query node, a first-hop semantic diffusion is performed along the directed edge connection. For the neighboring nodes reached, their original semantic features are combined with the semantic features of the query node according to the weight of the connecting edge to generate the first-hop path fusion feature, and the cumulative path weight of each neighboring node is recorded.

[0018] For each neighboring node corresponding to the first-hop path fusion feature, it is determined whether its cumulative path weight meets the semantic transmission validity threshold. For nodes that meet the condition, the next-hop semantic diffusion is continued, and for nodes that do not meet the condition, propagation truncation is performed. In the next-hop diffusion, the path fusion feature of the previous hop is used as the input feature of the current hop for a new round of weighted combination, and the cumulative path weight is iteratively updated until all propagation paths are truncated or the preset propagation layer is reached, thus obtaining a multi-hop reachable node set.

[0019] For each node in the multi-hop reachable node set, its path fusion features are globally weighted and aggregated according to its final accumulated path weight to generate a path aggregation semantic representation.

[0020] A hybrid semantic query vector is constructed based on the fused semantic features and the extended semantic representation. This hybrid semantic query vector is then used to perform multi-granularity semantic matching in the data resource database to obtain a candidate retrieval result set, including:

[0021] Calculate the semantic complementarity between the fused semantic features and the extended semantic representation, determine the adaptive fusion weights based on the semantic complementarity, and perform weighted fusion of the fused semantic features and the extended semantic representation according to the adaptive fusion weights to construct a hybrid semantic query vector;

[0022] The hybrid semantic query vector is projected onto multiple semantic representation spaces of different granularities, generating local semantic matching vectors in the fine-grained semantic representation space and generating global semantic matching vectors in the coarse-grained semantic representation space.

[0023] The local semantic similarity is calculated using the local semantic matching vector and the fine-grained feature representation of each data resource in the data resource library. The global semantic similarity is calculated using the global semantic matching vector and the coarse-grained feature representation of each data resource. The local semantic similarity and the global semantic similarity are then adaptively weighted and combined to obtain a comprehensive matching score.

[0024] The data resources in the data resource library are sorted and filtered according to the comprehensive matching score to obtain a set of candidate search results.

[0025] A semantic confidence assessment mechanism is established. This mechanism calculates the comprehensive confidence level of cross-modal semantic consistency, semantic completeness, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector. The candidate retrieval result set is then weighted and ranked according to this confidence level to generate accurate retrieval results, including:

[0026] A semantic confidence assessment mechanism is constructed, which includes a cross-modal consistency assessment module, a semantic integrity assessment module, and a semantic interpretability assessment module. The cross-modal consistency assessment module quantifies the degree of collaboration and conflict between different modal semantic expressions by constructing an inter-modal semantic consistency constraint matrix, thereby obtaining a cross-modal consistency metric.

[0027] The semantic integrity assessment module is based on the semantic element dependency graph obtained by hybrid semantic query vector decomposition. It maps the semantic features of each data resource to the node space and calculates the coverage. It combines the implicit semantic path in the extended semantic representation to compensate for semantically missing nodes and obtains the semantic integrity metric value.

[0028] Based on the semantic interpretability assessment module, the semantic matching attribution chain is extracted, and the interpretability strength is quantified according to the semantic transmission transparency to obtain the semantic interpretability metric value.

[0029] According to the semantic confidence assessment mechanism, the cross-modal consistency metric, the semantic integrity metric, and the semantic interpretability metric are input into the confidence fusion function to calculate the comprehensive confidence of each data resource, and then sort them according to the comprehensive confidence to generate accurate search results.

[0030] According to the semantic confidence assessment mechanism, the cross-modal consistency metric, the semantic integrity metric, and the semantic interpretability metric are input into the confidence fusion function to calculate the comprehensive confidence of each data resource, including:

[0031] Analyze the semantic dependencies among cross-modal consistency measures, semantic integrity measures, and semantic interpretability measures; identify the confidence propagation paths and their propagation weights among different measures; and establish the association structure among the measure dimensions.

[0032] Based on the association structure, the association strength is propagated, and each metric is transmitted to the relevant metric along the confidence transmission path. The association enhancement adjustment is performed on each metric according to the transmission weight to obtain the adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric.

[0033] Analyze the modal distribution characteristics and semantic complexity of the hybrid semantic query vector, and determine the adaptive fusion weights of the adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric based on the modal distribution characteristics and the semantic complexity.

[0034] The adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric are weighted and combined according to the adaptive fusion weight input confidence fusion function to obtain the comprehensive confidence of each data resource.

[0035] Based on user interaction feedback with the precise search results, a semantic preference evolution pattern is extracted, and the edge weights of the semantic enhancement graph structure are adaptively adjusted using the semantic preference evolution pattern, including:

[0036] Collect user interaction feedback behavior sequences for each data resource in the precise search results, extract semantic feature patterns that users care about from the interaction feedback behavior sequences, and identify the evolution direction and rate of user semantic preferences by analyzing the semantic feature change trends of user interaction behavior in different time windows, thereby obtaining the semantic preference evolution pattern.

[0037] The semantic preference evolution pattern is mapped to a semantic enhancement graph structure, semantic association paths related to the semantic preference evolution pattern are identified, and the edge weights on the semantic association paths are enhanced and adjusted according to the evolution direction in the semantic preference evolution pattern. The adjustment range of the edge weights is determined according to the evolution rate, thus completing the adaptive adjustment of the edge weights of the semantic enhancement graph structure.

[0038] A second aspect of the present invention provides an electronic device, comprising:

[0039] processor;

[0040] Memory used to store processor-executable instructions;

[0041] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0042] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0043] The beneficial effects of this application are as follows:

[0044] This method enables deep semantic fusion and unified representation of multimodal input data. By extracting feature representations of each modality and projecting them onto a unified semantic space using cross-modal semantic mapping relationships, it effectively breaks down semantic barriers between different modalities, solves the semantic gap problem of heterogeneous data, and lays a unified semantic foundation for subsequent accurate matching.

[0045] The constructed semantically enhanced graph structure organically combines semantic features with the semantic relationships between data resources. Through a graph propagation mechanism, it performs multi-hop semantic reasoning, which can fully explore the deep semantic relationships hidden between data resources. This process not only enhances the semantic representation of a single data resource but also achieves the effective propagation and expansion of semantic information, significantly improving the richness of semantic representation and context awareness.

[0046] The hybrid semantic query vector, constructed based on fusion features and extended representations, combines the semantics of the data itself with the semantics of the associated context, making the expression of query intent more comprehensive and accurate. Using this vector for multi-granularity semantic matching allows for simultaneous comparison with the data repository at both the overall semantic level and the local detail level, improving the coverage and accuracy of the retrieval and ensuring the comprehensiveness and relevance of candidate results. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the method for precise processing and sharing of data resources based on multimodal semantic association analysis, as described in an embodiment of the present invention.

[0048] Figure 2 This is a flowchart illustrating the adaptive optimization process of multimodal semantic association data evolution in an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0051] Figure 1 This is a flowchart illustrating the method for precise processing and sharing of data resources based on multimodal semantic association analysis according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0052] The query request to be retrieved and its corresponding multimodal input data are obtained. Features are extracted from each modality in the multimodal input data to obtain the feature representation of each modality. Based on the cross-modal semantic mapping relationship, the feature representation of each modality is projected onto a unified semantic space to obtain fused semantic features.

[0053] A semantically enhanced graph structure is constructed, with the fused semantic features as node features and the semantic relationships between data resources as edge connections. Multi-hop semantic reasoning is performed through a graph propagation mechanism to obtain an extended semantic representation.

[0054] A hybrid semantic query vector is constructed based on the fused semantic features and the extended semantic representation, and multi-granular semantic matching is performed in the data resource library using the hybrid semantic query vector to obtain a candidate retrieval result set;

[0055] A semantic confidence evaluation mechanism is established. By calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector, the candidate retrieval result set is weighted and sorted by confidence to generate accurate retrieval results.

[0056] Based on the user's interactive feedback behavior on the precise search results, a semantic preference evolution pattern is extracted, and the edge weights of the semantic enhancement graph structure are adaptively adjusted using the semantic preference evolution pattern.

[0057] In one optional implementation, a semantically enhanced graph structure is constructed, using the fused semantic features as node features and the semantic relationships between data resources as edge connections. Multi-hop semantic reasoning is then performed through a graph propagation mechanism to obtain an extended semantic representation, including:

[0058] Each data resource and fused semantic feature in the data resource repository is used as a graph node. By calculating the cross-modal semantic coupling degree and semantic structure similarity between nodes, directed edge connections reflecting the semantic dependency strength are established, and edge weights based on semantic transmission credibility are assigned to each edge to construct a semantically enhanced graph structure.

[0059] In the semantic enhancement graph structure, starting from the query node, semantic diffusion is performed layer by layer along the edge weights. In each hop of propagation, the semantic features of the reached node are weighted and fused according to the path dependency. The propagation depth is dynamically truncated and controlled according to the accumulated path weights to obtain a set of multi-hop reachable nodes and their corresponding path aggregation semantic representation.

[0060] The path aggregation semantic representation and the fused semantic features are semantically aligned, and an extended semantic representation is generated through a graph propagation mechanism.

[0061] In the data resource repository, each data resource and its corresponding fused semantic features are mapped as independent nodes in a graph structure. Fusion semantic features are feature vectors obtained by projecting different modalities such as text, images, and audio onto a unified semantic space through cross-modal semantic mapping relationships. These vectors are typically set to 512 or 768 dimensions. To establish semantic associations between nodes, the cross-modal semantic coupling degree between any two nodes is calculated. Specifically, cosine similarity is used to measure the directional consistency of the two fused semantic feature vectors; a semantic association is considered to exist when the similarity exceeds a preset threshold of 0.6. Simultaneously, a semantic structure similarity index is introduced. This index quantifies the relative positional similarity of nodes within the knowledge structure by analyzing their attribute labels, classification levels, and contextual relationships in the original data resources. The cross-modal semantic coupling degree and semantic structure similarity are weighted and combined, with weight coefficients set to 0.7 and 0.3, respectively, to obtain the comprehensive semantic dependency strength. When the comprehensive dependency strength is greater than 0.5, a directed edge is established between the corresponding nodes, with the edge pointing towards the node with the higher semantic dependency strength.

[0062] Each edge is assigned a weight based on semantic propagation credibility, which comprehensively considers the stability of the source node's features, the reliability of the edge connection, and the semantic consistency of the target node. The stability of the source node's features is obtained by calculating the product of the frequency of the node being hit in historical queries and its average confidence level. The reliability of the edge connection is determined by statistically analyzing the proportion of times the edge is successfully propagated in the semantic inference path. The semantic consistency of the target node is evaluated by assessing the variance of the multi-path semantic information received by the target node; a smaller variance indicates higher consistency. The three factors are normalized and then multiplied to obtain an edge weight value ranging from 0 to 1.

[0063] When performing multi-hop semantic reasoning within the constructed semantically enhanced graph structure, the fused semantic features corresponding to the query request are used as the initial query node. Starting from the query node, semantic diffusion proceeds along edges with weights greater than 0.4. In the first hop propagation, all neighboring nodes directly connected to the query node are identified, and the fused semantic features of these neighboring nodes are summed element-wise with the weights of the connecting edges to obtain the aggregated semantic representation of the first hop. In the second hop propagation, the node reached in the first hop is used as the starting point, and propagation continues outwards. However, path dependency needs to be considered here, i.e., the cumulative path weight from the query node through the first hop node to the second hop node, which is calculated by multiplying the weights of each edge along the path. For the node reached in the second hop, its semantic features are weighted and fused according to the cumulative path weights.

[0064] To prevent semantic reasoning from spreading infinitely in the graph and introducing noise, a dynamic truncation control mechanism is implemented. After each hop, the cumulative weight of all paths in the current hop is calculated, and the average cumulative weight is determined. When the cumulative weight of a path falls below 30% of the average, propagation for that path is terminated. Simultaneously, a maximum propagation hop count of 3 hops is set; propagation is forcibly stopped when the propagation depth reaches 3 hops. Through dynamic truncation control, a multi-hop reachable node set is obtained, containing all nodes reachable from the query node within the effective propagation depth.

[0065] For each node in the multi-hop reachable node set, all valid paths to that node are collected. The cumulative weight of each path is used as a fusion coefficient, and the semantic features of each node along the path are weighted and averaged to obtain the path-aggregated semantic representation of that node. The path-aggregated semantic representation is semantically aligned with the initial fused semantic features, and an attention mechanism is used to calculate the relevance weight between the two. The final extended semantic representation is generated through weighted fusion. The extended semantic representation retains the core semantics of the original query while incorporating the association semantic information obtained from graph structure propagation.

[0066] In one optional implementation, in the semantic enhancement graph structure, starting from the query node, semantic diffusion is performed layer by layer along the edge weights. In each hop of propagation, the semantic features of the arriving nodes are weighted and fused according to path dependencies, and the propagation depth is dynamically truncated based on the accumulated path weights, resulting in a multi-hop reachable node set and its corresponding path aggregation semantic representation, including:

[0067] The node corresponding to the fused semantic features is set as the query node, and the propagation state and cumulative path weight of the query node are initialized.

[0068] Starting from the query node, a first-hop semantic diffusion is performed along the directed edge connection. For the neighboring nodes reached, their original semantic features are combined with the semantic features of the query node according to the weight of the connecting edge to generate the first-hop path fusion feature, and the cumulative path weight of each neighboring node is recorded.

[0069] For each neighboring node corresponding to the first-hop path fusion feature, it is determined whether its cumulative path weight meets the semantic transmission validity threshold. For nodes that meet the condition, the next-hop semantic diffusion is continued, and for nodes that do not meet the condition, propagation truncation is performed. In the next-hop diffusion, the path fusion feature of the previous hop is used as the input feature of the current hop for a new round of weighted combination, and the cumulative path weight is iteratively updated until all propagation paths are truncated or the preset propagation layer is reached, thus obtaining a multi-hop reachable node set.

[0070] For each node in the multi-hop reachable node set, its path fusion features are globally weighted and aggregated according to its final accumulated path weight to generate a path aggregation semantic representation.

[0071] When performing multi-hop semantic reasoning in a semantically enhanced graph structure, the node corresponding to the fused semantic features is set as the query node v. q Initialize the propagation state of this node. It integrates semantic feature vectors and sets cumulative path weights. This indicates that the starting node has full semantic propagation weights. In the graph structure, the weight e of the directed edges between nodes... ij This represents the semantic association strength between node i and node j. This weight value is determined by semantic similarity measurement or knowledge graph relationship strength, and its value ranges from 0 to 1.

[0072] From query node v q The first hop of semantic diffusion begins by traversing all directly connected neighboring nodes along the directed edges. For neighboring nodes Obtain the edge weight e qj , will node v j The original semantic features f j The first-hop path fusion feature is calculated by weighting and combining the semantic features of the query node. This weighted combination mechanism ensures that semantic propagation preserves both the original node features and incorporates path source information. Neighboring nodes v are recorded synchronously. j Cumulative path weight This weight reflects the semantic transmission strength of the node reached through the current path.

[0073] For each of the neighboring nodes obtained in the first hop, determine its cumulative path weight. Is it greater than the preset semantic transmission validity threshold? This threshold is typically set between 0.1 and 0.3. When When the node is considered to still have semantic propagation value, it is added to the queue of nodes to be propagated, preparing for the next hop of semantic propagation; when If the semantic transmission through the current path has decayed to the point that it is insufficient to support effective reasoning, the propagation of the node is truncated and no longer spreads from the node, but the fusion features of the node and its current path are retained for subsequent result aggregation.

[0074] The semantic diffusion process proceeds to the second hop and subsequent hops, involving each node v in the queue of nodes to be diffused. k fused the path features of its first hop As input features for a new round of diffusion. Traverse node v kThe set of neighboring nodes For neighboring node v m Calculate the second-hop path fusion features Simultaneously update the cumulative path weights The cumulative weight judgment and propagation truncation logic is repeatedly executed. Nodes that meet the conditions continue to propagate, while nodes that do not meet the conditions are truncated. This iterative process continues until all propagation paths are truncated due to cumulative weights falling below a threshold, or the preset maximum propagation layer L is reached. max It is usually set to 3 to 5 layers to avoid semantic overgeneralization.

[0075] After completing multi-hop semantic diffusion, the nodes visited across all hop counts are aggregated to form a set of multi-hop reachable nodes. This set contains nodes reached at different hop counts, and each node retains its final hop count path fusion feature and cumulative path weight. For each node v in the set... i Based on its cumulative path weight Its path fusion features Perform global weighted aggregation to generate a path aggregation semantic representation. ,in, Let l be the weight of the i-th element in the j-th iteration (or the j-th time step). j Let l be the length, scale factor, or other adjustment parameter of the i-th element, and l be the total number of hops or the maximum number of hops (the total number of hops for multi-hop semantic diffusion). Normalization ensures that the semantic contributions of different paths are reasonably balanced, and finally, an extended semantic representation that integrates multi-hop reasoning results is obtained.

[0076] In one optional implementation, a hybrid semantic query vector is constructed based on the fused semantic features and the extended semantic representation, and the hybrid semantic query vector is used to perform multi-granularity semantic matching in a data resource database to obtain a candidate retrieval result set including:

[0077] Calculate the semantic complementarity between the fused semantic features and the extended semantic representation, determine the adaptive fusion weights based on the semantic complementarity, and perform weighted fusion of the fused semantic features and the extended semantic representation according to the adaptive fusion weights to construct a hybrid semantic query vector;

[0078] The hybrid semantic query vector is projected onto multiple semantic representation spaces of different granularities, generating local semantic matching vectors in the fine-grained semantic representation space and generating global semantic matching vectors in the coarse-grained semantic representation space.

[0079] The local semantic similarity is calculated using the local semantic matching vector and the fine-grained feature representation of each data resource in the data resource library. The global semantic similarity is calculated using the global semantic matching vector and the coarse-grained feature representation of each data resource. The local semantic similarity and the global semantic similarity are then adaptively weighted and combined to obtain a comprehensive matching score.

[0080] The data resources in the data resource library are sorted and filtered according to the comprehensive matching score to obtain a set of candidate search results.

[0081] After completing the extended semantic representation, the construction of the hybrid semantic query vector first requires quantifying the degree of information complementarity between the fused semantic features and the extended semantic representation. Specifically, the cosine distance between the two feature vectors in the semantic space is calculated as the basic measure of complementarity, while the feature variance ratio is introduced as an information gain index. The semantic complementarity is expressed as a normalized combination of the difference in feature distribution and the difference in information entropy between the two vectors. When the complementarity is high, it indicates that the graph propagation mechanism has introduced significant new semantic information, and in this case, the extended semantic representation needs to be given a higher fusion weight.

[0082] The adaptive fusion weights map complementarity to a weight range using the Sigmoid function, setting a complementarity threshold of 0.6. When complementarity is below this value, the fusion weights favor the fusion semantic features; when complementarity is above this value, the fusion weights tilt towards the extended semantic representation. The final hybrid semantic query vector is obtained through a weighted summation, maintaining the same vector dimension as the original semantic feature space, typically 512 or 768 dimensions.

[0083] For multi-granularity semantic matching, the hybrid semantic query vector is projected onto three semantic representation spaces of different granularities. The fine-grained space focuses on local semantic details, employing a high-dimensional representation of 1024. A fully connected layer maps the hybrid vector to this space, imposing L2 normalization constraints. The coarse-grained space captures global semantic topics, using a low-dimensional representation of 256, achieved through dimensionality reduction projection. The medium-granularity space serves as a transition layer, with a dimension of 512, used to balance local and global semantic information.

[0084] In the fine-grained matching process, the fine-grained feature representations of each resource in the data resource repository are pre-stored after undergoing the same projection transformation. Local semantic similarity calculation employs modified cosine similarity, introducing a position-weighted mechanism to align the query vector and resource vector across all dimensions in the high-dimensional space. For dimensional components with dimensional deviations exceeding a set threshold, attenuation weights are applied to prevent noisy dimensions from affecting matching accuracy.

[0085] In coarse-grained matching, the global semantic matching vector and the coarse-grained feature representation of the resource are multiplied by a dot product to obtain the global semantic similarity. This similarity reflects the degree of agreement between the query intent and the overall topic of the resource. To improve robustness, a temperature coefficient is applied to the dot product result, with the temperature parameter set to 0.07 to make the similarity distribution smoother.

[0086] The calculation of the comprehensive matching score requires adaptively determining the combined weights of local and global similarity, dynamically adjusting the weight allocation based on the semantic complexity of the query request: when the query contains multiple semantic concepts, the local matching weight is increased to 0.7; when the query has a single semantic meaning, the global matching weight is increased to 0.6. The combined weights are calculated through an attention mechanism, using the semantic entropy of the query vector as the input feature for the attention score. The final comprehensive matching score is expressed as a weighted sum of local and global similarity.

[0087] All resources in the database are sorted in descending order based on their comprehensive matching scores. A score threshold of 0.5 is set to filter out resources below this threshold. The size of the candidate set is also limited, typically consisting of the top 100 high-scoring resources. For resources with similar scores, a time decay factor is introduced as a secondary sorting criterion, prioritizing the return of more recently updated data resources to ensure the timeliness and accuracy of the search results.

[0088] In one optional implementation, a semantic confidence evaluation mechanism is established. This mechanism calculates the comprehensive confidence levels of cross-modal semantic consistency, semantic integrity, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector. The candidate retrieval result set is then weighted and ranked according to confidence levels to generate accurate retrieval results, including:

[0089] A semantic confidence assessment mechanism is constructed, which includes a cross-modal consistency assessment module, a semantic integrity assessment module, and a semantic interpretability assessment module. The cross-modal consistency assessment module quantifies the degree of collaboration and conflict between different modal semantic expressions by constructing an inter-modal semantic consistency constraint matrix, thereby obtaining a cross-modal consistency metric.

[0090] The semantic integrity assessment module is based on the semantic element dependency graph obtained by hybrid semantic query vector decomposition. It maps the semantic features of each data resource to the node space and calculates the coverage. It combines the implicit semantic path in the extended semantic representation to compensate for semantically missing nodes and obtains the semantic integrity metric value.

[0091] Based on the semantic interpretability assessment module, the semantic matching attribution chain is extracted, and the interpretability strength is quantified according to the semantic transmission transparency to obtain the semantic interpretability metric value.

[0092] According to the semantic confidence assessment mechanism, the cross-modal consistency metric, the semantic integrity metric, and the semantic interpretability metric are input into the confidence fusion function to calculate the comprehensive confidence of each data resource, and then sort them according to the comprehensive confidence to generate accurate search results.

[0093] After obtaining the candidate search result set, a semantic confidence evaluation mechanism is constructed to perform a refined evaluation of the search results. This mechanism consists of three core modules working together: a cross-modal consistency evaluation module, a semantic integrity evaluation module, and a semantic interpretability evaluation module.

[0094] The cross-modal consistency assessment module constructs a semantic consistency constraint matrix for the semantic expressions of different modalities such as text, images, and videos in the candidate results. The rows and columns of the constraint matrix correspond to different modalities, and the matrix elements are obtained by calculating the cosine similarity of the eigenvectors between the modalities. When an image in a data resource depicts an "industrial robot assembly scene," while the text describes it as "robotic arm grasping operation," the two are semantically highly consistent, and the corresponding matrix element value is close to 1. If the image shows static equipment while the text describes a dynamic process, there is a semantic conflict, and the matrix element value decreases. By performing eigenvalue decomposition on the constraint matrix, the magnitude of the principal eigenvalues ​​reflects the degree of cooperation between modalities, while the dispersion of the secondary eigenvalues ​​represents the degree of conflict. The weighted difference between the degree of cooperation and the degree of conflict is normalized to obtain a cross-modal consistency metric value, ranging from 0 to 1.

[0095] The semantic integrity assessment module first decomposes the hybrid semantic query vector into semantic elements. The query vector is broken down into several semantic element sub-vectors using an attention mechanism, each sub-vector corresponding to a key dimension of the query intent. These semantic elements are then used to construct a dependency graph, where nodes represent semantic elements and edge weights represent the dependency strength between elements. For candidate data resources, the similarity between their semantic feature vectors and each semantic element sub-vector is calculated. Element nodes with a similarity exceeding a threshold of 0.6 are marked as covered. The ratio of the number of covered nodes to the total number of nodes is used as the initial coverage rate. For uncovered nodes, a compensation assessment is performed using multi-hop inference paths in the extended semantic representation: if a path exists that can connect a covered node to an uncovered node within two hops, it is included in the partial coverage rate with a path length attenuation coefficient of 0.8 or 0.6. The semantic integrity metric is obtained by combining the direct coverage rate and the compensated coverage rate.

[0096] The semantic interpretability assessment module generates an attribution chain by tracing the semantic matching process in reverse. This attribution chain records the semantic transmission path from the query vector to the retrieval result, including key regions of feature extraction, the sequence of activated nodes in graph propagation, and the distribution of attention weights during matching. For each link in the attribution chain, the variance of its feature contribution is calculated; a smaller variance indicates that the semantic transmission at that link is concentrated on explicit features, indicating strong interpretability. A weighted average of the interpretability strengths of each link in the attribution chain is then calculated, with the weights determined by the importance of each link in the overall matching process, resulting in a semantic interpretability metric.

[0097] The three metrics are input into a confidence fusion function for comprehensive calculation. The fusion function adopts a weighted product form, with the weight coefficients for cross-modal consistency, semantic integrity, and semantic interpretability metrics set to 0.4, 0.35, and 0.25, respectively, reflecting the priority given to modal synergy. After calculating the comprehensive confidence of each candidate data resource, they are sorted in descending order of confidence value, with results having a confidence level higher than 0.7 presented first, generating the final accurate search results for users to access.

[0098] In one optional implementation, the cross-modal consistency metric, the semantic integrity metric, and the semantic interpretability metric are input into a confidence fusion function according to the semantic confidence assessment mechanism to calculate the overall confidence of each data resource, including:

[0099] Analyze the semantic dependencies among cross-modal consistency measures, semantic integrity measures, and semantic interpretability measures; identify the confidence propagation paths and their propagation weights among different measures; and establish the association structure among the measure dimensions.

[0100] Based on the association structure, the association strength is propagated, and each metric is transmitted to the relevant metric along the confidence transmission path. The association enhancement adjustment is performed on each metric according to the transmission weight to obtain the adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric.

[0101] Analyze the modal distribution characteristics and semantic complexity of the hybrid semantic query vector, and determine the adaptive fusion weights of the adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric based on the modal distribution characteristics and the semantic complexity.

[0102] The adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric are weighted and combined according to the adaptive fusion weight input confidence fusion function to obtain the comprehensive confidence of each data resource.

[0103] When calculating the overall confidence level of each data resource, a deep analysis of the semantic dependencies among the three measurement dimensions is first performed. The cross-modal consistency metric reflects the alignment degree of different modalities in the semantic space, and its value directly affects the credibility judgment of semantic integrity. The semantic integrity metric characterizes the coverage of the query semantics contained in the search results; higher integrity increases the baseline threshold for interpretability assessment. The semantic interpretability metric measures the clarity of the logical connection between the search results and the query intent, and its confidence level inversely corrects the weight allocation of cross-modal consistency. A directed dependency graph is constructed to represent the association structure of these three measurement dimensions. Nodes in the graph represent each measurement dimension, and directed edges represent confidence propagation paths. Edge weights are determined by statistical analysis of the historical correlation between the measurement values. Specifically, the mutual information of any two measurement values ​​in a large number of samples is calculated. When the mutual information exceeds a preset threshold, a propagation path is established, and the propagation weight is set to the normalized mutual information value.

[0104] After the association structure is established, a confidence strength propagation process is executed. For each data resource, its initial cross-modal consistency metric is used as the source node activation value and propagated along the propagation path in the dependency graph to the semantic integrity metric node. The propagation calculation adopts a weighted accumulation method, where the enhancement value of the receiving node is equal to the product of the source node's metric value and the propagation weight. Similarly, the semantic integrity metric propagates the confidence influence to the interpretability metric node, and the interpretability metric also adjusts the cross-modal consistency metric value through the feedback path. After multiple rounds of iterative propagation until the activation values ​​of each node converge, the obtained values ​​are the adjusted metric values ​​for each dimension. This propagation mechanism ensures that the confidence assessments between metric dimensions are mutually calibrated, avoiding excessive interference from outliers in a single dimension on the overall judgment.

[0105] To achieve adaptive weight allocation, a deep analysis of the inherent characteristics of the hybrid semantic query vector is required. Modal distribution features are obtained by statistically analyzing the energy proportion of each modal component in the query vector. When textual modal features dominate, the fusion weight of the adjusted cross-modal consistency metric should be appropriately reduced, as semantic integrity relies more on textual semantic coverage. Semantic complexity is quantified by calculating the entropy value of the query vector in the semantic space. A high entropy value indicates that the query intent contains multiple semantic branches, in which case the weight of the adjusted semantic interpretability metric needs to be increased to strengthen logical association judgment. Specifically, the weight calculation uses a normalized exponential function, taking the modal distribution bias and semantic entropy value as input parameters, and outputting fusion weight coefficients in three dimensions. The weight coefficients satisfy the constraint that their sum is 1, ensuring the stability of the overall confidence score's numerical range.

[0106] The confidence fusion function employs a non-linear weighted combination, summing the three adjusted metrics after multiplying them by their corresponding adaptive weights. To enhance robustness, the fusion function introduces a saturated non-linear mapping; when a metric is abnormally high, its contribution is compressed using a logarithmic function to prevent extreme values ​​from dominating the final result. The fused overall confidence score is mapped to the range of 0 to 1, with values ​​closer to 1 indicating a higher semantic match between the data resource and the query request. This calculation process is sequentially executed for all data resources in the candidate search result set, ultimately generating precise search results by sorting them in descending order of overall confidence score, providing users with the sequence of data resources offering the best semantic match.

[0107] In one optional implementation, extracting a semantic preference evolution pattern based on the user's interactive feedback behavior to the precise search results, and adaptively adjusting the edge weights of the semantic enhancement graph structure using the semantic preference evolution pattern includes:

[0108] Collect user interaction feedback behavior sequences for each data resource in the precise search results, extract semantic feature patterns that users care about from the interaction feedback behavior sequences, and identify the evolution direction and rate of user semantic preferences by analyzing the semantic feature change trends of user interaction behavior in different time windows, thereby obtaining the semantic preference evolution pattern.

[0109] The semantic preference evolution pattern is mapped to a semantic enhancement graph structure, semantic association paths related to the semantic preference evolution pattern are identified, and the edge weights on the semantic association paths are enhanced and adjusted according to the evolution direction in the semantic preference evolution pattern. The adjustment range of the edge weights is determined according to the evolution rate, thus completing the adaptive adjustment of the edge weights of the semantic enhancement graph structure.

[0110] like Figure 2 As shown, the method includes:

[0111] The collection of interactive feedback behavior sequences is achieved by deploying behavior listeners at the user interface layer. These listeners capture multi-dimensional interaction signals from users regarding click events, dwell time, favorite actions, sharing actions, and skip actions for each data resource in the precise search results. The collected behavior data is then arranged in chronological order by timestamp to form the interactive feedback behavior sequence. Each behavior element contains the behavior type, trigger time, and corresponding resource identifier.

[0112] When extracting semantic feature patterns from the sequence of interactive feedback behaviors, the associated fused semantic features are extracted from the data resources of user interaction. The semantic feature vectors corresponding to high-frequency interactive resources are clustered to identify the semantic clusters that users focus on. These semantic clusters reflect the user's core concerns. For resources whose dwell time exceeds a preset threshold, their semantic features are assigned higher weight coefficients, and a set of semantic feature patterns is formed through weighted aggregation.

[0113] The identification of semantic preference evolution patterns is achieved by setting a sliding time window, dividing the sequence of interactive feedback behaviors into multiple time windows, with the duration of each window dynamically configured according to the application scenario. The cosine similarity change of semantic feature patterns within adjacent time windows is calculated. A continuously increasing similarity trend is identified as a direction of semantic preference enhancement, while a decreasing similarity trend is identified as a direction of semantic preference decay. The evolution rate is quantified by the rate of change of the Euclidean distance between semantic feature vectors of adjacent windows; a larger rate of change indicates a faster shift in user preferences.

[0114] When semantic preference evolution patterns are mapped to a semantic enhancement graph structure, semantic clusters in the evolution patterns are semantically matched with nodes in the graph structure to identify a set of nodes highly correlated with the evolution patterns. Starting from these nodes, a breadth-first search is performed along the edges of the graph structure to extract all paths connecting these nodes; these paths constitute a set of semantically associated paths. For semantic clusters whose evolution direction is enhancement, the edge weights on their associated paths are multiplied by an enhancement factor, with the enhancement factor ranging from 1.2 to 1.8. For semantic clusters whose evolution direction is decay, the corresponding edge weights are multiplied by a decay factor, with the decay factor ranging from 0.5 to 0.8.

[0115] The adjustment range of edge weights is adaptively calculated based on the evolution rate, which is normalized to the range of zero to one. This normalized value is directly used as the adjustment range coefficient. When the evolution rate is high, the adjustment range of edge weights increases accordingly, enabling the graph structure to quickly respond to changes in user preferences. The adjusted edge weights are smoothed to avoid drastic fluctuations. An exponential moving average method is used to fuse historical weights with the current adjusted weights, with a smoothing coefficient set to 0.3 to ensure the stability of the graph structure's evolution. After the edge weights are updated, the semantically enhanced graph structure can provide semantic reasoning capabilities that better match users' personalized preferences in subsequent queries.

[0116] The method further includes:

[0117] When users interact with the returned data resources on the search results display interface, the front-end listening module captures each user action in real time through event listeners. These actions include the user's selection behavior of clicking on a data resource to enter the details page, recording the association between the unique identifier of the data resource and the user's identifier; the duration of the user's stay on the data resource details page or list page, recorded by the timer module from page loading to the user leaving or switching, with millisecond precision; and the user's operation sequence information, i.e., recording the timestamp of each operation, stored in Unix timestamp format. The listening module also captures the user's scroll depth, download or favorite operations, and the user's behavior of skipping certain data resources in the search results and selecting other resources. All this raw interaction data is stored in a distributed log collection system in the form of structured logs. The log format includes five core fields: user identifier, data resource identifier, operation type, timestamp, and operation details. The log collection module adopts an asynchronous batch writing mechanism, processing no less than 10,000 log entries per second, with a write latency controlled within 50 milliseconds.

[0118] After the log data is processed by the cleaning module, invalid records with a dwell time of 0, dirty data with disordered timestamps, and incomplete records with missing operation type fields are removed. The cleaned valid logs are grouped by user identifiers to form a user interaction feedback behavior sequence. This sequence is arranged in ascending order of timestamps, retaining all interaction records from the past 3 months. After the behavior sequence is constructed, the semantic feature extraction module performs semantic feature mapping on the data resources associated with each interaction record in the sequence, calls the semantic enhancement graph structure, and queries the node corresponding to the data resource in the graph structure and its connected semantic feature nodes. The semantic feature nodes contain information such as the data resource's topic tags, domain classification, key entities, and attribute dimensions. The extraction module counts the set of semantic feature nodes associated with the data resources that users frequently interact with in the behavior sequence, and uses the frequency of occurrence of these feature nodes as a preliminary indicator of user attention intensity.

[0119] To capture the dynamic changes in user semantic preferences, the evolutionary analysis module introduces a time window mechanism to segment and analyze behavioral sequences. The granularity of the time window is set to one week, and the window length is dynamically adjusted based on user interaction frequency. For high-frequency users, the window length is shortened to three days, while for low-frequency users, it is extended to two weeks. For each time window, the module statistically analyzes the distribution of semantic feature nodes that users focus on within that window and calculates the normalized weight of each semantic feature node. The weight is calculated by dividing the number of times the feature node appears within the window by the sum of the number of times all feature nodes appear within the window, resulting in a normalized value between 0 and 1. The normalized weights of the same semantic feature node within different time windows are arranged chronologically to form a weight time-series sequence for that feature node. Trend analysis is performed on the weight time-series sequence. A linear regression method is used to fit the sequence data, and the slope of the regression line is calculated as an indicator of the evolutionary direction of the semantic feature. A positive slope indicates that the user's attention to the semantic feature increases over time, a negative slope indicates that the attention decreases, and a slope close to 0 indicates that the attention remains stable. The numerical range of the evolution direction is between -1 and +1, and the larger the absolute value, the more significant the evolution trend.

[0120] The evolutionary analysis module also calculates an evolution rate index to quantify the speed of semantic preference changes. The evolution rate is obtained by analyzing the magnitude of changes in semantic feature weights between adjacent time windows. Specifically, it calculates the weight difference between every two adjacent windows, takes the absolute value of all differences, and averages them to obtain the average change magnitude as the baseline value of the evolution rate. This baseline value reflects the overall volatility of user preferences; a larger value indicates a faster change in preferences. To improve the sensitivity of the evolution rate, the module introduces a weighting mechanism, assigning higher weight to the magnitude of changes in recent time windows. The weighting coefficient decays exponentially with the distance of the time window from the current moment, with a decay factor set to 0.9. The weighted evolution rate value ranges from 0 to 1, where 0 represents no change and 1 represents drastic change.

[0121] The evolution direction and evolution rate together constitute the user's semantic preference evolution pattern. This pattern is stored in a structured data format, containing five fields: user identifier, semantic feature node identifier, evolution direction value, evolution rate value, and timestamp. The evolution pattern data is updated in real time. Whenever a user generates a new interaction and meets the time window update conditions, the evolution pattern is recalculated. The update frequency is adaptively adjusted according to user activity: once a day for active users and once a week for inactive users. After the evolution pattern calculation is completed, the results are synchronized to the semantic enhancement graph structure adjustment module.

[0122] The adjustment module first analyzes the evolution pattern data to identify semantic feature nodes with a positive evolution direction and an evolution rate exceeding a set threshold. These nodes represent the semantic domains that the user is currently enhancing their focus on. The threshold is set to 0.3; feature nodes below this value are considered to have insignificant preference changes. For the identified high-evolution feature nodes, the module traces all edges connected to the feature node in the semantic enhancement graph structure. The edge weights reflect the strength and importance of the connection. The adjustment module enhances the weights of the relevant edges based on the evolution direction values. The adjustment strategy involves multiplying the edge weights by an amplification factor of 1. This amplification factor is determined by both the evolution direction and the evolution rate, and is calculated as 1 plus the product of the evolution direction and evolution rate values. This factor ranges from 1 to 1.2 to ensure a moderate weight adjustment.

[0123] For semantic feature nodes with a negative evolution direction, indicating a weakening user interest in that domain, the adjustment module performs a decay adjustment on the relevant edge weights. The decay strategy involves multiplying the edge weights by a reduction factor. The reduction factor is calculated by subtracting the product of the absolute value of the evolution direction and the evolution rate from 1, with a value ranging from 0.8 to 1. After the edge weight adjustment is complete, the module normalizes all adjusted edge weights to ensure that the sum of the weights of all edges originating from the same node remains constant. The normalization method involves calculating the sum of the weights of all outgoing edges from a given node, dividing the weight of each edge by this sum, and obtaining the new normalized weight value.

[0124] The adjusted semantically enhanced graph structure takes effect immediately and is applied to the processing flow of subsequent user-submitted query requests. When a user initiates a new data retrieval request, the multi-hop semantic reasoning module performs reasoning tasks based on the updated graph structure. The graph traversal algorithm prioritizes expanding high-weight edges. Since the edge weights have been dynamically adjusted according to the user's preference evolution pattern, the reasoning path naturally tends towards the semantic direction that the user is currently interested in, and the ranking position of relevant data resources in the search results is correspondingly improved. The graph structure adjustment process adopts an incremental update mechanism, modifying only the edge weights affected by the evolution pattern, while the weights of unaffected edges remain unchanged. The time complexity of the update operation is linearly related to the number of edges to be adjusted, and the time consumption of a single update is controlled within 100 milliseconds.

[0125] For multi-user scenarios, the adjustment module isolates the evolution patterns of different users. Each user has an independent personalized semantically enhanced graph structure view. The view is overlaid with a user-specific edge weight adjustment layer on top of the shared base graph structure. During queries, the personalized view is used first, and any uncovered parts fall back to the base graph structure. The personalized view is stored in a sparse matrix format, recording only the edge weight modifications that differ from the base graph structure, keeping storage space overhead within 5% of the base graph structure. View updates employ a copy-on-write mechanism. When a change in user preferences triggers a view update, the part to be modified is first copied to a temporary space. After the modification is completed, the old view is atomically replaced, ensuring that the query process is not interfered with by update operations.

[0126] Historical records of edge weight adjustments are fully saved to the audit log. The log includes fields such as adjustment timestamp, user identifier, affected edge identifier, weight values ​​before and after the adjustment, and trigger evolution mode identifier, supporting backtracking analysis and anomaly diagnosis. When an edge weight adjustment for a user is detected as causing an abnormal decline in search performance, the rollback mechanism automatically restores the edge weight configuration to the previous stable version, with the rollback operation taking no more than 50 milliseconds. Anomaly detection rules are based on identifying sudden changes in user satisfaction metrics. If a user's click-through rate for five consecutive queries is below 50% of the historical average, an anomaly alarm and automatic rollback process are triggered.

[0127] A second aspect of the present invention provides an electronic device, comprising:

[0128] processor;

[0129] Memory used to store processor-executable instructions;

[0130] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0131] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0132] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for precise processing and sharing of data resources based on multimodal semantic association analysis, characterized in that: include: The query request to be retrieved and its corresponding multimodal input data are obtained. Features are extracted from each modality in the multimodal input data to obtain the feature representation of each modality. Based on the cross-modal semantic mapping relationship, the feature representation of each modality is projected onto a unified semantic space to obtain fused semantic features. A semantically enhanced graph structure is constructed, with the fused semantic features as node features and the semantic relationships between data resources as edge connections. Multi-hop semantic reasoning is performed through a graph propagation mechanism to obtain an extended semantic representation. A hybrid semantic query vector is constructed based on the fused semantic features and the extended semantic representation, and multi-granular semantic matching is performed in the data resource library using the hybrid semantic query vector to obtain a candidate retrieval result set; A semantic confidence evaluation mechanism is established. By calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector, the candidate retrieval result set is weighted and sorted by confidence to generate accurate retrieval results. Based on user interaction feedback with the precise search results, a semantic preference evolution pattern is extracted, and the edge weights of the semantic enhancement graph structure are adaptively adjusted using the semantic preference evolution pattern, including: Collect user interaction feedback behavior sequences for each data resource in the precise search results, extract semantic feature patterns that users care about from the interaction feedback behavior sequences, and identify the evolution direction and rate of user semantic preferences by analyzing the semantic feature change trends of user interaction behavior in different time windows, thereby obtaining the semantic preference evolution pattern. The semantic preference evolution pattern is mapped to a semantic enhancement graph structure, semantic association paths related to the semantic preference evolution pattern are identified, and the edge weights on the semantic association paths are enhanced and adjusted according to the evolution direction in the semantic preference evolution pattern. The adjustment range of the edge weights is determined according to the evolution rate, thus completing the adaptive adjustment of the edge weights of the semantic enhancement graph structure.

2. The method according to claim 1, characterized in that, A semantically enhanced graph structure is constructed, using the fused semantic features as node features and the semantic relationships between data resources as edge connections. Multi-hop semantic reasoning is then performed through a graph propagation mechanism to obtain an extended semantic representation, including: Each data resource and fused semantic feature in the data resource repository is used as a graph node. By calculating the cross-modal semantic coupling degree and semantic structure similarity between nodes, directed edge connections reflecting the semantic dependency strength are established, and edge weights based on semantic transmission credibility are assigned to each edge to construct a semantically enhanced graph structure. In the semantic enhancement graph structure, starting from the query node, semantic diffusion is performed layer by layer along the edge weights. In each hop of propagation, the semantic features of the reached node are weighted and fused according to the path dependency. The propagation depth is dynamically truncated and controlled according to the accumulated path weights to obtain a set of multi-hop reachable nodes and their corresponding path aggregation semantic representation. The path aggregation semantic representation and the fused semantic features are semantically aligned, and an extended semantic representation is generated through a graph propagation mechanism.

3. The method according to claim 2, characterized in that, In the semantically enhanced graph structure, starting from the query node, semantic diffusion is performed layer by layer along the edge weights. In each hop of propagation, the semantic features of the arriving nodes are weighted and fused according to path dependencies. The propagation depth is dynamically truncated based on the accumulated path weights, resulting in a set of multi-hop reachable nodes and their corresponding path-aggregated semantic representations, including: The node corresponding to the fused semantic features is set as the query node, and the propagation state and cumulative path weight of the query node are initialized. Starting from the query node, a first-hop semantic diffusion is performed along the directed edge connection. For the neighboring nodes reached, their original semantic features are combined with the semantic features of the query node according to the weight of the connecting edge to generate the first-hop path fusion feature, and the cumulative path weight of each neighboring node is recorded. For each neighboring node corresponding to the first-hop path fusion feature, it is determined whether its cumulative path weight meets the semantic transmission validity threshold. For nodes that meet the condition, the next-hop semantic diffusion is continued, and for nodes that do not meet the condition, propagation truncation is performed. In the next-hop diffusion, the path fusion feature of the previous hop is used as the input feature of the current hop for a new round of weighted combination, and the cumulative path weight is iteratively updated until all propagation paths are truncated or the preset propagation layer is reached, thus obtaining a multi-hop reachable node set. For each node in the multi-hop reachable node set, its path fusion features are globally weighted and aggregated according to its final accumulated path weight to generate a path aggregation semantic representation.

4. The method according to claim 1, characterized in that, A hybrid semantic query vector is constructed based on the fused semantic features and the extended semantic representation. This hybrid semantic query vector is then used to perform multi-granularity semantic matching in the data resource database to obtain a candidate retrieval result set, including: Calculate the semantic complementarity between the fused semantic features and the extended semantic representation, determine the adaptive fusion weights based on the semantic complementarity, and perform weighted fusion of the fused semantic features and the extended semantic representation according to the adaptive fusion weights to construct a hybrid semantic query vector; The hybrid semantic query vector is projected onto multiple semantic representation spaces of different granularities, generating local semantic matching vectors in the fine-grained semantic representation space and generating global semantic matching vectors in the coarse-grained semantic representation space. The local semantic similarity is calculated using the local semantic matching vector and the fine-grained feature representation of each data resource in the data resource library. The global semantic similarity is calculated using the global semantic matching vector and the coarse-grained feature representation of each data resource. The local semantic similarity and the global semantic similarity are then adaptively weighted and combined to obtain a comprehensive matching score. The data resources in the data resource library are sorted and filtered according to the comprehensive matching score to obtain a set of candidate search results.

5. The method according to claim 1, characterized in that, A semantic confidence assessment mechanism is established. This mechanism calculates the comprehensive confidence level of cross-modal semantic consistency, semantic completeness, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector. The candidate retrieval result set is then weighted and ranked according to this confidence level to generate accurate retrieval results, including: A semantic confidence assessment mechanism is constructed, which includes a cross-modal consistency assessment module, a semantic integrity assessment module, and a semantic interpretability assessment module. The cross-modal consistency assessment module quantifies the degree of collaboration and conflict between different modal semantic expressions by constructing an inter-modal semantic consistency constraint matrix, thereby obtaining a cross-modal consistency metric. The semantic integrity assessment module is based on the semantic element dependency graph obtained by hybrid semantic query vector decomposition. It maps the semantic features of each data resource to the node space and calculates the coverage. It combines the implicit semantic path in the extended semantic representation to compensate for semantically missing nodes and obtains the semantic integrity metric value. Based on the semantic interpretability assessment module, the semantic matching attribution chain is extracted, and the interpretability strength is quantified according to the semantic transmission transparency to obtain the semantic interpretability metric value. According to the semantic confidence assessment mechanism, the cross-modal consistency metric, the semantic integrity metric, and the semantic interpretability metric are input into the confidence fusion function to calculate the comprehensive confidence of each data resource, and then sort them according to the comprehensive confidence to generate accurate search results.

6. The method according to claim 5, characterized in that, According to the semantic confidence assessment mechanism, the cross-modal consistency metric, the semantic integrity metric, and the semantic interpretability metric are input into the confidence fusion function to calculate the comprehensive confidence of each data resource, including: Analyze the semantic dependencies among cross-modal consistency measures, semantic integrity measures, and semantic interpretability measures; identify the confidence propagation paths and their propagation weights among different measures; and establish the association structure among the measure dimensions. Based on the association structure, the association strength is propagated, and each metric is transmitted to the relevant metric along the confidence transmission path. The association enhancement adjustment is performed on each metric according to the transmission weight to obtain the adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric. Analyze the modal distribution characteristics and semantic complexity of the hybrid semantic query vector, and determine the adaptive fusion weights of the adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric based on the modal distribution characteristics and the semantic complexity. The adjusted cross-modal consistency metric, the adjusted semantic integrity metric, and the adjusted semantic interpretability metric are weighted and combined according to the adaptive fusion weight input confidence fusion function to obtain the comprehensive confidence of each data resource.

7. A data resource precision processing and sharing system based on multimodal semantic association analysis, used to implement the method of any one of claims 1-6, characterized in that, include: The feature extraction unit is used to acquire the query request to be retrieved and its corresponding multimodal input data, extract features from each modality of the multimodal input data to obtain the feature representation of each modality, and project the feature representation of each modality onto a unified semantic space based on the cross-modal semantic mapping relationship to obtain fused semantic features. The semantic fusion unit is used to construct a semantically enhanced graph structure, using the fused semantic features as node features, the semantic relationships between data resources as edge connections, and performing multi-hop semantic reasoning through a graph propagation mechanism to obtain an extended semantic representation. The graph reasoning unit is used to construct a hybrid semantic query vector based on the fused semantic features and the extended semantic representation, and to use the hybrid semantic query vector to perform multi-granular semantic matching in the data resource library to obtain a candidate retrieval result set; The query construction unit is used to establish a semantic confidence evaluation mechanism. By calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity, and semantic interpretability between each data resource in the candidate retrieval result set and the hybrid semantic query vector, the candidate retrieval result set is weighted and sorted to generate accurate retrieval results. The matching retrieval unit is used to extract semantic preference evolution patterns based on the user's interactive feedback behavior to the precise retrieval results, and to adaptively adjust the edge weights of the semantic enhancement graph structure using the semantic preference evolution patterns.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images