Digital twin asset searching method, system and equipment and medium

Abstract

The invention discloses a digital twinborn asset searching method, system, equipment and medium, and particularly relates to the field of data identification, and the technical key points are as follows: obtaining digital twinborn asset multi-modal data, carrying out coding and semantic alignment on the digital twinborn asset multi-modal data based on a federated learning framework of cross-domain feature alignment, and obtaining the digital twinborn asset multi-modal data; the aligned multi-modal data features are obtained; triggering a time sequence semantic attention weight mechanism to dynamically update the multi-dimensional feature vector based on a pre-constructed update trigger rule in combination with the asset time sequence feature vector difference to obtain an updated multi-dimensional feature vector; according to the updated multi-dimensional feature vectors of the digital twin assets, constructing a semantic graph by using a graph neural network; and analyzing the obtained user query condition to obtain a query feature vector, calculating a similarity value between the query feature vector and the multi-dimensional feature vector of each digital twin asset in the semantic map, and obtaining a digital twin asset search result based on the similarity value.

Description

Technical Field

[0001] The present invention relates to the field of data identification, and in particular to a digital twin asset search method, system, device and medium. Background Art

[0002] Digital twin assets refer to full-factor digital mappings of physical assets (equipment, production lines, factories, etc.) constructed in virtual space through multi-source data fusion and model-driven methods. They possess multi-dimensional attributes such as geometric features, behavioral rules, state parameters, and life cycles, and support real-time simulation, predictive optimization, and decision-making collaboration. Typical application scenarios include: equipment operation and maintenance: rapid retrieval of historical fault digital twins of equipment of the same model to assist in real-time fault diagnosis; production line planning: searching for production line digital twins that are suitable for specific production processes based on geometric models and production capacity data; cross-enterprise collaboration: sharing digital twins of common components in the supply chain to avoid repeated modeling; its core value lies in realizing intelligent management of the entire asset life cycle through virtual-reality interaction.

[0003] Digital twins evolve in real time with the operating status of physical entities (such as equipment aging and operating condition adjustments). Existing methods rely on fixed feature engineering and cannot capture the dynamic changes in asset semantics, resulting in feature representation lagging behind the actual status and a low degree of match between retrieval results and real needs. At the same time, digital twin assets contain multimodal data such as geometric models, time series data, and text descriptions. Existing technologies have not achieved cross-modal semantic alignment and are difficult to handle complex queries based on "geometric structure similarity + real-time operating parameter matching", which limits multi-dimensional search capabilities.

[0004] Based on this, the present invention aims to provide a digital twin asset search method, system, device and medium to solve the above-mentioned related problems. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that the existing methods rely on fixed feature engineering and are unable to capture the dynamic changes of asset semantics, resulting in feature representation lagging behind the actual state and poor matching between retrieval results and real needs. The purpose is to provide a digital twin asset search method, system, device and medium, which encodes and semantically aligns the multimodal data of digital twin assets through a federated learning framework with cross-domain feature alignment to obtain unified semantic space features; based on pre-built update trigger rules combined with the difference of asset time series feature vectors, triggers the time series semantic attention weight mechanism to dynamically update the multi-dimensional feature vectors to dynamically adjust the fusion ratio of time series features and historical semantics, and realizes real-time evolution of features, thereby solving the technical problem that the existing methods rely on fixed feature engineering and are unable to capture the dynamic changes of asset semantics, resulting in feature representation lagging behind the actual state and poor matching between retrieval results and real needs; through the deep integration of dynamic semantic modeling, cross-modal federated learning and intelligent search strategies, the present invention breaks through the limitations of traditional methods in terms of dynamicity, cross-domain collaboration and privacy protection, and provides an innovative solution for efficient retrieval of digital twin assets in complex scenarios.

[0006] The present invention is achieved through the following technical solutions:

[0007] The present invention provides a digital twin asset search method, the method comprising:

[0008] Obtain multimodal data of digital twin assets, encode and semantically align the multimodal data of digital twin assets based on a federated learning framework for cross-domain feature alignment, and obtain aligned multimodal data features;

[0009] The aligned multimodal data features are integrated to obtain the initial multi-dimensional feature vector. Based on the pre-built update trigger rules and the asset time series feature vector difference, the time series semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector to obtain the updated multi-dimensional feature vector.

[0010] Based on the updated multi-dimensional feature vectors of digital twin assets, a semantic graph is constructed using graph neural networks.

[0011] The obtained user query conditions are parsed to obtain a query feature vector, and the similarity value between the query feature vector and the multi-dimensional feature vector of each digital twin asset in the semantic graph is calculated, and the digital twin asset search results are obtained based on the similarity value.

[0012] Furthermore, multimodal data features include asset geometric features, asset temporal features, asset semantic features, and asset association relationships.

[0013] Furthermore, we obtain the multimodal data of digital twin assets, encode and semantically align the multimodal data of digital twin assets based on the federated learning framework of cross-domain feature alignment, and obtain the aligned multimodal data features, specifically:

[0014] Use local digital twin asset data to train a local multimodal encoder, and use the multimodal encoder to encode the local digital twin asset data to obtain a multimodal data feature representation;

[0015] The central server aggregates multimodal data feature representations through the FedAvg algorithm and uses the graph attention network to learn global semantic associations and output global feature representations;

[0016] A bidirectional mapping function is constructed, and the bidirectional mapping function is combined with the contrastive learning loss function to perform semantic alignment on the multimodal data features based on the global feature representation to obtain the aligned multimodal data features.

[0017] Furthermore, the bidirectional mapping function is specifically: Among them, f(·) and g(·) represent bidirectional mapping functions; G i represents the asset geometric feature data of the i-th sample; T i Represents the asset time series characteristic data of the i-th sample; Indicates the features after the asset geometric feature data is mapped; Indicates the characteristics after mapping of asset time series feature data.

[0018] Furthermore, the contrastive learning loss function is specifically: Where j and i represent the sample index of asset multimodal data; τ represents the temperature hyperparameter.

[0019] Furthermore, based on the pre-built update trigger rules and the asset time series feature vector difference, the time series semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector to obtain the updated multi-dimensional feature vector, specifically:

[0020] When the difference in the asset time series feature vector is greater than the preset dynamic threshold, the time series semantic attention weight mechanism is used to adjust the fusion weight of the time series feature vector and the semantic feature vector in the multi-dimensional feature vector to obtain the adjusted fusion weight;

[0021] The multi-dimensional feature vector is updated using the adjusted fusion weight to obtain an updated multi-dimensional feature vector.

[0022] Furthermore, the similarity between the query feature vector and the multi-dimensional feature vectors of each digital twin asset in the semantic graph is calculated, specifically: Sim(Q,F i )=λ1·Cosine(Q,F i)+λ2·GraphSim(Q,F i ), where λ1 and λ2 represent weight parameters, λ1+λ2=1; Cosine(·) represents cosine similarity; GraphSim(·) represents graph structure similarity; Q represents the query feature vector; F i Represents the multi-dimensional feature vector of the i-th digital twin asset.

[0023] The present invention also provides a digital twin asset search system, which is used in any one of the digital twin asset search methods described above, and the system includes:

[0024] The first module is used to obtain multimodal data of digital twin assets. It encodes and semantically aligns the multimodal data of digital twin assets based on a federated learning framework for cross-domain feature alignment to obtain aligned multimodal data features.

[0025] The second module is used to fuse the aligned multimodal data features to obtain the initial multidimensional feature vector. Based on the pre-built update trigger rules and the asset time series feature vector difference, the temporal semantic attention weight mechanism is triggered to dynamically update the multidimensional feature vector to obtain the updated multidimensional feature vector.

[0026] The third module is used to construct a semantic graph using a graph neural network based on the updated multi-dimensional feature vectors of the digital twin assets;

[0027] The fourth module is used to parse the obtained user query conditions to obtain a query feature vector, calculate the similarity value between the query feature vector and the multi-dimensional feature vector of each digital twin asset in the semantic graph, and obtain the digital twin asset search results based on the similarity value.

[0028] The present invention also provides a computer device, comprising a system memory and a processor, wherein the system memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0029] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods described above are implemented.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] In the present invention, the multimodal data of digital twin assets are encoded and semantically aligned through a federated learning framework with cross-domain feature alignment to obtain unified semantic space features; based on pre-built update trigger rules combined with the asset time series feature vector difference, the time series semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector, so as to dynamically adjust the fusion ratio of time series features and historical semantics, realize the real-time evolution of features, and solve the technical problems that the existing methods rely on fixed feature engineering and cannot capture the dynamic changes of asset semantics, resulting in feature representation lagging behind the actual state and low matching degree between retrieval results and real needs; the present invention breaks through the limitations of traditional methods in dynamics, cross-domain collaboration and privacy protection through the deep integration of dynamic semantic modeling, cross-modal federated learning and intelligent search strategies, and provides an innovative solution for efficient retrieval of digital twin assets in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the examples. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be considered as limiting the scope. A person of ordinary skill in the art can also derive other relevant drawings based on these drawings without inventive effort. In the drawings:

[0033] Figure 1 This is a flowchart of a digital twin asset search method in this embodiment;

[0034] Figure 2 This is a module connection diagram of a digital twin asset search system in this embodiment;

[0035] Figure 3 This is a structural diagram of a computer device in this embodiment. DETAILED DESCRIPTION

[0036] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0038] The terms used in the descriptions of various examples in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in this disclosure encompasses any one and all possible combinations of the listed items.

[0039] Example 1

[0040] See also Figure 1 , Figure 1 A flowchart of a digital twin asset search method is shown, wherein the method includes:

[0041] S1: Obtain multimodal data of digital twin assets, encode and semantically align the multimodal data of digital twin assets based on a federated learning framework for cross-domain feature alignment, and obtain aligned multimodal data features;

[0042] Specifically, in this embodiment, the multimodal data of digital twin assets includes digital twin asset geometric data, digital twin asset time series data, digital twin asset semantic data, and digital twin asset association data. Among them, digital twin asset geometric data refers to the physical geometric structure obtained through CAD models or point cloud data; digital twin asset time series data refers to the real-time signal of the sensor; digital twin asset semantic data refers to the working condition description semantics extracted from the operation and maintenance documents; digital twin asset association data refers to the upstream and downstream dependencies or collaborative relationships of the equipment; multimodal data features include asset geometric features, asset time series features, asset semantic features, and asset association relationships;

[0043] Then, a federated learning framework based on cross-domain feature alignment is used to encode and semantically align the multimodal data of digital twin assets to obtain aligned multimodal data features. The federated learning framework for cross-domain feature alignment includes a local training layer, a model aggregation layer, and a cross-modal alignment layer. Specifically, in the local training layer, the local digital twin asset data is used to train the local multimodal encoder, and the multimodal encoder is used to encode the local digital twin asset data to obtain a multimodal data feature representation.

[0044] In the model aggregation layer, the central server aggregates multimodal data feature representations using the FedAvg algorithm and uses the graph attention network to learn global semantic associations to output global feature representations. At the same time, the central server triggers the contrastive learning loss function every 50 training cycles to ensure semantic consistency of cross-domain features.

[0045] A bidirectional mapping function is constructed in the cross-modal alignment layer. The bidirectional mapping function is combined with the contrastive learning loss function to semantically align the multimodal data features based on the global feature representation to obtain the aligned multimodal data features.

[0046] Among them, the bidirectional mapping function is specifically: Among them, f(·) and g(·) represent bidirectional mapping functions; G i represents the asset geometric feature data of the i-th sample; T i Represents the asset time series characteristic data of the i-th sample; Indicates the features after the asset geometric feature data is mapped; Indicates the characteristics after the asset time series feature data is mapped;

[0047] It should be noted that in this embodiment, the bidirectional mapping function is to force the homologous multimodal features (such as the geometric model and time series signal of the same device) to be in the same Close in space, heterogeneous multimodal features are far away; for example: the point cloud geometric feature G of the same bearing i With the timing characteristic signal T i , after mapping f(G i )、g(T i ) must be higher than the threshold value (such as 0.8); even if the geometric features of different bearings are similar in structure, if the timing signals are very different, the The spatial distance must be greater than a threshold (such as 0.5).

[0048] The contrastive learning loss function is specifically: Where j and i represent the sample index of asset multimodal data; τ represents the temperature hyperparameter.

[0049] It should be noted that, in this embodiment, the contrastive learning loss function adjustment mechanism is the feature space reshaping driven by positive and negative samples, specifically: positive sample pairs: different modal features of the same digital twin asset (such as and ), requiring the distance in Z space to be as close as possible; negative sample pairs: arbitrary modal features of different assets (such as and j≠i), requiring the distance to be as far as possible. Through gradient backpropagation, the parameters of the mapping functions f() and g(·) (such as the weight matrix) are adjusted to achieve the following: homologous modal features are brought closer: the dot product of the geometric and temporal features of the same asset in Z space increases (cosine similarity is improved); heterologous modal features are pushed apart: the dot product of the features of different assets is reduced, forming "semantic clusters" (multimodal features of the same asset are clustered into one cluster, and the intervals between clusters of different assets are increased); at the same time, in this embodiment, a large T (such as 1.0) is set in the initial training stage to allow the feature space to be loose and avoid falling into local optimality; as the training converges, T is gradually reduced (such as 0.1) to enhance feature discrimination. Mathematically, T controls the "temperature" of softmax. The smaller T is, the more significant the probability advantage of positive sample pairs.

[0050] S2: The aligned multimodal data features are integrated to obtain the initial multidimensional feature vector. Based on the pre-built update trigger rules and the asset time series feature vector difference, the temporal semantic attention weight mechanism is triggered to dynamically update the multidimensional feature vector to obtain the updated multidimensional feature vector.

[0051] Specifically, in this embodiment, based on the pre-built update trigger rules combined with the asset temporal feature vector difference, the temporal semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector to obtain an updated multi-dimensional feature vector. Specifically, when the asset temporal feature vector difference is greater than the preset dynamic threshold, the temporal semantic attention weight mechanism is used to adjust the fusion weight of the temporal feature vector and the semantic feature vector in the multi-dimensional feature vector to obtain the adjusted fusion weight; the multi-dimensional feature vector is updated using the adjusted fusion weight to obtain an updated multi-dimensional feature vector.

[0052] It should be noted that, in this embodiment, when the difference of the asset time series feature vector is less than or equal to the preset dynamic threshold, the time series semantic attention weight mechanism is not triggered; the preset dynamic threshold ∈=β·σ t , where β represents the adjustment coefficient, σ t represents the time series standard deviation; when |T t -T t-1 When |>∈, the temporal semantic attention weight mechanism is triggered; at the same time, the fusion weight calculation function is specifically: Where Q(·) and K(·) represent the query and key mapping functions of the asset time series feature vector and the historical asset semantic feature vector, respectively; represents the scaling factor; N represents the sequence length of the historical semantic features, and the sum of the weights is 1, which represents the degree of dependence of the current time series feature on each historical semantic feature. At the same time, in this embodiment, when the device is operating normally (time series data is stable), α tReduce, semantic features dominate (such as functional description); when equipment failure warning (time series data mutation), α t Increase, time series features dominate (such as real-time signals of abnormal vibration); based on attention weight α t , the time series feature T t With historical semantics S t-1 Fusion into dynamic semantic vectors, specifically:

[0053] S3: Based on the updated multi-dimensional feature vectors of the digital twin assets, a graph neural network is used to construct a semantic graph.

[0054] S4: Analyze the obtained user query conditions to obtain a query feature vector, calculate the similarity value between the query feature vector and the multi-dimensional feature vectors of each digital twin asset in the semantic graph, and obtain the digital twin asset search results based on the similarity value.

[0055] Specifically, in this embodiment, the user enters a query condition (such as "find a spare bearing digital twin that matches the vibration mode of the XX production line"), the user query condition is parsed by keywords and a query feature vector Q is generated; the similarity value between the query feature vector and the multi-dimensional feature vector of each digital twin asset in the semantic graph is calculated by traversing the semantic graph, specifically: Sim(Q, F i )=λ1·Cosine(Q,F i )+λ2·GraphSim(Q,F i ), where λ1 and λ2 represent weight parameters, λ1+λ2=1; Cosine(·) represents cosine similarity; GraphSim(·) represents graph structure similarity; Q represents the query feature vector; F i Represents the multi-dimensional feature vector of the i-th digital twin asset; finally, the digital twin asset search results are obtained based on the similarity value.

[0056] At the same time, it should be noted that in this embodiment, the reinforcement learning module is also triggered to adjust the λ1 / λ2 weights based on user interaction feedback (such as clicking on an asset to view details) to optimize the next search result. The reinforcement learning module first includes building a reinforcement learning environment, quantifying interaction feedback, and adjusting weights. Specifically,

[0057] 1. Construction of a multi-dimensional representation of the state: Define the state vector St = [Qt, Ht, Wt, Ct], which contains four core elements: Current query feature Qt: the feature vector after user input parsing (e.g., the frequency domain amplitude distribution of the time series feature corresponding to "vibration mode"); Historical interaction record Ht: the interaction sequence of the past N searches (e.g., click asset ID, dwell time, and reuse results, stored using a sliding window); Current weight parameter Wt = [λ, μ, v]: the key weight in similarity calculation (λ is the feature vector similarity weight, μ is the graph structure similarity weight, and v is the semantic label matching weight, where λ + μ + v = 1); Real-time operating condition context Ct: the real-time operating parameters of the target asset (e.g., temperature and load, which influence the direction of weight adjustment; for example, high-load scenarios should focus on time series features);

[0058] 2. Action Discretization Design: Design three types of atomic action sets A = {a1, a2, a3}, with corresponding weight adjustment strategies: Feature weight fine-tuning a1: Perturb λ by ±5% (e.g., from 0.6 to 0.63 or 0.57), prioritizing enhancing / weakening the influence of feature vector similarity; Graph weight fine-tuning a2: Perturb μ by ±8% (because graph structure similarity calculation includes historical collaborative data, it is more sensitive); Trigger federated update a3: When M consecutive search feedbacks are below a threshold, trigger the federated server to re-aggregate cross-domain features (e.g., M = 10);

[0059] 3. Reward layered quantification model: Design a three-level reward function based on user interaction behavior to solve the feedback sparseness problem: R t =ω1R click +ω2R duration +ω3R reuse -ω4R omission , where R click Indicates click reward, click on the candidate asset +1, do not click but view details +0.5, ignore -0.3; R duration Indicates the stay time reward, stay > 30 seconds + 0.2 / 10 seconds, < 10 seconds - 0.1 (obtained through front-end tracking); R reuse Indicates the reward for successful reuse. If the user completes asset reuse based on the search results (such as downloading a digital twin model), +2 is awarded, and if it fails, -1 is awarded. omission represents the missed detection penalty, which is the user manually input supplementary keywords (implicit missed detection) + the average number of clicks on historical similar queries × (-0.5); weight coefficients ω1 = 0.5, ω2 = 0.3, ω3 = 0.8, ω4 = 1.2 (calibrated through offline simulation);

[0060] 4. Weight adjustment algorithm based on Q-learning: (1) Q-value function and state transition: Define Q-value Q(St,a) as the expected discounted value of the cumulative reward after executing action a in state St: Where γ = 0.95 is a discount factor that encourages long-term returns; (2) Weight update strategy (ε-greedy algorithm): Exploration phase (ε = 0.2): randomly select actions with a probability of 20% (such as in the initialization phase or when user demand suddenly changes) to avoid falling into local optimality; Utilization phase (ε = 0.8): select the action with the largest current Q value, such as prioritizing the adjustment of the weight corresponding to the historical high reward (such as increasing λ when the user clicks on the asset with matching temporal features multiple times); (3) Adjustment process (taking click feedback as an example) is as follows: Feedback capture: the user clicks on the kth candidate asset (the higher the ranking, the higher the reward correction coefficient, such as an additional +0.5 for the first place); State update: the feature vector F of the clicked asset is updated. k Join H t , update W t W t +ΔW, where: Among them, α=0.01 is the learning rate, is the key dimension in the feature vector of the click asset that matches the query; Q value iteration: update the Q table through the Bellman equation, specifically: Q(s t ,a)←(1-α)Q(s t ,a)+α(R t +γmax a＇ Q(S t+1 , a′)), where Q(S t , a) means in state S t The state-action value function of executing action a is used to measure the expected future cumulative reward of the state-action pair; α represents the learning rate, which ranges from [0, 1]; R t Indicates that in state S t The immediate reward obtained after executing action a; Y represents the discount factor, and its value range is [0, 1]. The closer Y is to 0, the more attention is paid to the immediate reward; the closer Y is to 1, the more attention is paid to the long-term reward; S t+1 Indicates the next state after executing action a; max a′ Q(S t+1 , a′) indicates that in the next state S t+1 Under this condition, the Q value of all possible actions a′ is maximized, representing the optimal expected future cumulative reward of the next state.

[0061] Specifically, in this embodiment, the multimodal data of digital twin assets are encoded and semantically aligned through a federated learning framework with cross-domain feature alignment to obtain unified semantic space features; based on pre-built update trigger rules combined with the asset time series feature vector difference, the time series semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector, so as to dynamically adjust the fusion ratio of time series features and historical semantics, realize the real-time evolution of features, and solve the technical problems that the existing methods rely on fixed feature engineering and cannot capture the dynamic changes of asset semantics, resulting in feature representation lagging behind the actual state and the low matching degree between retrieval results and real needs; the present invention breaks through the limitations of traditional methods in dynamics, cross-domain collaboration and privacy protection through the deep integration of dynamic semantic modeling, cross-modal federated learning and intelligent search strategies, and provides an innovative solution for efficient retrieval of digital twin assets in complex scenarios.

[0062] Example 2

[0063] See also Figure 2 As shown, the present invention also provides a digital twin asset search system, which is used in any one of the digital twin asset search methods described above, and the system includes:

[0064] The first module 100 is used to obtain multimodal data of digital twin assets, encode and semantically align the multimodal data of digital twin assets based on a federated learning framework for cross-domain feature alignment, and obtain aligned multimodal data features;

[0065] The second module 200 is used to fuse the aligned multimodal data features to obtain an initial multidimensional feature vector, and trigger the temporal semantic attention weight mechanism to dynamically update the multidimensional feature vector based on the pre-built update trigger rule and the asset temporal feature vector difference, thereby obtaining an updated multidimensional feature vector;

[0066] The third module 300 is used to construct a semantic graph using a graph neural network based on the updated multi-dimensional feature vector of the digital twin asset;

[0067] The fourth module 400 is used to parse the obtained user query conditions to obtain a query feature vector, calculate the similarity value between the query feature vector and the multi-dimensional feature vectors of each digital twin asset in the semantic graph, and obtain digital twin asset search results based on the similarity value.

[0068] It should be noted that the modules in the system of Example 2 correspond to the steps in the method of Example 1. The steps in the method of Example 1 have been described in detail in Example 1, and the contents of the modules in the system will not be described in detail in this Example 2.

[0069] Example 3

[0070] See also Figure 3 As shown, this embodiment further provides a computer device, including a system memory 1005 and a processor 1001, wherein the system memory 1005 stores a computer program, and the processor 1001 implements the steps of any of the above methods when executing the computer program.

[0071] It should be noted that the processor 1001 is configured to execute the steps of the above method embodiments according to the instructions in the program code. Alternatively, the processor 1001 implements the functions of the modules / units in the above system / device embodiments when executing the computer program.

[0072] Specifically, in this embodiment, the computer program may be divided into one or more modules / units, one or more modules / units being stored in the system memory 1005 and executed by the processor 1001 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0073] The terminal device may be a computing device such as a desktop computer, laptop, PDA, or cloud server. The terminal device may include, but is not limited to, a processor 1001 and a system memory 1005. Those skilled in the art will appreciate that this does not limit the terminal device and may include more or fewer components than shown, or a combination of certain components, or different components. For example, the terminal device may also include an input / output device 1003, a network access device 1002, a bus 1006, and the like.

[0074] The processor 1001 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0075] The system memory 1005 can be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. The system memory 1005 can also be the storage device 1004 of the terminal device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal device. Furthermore, the system memory 1005 can also include both the internal storage unit of the terminal device and the storage device 1004. The system memory 1005 is used to store computer programs and other programs and data required by the terminal device. The system memory 1005 can also be used to temporarily store data that has been output or is about to be output.

[0076] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0077] Example 4

[0078] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0079] Among them, the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, system or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk. Random Access Memory (RAM), Read-Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer-readable storage medium known in the art.

[0080] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (ASIC). In an embodiment of the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device.

[0081] Example 5

[0082] This embodiment further provides a computer program product comprising instructions. When the instructions are executed by a computer device cluster, the computer device cluster executes the method described in Embodiment 1.

[0083] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A digital twin asset search method, characterized in that the method include: Obtain multimodal data of digital twin assets, encode and semantically align the multimodal data of digital twin assets based on a federated learning framework for cross-domain feature alignment, and obtain aligned multimodal data features; The aligned multimodal data features are integrated to obtain the initial multi-dimensional feature vector. Based on the pre-built update trigger rules and the asset time series feature vector difference, the time series semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector to obtain the updated multi-dimensional feature vector. Based on the updated multi-dimensional feature vectors of digital twin assets, a semantic graph is constructed using graph neural networks. The obtained user query conditions are parsed to obtain a query feature vector, and the similarity value between the query feature vector and the multi-dimensional feature vector of each digital twin asset in the semantic graph is calculated, and the digital twin asset search results are obtained based on the similarity value.

2. A digital twin asset search method according to claim 1, characterized in that: Multimodal data features include asset geometric features, asset temporal features, asset semantic features, and asset association relationships.

3. A digital twin asset search method according to claim 1, characterized in that: Obtain multimodal data of digital twin assets, encode and semantically align the multimodal data of digital twin assets based on the federated learning framework of cross-domain feature alignment, and obtain the aligned multimodal data features, specifically: Use local digital twin asset data to train a local multimodal encoder, and use the multimodal encoder to encode the local digital twin asset data to obtain a multimodal data feature representation; The central server aggregates multimodal data feature representations through the FedAvg algorithm and uses the graph attention network to learn global semantic associations and output global feature representations; A bidirectional mapping function is constructed, and the bidirectional mapping function is combined with the contrastive learning loss function to perform semantic alignment on the multimodal data features based on the global feature representation to obtain the aligned multimodal data features.

4. A digital twin asset search method according to claim 3, characterized in that: The bidirectional mapping function is specifically: Among them, f(·) and g(·) represent bidirectional mapping functions; G i represents the asset geometric feature data of the i-th sample; T i Represents the asset time series characteristic data of the i-th sample; Indicates the features after the asset geometric feature data is mapped; Indicates the characteristics after mapping of asset time series feature data.

5. A digital twin asset search method according to claim 4, characterized in that: The contrastive learning loss function is specifically: Where j and i represent the sample index of asset multimodal data; τ represents the temperature hyperparameter.

6. A digital twin asset search method according to claim 1, characterized in that: Based on the pre-built update trigger rules and the asset time series feature vector difference, the time series semantic attention weight mechanism is triggered to dynamically update the multi-dimensional feature vector to obtain the updated multi-dimensional feature vector, specifically: When the difference in the asset time series feature vector is greater than the preset dynamic threshold, the time series semantic attention weight mechanism is used to adjust the fusion weight of the time series feature vector and the semantic feature vector in the multi-dimensional feature vector to obtain the adjusted fusion weight; The multi-dimensional feature vector is updated using the adjusted fusion weight to obtain an updated multi-dimensional feature vector.

7. A digital twin asset search method according to claim 1, characterized in that: Calculate the similarity between the query feature vector and the multi-dimensional feature vectors of each digital twin asset in the semantic graph, specifically: Sim(Q,F i )=λ1·Cosine(Q,F i )+λ2·GraphSim(Q,F i ), where λ1 and λ2 represent weight parameters, λ1+λ2=1; Cosine(·) represents cosine similarity; GraphSim(·) represents graph structure similarity; Q represents the query feature vector; F i Represents the multi-dimensional feature vector of the i-th digital twin asset.

8. A digital twin asset search system, characterized in that: The system is used in a digital twin asset search method according to any one of claims 1 to 7, and the system includes: The first module is used to obtain multimodal data of digital twin assets. It encodes and semantically aligns the multimodal data of digital twin assets based on a federated learning framework for cross-domain feature alignment to obtain aligned multimodal data features. The second module is used to fuse the aligned multimodal data features to obtain the initial multidimensional feature vector. Based on the pre-built update trigger rules and the asset time series feature vector difference, the temporal semantic attention weight mechanism is triggered to dynamically update the multidimensional feature vector to obtain the updated multidimensional feature vector. The third module is used to construct a semantic graph using a graph neural network based on the updated multi-dimensional feature vectors of the digital twin assets; The fourth module is used to parse the obtained user query conditions to obtain a query feature vector, calculate the similarity value between the query feature vector and the multi-dimensional feature vector of each digital twin asset in the semantic graph, and obtain the digital twin asset search results based on the similarity value.

9. A computer device comprising a system memory and a processor, wherein the system memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images