Data processing methods, apparatus, equipment and storage media

Abstract

This application relates to a data processing method, apparatus, device, and storage medium. The method includes: first, determining the semantic similarity between target data in a first business system and each candidate data in a second business system; then, determining the structural similarity between the target data and each candidate data, wherein the structural similarity indicates the similarity between the target data and other data in the first business system, and between the candidate data and other data in the second business system; finally, determining a similarity judgment result based on the semantic and structural similarity of each candidate data, and determining data corresponding to the target data from multiple candidate data based on the similarity judgment result. This method enables data association between different business systems without manual comparison, resulting in higher efficiency.

Description

Technical Field

[0001] This application relates to the field of IT and software development technology, and in particular to a data processing method, apparatus, device and storage medium. Background Technology

[0002] As the complexity of the company's business systems and the diversity of its products increase, a large amount of master data is used in the business system applications. However, the way master data is represented may be different between different business systems, which makes it impossible to associate the master data between different business systems when integrating master data.

[0003] Currently, in order to link the master data of different business systems, it is necessary to compare the master data of different business systems one by one, which is time-consuming and inefficient. Summary of the Invention

[0004] Therefore, it is necessary to provide a more efficient data processing method, apparatus, device, and storage medium to address the aforementioned technical problems.

[0005] Firstly, this application provides a data processing method. The method includes: determining the semantic similarity between target data in a first business system and each candidate data in a second business system; determining the structural similarity between the target data and each candidate data, wherein the structural similarity indicates the similarity between the target data and other data in the first business system, and between the candidate data and other data in the second business system; determining a similarity judgment result based on the semantic and structural similarities corresponding to each candidate data; and determining data that corresponds to the target data from multiple candidate data based on the similarity judgment result.

[0006] In one embodiment, determining the semantic similarity between target data in the first business system and each candidate data in the second business system includes: determining first data associated with the target data from the data included in the first business system, and generating a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each candidate data, determining second data associated with the candidate data from the data included in the second business system, and generating a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; and determining the similarity between the first concept tree and each second concept tree to obtain the semantic similarity between the target data and each candidate data.

[0007] In one embodiment, determining the structural similarity between the target data and each candidate data includes: generating a first association graph based on the association between the target data and each first data, and generating a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data; for each candidate data, generating a second association graph based on the association between the candidate data and each second data, and generating a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data; inputting the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model, and obtaining the structural similarity between the target data and each candidate data based on the output of the graph neural network model.

[0008] In one embodiment, the similarity determination result is determined based on the semantic similarity and structural similarity corresponding to each candidate data, including: performing a fusion process on the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and determining the similarity determination result based on the comprehensive similarity corresponding to each candidate data.

[0009] In one embodiment, the semantic similarity and structural similarity corresponding to each candidate data are fused to obtain a comprehensive similarity corresponding to each candidate data, and a similarity judgment result is determined based on the comprehensive similarity corresponding to each candidate data. This includes: constructing a semantic similarity matrix based on the semantic similarity corresponding to each candidate data; constructing a structural similarity matrix based on the structural similarity corresponding to each candidate data; performing a weighted summation of the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each candidate data; and using the comprehensive similarity matrix as the similarity judgment result.

[0010] In one embodiment, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. Based on the similarity determination result, data that corresponds to the target data is determined from the multiple candidate data, including: if the maximum value of the similarity is greater than a preset threshold, the candidate data corresponding to the maximum value of each similarity is determined as data that corresponds to the target data.

[0011] In one embodiment, determining the similarity between the first concept tree and each of the second concept trees includes: determining the similarity between the first concept tree and each of the second concept trees according to a similarity algorithm, wherein the similarity algorithm includes at least one of an edit distance algorithm, a common substring algorithm, a deep learning model, or a text embedding model.

[0012] Secondly, this application also provides a data processing apparatus. The apparatus includes:

[0013] The first determining module is used to determine the semantic similarity between the target data in the first business system and each candidate data in the second business system.

[0014] The second determining module is used to determine the structural similarity between the target data and each candidate data respectively. The structural similarity is used to indicate the similarity between the target data and other data in the first business system and between the candidate data and other data in the second business system.

[0015] The third determining module is used to determine the similarity judgment result based on the semantic similarity and structural similarity corresponding to each candidate data.

[0016] The fourth determination module is used to determine the data that corresponds to the target data from multiple candidate data based on the similarity determination results.

[0017] In one embodiment, the first determining module is specifically configured to determine first data associated with the target data from the data included in the first business system, and generate a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each candidate data, determine second data associated with the candidate data from the data included in the second business system, and generate a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; and determine the similarity between the first concept tree and each second concept tree respectively to obtain the semantic similarity between the target data and each candidate data.

[0018] In one embodiment, the second determining module is specifically configured to generate a first association graph based on the association relationship between the target data and each first data, and generate a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data; for each candidate data, generate a second association graph based on the association relationship between the candidate data and each second data, and generate a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data; input the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model, and obtain the structural similarity between the target data and each candidate data based on the output of the graph neural network model.

[0019] In one embodiment, the third determining module is specifically used to perform fusion processing on the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and to determine the similarity judgment result based on the comprehensive similarity corresponding to each candidate data.

[0020] In one embodiment, the third determining module is specifically used to construct a semantic similarity matrix based on the semantic similarity corresponding to each candidate data; construct a structural similarity matrix based on the structural similarity corresponding to each candidate data; perform weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each candidate data; and use the comprehensive similarity matrix as the similarity determination result.

[0021] In one embodiment, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. The fourth determining module is specifically used to determine the candidate data corresponding to the maximum value of each similarity as data that has a corresponding relationship with the target data if the maximum value of the similarity is greater than a preset threshold.

[0022] In one embodiment, the first determining module is specifically used to determine the similarity between the first concept tree and each of the second concept trees according to a similarity algorithm, wherein the similarity algorithm includes at least one of the edit distance algorithm, common substring algorithm, deep learning model or text embedding model.

[0023] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the data processing methods described in the first aspect.

[0024] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements any of the data processing methods described in the first aspect.

[0025] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements any of the data processing methods described in the first aspect.

[0026] The aforementioned data processing method, apparatus, device, and storage medium first determine the semantic similarity between target data in the first business system and each candidate data in the second business system. Then, they determine the structural similarity between the target data and each candidate data, where structural similarity indicates the similarity between the target data and other data in the first business system, and between the candidate data and other data in the second business system. Finally, based on the semantic and structural similarity of each candidate data, a similarity judgment result is determined. Based on the similarity judgment result, data corresponding to the target data is identified from multiple candidate data. In this way, after determining the textual and structural similarity between the target data in the first business system and the candidate data in the second business system, the associated target data and candidate data are obtained, achieving data association between different business systems without manual comparison, thus improving efficiency. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a data processing method in one embodiment;

[0028] Figure 2 This is a flowchart illustrating the data processing method in another embodiment;

[0029] Figure 3 Here is a flowchart of semantic similarity calculation in another embodiment;

[0030] Figure 4 This is a flowchart illustrating the data processing method in another embodiment;

[0031] Figure 5 This is a flowchart of structural similarity calculation in another embodiment;

[0032] Figure 6 This is a flowchart illustrating the data processing method in another embodiment;

[0033] Figure 7 This is a flowchart illustrating the data processing method in another embodiment;

[0034] Figure 8 This is a flowchart illustrating the data processing method in another embodiment;

[0035] Figure 9 This is a structural block diagram of a data processing device in one embodiment;

[0036] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0038] In one embodiment, such as Figure 1 As shown, a data processing method is provided. Taking its application to a terminal as an example, it can be understood that this method can also be applied to a server, and furthermore, to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The method includes the following steps:

[0039] Step 101: Determine the semantic similarity between the target data in the first business system and each candidate data in the second business system.

[0040] In this system, the first business system and the second business system are different business systems. The target data can be the master data in the first business system, and the candidate data can be the master data in the second business system. Both the first and second business systems can include multiple master data sets. Optionally, the master data can be the same data across different business systems, but represented differently. For example, to represent gender, the first business system uses "F (female)" and "M (male)," while the second business system uses "1000 (female)" and "2000 (male)." Therefore, when integrating different business systems, it is necessary to integrate the master data, that is, to associate the master data of different business systems.

[0041] Semantic similarity, also known as text similarity, can optionally be determined by extracting semantic features between the target data in the first business system and the candidate data in the second business system.

[0042] Step 102: Determine the structural similarity between the target data and each candidate data.

[0043] Structural similarity is used to indicate the similarity between target data and other data in the first business system, and between candidate data and other data in the second business system. The relationship between target data and other data in the first business system can be determined using data such as business models and association tables in the first business system, and the relationship between candidate data and other data in the second business system can be determined using data such as business models and association tables in the second business system. Then, structural similarity is calculated by extracting the structural features of the relationship between target data and other data in the first business system, and the structural features of the relationship between candidate data and other data in the second business system.

[0044] Step 103: Determine the similarity judgment result based on the semantic similarity and structural similarity corresponding to each candidate data.

[0045] After determining the semantic similarity and structural similarity between the target data and each candidate data, optionally, the determined semantic similarity and structural similarity can be fused and calculated to determine the similarity judgment result.

[0046] Step 104: Based on the similarity determination results, determine the data that corresponds to the target data from multiple candidate data.

[0047] Each similarity score represents the similarity between the target data and each candidate data. Based on the similarity scores, candidate data that corresponds to the target data can be identified. By sequentially judging multiple target data in the first business system, candidate data that corresponds to each target data can be obtained, thus completing the integration of the master data.

[0048] In the above embodiments, firstly, the semantic similarity between the target data in the first business system and each candidate data in the second business system is determined. Then, the structural similarity between the target data and each candidate data is determined, where structural similarity indicates the similarity between the target data and other data in the first business system, and between the candidate data and other data in the second business system. Finally, based on the semantic and structural similarity of each candidate data, a similarity judgment result is determined. Based on the similarity judgment result, data corresponding to the target data is determined from multiple candidate data. In this way, after determining the textual and structural similarity between the target data in the first business system and the candidate data in the second business system, the associated target data and candidate data are obtained, realizing the association of data between different business systems without manual comparison, thus improving efficiency. Furthermore, simultaneously determining the candidate data associated with the target data through semantic and structural similarity further increases the accuracy of data association.

[0049] In one embodiment, the steps for determining semantic similarity are as follows: Figure 2 As shown, it includes:

[0050] Step 201: Determine the first data associated with the target data from the data included in the first business system, and generate a first concept tree corresponding to the target data and the first data based on the business structure of the first business system.

[0051] Optionally, the first data refers to relevant data associated with the target data in the first business system. For example, when the target data is a product type, the relevant data may include the product itself, the attribute values ​​of the product type, etc. The first data and the target data are extracted from the first business system, and then a first concept tree is generated based on the business structure.

[0052] Step 202: For each candidate data, determine the second data associated with the candidate data from the data included in the second business system, and generate a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system.

[0053] Optionally, the second data is the relevant data associated with the candidate data in the second business system. The second data and candidate data are extracted from the second business system, and then a second concept tree can be generated according to the business structure.

[0054] Step 203: Determine the similarity between the first concept tree and each of the second concept trees to obtain the semantic similarity between the target data and each candidate data.

[0055] Based on the similarity algorithm, the similarity between the first concept tree and each of the second concept trees is determined. The similarity algorithm includes at least one of the following: edit distance algorithm, common substring algorithm, deep learning model or text embedding model.

[0056] After determining the first and second concept trees, the semantic similarity between the target data and each candidate data is determined according to a similarity algorithm. Optionally, the similarity algorithm can include edit distance algorithms, common substring algorithms, or pre-trained open-source deep learning models such as fastText, as well as text embedding models such as word2Vec, BERT-based embedding models, etc. Optionally, due to different inputs, the first and second concept trees can be processed to meet the input requirements of different similarity algorithms.

[0057] Optional, such as Figure 3 As shown, the first concept tree generated from the target data in the first business system and the second concept tree generated from the candidate data in the second business system are input into different similarity algorithms to calculate the similarity and obtain the semantic similarity matrix.

[0058] In the above embodiments, by obtaining the target data and the first concept tree generated from the first data in the first business system, and obtaining the candidate data and the second concept tree generated from the second data in the second business system, and then performing similarity calculations according to different similarity algorithms, a semantic similarity matrix is ​​obtained, resulting in a more accurate semantic similarity.

[0059] In one embodiment, the steps for determining structural similarity are as follows: Figure 4 As shown, it includes:

[0060] Step 401: Generate a first association graph based on the relationship between the target data and each first data, and generate a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data.

[0061] Optionally, based on the relationships between the target data and each piece of first data, and the business model of the first business system, the relationships between tables in the first business system, and related tables, etc., are used as graph relationships, and the data of the business model tables are used as graph nodes, a first relationship graph is generated. Optionally, the data content of the target data and the data content of the first data, i.e., the fields of the tables, are used as attributes of the graph nodes, and the field values ​​are used as attribute values ​​of the graph nodes. For text content, a text embedding model can be used to process it, and then it is used as the attribute value content of the graph nodes of the first relationship graph, i.e., generating the first data attribute matrix corresponding to the first relationship graph.

[0062] Step 402: For each candidate data, generate a second association graph based on the association relationship between the candidate data and each second data, and generate a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data.

[0063] Optionally, for each candidate data, as shown in the above embodiments, a second association graph and a second data attribute matrix corresponding to the second association graph are generated.

[0064] Step 403: Input the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into the graph neural network model, and obtain the structural similarity between the target data and each candidate data according to the output of the graph neural network model.

[0065] The first association graph, the first data attribute matrix, each of the second association graphs, and each of the second data attribute matrices are input into a pre-trained graph neural network model. The structural similarity between the target data and each candidate data point is then obtained based on the output of the graph neural network model. Optionally, the graph neural network model can be trained using a pre-defined training set after annotation.

[0066] Optional, such as Figure 5 As shown, the first business system generates a first relationship graph (graph1 in the figure), and the second business system generates a second relationship graph (graph2 in the figure). Then, the graph is input into the graph neural network model. According to the internal processing of the graph neural network model, the structural similarity vector is output. Optionally, the similarity vector can be extended into a structural similarity matrix.

[0067] In the above embodiments, a first association graph is generated by acquiring data from the first business system and a second association graph is generated by acquiring data from the second business system. Then, the structural features are extracted by inputting the graph neural network model to obtain the structural similarity between the two graphs. The structural similarity of the features of the target data and candidate data, including related data, is generated by the graph neural network model to achieve more accurate similarity calculation.

[0068] In the embodiments of this application, the similarity determination result is determined based on the semantic similarity and structural similarity corresponding to each candidate data, including: performing a fusion processing on the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and determining the similarity determination result based on the comprehensive similarity corresponding to each candidate data.

[0069] Optional, specific fusion processing steps are as follows: Figure 6 As shown, it includes:

[0070] Step 601: Construct a semantic similarity matrix based on the semantic similarity of each candidate data.

[0071] Optional, such as Figure 3 As shown, the target data and each candidate data are generated into concept trees and input into different similarity algorithms to obtain the semantic similarity between the target data and each candidate data under different similarity algorithms, and a semantic similarity matrix is ​​constructed.

[0072] Step 602: Construct a structural similarity matrix based on the structural similarity of each candidate data.

[0073] Optional, such as Figure 5 As shown, the graph of the relationship between the target data and each candidate data is input into the graph neural network to obtain the vector of graph similarity between the target data and each candidate data. The vector is then extended to the same dimension as the semantic similarity matrix to construct the structural similarity matrix.

[0074] Step 603: Perform a weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity of each candidate data, and use the comprehensive similarity matrix as the similarity determination result.

[0075] Optionally, the semantic similarity matrix and structural similarity matrix obtained above are assigned corresponding weights and then added together to obtain the comprehensive similarity matrix between the target data and each candidate data. The comprehensive similarity matrix includes the comprehensive similarity between the target data and each candidate data, and the comprehensive similarity matrix is ​​used as the similarity determination result.

[0076] Optionally, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. Based on the similarity determination results, data that corresponds to the target data is determined from the multiple candidate data, including: if the maximum value of the similarity is greater than a preset threshold, the candidate data corresponding to the maximum value of each similarity is determined as the data that corresponds to the target data.

[0077] Optionally, the maximum value among the similarities is compared with a preset threshold. If it exceeds the preset threshold, the candidate data corresponding to the maximum value is determined as data that corresponds to the target data. The corresponding data pair is output. For example, taking one master data from the first business system and the second business system mentioned above as an example, the output is "First Business System-F-Female-Similar-Second Business System-1000-Female". Optionally, the weight values ​​of semantic similarity and structural similarity can be dynamically adjusted according to the accuracy of the finally output corresponding data pairs to improve the accuracy of matching.

[0078] In the above embodiments, the final comprehensive similarity is obtained by fusing semantic similarity and structural similarity. Then, candidate data associated with the target data are determined based on the comprehensive similarity and a preset threshold. The weight value can be dynamically adjusted according to the result of the main data association to improve the accuracy of the main data association.

[0079] In the embodiments of this application, please refer to Figure 7 The flowchart illustrates a data processing method provided in an embodiment of this application, which includes the following steps:

[0080] Step 701: Determine the first data associated with the target data from the data included in the first business system, and generate a first concept tree corresponding to the target data and the first data based on the business structure of the first business system.

[0081] Step 702: For each candidate data, determine the second data associated with the candidate data from the data included in the second business system, and generate a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system.

[0082] Step 703: Determine the similarity between the first concept tree and each of the second concept trees to obtain the semantic similarity between the target data and each candidate data.

[0083] Step 704: Generate a first association graph based on the relationship between the target data and each first data, and generate a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data.

[0084] Step 705: For each candidate data, generate a second association graph based on the association relationship between the candidate data and each second data, and generate a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data.

[0085] Step 706: Input the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into the graph neural network model, and obtain the structural similarity between the target data and each candidate data according to the output of the graph neural network model.

[0086] Step 707: Construct a semantic similarity matrix based on the semantic similarity of each candidate data.

[0087] Step 708: Construct a structural similarity matrix based on the structural similarity of each candidate data.

[0088] Step 709: Perform a weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity of each candidate data, and use the comprehensive similarity matrix as the similarity determination result.

[0089] Step 710: Based on the similarity determination results, determine the data that corresponds to the target data from multiple candidate data.

[0090] In one embodiment, a data processing method is provided, such as Figure 8 As shown, the process involves obtaining associated data with the target data from the first business system to generate a first concept tree, a first association graph, and a first data attribute matrix; obtaining associated data with the candidate data from the second business system to generate a second concept tree, a second association graph, and a second data attribute matrix; then inputting the first and second concept trees into a similarity algorithm to obtain a semantic similarity matrix; inputting the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model to obtain a similarity vector, which is then expanded into a structural similarity matrix; finally, the semantic similarity matrix and the structural similarity matrix are weighted and summed; and then, based on a preset threshold, the correspondence between the master data of the first and second business systems, i.e., master data association pairs, is output.

[0091] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0092] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0093] In one embodiment, such as Figure 9 As shown, a data processing device 900 is provided, including: a first determining module 901, a second determining module 902, a third determining module 903, and a fourth determining module 904, wherein:

[0094] The first determining module 901 is used to determine the semantic similarity between the target data in the first business system and each candidate data in the second business system, respectively.

[0095] The second determining module 902 is used to determine the structural similarity between the target data and each candidate data respectively. The structural similarity is used to indicate the similarity between the target data and other data in the first business system and between the candidate data and other data in the second business system.

[0096] The third determining module 903 is used to determine the similarity judgment result based on the semantic similarity and structural similarity corresponding to each candidate data.

[0097] The fourth determining module 904 is used to determine the data that corresponds to the target data from multiple candidate data based on the similarity determination result.

[0098] In one embodiment, the first determining module 901 is specifically configured to: determine first data associated with the target data from the data included in the first business system; generate a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each candidate data, determine second data associated with the candidate data from the data included in the second business system; generate a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; and determine the similarity between the first concept tree and each second concept tree to obtain the semantic similarity between the target data and each candidate data.

[0099] In one embodiment, the second determining module 902 is specifically configured to generate a first association graph based on the association relationship between the target data and each first data, and generate a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data; for each candidate data, generate a second association graph based on the association relationship between the candidate data and each second data, and generate a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data; input the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model, and obtain the structural similarity between the target data and each candidate data based on the output of the graph neural network model.

[0100] In one embodiment, the third determining module 903 is specifically used to perform fusion processing on the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and to determine the similarity judgment result based on the comprehensive similarity corresponding to each candidate data.

[0101] In one embodiment, the third determining module 903 is specifically used to construct a semantic similarity matrix based on the semantic similarity corresponding to each candidate data; construct a structural similarity matrix based on the structural similarity corresponding to each candidate data; perform weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each candidate data; and use the comprehensive similarity matrix as the similarity determination result.

[0102] In one embodiment, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. The fourth determining module 904 is specifically used to determine the candidate data corresponding to the maximum value of each similarity as data that has a corresponding relationship with the target data if the maximum value of the similarity is greater than a preset threshold.

[0103] In one embodiment, the first determining module 901 is specifically used to determine the similarity between the first concept tree and each of the second concept trees according to a similarity algorithm, wherein the similarity algorithm includes at least one of the edit distance algorithm, common substring algorithm, deep learning model or text embedding model.

[0104] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0105] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data processing method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0106] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0107] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: determining the semantic similarity between target data in a first business system and each candidate data in a second business system; determining the structural similarity between the target data and each candidate data, wherein the structural similarity is used to indicate the similarity between the target data and other data in the first business system and the similarity between the candidate data and other data in the second business system; determining a similarity judgment result based on the semantic similarity and structural similarity corresponding to each candidate data; and determining data that corresponds to the target data from multiple candidate data based on the similarity judgment result.

[0108] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining first data associated with the target data from the data included in the first business system, and generating a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each candidate data, determining second data associated with the candidate data from the data included in the second business system, and generating a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; and determining the similarity between the first concept tree and each second concept tree respectively to obtain the semantic similarity between the target data and each candidate data.

[0109] In one embodiment, when the processor executes the computer program, it further performs the following steps: generating a first association graph based on the association relationship between the target data and each first data, and generating a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data; for each candidate data, generating a second association graph based on the association relationship between the candidate data and each second data, and generating a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data; inputting the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model, and obtaining the structural similarity between the target data and each candidate data based on the output of the graph neural network model.

[0110] In one embodiment, when the processor executes the computer program, it further performs the following steps: fusing the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and determining the similarity judgment result based on the comprehensive similarity corresponding to each candidate data.

[0111] In one embodiment, when the processor executes the computer program, it further performs the following steps: constructing a semantic similarity matrix based on the semantic similarity corresponding to each candidate data; constructing a structural similarity matrix based on the structural similarity corresponding to each candidate data; performing a weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each candidate data; and using the comprehensive similarity matrix as the similarity determination result.

[0112] In one embodiment, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. When the processor executes the computer program, it further implements the following steps: if the maximum value among the similarities is greater than a preset threshold, the candidate data corresponding to the maximum value among the similarities is determined as data that has a corresponding relationship with the target data.

[0113] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the similarity between the first concept tree and each of the second concept trees according to a similarity algorithm, wherein the similarity algorithm includes at least one of an edit distance algorithm, a common substring algorithm, a deep learning model, or a text embedding model.

[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: determining the semantic similarity between target data in a first business system and each candidate data in a second business system; determining the structural similarity between the target data and each candidate data, wherein the structural similarity is used to indicate the similarity between the target data and other data in the first business system and the similarity between the candidate data and other data in the second business system; determining a similarity judgment result based on the semantic similarity and structural similarity corresponding to each candidate data; and determining data that corresponds to the target data from multiple candidate data based on the similarity judgment result.

[0115] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: determining first data associated with the target data from the data included in the first business system, and generating a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each candidate data, determining second data associated with the candidate data from the data included in the second business system, and generating a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; and determining the similarity between the first concept tree and each second concept tree to obtain the semantic similarity between the target data and each candidate data.

[0116] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: generating a first association graph based on the association relationship between the target data and each first data, and generating a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data; for each candidate data, generating a second association graph based on the association relationship between the candidate data and each second data, and generating a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data; inputting the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model, and obtaining the structural similarity between the target data and each candidate data based on the output of the graph neural network model.

[0117] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: fusing the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and determining the similarity judgment result based on the comprehensive similarity corresponding to each candidate data.

[0118] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing a semantic similarity matrix based on the semantic similarity corresponding to each candidate data; constructing a structural similarity matrix based on the structural similarity corresponding to each candidate data; performing a weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each candidate data; and using the comprehensive similarity matrix as the similarity determination result.

[0119] In one embodiment, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. When the computer program is executed by the processor, it further implements the following steps: if the maximum value among the similarities is greater than a preset threshold, the candidate data corresponding to the maximum value among the similarities is determined as data that has a corresponding relationship with the target data.

[0120] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the similarity between the first concept tree and each of the second concept trees according to a similarity algorithm, wherein the similarity algorithm includes at least one of an edit distance algorithm, a common substring algorithm, a deep learning model, or a text embedding model.

[0121] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: determining the semantic similarity between target data in a first business system and each candidate data in a second business system; determining the structural similarity between the target data and each candidate data, wherein the structural similarity is used to indicate the similarity between the target data and other data in the first business system and the similarity between the candidate data and other data in the second business system; determining a similarity judgment result based on the semantic and structural similarity corresponding to each candidate data; and determining data that corresponds to the target data from multiple candidate data based on the similarity judgment result.

[0122] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: determining first data associated with the target data from the data included in the first business system, and generating a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each candidate data, determining second data associated with the candidate data from the data included in the second business system, and generating a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; and determining the similarity between the first concept tree and each second concept tree to obtain the semantic similarity between the target data and each candidate data.

[0123] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: generating a first association graph based on the association relationship between the target data and each first data, and generating a first data attribute matrix corresponding to the first association graph based on the data content of the target data and the data content of each first data; for each candidate data, generating a second association graph based on the association relationship between the candidate data and each second data, and generating a second data attribute matrix corresponding to the second association graph based on the data content of the candidate data and the data content of each second data; inputting the first association graph, the first data attribute matrix, each second association graph, and each second data attribute matrix into a graph neural network model, and obtaining the structural similarity between the target data and each candidate data based on the output of the graph neural network model.

[0124] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: fusing the semantic similarity and structural similarity corresponding to each candidate data to obtain the comprehensive similarity corresponding to each candidate data, and determining the similarity judgment result based on the comprehensive similarity corresponding to each candidate data.

[0125] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing a semantic similarity matrix based on the semantic similarity corresponding to each candidate data; constructing a structural similarity matrix based on the structural similarity corresponding to each candidate data; performing a weighted summation on the semantic similarity matrix and the structural similarity matrix to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each candidate data; and using the comprehensive similarity matrix as the similarity determination result.

[0126] In one embodiment, the comprehensive similarity matrix includes multiple similarities corresponding to each candidate data. When the computer program is executed by the processor, it further implements the following steps: if the maximum value among the similarities is greater than a preset threshold, the candidate data corresponding to the maximum value among the similarities is determined as data that has a corresponding relationship with the target data.

[0127] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the similarity between the first concept tree and each of the second concept trees according to a similarity algorithm, wherein the similarity algorithm includes at least one of an edit distance algorithm, a common substring algorithm, a deep learning model, or a text embedding model.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0130] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized by, The method comprises: respectively determining semantic similarity between target data in a first business system and each candidate data in a second business system; the semantic similarity is text similarity; respectively determining structural similarity between the target data and each of the candidate data, the structural similarity being used to indicate similarity between an association relationship of the target data with other data in the first business system and an association relationship of the candidate data with other data in the second business system; determining a similarity determination result according to the semantic similarity and the structural similarity corresponding to each of the candidate data; determining data having a corresponding relationship with the target data from a plurality of the candidate data according to the similarity determination result; the respective determination of semantic similarity between target data in a first business system and each candidate data in a second business system comprises: determining first data associated with the target data from data included in the first business system, and generating a first concept tree corresponding to the target data and the first data according to a business structure of the first business system; for each of the candidate data, determining second data associated with the candidate data from data included in the second business system, and generating a second concept tree corresponding to the candidate data and the second data according to a business structure of the second business system; respectively determining similarity between the first concept tree and each of the second concept trees to obtain semantic similarity between the target data and each of the candidate data; the respective determination of structural similarity between the target data and each of the candidate data comprises: generating a first association relationship graph according to an association relationship between the target data and each of the first data, and generating a first data attribute matrix corresponding to the first association relationship graph according to data content of the target data and data content of each of the first data; for each of the candidate data, generating a second association relationship graph according to an association relationship between the candidate data and each of the second data, and generating a second data attribute matrix corresponding to the second association relationship graph according to data content of the candidate data and data content of each of the second data; inputting the first association relationship graph, the first data attribute matrix, each of the second association relationship graphs and each of the second data attribute matrices into a graph neural network model, and obtaining structural similarity between the target data and each of the candidate data according to an output of the graph neural network model.

2. The method of claim 1, wherein, the determination of a similarity determination result according to the semantic similarity and the structural similarity corresponding to each of the candidate data comprises: performing fusion processing on the semantic similarity and the structural similarity corresponding to each of the candidate data to obtain comprehensive similarity corresponding to each of the candidate data, and determining the similarity determination result according to the comprehensive similarity corresponding to each of the candidate data.

3. The method of claim 2, wherein, The semantic similarity and the structural similarity corresponding to each of the candidate data are fused to obtain a comprehensive similarity corresponding to each of the candidate data, and the similarity determination result is determined according to the comprehensive similarity corresponding to each of the candidate data, and the similarity determination result is determined according to the comprehensive similarity corresponding to each of the candidate data, including: A semantic similarity matrix is constructed according to the semantic similarity corresponding to each of the candidate data; A structural similarity matrix is constructed according to the structural similarity corresponding to each of the candidate data; The semantic similarity matrix and the structural similarity matrix are weighted and summed to obtain a comprehensive similarity matrix composed of the comprehensive similarity corresponding to each of the candidate data, and the comprehensive similarity matrix is taken as the similarity determination result.

4. The method of claim 3, wherein, The comprehensive similarity matrix includes a plurality of similarities corresponding to each of the candidate data, and the data having a corresponding relationship with the target data is determined from the plurality of candidate data according to the similarity determination result, including: If the maximum value in the similarity is greater than a preset threshold, the candidate data corresponding to the maximum value in each of the similarities is determined as the data having a corresponding relationship with the target data.

5. The method of claim 1, wherein, The similarity between the first concept tree and each of the second concept trees is determined respectively, including: The similarity between the first concept tree and each of the second concept trees is determined according to a similarity algorithm, and the similarity algorithm includes at least one of an edit distance algorithm, a common substring algorithm, a deep learning model or a text embedding model.

6. A data processing apparatus, characterized by, The device includes: A first determination module is configured to determine the semantic similarity between the target data in the first business system and each of the candidate data in the second business system respectively; the semantic similarity is a text similarity; A second determination module is configured to determine the structural similarity between the target data and each of the candidate data, and the structural similarity is used to indicate the similarity between the association relationship of the target data and other data in the first business system and the association relationship of the candidate data and other data in the second business system; A third determination module is configured to determine a similarity determination result according to the semantic similarity and the structural similarity corresponding to each of the candidate data; A fourth determination module is configured to determine the data having a corresponding relationship with the target data from the plurality of candidate data according to the similarity determination result; The first determination module is specifically configured to determine first data associated with the target data from data included in the first business system, and generate a first concept tree corresponding to the target data and the first data according to the business structure of the first business system; for each of the candidate data, determine second data associated with the candidate data from data included in the second business system, and generate a second concept tree corresponding to the candidate data and the second data according to the business structure of the second business system; the similarity between the first concept tree and each of the second concept trees is determined respectively to obtain the semantic similarity between the target data and each of the candidate data. The second determining module is specifically configured to generate a first association graph according to the association relationship between the target data and each first data, and generate a first data attribute matrix corresponding to the first association graph according to the data content of the target data and the data content of each first data; for each candidate data, generate a second association graph according to the association relationship between the candidate data and each second data, and generate a second data attribute matrix corresponding to the second association graph according to the data content of the candidate data and the data content of each second data; input the first association graph, the first data attribute matrix, each second association graph and each second data attribute matrix into a graph neural network model, and obtain the structural similarity between the target data and each candidate data according to the output of the graph neural network model.

7. The apparatus of claim 6, wherein, The third determining module is specifically configured to fuse the semantic similarity and the structural similarity corresponding to each candidate data to obtain a comprehensive similarity corresponding to each candidate data, and determine the similarity determination result according to the comprehensive similarity corresponding to each candidate data.

8. The apparatus of claim 6, wherein, The first determining module is specifically configured to determine the similarity between the first concept tree and each second concept tree according to a similarity algorithm, and the similarity algorithm includes at least one of an edit distance algorithm, a common substring algorithm, a deep learning model or a text embedding model. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the method in any one of claims 1 to 5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1 to 5.

Images