Data fusion method, control system and medium

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides a data fusion method, a control system and a medium, and the method comprises the steps: obtaining newly-added data of a first data table; and searching in a second data table based on a preset matching rule according to the newly added data to obtain associated data. According to the data fusion method, the data fusion speed can be greatly increased.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically provides a data fusion method, a control system and a medium. Background Art

[0002] Conventional data fusion requires people to create different types of data fusion tasks during the implementation process to build a data warehouse based on various types of data according to the business. In industrial scenarios, the data involved is asset-centric, including time series data of equipment assets, P&ID drawing data, image and video data. For each new data uploaded, the data needs to be parsed and then associated with the existing data, so the data fusion process is a labor-intensive implementation process.

[0003] The object of data fusion operation can be considered as multiple data tables, which contain at least one identical item. The data fusion process searches based on the identical item. The fusion process is generally implemented based on SQL, such as extracting matching content from multiple data tables for Join operation. Assuming there are N instances, each instance can be time series data, drawings or images. For a newly added instance, if it is compared in sequence, it needs to be compared N times; the current fusion method is that the first table has N records and the second table has N records. A complete comparison is N*N comparisons. If three tables are compared, the third table also has N records, and a complete comparison is N*N*N times. This operation is a complete comparison, comparing all possible parts. After these SQLs are built, some tasks will be set to execute these SQLs regularly, so as to continuously merge the data of multiple original tables.

[0004] The existing data fusion method is to make a complete comparison of all possibilities based on the existing data, which is time-consuming and labor-intensive.

[0005] In view of this situation, it is necessary to develop a data fusion method that can quickly and efficiently associate data. Summary of the invention

[0006] In order to overcome the above defects, the present invention proposes a data fusion method, a control system and a medium, which can greatly improve the speed of data fusion.

[0007] In a first aspect, the present invention provides a method for data fusion, comprising:

[0008] Obtaining the newly added data to be associated in the first data table;

[0009] According to the newly added data to be associated, searching in the second data table based on a preset matching rule to obtain data most similar to the newly added data to be associated;

[0010] It is determined whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold. If so, the most similar data is associated data of the newly added data to be associated.

[0011] In a technical solution of the above data fusion method, searching in the second data table based on the newly added data to be associated based on a preset matching rule to obtain the data most similar to the newly added data to be associated includes:

[0012] S210, obtaining a group of data in the second data table, and obtaining a group of data that is most similar to the newly added data to be associated according to a first preset matching rule;

[0013] S220, obtaining the number of data included in the group containing the data most similar to the newly added data to be associated, and if the number is greater than 1, further grouping the group, and obtaining the group containing the data most similar to the newly added data to be associated after grouping according to the second preset matching rule;

[0014] S220 is repeatedly executed until the data most similar to the newly added data to be associated is obtained.

[0015] In a technical solution of the above data fusion method, the determining whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold comprises:

[0016] Obtaining the distance between the most similar data encoding vector and the newly added data encoding vector to be associated;

[0017] It is determined whether the distance is less than a preset threshold.

[0018] In a technical solution of the above data fusion method, if the condition is satisfied, the most similar data is the associated data of the newly added data to be associated, including:

[0019] If the distance is less than a preset threshold, the most similar data is associated data of the newly added data to be associated.

[0020] In a technical solution of the above data fusion method, obtaining the group of data that is most similar to the newly added data to be associated according to the first preset matching rule includes:

[0021] Get the cluster center of each group of data after grouping;

[0022] Obtain the distance D between each cluster center and the newly added data encoding vector respectively;

[0023] Get the maximum distance R between each group of data and the cluster center of the group of data respectively;

[0024] Obtaining the distance B between the cluster center of the data in the second data table and the encoding vector of the newly added data to be associated;

[0025] Determine whether the sum of each group B and R after grouping is greater than the corresponding D;

[0026] If it is greater, then this group is the group containing the data most similar to the newly added data to be associated.

[0027] In a technical solution of the above data fusion method, the group of data that is most similar to the newly added data to be associated after being grouped according to the second preset matching rule includes:

[0028] Get the cluster center of each group of data after grouping;

[0029] Obtain the distance D between the cluster center of each group of data and the encoding vector of the newly added data to be associated;

[0030] Get the maximum distance R between each group of data and the cluster center of the group of data respectively;

[0031] Obtain the distance B between the cluster center of the group of data that is most similar to the newly added data to be associated before grouping and the encoding vector of the newly added data to be associated;

[0032] Calculate whether the sum of B and R in each group after grouping is greater than D;

[0033] If it is greater, then this group is the group containing the data most similar to the newly added data to be associated.

[0034] In a technical solution of the above data fusion method, after the step of obtaining the newly added data of the first data table, the method further includes:

[0035] Obtaining a coding vector of the newly added data to be associated;

[0036] Get the encoding vector of the second data table data.

[0037] In a technical solution of the above data fusion method, obtaining the encoding vector of the newly added data to be associated includes:

[0038] Obtain the character string of the newly added data to be associated;

[0039] Encoding the character string to obtain an encoding vector of the newly added data to be associated;

[0040] The step of obtaining the encoding vector of the second data table data comprises:

[0041] Respectively obtain character strings of the second data table data;

[0042] The character strings are respectively encoded to obtain encoding vectors of the second data table data.

[0043] In a technical solution of the above-mentioned data fusion method, the newly added data to be associated includes newly added equipment data, P&ID drawings, images or videos.

[0044] In a second aspect, a control device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the data fusion method described in any one of the technical solutions of the above-mentioned data fusion method.

[0045] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the data fusion method described in any one of the technical solutions of the above-mentioned data fusion method.

[0046] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0047] In the technical solution of the present invention, the present invention performs data fusion on each newly added data to be associated, and realizes the association of the newly added data. Compared with the existing fusion method of complete comparison, the data fusion efficiency is greatly improved.

[0048] In the technical solution for implementing the present invention, the present invention determines the similarity of two data through search based on preset matching rules. After completing the search, the most similar data corresponding to the newly added data to be associated will be associated, and there is no need to manually create a data fusion task for association. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0050] Figure 1 is a flow chart of main steps of a method for data fusion according to an embodiment of the present invention;

[0051] Figure 2 is a flowchart of the main steps of searching in the second data table based on the newly added data to be associated based on a preset matching rule to obtain data most similar to the newly added data to be associated according to an embodiment of the present invention;

[0052] Figure 3 is a flowchart of main steps for obtaining a group of data that is most similar to the newly added data to be associated according to a first preset matching rule according to an embodiment of the present invention;

[0053] Figure 4 It is a flowchart of the main steps of obtaining the group of data that is most similar to the newly added data to be associated after grouping according to the second preset matching rule according to one embodiment of the present invention;

[0054] Figure 5 is a flowchart of main steps for determining whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold according to an embodiment of the present invention;

[0055] Figure 6 It is a flowchart of main steps of, if the conditions are met, the most similar data is the associated data of the newly added data to be associated according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0057] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, and the like. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and may include only A, only B, or A and B. The singular terms "one" and "the" may also include plural forms.

[0058] In the prior art, an application scenario is used as an example. The first data table A has 100 data, and the second data table B has 500 data. When the two data tables are merged, each data in the first data table needs to be compared with the 500 data in the second data table, and each data needs to be compared 500 times. When the 100 data are completely merged, all of them need to be compared to find the data corresponding to each data in the first data table, a total of 500×100 comparisons. Obviously, this workload is huge.

[0059] Table A is the basic attribute data of the equipment, including the equipment ID, name, purchase date, model and manufacturer of different equipment. Table B is the equipment operation data, specifically the data parameter data monitored by sensors in real time when different equipment is running, including the first sensor parameters, the second sensor parameters and the third sensor parameters.

[0060] Table A Basic properties of equipment

[0061] Device ID name Purchase Date model Manufacturer 001 N001 T001 L001 S001 002 N002 T002 L002 S002 003 N003 T003 L003 S003 ...... ...... ...... ...... ...... 099 N099 T099 L099 S099 100 N100 T100 L100 S100

[0062] Table B Equipment operation data (real-time data detected by sensors)

[0063]

[0064] In the prior art, the method for associating the data in Table A and Table B is:

[0065] To associate the data with the same ID in Table A and Table B, we create SQL: t1.id = t2.id. Assuming that 200 associated data can be obtained, we associate the entire data of Table A and Table B and write SQL statements, such as SQL: tA.id = tB.id. After association, see Table C. After association, there are 7 types of data, including name, purchase date, model, manufacturer, first sensor parameter, second sensor parameter, and third sensor parameter, as shown in Table C.

[0066] Table C

[0067]

[0068] The existing data association is to associate batches of data. In reality, new data will be continuously inserted. For the newly added data device ID 135:

[0069] Device ID name Purchase Date model Manufacturer 135 N135 T135 L135 S135

[0070] The existing technology is to add it to the data table first. At this time, there are 101 data in Table A. Since the data is not inserted in order, the newly added data device ID 135 will be randomly inserted into the first 100 data. In this way, 201 data in Table A need to be re-associated. When each data is associated, it needs to be compared 500 times, and the device ID of the 500 data in Table B needs to be found.

[0071] The present invention substantially changes the method of the prior art of complete possibility comparison. The present invention abandons the existing method of complete possibility comparison of batch and static data, and adopts dynamic and real-time data fusion for each new data, and no longer needs to manually create data fusion tasks for association, so that the data in the first data table has associated data after data fusion.

[0072] Reference Figure 1 The present invention provides a data fusion method, comprising:

[0073] S1, obtaining the newly added data to be associated in the first data table;

[0074] S2, searching in the second data table according to the newly added data to be associated based on a preset matching rule to obtain data most similar to the newly added data to be associated;

[0075] S3, judging whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold; if so, the most similar data is associated data of the newly added data to be associated.

[0076] In the present invention, the most similar data includes the same data and similar data. For example, the device ID 135 in Table A and the device ID 135 in Table B are the same data, and the device ID 135 in Table A and the device ID 134 in Table B are similar data.

[0077] In one embodiment, obtaining the newly added data to be associated in the first data table includes:

[0078] The newly added data in the first data table is obtained in real time and associated. The data is merged before the newly added data is added to the first data table, so that the data in the first data table can be merged in time to achieve data association. Since the newly added data is associated before being inserted into the first data table, the newly added data in the first data table is recorded as newly added data to be associated.

[0079] If the new data is not fused in time, and the fusion is performed after the new data is added to the first data table, the fusion workload will be greatly increased. This is because even if the existing data in the first data table are all fused data, the addition of the new data will be randomly inserted into the existing data, and the new data added to the first data table cannot be fused separately. All the data in the first data table, including the existing fused data and the unfused new data, still need to be fused. This is time-consuming and labor-intensive and causes repeated operations.

[0080] Reference Figure 2 , in S2, according to the newly added data to be associated, searching in the second data table based on a preset matching rule, obtaining the data most similar to the newly added data to be associated includes:

[0081] S21, obtaining a group of data in the second data table, and obtaining a group of data that is most similar to the newly added data to be associated according to a first preset matching rule;

[0082] S22, obtaining the number of data included in the group containing the data most similar to the newly added data to be associated, and if the number is greater than 1, further grouping the group, and obtaining the group containing the data most similar to the newly added data to be associated after grouping according to the second preset matching rule;

[0083] S22 is repeatedly executed until the data most similar to the newly added data to be associated is obtained.

[0084] The method of obtaining the number of data contained in the group containing the data most similar to the newly added data to be associated, if it is greater than 1, then continuing to group the group as follows: calculating the number of data contained in each grouping, the purpose of grouping is to group until there is only one data, that is, to the last layer.

[0085] The data most similar to the newly added data to be associated is obtained, and the data in the second data table that is most similar to the newly added data to be associated is found. For a data table, there must be the most similar data in the entire data. By continuously grouping and reducing the data in the group, the data most similar to the newly added data to be associated is found in the second data table.

[0086] In one embodiment, referring to Figure 3 The group of data that is most similar to the newly added data to be associated obtained according to the first preset matching rule in S21 includes:

[0087] S211, obtaining the cluster center of each group of data after grouping;

[0088] S212, respectively obtaining the distance D between each cluster center and the newly added data encoding vector;

[0089] S213, respectively obtaining the maximum distance R between each group of data and the cluster center of the group of data;

[0090] S214, obtaining a distance B between the cluster center of the data in the second data table and the encoding vector of the newly added data to be associated;

[0091] S215, determining whether the sum of each group B and R after grouping is greater than the corresponding D;

[0092] S216: If it is greater than, the group is the group containing the data most similar to the newly added data to be associated.

[0093] Application Example 1: The technical route of the present invention is: group N vectors, each group has a virtual vector representing the center point of this group, and compare the vector of any newly added instance X with the virtual center point of each group to determine whether the vector most similar to X is in this group (judged according to condition 1). If it is in this group, continue to repeat the above steps in the subgroups of this group until a matching vector that meets the conditions is found.

[0094] Condition 1:

[0095] M: the center point of the virtual grouping, i.e. the cluster center;

[0096] R: The radius of the group, defined as the maximum distance from all other vectors in the group to M);

[0097] B: The distance between the vector most similar to X and X (the vector most similar to X is represented by the cluster center of the previous grouping);

[0098] D: the distance between X and M;

[0099] If B+R>D, then the vector with a smaller distance to X exists in this group.

[0100] The method of the present invention is described by taking a calculation process as an example. The newly added data to be associated is denoted as X, the cluster center of the entire data in the first data table is denoted as M0, and M0X represents the distance between the cluster center M0 of the entire data in the first data table and the newly added data to be associated X. Other letter combinations are deduced in the same way.

[0101] The first step of the judgment process of the present invention is to obtain the grouping of the data in the second data table, and to start the judgment with M0 as the data most similar to X.

[0102] The first data table is first divided into two groups, namely group 1 and group 2. The parameters of group 1 are recorded as M1, R1, D1, and the parameters of group 2 are recorded as M2, R2, D2;

[0103] Determine whether it is in group 1: whether M0X+R1 is greater than XM1;

[0104] Determine whether it is in group 2: Determine whether M0X+R2 is greater than XM2;

[0105] If M0X+R1 is greater than XM1, it is in the first group.

[0106] Application Example 2:

[0107] The above-mentioned Table A, Table B and newly added data to be associated (device ID 135) are used as an example for description.

[0108] For each new data that is not added to Table A, perform an association.

[0109] The device ID is converted into a code vector and the distance is calculated based on the code vector.

[0110] For a more intuitive purpose, this example uses real numbers to represent the distance between encoding vectors.

[0111] The cluster center of the device ID in Table B is 500 away from the center of the newly added data point in Table A;

[0112] The data in Table B is divided into 5 groups, each with 100 data, and there are 100 cluster centers.

[0113] Table B Equipment operation data (real-time data detected by sensors)

[0114]

[0115]

[0116] The second group calculates the D, R, and B values:

[0117] The distance between the second group of data centers and the newly added data to be associated is D=125;

[0118] The maximum distance between the second group of data and the cluster center of this group of data is R=32;

[0119] Before grouping, the distance B between the center of the group of data most similar to the newly added data to be associated (the center of the entire data in table B) and the newly added data to be associated is 138.

[0120] 138+32>125, satisfying B+R>D, and the data most similar to the newly added data to be associated is in the second group.

[0121] The number of groups and the amount of data contained in each group can be set randomly or as a fixed value.

[0122] In one embodiment, referring to Figure 4 The group of data that is most similar to the newly added data to be associated after being grouped according to the second preset matching rule in S22 includes:

[0123] S221, obtaining the cluster center of each group of data after grouping;

[0124] S222, respectively obtaining the distance D between the cluster center of each group of data and the encoding vector of the newly added data to be associated;

[0125] S223, respectively obtaining the maximum distance R between each group of data and the cluster center of the group of data;

[0126] S224, obtaining the distance B between the cluster center of the group of data that is most similar to the newly added data to be associated before grouping and the encoding vector of the newly added data to be associated;

[0127] S225, respectively calculating whether the sum of B and R in each group after grouping is greater than D;

[0128] S226: If it is greater than, then the group is the group containing the data most similar to the newly added data to be associated.

[0129] Application Example 1:

[0130] The first group is further divided into two groups, namely the third group and the fourth group. The parameters of the third group are recorded as M3, R3, D3, and the parameters of the fourth group are recorded as M4, R4, D4.

[0131] Determine whether it is in the third group: determine whether M1X+R3 is greater than XM3; Determine whether it is in the fourth group: determine whether M1X+R4 is greater than XM4;

[0132] If M1X+R3 is greater than XM3, then it is in Group 3;

[0133] Group 3 is further divided into two groups, namely Group 5 and Group 6. The parameters of Group 5 are recorded as M5, R5, D5, and the parameters of Group 6 are recorded as M6, R6, D6;

[0134] Continue to determine whether it is in the 5th or 6th group. To determine whether it is in the 5th group: determine whether M3X+R5 is greater than XM5. To determine whether it is in the 6th group: determine whether M3X+R6 is greater than XM6.

[0135] If M3X+R6 is greater than XM6, the most similar data is the sixth group of data. If the sixth group of data is a data point, then this data point is the data most similar to X, recorded as Y.

[0136] Application Example 2:

[0137] Continue to group and judge in the second group, divide into 2 groups, 50 data in each group:

[0138]

[0139] Group 1 calculates the D, R, and B values:

[0140] The distance between the first group of data centers and the newly added data to be associated is D=101;

[0141] The maximum distance between the first group of data and the cluster center of this group of data is R=23;

[0142] Before grouping, the distance between the center of the group of data most similar to the newly added data to be associated (the data center of the second group) and the newly added data to be associated is B=106.

[0143] 106+23>101, satisfying B+R>D, and the data most similar to the newly added data to be associated is in the first group.

[0144] Continue to group the first group into 2 groups, with 25 data in each group

[0145]

[0146] Group (2) calculates the D, R, and B values:

[0147] The distance between the data center of the second group and the newly added data to be associated is D=101;

[0148] The maximum distance between the second group of data and the cluster center of this group of data is R=23;

[0149] The distance between the center of the group of data that is most similar to the newly added data to be associated before grouping (the data center of the first group) and the newly added data to be associated is B = 106

[0150] 106+23>101, satisfying B+R>D, and the data most similar to the newly added data to be associated is in group (2).

[0151] Continue to group group (2), with 1 data in each group, for a total of 25 groups

[0152]

[0153] Group ⑩ calculates the D, R, and B values:

[0154] The distance between the ⑩th group of data centers and the newly added data to be associated is D=30;

[0155] The maximum distance between the ⑩th group of data and the cluster center of this group of data is R=1;

[0156] Before grouping, the distance between the center of the group of data most similar to the newly added data to be associated (the center of the (2nd) group) and the newly added data to be associated is B=32.

[0157] 32+1>30, satisfying B+R>D, then the data most similar to the newly added data is in the ⑩ group.

[0158] In order to determine whether the most similar data is similar enough, further determination is required. In one embodiment, referring to Figure 5 In S3, the determining whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold comprises:

[0159] S31, obtaining the distance between the most similar data encoding vector and the newly added data encoding vector to be associated;

[0160] S32, determining whether the distance is less than a preset threshold.

[0161] The above text finds the data in the second data table that is most similar to the newly added data to be associated. However, whether this data is similar enough to the newly added data to be associated needs to be judged by S31-S32.

[0162] If the distance between the data most similar to the newly added data to be associated found through continuous grouping and the newly added data to be associated is close enough, indicating that they are similar enough, then the data will be used as associated data; if it is not greater than the preset threshold, it means that the data most similar to the newly added data to be associated is the data most similar to the newly added data to be associated in the second data table, but is still not similar enough to the newly added data to be associated, then it is chosen not to associate the data.

[0163] Reference Figure 6 In one embodiment, in S3, if the condition that the most similar data is the associated data of the newly added data to be associated includes:

[0164] S33: If the distance is less than a preset threshold, the most similar data is associated data of the newly added data to be associated.

[0165] Application Example 1:

[0166] Find the most similar data Y and calculate the XY distance. If the XY distance is less than the threshold, the data is associated. If the XY distance is greater than the threshold, the data cannot be associated.

[0167] Application example 2: The threshold of D is set to 35, and 30 is less than 35. Then the data in group ⑩ is the associated data of the newly added data to be associated.

[0168] If the threshold of D is set to 25, then 30 is greater than 25, and the data in the ⑩th group is not associated data of the newly added data to be associated.

[0169] Assuming that the device ID of the data found that is most similar to the newly added data to be associated is ID134.5, then the vector distance between the ID135 data and the newly added data to be associated ID135 is calculated, and the distance is 4. If the preset threshold is 5, the ID134.5 data in the second data table can be associated with the newly added data to be associated ID135. If the preset threshold is 2, the ID134.5 data in the second data table cannot be associated with the newly added data to be associated ID135.

[0170] In one embodiment, after the step of obtaining the newly added data of the first data table, the method further includes:

[0171] Obtaining a coding vector of the newly added data to be associated;

[0172] Get the encoding vector of the second data table data.

[0173] In one embodiment, obtaining the encoding vector of the newly added data to be associated includes:

[0174] Obtain the character string of the newly added data to be associated;

[0175] Encoding the character string to obtain an encoding vector of the newly added data to be associated;

[0176] The step of obtaining the encoding vector of the second data table data comprises:

[0177] Respectively obtain character strings of the second data table data;

[0178] The character strings are respectively encoded to obtain encoding vectors of the second data table data.

[0179] Each data is represented by a code, which can be a vector. The similarity between two vectors is defined as the distance between the two vectors.

[0180] The distance calculation can be obtained using Euclidean distance.

[0181] The cluster centers of the data set can be obtained using the kmeans algorithm.

[0182] In one embodiment, the newly added data to be associated includes newly added equipment data, P&ID drawings, images or videos.

[0183] The newly added device data includes device ID, device information parameters, device attribute parameters, etc.

[0184] In industrial scenarios, the data involved is asset-centric, and the so-called assets are equipment.

[0185] In view of the fact that industrial data is asset-centric, the present invention automatically completes the data fusion process based on search. By setting matching fields, such as the device ID in time series data, the tag recognized by OCR in P&ID drawings, and the label recognized by images or videos, ID, tag, and label are uniformly considered as variable-length strings and encoded by embedding.

[0186] In industrial scenarios, the data involved is asset-centric, and the so-called assets are equipment.

[0187] In view of the fact that industrial data is asset-centric, the present invention automatically completes the data fusion process based on search. By setting matching fields, such as the device ID in time series data, the tag recognized by OCR in P&ID drawings, and the label recognized by images or videos, ID, tag, and label are uniformly considered as variable-length strings and encoded by embedding.

[0188] The present invention determines the similarity of two data by searching based on the distance between vectors and the condition B+R>D. After the search is completed, the data corresponding to the newly added data X and the data corresponding to the most similar vector found will be associated, and there is no need to manually create a data fusion task for association. Finally, multiple different P&ID drawings can be associated; by selecting a device in a drawing or image, the data of the device can be automatically extracted.

[0189] The method of the present invention accelerates the data fusion process through similarity search. Users no longer need to write SQL, but instead perform comparisons through preset matching rules. By adopting a method of comparing each newly added data, the fusion speed of the present invention can be greatly improved compared to the prior art of performing a complete comparison of all possibilities.

[0190] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0191] Furthermore, the present invention also provides a control device. In an embodiment of a control device according to the present invention, the control device includes a processor and a storage device, the storage device can be configured to store a program for executing the data fusion method of the above method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the data fusion method of the above method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The control device can be a control device device formed by various electronic devices.

[0192] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the data fusion method of the above method embodiment, and the program can be loaded and run by the processor to implement the above data fusion method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0193] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0194] Those skilled in the art will appreciate that the modules in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present invention.

[0195] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A data fusion method, characterized in that: include: Obtaining the newly added data to be associated in the first data table; According to the newly added data to be associated, searching in the second data table based on a preset matching rule to obtain data most similar to the newly added data to be associated; It is determined whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold. If so, the most similar data is associated data of the newly added data to be associated.

2. The method according to claim 1, characterized in that The searching in the second data table based on the newly added data to be associated based on the preset matching rule to obtain the data most similar to the newly added data to be associated includes: S210, obtaining a group of data in the second data table, and obtaining a group of data that is most similar to the newly added data to be associated according to a first preset matching rule; S220, obtaining the number of data included in the group containing the data most similar to the newly added data to be associated, and if the number is greater than 1, further grouping the group, and obtaining the group containing the data most similar to the newly added data to be associated after grouping according to the second preset matching rule; S220 is repeatedly executed until the data most similar to the newly added data to be associated is obtained.

3. The method according to claim 2, characterized in that The determining whether the similarity between the newly added data to be associated and the most similar data meets a preset threshold comprises: Obtaining the distance between the most similar data encoding vector and the newly added data encoding vector to be associated; It is determined whether the distance is less than a preset threshold.

4. The method according to claim 3, characterized in that If the condition is satisfied, the most similar data is the associated data of the newly added data to be associated, including: If the distance is less than a preset threshold, the most similar data is associated data of the newly added data to be associated.

5. The method according to claim 2, characterized in that: The group of data that is most similar to the newly added data to be associated obtained according to the first preset matching rule includes: Get the cluster center of each group of data after grouping; Obtain the distance D between each cluster center and the newly added data encoding vector respectively; Get the maximum distance R between each group of data and the cluster center of the group of data respectively; Obtaining the distance B between the cluster center of the data in the second data table and the encoding vector of the newly added data to be associated; Determine whether the sum of each group B and R after grouping is greater than the corresponding D; If it is greater, then this group is the group containing the data most similar to the newly added data to be associated.

6. The method according to claim 3, characterized in that The group of data that is most similar to the newly added data to be associated after being grouped according to the second preset matching rule includes: Get the cluster center of each group of data after grouping; Obtain the distance D between the cluster center of each group of data and the encoding vector of the newly added data to be associated; Get the maximum distance R between each group of data and the cluster center of the group of data respectively; Obtain the distance B between the cluster center of the group of data that is most similar to the newly added data to be associated before grouping and the encoding vector of the newly added data to be associated; Calculate whether the sum of B and R in each group after grouping is greater than D; If it is greater, then this group is the group containing the data most similar to the newly added data to be associated.

7. The method according to claim 1, characterized in that After the step of obtaining the newly added data of the first data table, the method further includes: Obtaining a coding vector of the newly added data to be associated; Get the encoding vector of the second data table data.

8. The method according to claim 7, characterized in that The step of obtaining the encoding vector of the newly added data to be associated includes: Obtain the character string of the newly added data to be associated; Encoding the character string to obtain an encoding vector of the newly added data to be associated; The step of obtaining the encoding vector of the second data table data comprises: Respectively obtain character strings of the second data table data; The character strings are respectively encoded to obtain encoding vectors of the second data table data.

9. The method according to claim 1, characterized in that: The newly added data to be associated includes newly added equipment data, P&ID drawings, images or videos.

10. A control device, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and executed by the processor to execute the data fusion method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the data fusion method according to any one of claims 1 to 9.

Images