A conflict data positioning method, device, equipment and readable storage medium

Abstract

This application discloses a method, apparatus, device, and readable storage medium for locating conflict data. The method includes: pre-determining multiple levels of conflict categories based on the data of each historical user on various platforms; determining the conflict category to which the current data belongs on each platform based on the current user's current data on each platform and the center of each level of conflict category; comparing the level of the conflict category to which the current data belongs on each platform with a preset level, and identifying the current data on platforms whose conflict category level is higher than the preset level as conflict data. The above technical solution disclosed in this application, by determining multiple levels of conflict categories based on historical user data, determining the conflict category to which the current user's data belongs on each platform, and determining the current user's conflict data based on the level of the conflict category, achieves automatic location of the current user's conflict data across multiple platforms, improving the efficiency and accuracy of conflict data location.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, device, and readable storage medium for locating conflict data. Background Technology

[0002] User data from across departments is typically multi-typed and uncertain, and inconsistencies arise due to the unique recording methods of different platforms. Digital information conflict is an unavoidable consequence of cross-departmental service environments and a significant obstacle to the unification and standardization of digital information.

[0003] Currently, for cross-platform user data, conflicting data is typically identified manually through comparison and other methods to facilitate data processing and minimize inconsistencies, thereby improving the quality of cross-platform data integration. However, manually identifying conflicting data results in low efficiency and accuracy.

[0004] In conclusion, improving the accuracy and efficiency of conflict data location across platforms is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a conflict data location method, apparatus, device and readable storage medium to improve the accuracy and efficiency of conflict data location across platforms.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A conflict data location method, comprising:

[0008] Multiple levels of conflict categories are determined in advance based on the data of each of the multiple historical users on each platform;

[0009] Based on the current user's current data on each platform and the center of each level of the conflict class, determine the conflict class to which the current data on each platform belongs;

[0010] The conflict class level of the current data on each of the platforms is compared with a preset level, and the current data on the platform whose conflict class level is higher than the preset level is identified as conflict data.

[0011] Preferably, multiple levels of conflict categories are determined in advance based on the data of each historical user on each platform, including:

[0012] The distance between the data of each historical user on the j-th platform and the data on the other platforms is pre-calculated to obtain the distance group corresponding to the data of each historical user on the j-th platform; where j = 1, 2, ..., m, and m is the number of platforms;

[0013] Based on the distance groups corresponding to the data of each historical user on each of the platforms, a clustering algorithm is used to obtain multiple levels of conflict classes;

[0014] Based on the current user's data on each platform and the center of each conflict class at each level, determine the conflict class to which the current data on each platform belongs, including:

[0015] Calculate the distance between the current data of the current user on the j-th platform and the current data on the other platforms to obtain the distance group corresponding to the current data of the current user on the j-th platform;

[0016] Calculate the distance between the distance group corresponding to the current data of the current user on the j-th platform and the center of the conflict class at each level, and determine the conflict class corresponding to the minimum distance as the conflict class to which the current data of the current user on the j-th platform belongs.

[0017] Preferably, after identifying current data in platforms whose conflict category level is higher than a preset level as conflict data, the method further includes:

[0018] Extract a first data attribute table from each of the conflicting data of the current user, and extract a second data attribute table from any of the non-conflicting data of the current user;

[0019] Compare the attribute fields in each of the first data attribute tables with the corresponding attribute fields in the second data attribute table;

[0020] If there is an attribute field in the first data attribute table that is inconsistent with the corresponding attribute field in the second data attribute table, then the first data attribute table that is inconsistent with the corresponding attribute field in the second data attribute table is determined to be a data attribute table with attribute conflict.

[0021] Determine the time when conflicting data was generated based on the attribute table of data with conflicting attributes.

[0022] Preferred options also include:

[0023] If the attribute field in the first data attribute table is consistent with the corresponding attribute field in the second data attribute table, then calculate the similarity between the attribute field in the first data attribute table and the corresponding attribute field in the second data attribute table.

[0024] Determine whether the similarity is greater than a threshold. If not, determine that there is an attribute field conflict between the first data attribute table and the second data attribute table, and determine the first data attribute table that has an attribute field conflict with the second data attribute table as the data attribute table with attribute conflict.

[0025] The time of conflict is determined based on the data attribute table where there are attribute conflicts.

[0026] Preferably, calculating the similarity between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table includes:

[0027] Calculate the similarity between the attribute values ​​of the attribute fields in the first data attribute table and the attribute values ​​of the corresponding attribute fields in the second data attribute table.

[0028] Preferably, calculating the similarity between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table includes:

[0029] The character type semantic similarity between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table is calculated based on the order of the attribute values ​​corresponding to the attribute fields.

[0030] A conflict data location device, comprising:

[0031] The first determining module is used to pre-determine multiple levels of conflict classes based on the data of each of the multiple historical users on each platform;

[0032] The second determining module is used to determine the conflict class to which the current data on each platform belongs based on the current data of the current user on each platform and the center of the conflict class at each level;

[0033] The comparison module is used to compare the level of the conflict class to which the current data belongs on each of the platforms with a preset level, and to determine the current data in platforms whose conflict class level is higher than the preset level as conflict data.

[0034] Preferably, the first determining module includes:

[0035] The first calculation unit is used to pre-calculate the distance between the data of each historical user on the j-th platform and the data on the other platforms, so as to obtain the distance group corresponding to the data of each historical user on the j-th platform; where j = 1, 2, ..., m, and m is the number of platforms;

[0036] The unit is used to obtain multiple levels of conflict classes based on the distance groups corresponding to the data of each historical user on each of the platforms using a clustering algorithm.

[0037] The second determining module includes:

[0038] The second calculation unit is used to calculate the distance between the current data of the current user on the j-th platform and the current data on the other platforms, so as to obtain the distance group corresponding to the current data of the current user on the j-th platform.

[0039] The third calculation unit is used to calculate the distance between the distance group corresponding to the current data of the current user on the j-th platform and the center of the conflict class at each level, and to determine the conflict class corresponding to the minimum distance as the conflict class to which the current data of the current user on the j-th platform belongs.

[0040] A conflict data location device, comprising:

[0041] Memory, used to store computer programs;

[0042] A processor, configured to implement the steps of the conflict data localization method as described in any of the preceding claims when executing the computer program.

[0043] A readable storage medium storing a computer program that, when executed by a processor, implements the steps of the conflict data location method as described in any of the preceding claims.

[0044] This application provides a conflict data location method, apparatus, device, and readable storage medium. The method includes: pre-determining multiple conflict classes based on the data of each historical user on each platform; determining the conflict class to which the current data on each platform belongs based on the current user's current data on each platform and the center of each conflict class; comparing the level of the conflict class to which the current data on each platform belongs with a preset level, and determining the current data on platforms whose conflict class level is higher than the preset level as conflict data.

[0045] The technical solution disclosed in this application determines multiple levels of conflict categories in advance based on the data of each historical user on each platform. It then determines the conflict category of the current data on each platform based on the current user's current data on each platform and the center of each pre-determined conflict category. The current data on platforms whose conflict category level is higher than the preset level is identified as conflict data. This achieves automatic determination of the current user's conflict data on multiple platforms without human intervention, thereby improving the efficiency of conflict data location and avoiding the influence of human factors, thus improving the accuracy of conflict data location. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 A flowchart illustrating a conflict data location method provided in this application embodiment;

[0048] Figure 2 A flowchart illustrating another conflict data location method provided in this application embodiment;

[0049] Figure 3 This is a schematic diagram of the structure of a conflict data location device provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of a conflict data location device provided in an embodiment of this application. Detailed Implementation

[0051] Due to the differences in service recipients across departments, the diversity of service types, and the inconsistency of operational rules within enterprises, it is impossible to establish an absolutely standardized solution for integrating user digital information across service platforms in a cross-departmental environment. This easily leads to inconsistencies in the same attribute digital information data of individuals across different departmental platforms, ultimately resulting in unreliable audits during subsequent service applications. It can be said that digital information conflict is an inevitable product of the cross-departmental service environment and a significant obstacle to the unification and standardization of digital information data. Cross-departmental services refer to the multi-level, multi-layered service departments within an enterprise, each providing comprehensive and high-quality services to users in parallel, due to detailed division of labor. Digital information conflict arises when different departments within an enterprise provide services and collect user data; differences in recording methods, frequency, and evaluation methods lead to significant discrepancies in the representation of similar attributes between different departments. For example, the differences in personal information scoring of the same user's deposit records by different branches of the Bank of China.

[0052] Currently, for cross-platform user data, conflicting data is typically identified manually through comparison and other methods to facilitate data processing and minimize inconsistencies, thereby improving the quality of cross-platform data integration. However, manually identifying conflicting data results in low efficiency and accuracy.

[0053] Therefore, this application provides a conflict data location method, apparatus, device, and readable storage medium to improve the accuracy and efficiency of conflict data location across platforms.

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] See Figure 1 The diagram illustrates a flowchart of a conflict data localization method provided in an embodiment of this application. This conflict data localization method may include:

[0056] S11: Based on the data of each historical user on each platform, determine multiple levels of conflict categories in advance.

[0057] In this application, firstly, the data of each historical user in the cross-department service on each platform of the cross-department service can be obtained in advance. Then, based on the data of each historical user on each platform, multiple levels of conflict categories are determined, and the center of each level of conflict category is determined. The higher the level of the conflict category, the more obvious the conflict between the data on its corresponding platform and the data on other platforms, that is, the higher the conflict.

[0058] For example, conflict classes can be identified at three levels: Level 1, Level 2, and Level 3, with the levels of these three conflict classes increasing sequentially. Of course, conflict classes at other levels can also be identified, but this application does not limit this.

[0059] S12: Based on the current user's current data on each platform and the center of each level of conflict class, determine the conflict class to which the current data on each platform belongs.

[0060] Based on step S11, the current data of the current user on each platform of the cross-department service is obtained. Then, based on the current data of the current user on each platform and the center of each level of conflict class, the conflict class to which the current user's current data on each platform belongs is determined, that is, to determine which level of conflict class the current user's current data on each platform belongs to as determined in step S11.

[0061] S13: Compare the conflict class level of the current data on each platform with the preset level, and identify the current data on platforms whose conflict class level is higher than the preset level as conflict data.

[0062] Based on step S12, the conflict class level of the current user's current data on each platform is compared with the preset level, where the preset level is specifically a level set in advance based on actual experience.

[0063] If the conflict class level of the current user's current data on a certain platform (or platforms) is equal to or lower than the preset level, it indicates that the conflict of the current user's current data on that platform (or platforms) is relatively small (relative to the current user's current data on other platforms), or even almost non-conflicting. In this case, the current data on platforms with a conflict class level equal to or lower than the preset level can be identified as non-conflicting data, and the platform where the non-conflicting data is located can be identified as a non-conflicting platform (for the current user).

[0064] If the conflict class level of the current user's current data on a certain platform (or platforms) is higher than the preset level mentioned above, it indicates that the conflict of the current user's current data on that platform (or platforms) is relatively large. Therefore, the current data on the platform whose conflict class level is higher than the preset level is identified as conflict data, and the platform where the conflict data is located is identified as the conflict platform (relative to the current user).

[0065] The above process can identify which data on various platforms is conflicting and which is not, eliminating the need for manual searching and location. This reduces manpower consumption and costs, and improves the efficiency and accuracy of conflict data location.

[0066] The technical solution disclosed in this application determines multiple levels of conflict categories in advance based on the data of each historical user on each platform. It then determines the conflict category of the current data on each platform based on the current user's current data on each platform and the center of each pre-determined conflict category. The current data on platforms whose conflict category level is higher than the preset level is identified as conflict data. This achieves automatic determination of the current user's conflict data on multiple platforms without human intervention, thereby improving the efficiency of conflict data location and avoiding the influence of human factors, thus improving the accuracy of conflict data location.

[0067] This application provides a conflict data location method that pre-determines multiple levels of conflict categories based on the data of each historical user on various platforms, which may include:

[0068] Pre-calculate the distance between the data of each historical user on the j-th platform and the data on all other platforms to obtain the distance group corresponding to the data of each historical user on the j-th platform; where j = 1, 2, ..., m, and m is the number of platforms;

[0069] Based on the distance groups corresponding to the data of each historical user on each platform, a clustering algorithm is used to obtain multiple levels of conflict classes;

[0070] Based on the current user data on each platform and the center of each level of conflict class, determine the conflict class to which the current data on each platform belongs, which may include:

[0071] Calculate the distance between the current user's current data on the j-th platform and the current data on all other platforms to obtain the distance group corresponding to the current user's current data on the j-th platform;

[0072] Calculate the distance between the distance group corresponding to the current data of the current user on the j-th platform and the center of the conflict class at each level, and determine the conflict class corresponding to the minimum distance as the conflict class to which the current data of the current user on the j-th platform belongs.

[0073] In this application, the specific implementation process for determining multiple levels of conflict classes in advance based on the data of each historical user on each platform is as follows:

[0074] 1) Let i = 1;

[0075] 2) Let j = 1;

[0076] 3) Calculate the distance between the data of the i-th historical user on the j-th platform and the data on all other platforms except the j-th platform (the distance mentioned here is Euclidean distance), and use the distance between the data of the i-th historical user on the j-th platform and the data on the other platforms to form the distance group corresponding to the data of the i-th user on the j-th platform;

[0077] 4) Determine if j is equal to m (where m is the number of platforms). If not, set j = j + 1 and return to step 3). If yes, execute step 5.

[0078] 5) Determine if i is equal to n (where n is the number of historical users obtained in step S11). If not, then i = i + 1 and return to step 2). If yes, then execute step 6.

[0079] 6) Based on the distance groups corresponding to the data of each historical user on each platform, a clustering algorithm is used to obtain multiple levels of conflict classes. At the same time, the center of each conflict class is also obtained. That is, multiple levels of conflict classes and the center of each level of conflict class are obtained by using unsupervised clustering.

[0080] Specifically, steps 1)-5) above yield distance groups corresponding to the data of each historical user on each platform, and step 6) determines multiple levels of conflict categories. That is, for the first historical user and m platforms, the distance between the first user's data on the first platform and the first user's data on the second platform can be calculated. Calculate the distance between the data of the first user on the first platform and the data of the first user on the third platform. ...Calculate the distance between the data of the first user on the first platform and the data of the first user on the m-th platform. use These m-1 distances yield the distance group corresponding to the data of the first user on the first platform. Similarly, we can obtain the distance group corresponding to the data of the first user on the second platform. ...the distance group corresponding to the data of the first user on the m-th platform Similarly, we can obtain the distance groups corresponding to the data of each historical user on each platform. Then, we can use a clustering algorithm to cluster the distance groups based on the distance groups corresponding to the data of each historical user on each platform, thereby obtaining multiple conflict classes.

[0081] Based on the above, the specific process for determining the conflict class to which the current data belongs on each platform, according to the current user's data on each platform and the center of each level of conflict class, is as follows:

[0082] a) Let j = 1;

[0083] b) Calculate the distance between the current user's current data on the j-th platform and the current data on the other platforms to obtain the distance group corresponding to the current user's current data on the j-th platform;

[0084] c) Calculate the distance between the distance group corresponding to the current data on the j-th platform and the center of the conflict class at each level;

[0085] d) Determine the minimum distance from the distance group corresponding to the current data on the j-th platform and the center of each level of conflict class, and determine the conflict class corresponding to the minimum distance as the conflict class to which the current user's current data belongs on the j-th platform;

[0086] e) Determine if j is equal to m. If not, set j = j + 1 and return to step b). If yes, end.

[0087] For example: Suppose there are three levels of conflict classes: Level 1, Level 2, and Level 3. If the distance between the distance group corresponding to the current user's current data on Level 1 and the center of the Level 2 conflict class is the smallest, then it can be determined that the current user's current data on Level 1 belongs to the Level 2 conflict class.

[0088] The above process enables the classification of conflict levels and the determination of the conflict level of the current user's current data on each platform, so as to locate conflicting data in the current data corresponding to the current user.

[0089] For details, please refer to [link / reference]. Figure 2 The diagram illustrates a flowchart of another conflict data location method provided in an embodiment of this application. After identifying current data in a platform whose conflict class level is higher than a preset level as conflict data, the conflict data location method provided in this application may further include:

[0090] Extract the first data attribute table from each conflicting data of the current user, and extract the second data attribute table from any non-conflicting data of the current user;

[0091] Compare the attribute fields in each of the first data attribute tables with the corresponding attribute fields in the second data attribute tables;

[0092] If there are attribute fields in the first data attribute table that are inconsistent with the corresponding attribute fields in the second data attribute table, then the first data attribute table that is inconsistent with the corresponding attribute fields in the second data attribute table is determined to be a data attribute table with attribute conflict.

[0093] Determine the time when conflicting data was generated based on the attribute table of data with conflicting attributes.

[0094] In this application, after identifying current data in platforms whose conflict class level is higher than a preset level as conflict data, a first data attribute table can be extracted from each conflict data of the current user. That is, the first data attribute table is extracted from the current data of the current user in each conflicting platform (for the current user). A second data attribute table is also extracted from any non-conflicting data of the current user. That is, the second data attribute table is extracted from the current data of the current user in any non-conflicting platform (for the current user). The data attribute table contains information such as attribute fields, attribute values ​​of the attribute fields, and the order of the attribute values.

[0095] Then, the attribute fields in each extracted first data attribute table are compared with the corresponding attribute fields in the extracted second data attribute table. If there is a discrepancy between the attribute fields in the first and second data attribute tables, it indicates an attribute field conflict in the first data attribute table. In this case, the first data attribute table that is inconsistent with the corresponding attribute fields in the second data attribute table is identified as the data attribute table with attribute conflict. Then, based on the data attribute table with attribute conflict, the interpretability analysis algorithm of the neural network is used to determine the time when the conflicting data was generated, that is, to determine at what time on the platform where the user extracted the data attribute table with attribute conflict, the data conflict occurred.

[0096] In this application, coarse detection of conflict data can be achieved by determining and judging conflict class levels. Fine detection of conflict data can be achieved by extracting data from data tables and comparing attribute fields. Furthermore, by combining coarse and fine detection, the coarse detection can filter out the vast majority of detection tasks, allowing fine detection to analyze a small portion. Finally, for identified conflict data, a method for quickly locating conflict data sources (i.e., the time it takes to locate conflict data) can be designed by combining neural network interpretability analysis algorithms, so that relevant personnel can obtain conflict data more accurately.

[0097] The conflict data location method provided in this application embodiment may further include:

[0098] If the attribute field in the first data attribute table is consistent with the corresponding attribute field in the second data attribute table, then calculate the similarity between the attribute field in the first data attribute table and the corresponding attribute field in the second data attribute table.

[0099] Determine if the similarity is greater than the threshold. If not, determine that there is an attribute field conflict between the first data attribute table and the second data attribute table, and determine the first data attribute table that has an attribute field conflict with the second data attribute table as the data attribute table with attribute conflict.

[0100] The time of conflict is determined based on the data attribute table where there are attribute conflicts.

[0101] In this application, if an attribute field in the first data attribute table matches the corresponding attribute field in the second data attribute table, then, to improve the granularity and accuracy of conflict data determination, the similarity between the attribute fields in the first and second data attribute tables is calculated. Then, it is determined whether the calculated similarity is greater than a threshold. If the calculated similarity is greater than the threshold, it is determined that there is no attribute conflict; if the calculated similarity is not greater than the threshold, it is determined that there is an attribute field conflict in the first data attribute table. In this case, the first data attribute table that has an attribute field conflict with the second data attribute table is identified as the data attribute table with attribute conflict. Subsequently, based on the data attribute table with attribute conflict, and combined with a neural network interpretability analysis algorithm, the time of occurrence of the conflict data is determined, that is, at what time on the platform where the user extracted the data attribute table with attribute conflict, the data conflict occurred.

[0102] The above process enables refined conflict detection based on semantic similarity, thereby further improving the accuracy of conflict data localization.

[0103] This application provides a conflict data location method that calculates the similarity between attribute fields in a first data attribute table and corresponding attribute fields in a second data attribute table, which may include:

[0104] Calculate the similarity between the attribute values ​​of the attribute fields in the first data attribute table and the attribute values ​​of the corresponding attribute fields in the second data attribute table.

[0105] In this application, when calculating the similarity between attribute fields in the first data attribute table and corresponding attribute fields in the second data attribute table, the similarity can be specifically calculated based on the order of the attribute values ​​corresponding to the attribute fields.

[0106]

[0107] Calculate the similarity ε1 between the attribute values ​​of the attribute fields in the first data attribute table and the attribute values ​​of the corresponding attribute fields in the second data attribute table, where x i For the attribute value of an attribute field in the first data attribute table, Max(x) i Min(x) represents the maximum attribute value. i ) represents the minimum attribute value, N represents the number of attribute values, and x and y represent two attribute values ​​of the same attribute field in the first and second data attribute tables, respectively.

[0108] The above methods can be used to calculate the semantic degree of numerical types and improve the accuracy of similarity calculation.

[0109] This application provides a conflict data location method that calculates the similarity between attribute fields in a first data attribute table and corresponding attribute fields in a second data attribute table, which may include:

[0110] The semantic similarity of character types between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table is calculated based on the order of the attribute values ​​corresponding to the attribute fields.

[0111] In this application, when calculating the similarity between attribute fields in the first data attribute table and corresponding attribute fields in the second data attribute table, it is also possible to utilize...

[0112]

[0113] Calculate the character type semantic similarity ε2 between the attribute field in the first data attribute table and the corresponding attribute field in the second data attribute table, where n(x) is the order of x in the attribute value, n(y) is the order of y in the attribute value, and x and y are two attribute values ​​of the same attribute field in the first data attribute table and the second data attribute table, respectively.

[0114] The above process can be used to calculate the semantic similarity of character types and improve the accuracy of similarity calculation.

[0115] This application also provides a conflict data location device, see [link to relevant documentation]. Figure 3 It shows a schematic diagram of the structure of a conflict data localization device provided in an embodiment of this application, which may include:

[0116] The first determining module 31 is used to pre-determine multiple levels of conflict categories based on the data of each historical user on each platform among multiple historical users;

[0117] The second determining module 32 is used to determine the conflict class to which the current data on each platform belongs based on the current data of the current user on each platform and the center of the conflict class at each level;

[0118] The comparison module 33 is used to compare the level of the conflict class to which the current data belongs on each platform with the preset level, and to identify the current data in the platform whose conflict class level is higher than the preset level as conflict data.

[0119] This application provides a conflict data location device, wherein the first determining module 31 may include:

[0120] The first calculation unit is used to pre-calculate the distance between the data of each historical user on the j-th platform and the data on the other platforms, so as to obtain the distance group corresponding to the data of each historical user on the j-th platform; where j = 1, 2, ..., m, and m is the number of platforms;

[0121] The unit is used to obtain multiple levels of conflict classes based on the distance groups corresponding to the data of each historical user on each platform, using a clustering algorithm.

[0122] The second determining module 32 may include:

[0123] The second calculation unit is used to calculate the distance between the current data of the current user on the j-th platform and the current data on the other platforms, so as to obtain the distance group corresponding to the current data of the current user on the j-th platform.

[0124] The third calculation unit is used to calculate the distance between the distance group corresponding to the current data of the current user on the j-th platform and the center of the conflict class at each level, and to determine the conflict class corresponding to the minimum distance as the conflict class to which the current data of the current user on the j-th platform belongs.

[0125] The conflict data location device provided in this application embodiment may further include:

[0126] The extraction module is used to extract the first data attribute table from each conflicting data of the current user, and to extract the second data attribute table from any non-conflicting data of the current user.

[0127] The comparison module is used to compare the attribute fields in each of the first data attribute tables with the corresponding attribute fields in the second data attribute table;

[0128] The third determining module is used to determine the first data attribute table that is inconsistent with the corresponding attribute field in the second data attribute table as a data attribute table with attribute conflict if there is an inconsistency between the attribute field in the first data attribute table and the corresponding attribute field in the second data attribute table.

[0129] The fourth determination module is used to determine the time when conflicting data was generated based on the data attribute table where there are attribute conflicts.

[0130] The conflict data location device provided in this application embodiment may further include:

[0131] The calculation module is used to calculate the similarity between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table if the attribute fields in the first data attribute table are consistent with the corresponding attribute fields in the second data attribute table.

[0132] The fifth determination module is used to determine whether the similarity is greater than the threshold. If not, it is determined that there is an attribute field conflict between the first data attribute table and the second data attribute table, and the first data attribute table that has an attribute field conflict with the second data attribute table is determined as the data attribute table with attribute conflict.

[0133] The sixth module is used to determine the time of conflict based on the data attribute table where there are attribute conflicts.

[0134] This application provides a conflict data location device, the calculation module of which may include:

[0135] The first calculation unit is used to calculate the similarity between the attribute values ​​of the attribute fields in the first data attribute table and the attribute values ​​of the corresponding attribute fields in the second data attribute table.

[0136] This application provides a conflict data location device, the calculation module of which may include:

[0137] The second calculation unit is used to calculate the character type semantic similarity between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table according to the order of the attribute values ​​corresponding to the attribute fields.

[0138] This application also provides a conflict data location device, see [link to relevant documentation]. Figure 4 It shows a schematic diagram of the structure of a conflict data location device provided in an embodiment of this application, which may include:

[0139] Memory 41 is used to store computer programs;

[0140] When processor 42 executes a computer program stored in memory 41, it can perform the following steps:

[0141] Based on the data of each historical user on each platform, multiple conflict categories are determined in advance; based on the current data of the current user on each platform and the center of each conflict category, the conflict category to which the current data on each platform belongs is determined; the level of the conflict category to which the current data on each platform belongs is compared with the preset level, and the current data on the platform whose conflict category level is higher than the preset level is determined as conflict data.

[0142] This application embodiment also provides a readable storage medium storing a computer program, which, when executed by a processor, can perform the following steps:

[0143] Based on the data of each historical user on each platform, multiple conflict categories are determined in advance; based on the current data of the current user on each platform and the center of each conflict category, the conflict category to which the current data on each platform belongs is determined; the level of the conflict category to which the current data on each platform belongs is compared with the preset level, and the current data on the platform whose conflict category level is higher than the preset level is determined as conflict data.

[0144] The readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] For a description of the relevant parts of the conflict data location device, equipment and readable storage medium provided in this application, please refer to the detailed description of the corresponding parts in the conflict data location method provided in the embodiments of this application, and will not be repeated here.

[0146] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0147] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for locating conflict data, characterized in that, include: Multiple levels of conflict categories are determined in advance based on the data of each of the multiple historical users on each platform; Based on the current user's current data on each platform and the center of each level of the conflict class, determine the conflict class to which the current data on each platform belongs; The conflict class level of the current data on each platform is compared with the preset level, and the current data on the platform whose conflict class level is higher than the preset level is identified as conflict data. The data is cross-platform user data. The higher the level of the conflict category, the more obvious the conflict between the data on the corresponding platform and the data on other platforms. This involves pre-determining multiple levels of conflict categories based on the data of each historical user across various platforms, including: The distance between the data of each historical user on the j-th platform and the data on the other platforms is pre-calculated to obtain the distance group corresponding to the data of each historical user on the j-th platform; where j=1,2,…,m, and m is the number of platforms; Based on the distance groups corresponding to the data of each historical user on each of the platforms, a clustering algorithm is used to obtain multiple levels of conflict classes; Based on the current user's data on each platform and the center of each conflict class at each level, determine the conflict class to which the current data on each platform belongs, including: Calculate the distance between the current data of the current user on the j-th platform and the current data on the other platforms to obtain the distance group corresponding to the current data of the current user on the j-th platform; Calculate the distance between the distance group corresponding to the current data of the current user on the j-th platform and the center of the conflict class at each level, and determine the conflict class corresponding to the minimum distance as the conflict class to which the current data of the current user on the j-th platform belongs; Wherein, the distance between the data of each historical user on the j-th platform and the data on the remaining platforms is the Euclidean distance; Specifically, based on the distance groups corresponding to the data of each historical user on each of the platforms, a clustering algorithm is used to obtain multiple levels of conflict classes, including: Based on the distance groups corresponding to the data of each historical user on each platform, multiple levels of conflict classes and the centers of each level of conflict classes are obtained by using unsupervised clustering.

2. The conflict data location method according to claim 1, characterized in that, After identifying current data in platforms whose conflict category level is higher than the preset level as conflict data, the following is also included: Extract a first data attribute table from each of the conflicting data of the current user, and extract a second data attribute table from any of the non-conflicting data of the current user; Compare the attribute fields in each of the first data attribute tables with the corresponding attribute fields in the second data attribute table; If there is an attribute field in the first data attribute table that is inconsistent with the corresponding attribute field in the second data attribute table, then the first data attribute table that is inconsistent with the corresponding attribute field in the second data attribute table is determined to be a data attribute table with attribute conflict. Determine the time when conflicting data was generated based on the attribute table of data with conflicting attributes.

3. The conflict data location method according to claim 2, characterized in that, Also includes: If the attribute field in the first data attribute table is consistent with the corresponding attribute field in the second data attribute table, then calculate the similarity between the attribute field in the first data attribute table and the corresponding attribute field in the second data attribute table. Determine whether the similarity is greater than a threshold. If not, determine that there is an attribute field conflict between the first data attribute table and the second data attribute table, and determine the first data attribute table that has an attribute field conflict with the second data attribute table as the data attribute table with attribute conflict. The time of conflict is determined based on the data attribute table where there are attribute conflicts.

4. The conflict data location method according to claim 3, characterized in that, Calculating the similarity between attribute fields in the first data attribute table and corresponding attribute fields in the second data attribute table includes: Calculate the similarity between the attribute values ​​of the attribute fields in the first data attribute table and the attribute values ​​of the corresponding attribute fields in the second data attribute table.

5. The conflict data location method according to claim 3, characterized in that, Calculating the similarity between attribute fields in the first data attribute table and corresponding attribute fields in the second data attribute table includes: The character type semantic similarity between the attribute fields in the first data attribute table and the corresponding attribute fields in the second data attribute table is calculated based on the order of the attribute values ​​corresponding to the attribute fields.

6. A conflict data location device, characterized in that, include: The first determining module is used to pre-determine multiple levels of conflict classes based on the data of each of the multiple historical users on each platform; The second determining module is used to determine the conflict class to which the current data on each platform belongs based on the current data of the current user on each platform and the center of the conflict class at each level; The comparison module is used to compare the level of the conflict class to which the current data belongs on each platform with a preset level, and to determine the current data in the platform whose conflict class level is higher than the preset level as conflict data. The data is cross-platform user data. The higher the level of the conflict category, the more obvious the conflict between the data on the corresponding platform and the data on other platforms. The first determining module includes: The first calculation unit is used to pre-calculate the distance between the data of each historical user on the j-th platform and the data on the other platforms, so as to obtain the distance group corresponding to the data of each historical user on the j-th platform; where j=1,2,…,m, and m is the number of platforms; The unit is used to obtain multiple levels of conflict classes based on the distance groups corresponding to the data of each historical user on each of the platforms using a clustering algorithm. The second determining module includes: The second calculation unit is used to calculate the distance between the current data of the current user on the j-th platform and the current data on the other platforms, so as to obtain the distance group corresponding to the current data of the current user on the j-th platform. The third calculation unit is used to calculate the distance between the distance group corresponding to the current data of the current user on the j-th platform and the center of the conflict class at each level, and to determine the conflict class corresponding to the minimum distance as the conflict class to which the current data of the current user on the j-th platform belongs; Wherein, the distance between the data of each historical user on the j-th platform and the data on the remaining platforms is the Euclidean distance; Specifically, based on the distance groups corresponding to the data of each historical user on each of the platforms, a clustering algorithm is used to obtain multiple levels of conflict classes, including: Based on the distance groups corresponding to the data of each historical user on each platform, multiple levels of conflict classes and the centers of each level of conflict classes are obtained by using unsupervised clustering.

7. A conflict data location device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the conflict data localization method as described in any one of claims 1 to 5 when executing the computer program.

8. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the conflict data location method as described in any one of claims 1 to 5.

Images