A data transmission method, device, medium and equipment

By comparing the verification results and the degree of data demand to determine the data transmission method, the inefficiency and high cost problems caused by inconsistency in data transmission are solved, and the data transmission efficiency is improved while ensuring data integrity.

CN118714149BActive Publication Date: 2024-11-08HANGZHOU YSCREDIT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411210936.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-08
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

During data transmission, due to frequent data updates, concurrent operations on multiple clients and network delays, the target application data table provided by the target client may be inconsistent with the corresponding data table stored on the server side, resulting in inefficient data transmission and The cost is high and cannot meet the needs of data integrity.

Method used

By obtaining the comparison verification results between the target client and the server, determine the exception field name, and calculate the overall demand level based on the number of selections, number of exceptions, data volume and update times of each exception field name, select the appropriate data sending method , send overall data or abnormal related data to improve efficiency.

Benefits of technology

While ensuring data validity, the efficiency of data transmission is improved, the waste of network resources and time is reduced, and the client's needs for data integrity are met.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118714149B_ABST
    Figure CN118714149B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data processing, and in particular to a data sending method, device, medium and equipment, comprising: when the comparison and verification result between target application data and a first intermediate data table is inconsistent, according to the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name, the first quantity, the total data volume and the number of data updates, analyzing the data sending effects corresponding to the data sending form of sending the first intermediate data as a whole to the target client each time and the data sending form of sending the application data part corresponding to the abnormal related data in the target application data table in the first intermediate data table to the target client, respectively, obtaining the overall demand degree to characterize the priority of sending the first intermediate data as a whole to the target client, thereby determining the data sending mode and sending the data to be sent to the target client, thereby improving the efficiency of data sending while ensuring the validity of the data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data sending method, device, medium and equipment. Background Art

[0002] In today's highly digital business environment, accurate and timely data transmission is crucial to business operations and decision-making. With the continuous expansion of business and the rapid growth of data volume, data interaction between different systems and clients has become increasingly complex.

[0003] For example, in the e-commerce industry, the target client (such as mobile applications, web clients) needs to synchronize data with the server to ensure that users can obtain the latest and accurate product information, order status, etc. In the financial field, the consistency and timeliness of transaction data are even more related to major interests and risks. Data differences between the client and the server may lead to serious problems such as transaction errors and risk assessment errors. In the logistics industry, if there is a deviation in the synchronization of cargo tracking information, inventory data, etc. between the client and the server, it will affect the efficiency of logistics and customer satisfaction.

[0004] However, due to factors such as frequent data updates, concurrent operations of multiple clients, and network delays, the target application data table provided by the target client and the corresponding data table stored on the server may be inconsistent.

[0005] After comparing and verifying the target application data table provided by the target client and the corresponding data table stored on the server, the required data can be sent to the target client based on the comparison and verification results. Traditional data sending methods are usually simple and crude. One data sending method is to send all data. Although it can provide complete and accurate data to the target client, it will cause a lot of network resources and time waste, and reduce the efficiency of data sending; another data sending method is to send only part of the key data, which can improve the efficiency of data sending and reduce the cost of data sending, but it may not meet the client's requirements for data integrity, resulting in poor data sending effect.

[0006] Therefore, how to determine the method for the server to send data to the client so as to improve the efficiency of data transmission while ensuring the validity of the data is an urgent problem to be solved. Summary of the invention

[0007] In view of the above technical problem, the technical solution adopted by the present invention is a data sending method, which includes the following steps:

[0008] S100, obtain the target application data table provided by the target client, the first intermediate data table corresponding to the target application data table stored on the server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, the reference field name corresponding to each target field name in the target application data table, and the comparison and verification results between the target application data table provided by the target client and the first intermediate data table, wherein the comparison and verification results include consistency and inconsistency.

[0009] S200, if the comparison and verification result is inconsistent, obtain the first number of comparison and verification results provided by the target client within a preset time period, the total amount of data corresponding to the first intermediate data table, and the number of data updates corresponding to the first intermediate data table within the preset time period.

[0010] S300, for any target field name in the target application data table, if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table, the current target field name is determined as an abnormal field name.

[0011] S400, obtaining the overall demand degree of the target client for the first intermediate data table according to the reference field name corresponding to each abnormal field name, the selection times and abnormal times corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the data update times.

[0012] S500, determining a data sending mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data sending mode includes sending overall data and sending abnormality-related data.

[0013] S600, acquiring a data table to be sent according to a data sending mode, a first intermediate data table, and a reference field name corresponding to each abnormal field name.

[0014] S700, sending the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

[0015] The present invention also provides a data sending device, the data sending device comprising:

[0016] The first data acquisition module is used to obtain the target application data table provided by the target client, the first intermediate data table corresponding to the target application data table stored on the server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, the reference field name corresponding to each target field name in the target application data table, and the comparison and verification results between the target application data table provided by the target client and the first intermediate data table, wherein the comparison and verification results include consistency and inconsistency.

[0017] The second data acquisition module is used to obtain the first number of comparison and verification results provided by the target client within a preset time period, the total amount of data corresponding to the first intermediate data table, and the number of data updates corresponding to the first intermediate data table within the preset time period if the comparison and verification results are inconsistent.

[0018] The abnormal field name determination module is used to determine the current target field name as an abnormal field name for any target field name in the target application data table if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table.

[0019] The overall demand degree acquisition module is used to obtain the overall demand degree of the target client for the first intermediate data table based on the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates.

[0020] The data sending mode determination module is used to determine the data sending mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data sending mode includes sending overall data and sending abnormal related data.

[0021] The module for acquiring the data table to be sent is used to acquire the data table to be sent according to the data sending mode, the first intermediate data table and the reference field name corresponding to each abnormal field name.

[0022] The data sending module is used to send the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

[0023] The present invention also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned data sending method.

[0024] The present invention also provides an electronic device, comprising a processor and the above-mentioned non-transitory computer-readable storage medium.

[0025] The present invention has at least the following beneficial effects: if the comparison verification result is inconsistent, then for any target field name in the target application data table, if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table, the current target field name is determined as an abnormal field name, and the overall demand degree of the target client for the first intermediate data table is obtained according to the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates, and the corresponding number of the first intermediate data table is determined according to the overall demand degree and the preset overall demand degree threshold. The data sending method is used, and the data table to be sent is obtained according to the data sending method, the first intermediate data table and the reference field name corresponding to each abnormal field name, and the data table to be sent is sent to the target client. It can be seen that by analyzing the data sending effect between the data sending form of sending the first intermediate data table as a whole to the target client each time, and the data sending form of sending the abnormal related data in the target application data table to the application data part corresponding to the first intermediate data table is sent to the target client, the priority of sending the first intermediate data table as a whole to the target client is measured, and the data sending method is determined accordingly and the corresponding data to be sent is sent to the target client, which improves the efficiency of data sending while ensuring the validity of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A flowchart of a data sending method provided in Embodiment 1 of the present invention;

[0028] Figure 2 A flowchart of a data synchronization method provided in Embodiment 2 of the present invention;

[0029] Figure 3 A flowchart of a data synchronization method based on a hash algorithm provided in Embodiment 3 of the present invention;

[0030] Figure 4 A structural diagram of a data sending device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It is understood that, where appropriate, the above-mentioned terms used to distinguish similar objects can be interchanged so that the present invention can also implement other embodiments other than the above-mentioned illustrated embodiments or described embodiments. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] This embodiment provides a data transmission method, which includes the following steps: Figure 1 As shown:

[0035] S100, obtain the target application data table provided by the target client, the first intermediate data table corresponding to the target application data table stored on the server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, the reference field name corresponding to each target field name in the target application data table, and the comparison and verification results between the target application data table provided by the target client and the first intermediate data table, wherein the comparison and verification results include consistency and inconsistency.

[0036] Among them, the server side stores several initial application data tables. The target client can purchase data from a certain initial application data table on the server side based on its own needs to form its own target application data table, and filter out a first intermediate data table with higher similarity from the server side based on the target application data table, which is used to compare and verify the consistency of the target application data of the target client. Therefore, when abnormal data appears in the target client or the server side updates the data, the first intermediate data table as a whole or the abnormal related data in the first intermediate data table is sent to the target client to assist the target client in synchronously updating the target application data and meet various business needs of the target client.

[0037] For example, in an e-commerce platform, the commodity inventory data table on the server side needs to be synchronized with the client data table in each region to ensure that clients in each region can obtain accurate inventory information in real time to avoid overselling or inaccurate inventory. Another example is in the financial field, the transaction data on the server side needs to be synchronized with different clients (such as mobile terminals and web terminals) to ensure that users can obtain the latest and consistent transaction records on different terminals.

[0038] In this embodiment, the application data in the first intermediate data table can be regarded as data purchased by the target client from the server, and the target application data table can be regarded as the data table actually obtained by the target client based on the first intermediate data table and the data table obtained after the target client performs customized processing on the obtained data table, wherein the customized processing may include operations such as modifying the table name and modifying the field name.

[0039] The specific time of the preset time period can be set by the implementer according to the actual situation. For example, the preset time period can refer to a time period of one month, six months, one year, etc. before the current time point.

[0040] The first intermediate data table includes several reference field names, and the number of selections refers to the number of times the corresponding reference field name is purchased by the client within a preset time period. The number of exceptions refers to the number of times the corresponding reference field name is inconsistent with the corresponding application data in the target application data table when participating in data consistency verification within a preset time period.

[0041] In a specific implementation, when the comparison and verification result is consistent, the data related to the first intermediate data table is not sent to the target client to synchronously update the target data table.

[0042] In a specific implementation, S100 further includes the following steps:

[0043] S110, obtain several initial application data tables stored on the server side, first attribute information of each initial application data table, and target application data table provided by the target client and second attribute information of the target application data table, wherein the first attribute information includes an initial table name corresponding to the initial application data table, several initial field names and a field type corresponding to each initial field name, and the second attribute information includes a target table name of the target application data table, several target field names and a field type corresponding to each target field name.

[0044] S120: Acquire a matching degree between the target application data table and each initial application data table according to the second attribute information of the target application data table and the first attribute information of each initial application data table.

[0045] S130: Determine the initial application data table corresponding to the maximum matching degree as the designated application data table corresponding to the target application data table.

[0046] S140: Filter and obtain a first intermediate data table from the designated application data table according to the target field name corresponding to the target application data table and the initial field name corresponding to the designated application data table.

[0047] Among them, there may be differences in data information such as data table structure, data table naming, field naming, field type, etc. between the target client and the server.

[0048] Therefore, this embodiment obtains the first attribute information of each initial application data table, that is, the initial table name of the initial application data table, several initial field names and the field type corresponding to each initial field name, and the second attribute information of the target application data table, that is, the target table name of the target application data table, several target field names and the field type corresponding to each target field name, so as to determine the initial application data table with the greatest correlation with the target application data table by measuring the similarity of the table name, field name and field type between the initial application data table and the target application data table, as the basis for measuring data consistency.

[0049] The specific field type can be set by the implementer according to the actual situation. For example, the field type can include integer type, floating point type, character type, date and time type, Boolean type, etc.

[0050] The matching degree may represent the matching degree between the target application data table and the corresponding initial application data table, that is, the degree to which the target application data table is derived from the corresponding initial application data table.

[0051] In a specific implementation, S120 further includes the following steps:

[0052] S121 , for any initial application data table, according to the initial table name and the target table name of the current initial application data table, obtain a first matching score corresponding to the current initial application data table.

[0053] S122, according to a number of initial field names corresponding to the current initial application data table and a field type corresponding to each initial field name, and a number of target field names and a field type corresponding to each target field name, obtain a second matching score corresponding to the current initial application data table.

[0054] S123: Acquire a matching degree between the target application data table and the current initial application data table according to the first matching score and the second matching score.

[0055] The first matching score is used to characterize the similarity between the target application data table and the current initial application data table regarding the data table name, and the second matching score is used to characterize the similarity between the target application data table and the current initial application data table regarding the field name and field type.

[0056] In a specific implementation, S122 further includes the following steps:

[0057] S1221: For any target field name, calculate a first similarity between the current target field name and each initial field name in the current initial application data table.

[0058] S1222: Perform consistency matching on the field type corresponding to the current target field name and the field type corresponding to each initial field name in the current initial application data table, and obtain a second similarity between the current target field name and each initial field name in the current initial application data table.

[0059] S1223: Acquire a third similarity between the current target field name and each initial field name in the current initial application data table according to the first similarity and the second similarity.

[0060] S1224: Determine the maximum value of the third similarities between the current target field name and all the initial field names in the current initial application data table as the matching degree between the current initial application data table and the current target field name.

[0061] S1225, traverse all target field names, and determine the average value of the matching degree between the current initial application data table and all target field names as the second matching score corresponding to the current initial application data table.

[0062] Among them, the target field name and the initial field name are converted into corresponding word vectors according to a preset word vector model, and the first similarity between the target field name and the initial field name is represented according to the similarity between the word vectors.

[0063] The specific word vector model can be set by the implementer according to the actual situation. For example, the preset word vector model can refer to the word vector model such as Global Vectors for Word Representation (GloVe), Word2Vec and FastText. Among them, those skilled in the art know that any word vector model in the prior art falls within the protection scope of the present invention, and will not be repeated here.

[0064] If the field type corresponding to the target field name is consistent with the field type corresponding to the initial field name in the current initial application data table, the corresponding second similarity is determined to be 1. If the field type corresponding to the target field name is consistent with the field type corresponding to the initial field name in the current initial application data table, the corresponding second similarity is determined to be 0.

[0065] Based on the first mapping weight corresponding to the field name and the second preset weight corresponding to the field type, the first similarity and the second similarity between the current target field name and each initial field name in the current initial application data table are weightedly summed to obtain the corresponding third similarity.

[0066] The target client may have purchased some application data in the specified application data table to form the target application data table. Therefore, this embodiment filters the initial field names corresponding to the specified application data table according to the target field names corresponding to the target application data table and the initial field names corresponding to the specified application data table, and then obtains the first intermediate data table from the specified application data table as the basis for consistency verification and synchronous update of the target application data.

[0067] In the above, based on the second attribute information of the target application data table and the first attribute information of each initial application data table, the matching degree between the target application data table and each initial application data table is measured to characterize the degree to which the target application data table is derived from the corresponding initial application data table, and based on the matching degree, the designated application data table corresponding to the target application data is obtained, and the first intermediate data table is further screened from the designated application data table, which provides a verification object for consistency verification of the target application data of the target client, thereby improving the verification accuracy.

[0068] Among them, the target client may have purchased part of the application data in the specified application data table to form the target application data table. Therefore, this embodiment filters the initial field names corresponding to the specified application data table according to the target field names corresponding to the target application data table and the initial field names corresponding to the specified application data table, and then obtains the first intermediate data table from the specified application data table as the basis for consistency verification and synchronous update of the target application data.

[0069] In a specific implementation, S140 further includes the following steps:

[0070] S141 , for any target field name, obtaining a third similarity between the current target field name and each initial field name in the specified application data table.

[0071] S142, determining the initial field name corresponding to the largest third similarity in the specified application data table as the reference field name corresponding to the current target field name.

[0072] S143, traverse all target field names, and obtain the reference field name corresponding to each target field name.

[0073] S144, filtering out application data corresponding to each reference field name from the specified application data table.

[0074] S145, forming a first intermediate data table according to all the extracted application data.

[0075] In the above, by measuring the similarity between the target field name and the initial field name corresponding to the specified application data table, the initial field name with higher similarity to the target field name is screened out from all the initial field names corresponding to the specified application data table, and then the first intermediate data table is obtained by screening from the specified application data table as a basis for consistency verification and synchronous update of the target application data. While reducing the amount of data that needs to be verified, the interference of application data corresponding to initial field names with lower similarity on data consistency verification is eliminated, thereby improving the accuracy and efficiency of data synchronization, and further improving the accuracy of data transmission.

[0076] S200, if the comparison and verification result is inconsistent, obtain the first number of comparison and verification results provided by the target client within a preset time period, the total amount of data corresponding to the first intermediate data table, and the number of data updates corresponding to the first intermediate data table within the preset time period.

[0077] The first number of comparison verification results provided by the target client within the preset time period can represent the number of verification requests made by the target client to the server within the preset time period. The number of data updates refers to the number of times the server actively updates the corresponding application data in the first intermediate data table.

[0078] In this embodiment, when the first number is larger, the total data volume is larger, and the data update times are more, the cost of sending the first intermediate data as a whole to the target client each time is higher, and the cost of sending the application data part corresponding to the exception-related data in the target application data table in the first intermediate data table to the target client is lower.

[0079] Therefore, this embodiment obtains the first number of comparison and verification results provided by the target client within a preset time period, the total amount of data corresponding to the first intermediate data table, and the number of data updates corresponding to the first intermediate data table within the preset time period, to measure the cost of sending the first intermediate data as a whole to the target client each time, so as to determine the overall demand degree of the target client for the first intermediate data.

[0080] S300, for any target field name in the target application data table, if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table, the current target field name is determined as an abnormal field name.

[0081] Among them, by traversing all target field names, all abnormal field names can be filtered out from all target field names as the basis for data sending.

[0082] S400, obtaining the overall demand degree of the target client for the first intermediate data table according to the reference field name corresponding to each abnormal field name, the selection times and abnormal times corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the data update times.

[0083] In a specific implementation, S400 further includes the following steps:

[0084] S410, according to the reference field name corresponding to each abnormal field name, and the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the abnormality degrees corresponding to all abnormal field names are obtained.

[0085] S420: Obtain the importance corresponding to the first intermediate data table according to the first quantity, the total data volume, and the number of data updates.

[0086] S430: Obtain the overall demand level corresponding to the first intermediate data table according to the abnormality level and the importance level.

[0087] Among them, the higher the abnormality degree corresponding to the abnormal field name and the higher the importance corresponding to the first intermediate data table, the higher the possibility and frequency of verifying and synchronizing the target application data table based on the first intermediate data table. Correspondingly, compared with sending the application data part corresponding to the abnormal related data in the target application data table in the first intermediate data table to the target client, sending the entire first intermediate data table to the target client each time can provide the target client with complete and accurate data, meet the various application requirements corresponding to the target client, and avoid the target client being unable to use the corresponding data normally due to too low data validity.

[0088] On the contrary, the lower the abnormality degree corresponding to the abnormal field name and the lower the importance corresponding to the first intermediate data table, the lower the possibility and frequency of verifying and synchronizing the target application data table based on the first intermediate data table. Correspondingly, compared with sending the entire first intermediate data table to the target client every time, sending the abnormal related data in the target application data table to the target client in the application data part corresponding to the first intermediate data table can improve the efficiency of data sending, reduce the cost of data sending, and improve the timeliness of data synchronization update by the target client.

[0089] Therefore, based on the degree of abnormality and importance, the overall demand level corresponding to the first intermediate data is obtained, which is used to characterize the priority of sending the first intermediate data table as a whole to the target client, thereby determining the sending method of the first intermediate data according to the overall demand level, thereby improving the efficiency of data sending while ensuring the validity of the data.

[0090] In a specific implementation, S410 further includes the following steps:

[0091] S411, for any abnormal field name, according to the number of selections and the number of abnormalities corresponding to the reference field name corresponding to the current abnormal field name within a preset time period, obtain the abnormal sub-degree corresponding to the current abnormal field name.

[0092] S412, traverse all abnormal field names, and obtain the abnormal sub-level corresponding to each abnormal field name.

[0093] S413: The sum of all abnormal sub-degrees is determined as the abnormal degree corresponding to all abnormal field names.

[0094] Among them, the average number of selections and the average number of exceptions corresponding to all initial field names corresponding to all initial application data tables within a preset time period are obtained, the ratio of the number of selections to the average number of selections is determined as the corresponding reference number of selections, and the ratio of the number of exceptions to the average number of exceptions is determined as the corresponding reference number of exceptions, thereby unifying the scales of the reference number, reference number of selections and reference number of exceptions, and improving the accuracy of the degree of abnormality.

[0095] The second preset weight corresponding to the number of selections and the third preset weight corresponding to the number of exceptions are obtained, and the reference number of selections and the reference number of exceptions are weighted and summed according to the second preset weight and the third preset weight to obtain the abnormal sub-degree corresponding to each abnormal field name.

[0096] The specific values ​​of the second preset weight and the third preset weight can be set by the implementer according to actual conditions.

[0097] In the above, based on the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates, the priority of sending the first intermediate data table as a whole to the target client is measured, and the overall demand degree of the target client for the first intermediate data table is obtained, which is used to determine the sending method of the first intermediate data to achieve efficient data sending effect.

[0098] S500, determining a data sending mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data sending mode includes sending overall data and sending abnormality-related data.

[0099] In a specific implementation, S500 includes the following steps:

[0100] S510: If the overall demand level is greater than a preset overall demand level threshold, determine that the data sending method corresponding to the first intermediate data table is to send the overall data.

[0101] S520: If the overall demand level is less than or equal to a preset overall demand level threshold, determine that the data sending method corresponding to the first intermediate data table is to send abnormality-related data.

[0102] S600, acquiring a data table to be sent according to a data sending mode, a first intermediate data table, and a reference field name corresponding to each abnormal field name.

[0103] In a specific implementation, S600 further includes the following steps:

[0104] S610: If the data sending method is to send the entire data, the first intermediate data table is determined as the data table to be sent.

[0105] S620: If the data sending mode is to send exception-related data, the position identifier of each exception field name in the target application data table is obtained.

[0106] S630: Determine the application data corresponding to the reference field name corresponding to each abnormal field name in the first intermediate data table as the first data to be sent.

[0107] S640, acquiring second data to be sent corresponding to each first data to be sent according to each first data to be sent and the corresponding position identifier of the abnormal field name corresponding to each first data to be sent in the target application data table.

[0108] S650: compose a table of data to be sent according to all the second data to be sent.

[0109] The position identifier is used to indicate the position of the abnormal field name in the target application data table. For example, the position identifier corresponding to each abnormal field name may refer to the row number or column number corresponding to each abnormal field name in the target application data table.

[0110] In the above, according to the data sending mode, the first intermediate data table and the reference field name corresponding to each abnormal field name, the data table to be sent is obtained as the data basis for synchronizing the target application data.

[0111] S700, sending the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

[0112] If the data table to be sent is the entire first intermediate data table, the first intermediate data table is used to replace the target application data table, so that the target application data table is synchronized with the first intermediate data table.

[0113] If the data table to be sent consists of the second data to be sent, each first data to be sent is used to replace the application data corresponding to the corresponding abnormal field name in the target application data, so that the target application data table is synchronized with the first intermediate data table.

[0114] In the above, if the comparison verification result is inconsistent, then for any target field name in the target application data table, if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table, then the current target field name is determined as an abnormal field name, and the overall demand degree of the target client for the first intermediate data table is obtained according to the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates, and the data sender corresponding to the first intermediate data table is determined according to the overall demand degree and the preset overall demand degree threshold. According to the data sending method, the first intermediate data table and the reference field name corresponding to each abnormal field name, the data table to be sent is obtained, and the data table to be sent is sent to the target client. It can be seen that by analyzing the data sending effect between the data sending form of sending the first intermediate data table as a whole to the target client each time and the data sending form of sending the application data part corresponding to the abnormal related data in the target application data table in the first intermediate data table to the target client, the priority of sending the first intermediate data table as a whole to the target client is measured, and the data sending method is determined accordingly and the corresponding data to be sent is sent to the target client, which improves the efficiency of data sending while ensuring the validity of the data.

[0115] Embodiment 2

[0116] This embodiment 2 provides a data synchronization method, which includes the following steps: Figure 2 As shown:

[0117] S1, obtains several initial application data tables stored on the server side, the first attribute information of each initial application data table, and the target application data table provided by the target client and the second attribute information of the target application data table, wherein the first attribute information includes the initial table name corresponding to the initial application data table, several initial field names and the field type corresponding to each initial field name, and the second attribute information includes the target table name of the target application data table, several target field names and the field type corresponding to each target field name.

[0118] S2: Acquire a matching degree between the target application data table and each initial application data table according to the second attribute information of the target application data table and the first attribute information of each initial application data table.

[0119] S3, determining the initial application data table corresponding to the maximum matching degree as the designated application data table corresponding to the target application data table.

[0120] S4, according to the target field name corresponding to the target application data table and the initial field name corresponding to the designated application data table, a first intermediate data table is obtained by screening from the designated application data table.

[0121] S5. According to the first intermediate data table, obtain the data magnitude corresponding to the first intermediate data table.

[0122] The data magnitude may represent the amount of data in the first intermediate data table. This embodiment selects a processing method for the first intermediate data table according to the amount of data to balance the accuracy and efficiency of data verification.

[0123] In a specific implementation, S5 further includes the following steps:

[0124] S51, obtaining a second data volume of application data in the first intermediate data table.

[0125] S52, performing a logarithm operation on the second data volume to obtain a data volume level corresponding to the first intermediate data table.

[0126] S6, comparing the data level with a preset data level threshold to obtain a data comparison result.

[0127] The data comparison result is that the data magnitude is greater than the threshold, which is used to indicate that the data volume of the first intermediate data table is large, or the data magnitude is less than or equal to the threshold, which is used to indicate that the data volume of the first intermediate data table is small.

[0128] The specific value of the preset data level threshold can be set by the implementer according to the actual situation. For example, the preset data level threshold can be set to 5.

[0129] In the above, the data level is compared with the preset data level threshold to obtain a data comparison result, which is used to characterize the data volume of the first intermediate data table and provide a basis for selecting a processing method corresponding to the first intermediate data table.

[0130] S7, processing the first intermediate data table according to a preset data processing method corresponding to the data comparison result to obtain a second intermediate data table, wherein the preset data processing method includes keeping the first intermediate data table unchanged and performing group hash conversion on the first intermediate data table.

[0131] In a specific implementation, the data comparison result includes that the data magnitude is greater than the threshold of the application data and the data magnitude is less than or equal to the threshold of the application data, and S7 further includes the following steps:

[0132] S71, if the data comparison result is that the data magnitude is less than or equal to the threshold of the application data, the preset data processing method corresponding to the data comparison result is to keep the first intermediate data table unchanged and determine the first intermediate data table as the second intermediate data table.

[0133] S72, if the data comparison result is that the data magnitude is greater than the threshold of the application data, the preset data processing method corresponding to the data comparison result is to perform group hash conversion on the first intermediate data table to obtain the second intermediate data table, wherein S72 also includes the following steps:

[0134] S721, obtaining a first data volume of application data corresponding to each reference field name in the first intermediate data table.

[0135] S722: For any reference field name, obtain a reference group length corresponding to the current reference field name according to the first data volume corresponding to the current reference field name and a preset number of groups.

[0136] S723, grouping the application data corresponding to the current reference field name according to the reference group length corresponding to the current reference field name, and acquiring all reference application data groups corresponding to the current reference field name.

[0137] S724, process each reference application data group according to a preset hash algorithm to obtain reference hash data corresponding to each reference application data group.

[0138] S725, traverse all reference fields to obtain all reference hash data.

[0139] S726, forming a second intermediate data table according to all the reference hash data.

[0140] Among them, if the data comparison result is that the data level is less than or equal to the threshold of the application data, the first intermediate data table is kept unchanged, and the first intermediate data table is determined as the second intermediate data table. Correspondingly, the second intermediate data table is sent to the target client, that is, the first intermediate data table is directly sent to the target client, so as to assist the target client to directly verify and synchronously update the application data in the target application data table one by one according to the various application data in the first intermediate data table, thereby improving the accuracy of data synchronization.

[0141] If the data comparison result is that the data level is greater than the threshold of the application data, the application data corresponding to each reference field is grouped to obtain a reference application data group with a smaller data volume, and each reference application data group is hashed according to a preset hash algorithm. A second intermediate data table is formed according to all the reference hash data, and the second intermediate data table is sent to the target client to assist the target client in verifying and synchronously updating the hash data converted from the application data in the target application data table one by one according to each reference hash data in the second intermediate data table, thereby reducing the amount of data that needs to be verified and improving the efficiency of data synchronization.

[0142] Among them, those skilled in the art know that any hash algorithm in the prior art falls within the protection scope of the present invention, and will not be described in detail here.

[0143] In the above, the first intermediate data table is processed according to the data comparison result. When the amount of data in the first intermediate data table is small, the first intermediate data table is directly sent to the target client to assist the target client to directly verify and synchronously update the application data in the target application data table one by one according to the various application data in the first intermediate data table, thereby improving the accuracy of data synchronization. When the amount of data in the first intermediate data table is large, the application data corresponding to each reference field is grouped and hash-converted to assist the target client to verify and synchronously update the hash data converted from the application data in the target application data table one by one according to the various reference hash data in the second intermediate data table, thereby reducing the amount of data that needs to be verified and improving the efficiency of data synchronization.

[0144] S8, sending the second intermediate data table to the target client, wherein the target client is used to verify and synchronize the target application data table according to the second intermediate data table.

[0145] In a specific implementation, the target client is used to compare and verify the second intermediate data table and the target application data table, and update the target application data table according to the comparison and verification result, so that the updated target application data table is synchronized with the first intermediate data table.

[0146] Furthermore, when the first intermediate data table is kept unchanged to obtain the second intermediate data table, if the comparison and verification result shows that the second intermediate data table is consistent with the target application data table, the target application data table is kept unchanged. If the comparison and verification result shows that the second intermediate data table is inconsistent with the target application data table, the target application data table is updated according to the first intermediate data table, so that the updated target application data table is synchronized with the first intermediate data table, thereby meeting various business requirements corresponding to the target client.

[0147] When the first intermediate data table is grouped and hashed to obtain the second intermediate data table, if the comparison and verification result shows that the second intermediate data table is consistent with the target application data table, the target application data table remains unchanged. If the comparison and verification result shows that the second intermediate data table is inconsistent with the target application data table, the actual group length corresponding to each target field name is obtained according to the sub-data volume corresponding to each target field name in the target application data and the preset number of groups, and the application data in all target field names are grouped and hashed according to the preset hash algorithm and the actual group length corresponding to each target field name to obtain the actual hash data table, and the target hash data table and the actual hash data table are compared and verified, and the target application data table is updated according to the comparison and verification result, so that the updated target application data table is synchronized with the first intermediate data table, thereby meeting various business needs corresponding to the target client.

[0148] Further, if the comparison verification result is that the target hash data table and the actual hash data table are consistent, the target application data table is kept unchanged. If the comparison verification result is that the target hash data table and the actual hash data table are inconsistent, the inconsistent abnormal hash value and the application data corresponding to the abnormal hash value in the first intermediate data table are located, and the application data corresponding to the abnormal hash value in the target application data table is updated according to the application data corresponding to the abnormal hash value in the first intermediate data table, so that the updated target application data table is synchronized with the first intermediate data table.

[0149] In the above, according to the second attribute information of the target application data table and the first attribute information of each initial application data table, the matching degree between the target application data table and each initial application data table is obtained, and the initial application data table corresponding to the maximum matching degree is determined as the designated application data table corresponding to the target application data table. According to the target field name corresponding to the target application data table and the initial field name corresponding to the designated application data table, a first intermediate data table is screened from the designated application data table. According to the first intermediate data table, the data magnitude corresponding to the first intermediate data table is obtained, the data magnitude is compared with a preset data magnitude threshold, and a data comparison result is obtained. According to the data comparison result, the first intermediate data table is processed to obtain a second intermediate data table, and the second intermediate data table is sent to the target client. It can be seen that based on the second attribute information of the target application data table and Based on the similarity between the first attribute information of each initial application data table, the first intermediate data table is selected as the basis for consistency verification and synchronous update of the target application data. When the amount of data in the first intermediate data table is small, the first intermediate data table is directly sent to the target client to assist the target client to directly verify and synchronously update the application data in the target application data table one by one according to the various application data in the first intermediate data table, thereby improving the accuracy of data synchronization. When the amount of data in the first intermediate data table is large, the application data corresponding to each reference field is grouped and hashed, so as to assist the target client to verify and synchronously update the hash data converted from the application data in the target application data table one by one according to the various reference hash data in the second intermediate data table, thereby reducing the amount of data that needs to be verified and improving the efficiency of data synchronization.

[0150] Embodiment 3

[0151] The third embodiment provides a data synchronization method based on a hash algorithm. The data synchronization method based on a hash algorithm includes: Figure 3 As shown:

[0152] S10, obtain the target application data table provided by the target client, the first intermediate data table corresponding to the target application data table stored on the server, several reference field names in the first intermediate data table, the total data volume corresponding to the first intermediate data table and the corresponding number of data updates within a preset time period, and the corresponding number of selections and exceptions for each reference field name within the preset time period.

[0153] The specific time of the preset time period can be set by the implementer according to the actual situation. For example, the preset time period can refer to a time period such as one month, six months, or one year before the current time point.

[0154] With the expansion and diversification of enterprise business, the data exchange between the target client and the server is frequent and large in volume. When the target application data table is consistency verified and synchronously updated based on the first intermediate data table, the more total data volume corresponds to the first intermediate data table, the higher the number of data updates corresponding to the first intermediate data table within the preset time period, and the more selection times and exception times corresponding to the reference field name in the first intermediate data table within the preset time period, the higher the cost of verifying the application data in the first intermediate data table and the target application data table one by one, and the lower the cost of performing hash conversion on the application data in the first intermediate data table and the target application data table and then verifying it.

[0155] Therefore, this embodiment obtains the total amount of data corresponding to the first intermediate data table and the corresponding number of data updates within the preset time period, the corresponding number of selections and the number of exceptions for each reference field name within the preset time period, to measure the cost of verifying the application data in the first intermediate data table and the target application data table one by one, that is, the degree of need for hash conversion of the first intermediate data table, and selects the processing form of the first intermediate data table accordingly, so as to balance and improve the accuracy and efficiency of data synchronization.

[0156] S20, obtaining the hash conversion requirement degree corresponding to the first intermediate data table according to the number of data updates corresponding to the first intermediate data table, the number of selections and the number of exceptions corresponding to each reference field name within a preset time period.

[0157] In a specific implementation, S20 further includes the following steps:

[0158] S201, obtaining a first preset weight corresponding to the number of data updates, a second preset weight corresponding to the number of selections, and a third preset weight corresponding to the number of exceptions.

[0159] S202, respectively normalize the data update times, selection times and exception times to obtain corresponding data update reference times, reference selection times and reference exception times.

[0160] S203, obtaining the hash conversion requirement degree corresponding to the first intermediate data table according to the reference number of data updates, the reference number of selections, the reference number of exceptions, the first preset weight, the second preset weight and the third preset weight.

[0161] Among them, the average data update times corresponding to all initial application data tables stored on the server side within the preset time period, the average selection times and the average exception times corresponding to all initial field names corresponding to all initial application data tables within the preset time period are obtained, and the ratio of the data update times and the average data update times is determined as the corresponding data update reference times, the ratio of the selection times and the average selection times is determined as the corresponding reference selection times, and the ratio of the exception times and the average exception times is determined as the corresponding reference exception times, thereby unifying the scales of the reference times, reference selection times and reference exception times, and improving the accuracy of the hash conversion requirement degree.

[0162] The specific values ​​of the first preset weight, the second preset weight and the third preset weight can be set by the implementer according to actual conditions.

[0163] As described above, based on the number of data updates corresponding to the first intermediate data table, the number of selections and the number of exceptions corresponding to each reference field name within a preset time period, the degree of hash conversion demand corresponding to the first intermediate data table is obtained to analyze the cost of verifying the application data in the first intermediate data table and the target application data table one by one, and the cost of performing hash conversion and then verifying the application data in the first intermediate data table and the target application data table, and accordingly select the processing form for the first intermediate data table to balance and improve the accuracy and efficiency of data synchronization.

[0164] S30, determining a first data processing method corresponding to the first intermediate data table according to the hash conversion requirement and a preset requirement threshold, wherein the first data processing method includes keeping unchanged and performing hash conversion.

[0165] In a specific implementation, S30 further includes the following steps:

[0166] S301: If the hash conversion requirement is less than a preset requirement threshold, determine that the first data processing method corresponding to the first intermediate data table remains unchanged.

[0167] S302: If the hash conversion requirement is greater than or equal to a preset requirement threshold, determine that the first data processing method corresponding to the first intermediate data table is to perform hash conversion.

[0168] Among them, if the hash conversion requirement degree is less than the preset requirement degree threshold, it means that the cost of verifying the application data in the first intermediate data table and the target application data table one by one is less than the cost of verifying the application data in the first intermediate data table and the target application data table after hash conversion, then it is determined that the first data processing method corresponding to the first intermediate data table remains unchanged.

[0169] If the degree of hash conversion requirement is greater than or equal to a preset degree of requirement threshold, it means that the cost of verifying the application data in the first intermediate data table and the target application data table one by one is greater than or equal to the cost of verifying the application data in the first intermediate data table and the target application data table after hash conversion, then it is determined that the first data processing method corresponding to the first intermediate data table is hash conversion.

[0170] The specific value of the preset demand level threshold can be set by the implementer according to actual conditions.

[0171] S40, if the first data processing method is to perform hash conversion, then determine the second data processing method corresponding to the first intermediate data table according to the total data volume corresponding to the first intermediate data table and the preset data volume threshold, wherein the second data processing method includes overall hash conversion and grouped hash conversion.

[0172] In a specific implementation, S40 further includes the following steps:

[0173] S401: If the total data volume is less than a preset data volume threshold, determine that the second data processing method corresponding to the first intermediate data table is overall hash conversion.

[0174] S402: If the total data volume is greater than or equal to a preset data volume threshold, determine that the second data processing method corresponding to the first intermediate data table is group hash conversion.

[0175] Among them, when the amount of data to be hashed is too small, the amount of data information contained is small, and the probability of hash collision occurring during hash conversion is relatively high, that is, the probability that different data corresponds to the same hash value is relatively high.

[0176] Therefore, in order to improve the accuracy of the target hash data table, if the total data volume is less than the preset data volume threshold, the first intermediate data table can be hashed as a whole. If the total data volume is greater than or equal to the preset data volume threshold, the first intermediate data table can be grouped and then hashed, which can improve the efficiency of locating abnormal data during data consistency verification while ensuring the accuracy of the target hash data table, thereby comprehensively improving the accuracy and efficiency of data synchronization.

[0177] The specific value of the preset data volume threshold can be set by the implementer according to actual conditions.

[0178] As described above, when the total data volume is less than the preset data volume threshold, the first intermediate data table is hash converted as a whole; when the total data volume is greater than or equal to the preset data volume threshold, the first intermediate data table is grouped and then hash converted. While ensuring the accuracy of the target hash data table, the efficiency of locating abnormal data can be improved during data consistency verification, thereby comprehensively improving the accuracy and efficiency of data synchronization.

[0179] S50, if the second data processing method is group hash conversion, then according to the amount of sub-data corresponding to each reference field name and the preset number of groups, the target group length corresponding to each reference field name is obtained.

[0180] The specific value of the preset number of groups can be set by the implementer according to actual conditions.

[0181] Specifically, for any reference field name, the ratio of the amount of sub-data corresponding to the current reference field name and the preset number of groups is determined as the candidate group length corresponding to the current reference field name. The candidate group length is rounded up to obtain the target group length corresponding to the current reference field name, which is used to group the application data corresponding to the current reference field name.

[0182] S60, performing group hash conversion on the application data in each reference field name according to a preset hash algorithm and a target group length corresponding to each reference field name, and obtaining a target hash data table.

[0183] In a specific implementation, S60 further includes the following steps:

[0184] S601, for any reference field name, group the application data corresponding to the current reference field name according to the target group length corresponding to the current reference field name, and obtain all target application data groups corresponding to the current reference field name.

[0185] S602: Process each target application data group according to a preset hash algorithm to obtain hash conversion data corresponding to each target application data group.

[0186] S603, traverse all reference field names to obtain all hash conversion data.

[0187] S604, forming a target hash data table according to all hash conversion data.

[0188] Among them, those skilled in the art know that any hash algorithm in the prior art falls within the protection scope of the present invention, and will not be described in detail here.

[0189] As described above, according to the preset hash algorithm and the target group length corresponding to each reference field name, the application data in each reference field name is grouped and hashed to obtain the target hash data table as the basis for verifying and synchronizing the target application data table.

[0190] S70, sending the target hash data table to the target client, wherein the target client is used to verify and synchronize the target application data table according to the target hash data table.

[0191] In a specific embodiment, the target client is used to obtain the actual group length corresponding to each target field name based on the amount of sub-data corresponding to each target field name in the target application data and the preset number of groups, and perform group hash conversion on the application data in all target field names according to the preset hash algorithm and the actual group length corresponding to each target field name to obtain the actual hash data table, compare and verify the target hash data table and the actual hash data table, and update the target application data table based on the comparison and verification results, so that the updated target application data table and the first intermediate data table are synchronized.

[0192] Furthermore, if the comparison verification result is that the target hash data table and the actual hash data table are consistent, the target application data table remains unchanged. If the comparison verification result is that the target hash data table and the actual hash data table are inconsistent, the abnormal hash value and the application data corresponding to the abnormal hash value in the first intermediate data table are located, and the application data corresponding to the abnormal hash value in the target application data table is updated according to the application data corresponding to the abnormal hash value in the first intermediate data table, so that the updated target application data table is synchronized with the first intermediate data table, thereby meeting various business needs corresponding to the target client.

[0193] In a specific implementation, S40 also includes the following content: if the first data processing mode remains unchanged, sending the first intermediate data table to the target client, wherein the target client is used to verify and synchronize the target application data table according to the first intermediate data table.

[0194] S50 also includes the following content: if the second data processing method is overall hash conversion, the application data in the first intermediate data table is overall hash converted according to a preset hash algorithm to obtain a target hash data table.

[0195] In the above, according to the number of data updates corresponding to the first intermediate data table, the number of selections and the number of exceptions corresponding to each reference field name within a preset time period, the hash conversion requirement degree corresponding to the first intermediate data table is obtained, and according to the hash conversion requirement degree and the preset requirement degree threshold, the first data processing method corresponding to the first intermediate data table is determined. If the first data processing method is hash conversion, then according to the total data volume corresponding to the first intermediate data table and the preset data volume threshold, the second data processing method corresponding to the first intermediate data table is determined. If the second data processing method is group hash conversion, then according to the sub-data volume corresponding to each reference field name and the preset number of groups, the target group length corresponding to each reference field name is obtained. According to the preset hash algorithm and the target group length corresponding to each reference field name, group hash conversion is performed on the application data in each reference field name to obtain the target Hash data table, sends the target hash data table to the target client. It can be known that according to the number of data updates corresponding to the first intermediate data table, the number of selections and the number of exceptions corresponding to each reference field name within the preset time period, the cost of verifying the application data in the first intermediate data table and the target application data table one by one, and the cost of performing hash conversion and then verifying the application data in the first intermediate data table and the target application data table are analyzed. Based on this, the processing form of the first intermediate data table is selected, and when the total data volume is less than the preset data volume threshold, the first intermediate data table is hash converted as a whole. When the total data volume is greater than or equal to the preset data volume threshold, the first intermediate data table is grouped and then hash converted. While ensuring the accuracy of the target hash data table, the efficiency of locating abnormal data can be improved during data consistency verification, thereby comprehensively improving the accuracy and efficiency of data synchronization.

[0196] Embodiment 4

[0197] The fourth embodiment provides a data sending device, the data sending device includes: Figure 4 As shown:

[0198] The first data acquisition module 41 is used to obtain the target application data table provided by the target client, the first intermediate data table corresponding to the target application data table stored on the server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, the reference field name corresponding to each target field name in the target application data table, and the comparison and verification results between the target application data table provided by the target client and the first intermediate data table, wherein the comparison and verification results include consistency and inconsistency.

[0199] The second data acquisition module 42 is used to obtain the first number of comparison and verification results provided by the target client within a preset time period, the total amount of data corresponding to the first intermediate data table, and the number of data updates corresponding to the first intermediate data table within the preset time period if the comparison and verification results are inconsistent.

[0200] The abnormal field name determination module 43 is used to determine the current target field name as an abnormal field name for any target field name in the target application data table if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table.

[0201] The overall demand degree acquisition module 44 is used to obtain the overall demand degree of the target client for the first intermediate data table based on the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates.

[0202] The data sending mode determination module 45 is used to determine the data sending mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data sending mode includes sending overall data and sending abnormal related data.

[0203] The to-be-sent data table acquisition module 46 is used to acquire the to-be-sent data table according to the data transmission mode, the first intermediate data table and the reference field name corresponding to each abnormal field name.

[0204] The data sending module 47 is used to send the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

[0205] In a specific implementation, the first data acquisition module 41 further includes:

[0206] The first data acquisition submodule is used to acquire several initial application data tables stored on the server side, first attribute information of each initial application data table, and a target application data table provided by a target client and second attribute information of the target application data table, wherein the first attribute information includes an initial table name corresponding to the initial application data table, several initial field names and a field type corresponding to each initial field name, and the second attribute information includes a target table name of the target application data table, several target field names and a field type corresponding to each target field name.

[0207] The matching degree acquisition submodule is used to acquire the matching degree between the target application data table and each initial application data table according to the second attribute information of the target application data table and the first attribute information of each initial application data table.

[0208] The designated application data table acquisition submodule is used to determine the initial application data table corresponding to the maximum matching degree as the designated application data table corresponding to the target application data table.

[0209] The first intermediate data table acquisition submodule is used to filter and obtain the first intermediate data table from the designated application data table according to the target field name corresponding to the target application data table and the initial field name corresponding to the designated application data table.

[0210] In a specific implementation, the second data acquisition module 42 further includes:

[0211] The first matching score acquisition submodule is used to acquire, for any initial application data table, a first matching score corresponding to the current initial application data table according to the initial table name and the target table name of the current initial application data table.

[0212] The second matching score acquisition submodule is used to obtain the second matching score corresponding to the current initial application data table based on several initial field names corresponding to the current initial application data table and the field type corresponding to each initial field name, as well as several target field names and the field type corresponding to each target field name.

[0213] The matching degree acquisition submodule is used to acquire the matching degree between the target application data table and the current initial application data table according to the first matching score and the second matching score.

[0214] In a specific implementation, the overall demand level acquisition module 44 further includes:

[0215] The abnormality degree acquisition submodule is used to obtain the abnormality degrees corresponding to all abnormal field names based on the reference field name corresponding to each abnormal field name, and the number of selections and abnormalities corresponding to each reference field name within a preset time period.

[0216] The importance acquisition submodule is used to acquire the importance corresponding to the first intermediate data table according to the first quantity, the total data volume and the number of data updates.

[0217] The overall demand level acquisition submodule is used to acquire the overall demand level corresponding to the first intermediate data table according to the abnormality level and the importance level.

[0218] In a specific implementation, the abnormality degree acquisition submodule further includes:

[0219] The first abnormal sub-degree obtaining unit is used to obtain, for any abnormal field name, the abnormal sub-degree corresponding to the current abnormal field name according to the number of selections and the number of abnormalities corresponding to the reference field name corresponding to the current abnormal field name within a preset time period.

[0220] The second abnormal sub-degree obtaining unit is used to traverse all abnormal field names and obtain the abnormal sub-degree corresponding to each abnormal field name.

[0221] The abnormality degree acquisition unit is used to determine the sum of all abnormal sub-degrees as the abnormality degree corresponding to all abnormal field names.

[0222] In a specific implementation, the data transmission mode determination module 45 further includes:

[0223] The first data sending mode determining submodule is used to determine that the data sending mode corresponding to the first intermediate data table is to send the entire data if the overall demand level is greater than a preset overall demand level threshold.

[0224] The second data sending mode determining submodule is used to determine that the data sending mode corresponding to the first intermediate data table is to send abnormal related data if the overall demand level is less than or equal to a preset overall demand level threshold.

[0225] In a specific implementation, the module 46 for obtaining the data table to be sent further includes:

[0226] The first to-be-sent data table acquisition submodule is used to determine the first intermediate data table as the to-be-sent data table if the data sending mode is to send the entire data.

[0227] The location identifier acquisition submodule is used to obtain the location identifier of each abnormal field name in the target application data table if the data sending mode is to send abnormal related data.

[0228] The first data to be sent acquiring submodule is used to determine the application data corresponding to the reference field name corresponding to each abnormal field name in the first intermediate data table as the first data to be sent.

[0229] The second data to be sent acquisition submodule is used to acquire the second data to be sent corresponding to each first data to be sent according to the corresponding position identifier of each first data to be sent and the abnormal field name corresponding to each first data to be sent in the target application data table.

[0230] The second to-be-sent data table acquisition submodule is used to form a to-be-sent data table according to all the second to-be-sent data.

[0231] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0232] Embodiment 5

[0233] Embodiment 5 of the present invention provides a non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps:

[0234] S100, obtain the target application data table provided by the target client, the first intermediate data table corresponding to the target application data table stored on the server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, the reference field name corresponding to each target field name in the target application data, and the comparison and verification results between the target application data provided by the target client and the first intermediate data table, wherein the comparison and verification results include consistency and inconsistency.

[0235] S200, if the comparison and verification result is inconsistent, obtain the first number of comparison and verification results provided by the target client within a preset time period, the total amount of data corresponding to the first intermediate data table, and the number of data updates corresponding to the first intermediate data table within the preset time period.

[0236] S300, for any target field name in the target application data table, if the application data corresponding to the current target field name in the target application data is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table, the current target field name is determined as an abnormal field name.

[0237] S400, based on the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates, the overall demand degree of the target client for the first intermediate data is obtained.

[0238] S500, determining a data sending mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data sending mode includes sending overall data and sending abnormality-related data.

[0239] S600, acquiring a data table to be sent according to a data sending mode, a first intermediate data table, and a reference field name corresponding to each abnormal field name.

[0240] S700, sending the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

[0241] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchronization Link) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0242] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, and the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0243] Embodiment 6

[0244] A sixth embodiment of the present invention provides an electronic device, which includes a processor and the non-transitory computer-readable storage medium in the fifth embodiment of the present invention.

[0245] The above are only preferred embodiments of the present invention, and are not intended to limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A data transmission method, It is characterized in that The data sending method comprises the following steps: S100, obtaining a target application data table provided by a target client, a first intermediate data table corresponding to the target application data table stored on a server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, a reference field name corresponding to each target field name in the target application data table, and a comparison verification result between the target application data table provided by the target client and the first intermediate data table, wherein the comparison verification result includes consistency and inconsistency, the number of selections refers to the number of times the corresponding reference field name is purchased by the client within the preset time period, and the number of exceptions refers to the number of times the corresponding reference field name is inconsistent with the corresponding application data in the target application data table when participating in data consistency verification within the preset time period; S200, if the comparison and verification result is inconsistent, obtaining a first number of comparison and verification results provided by the target client within the preset time period, a total amount of data corresponding to the first intermediate data table, and a number of data updates corresponding to the first intermediate data table within the preset time period; S300, for any target field name in the target application data table, if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table, the current target field name is determined as an abnormal field name; S400, obtaining the overall demand degree of the target client for the first intermediate data table according to the reference field name corresponding to each abnormal field name, the selection times and abnormal times corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the data update times, wherein the overall demand degree is used to represent the priority of sending the first intermediate data table as a whole to the target client; S500, determining a data sending mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data sending mode includes sending overall data and sending abnormality-related data; S600, acquiring a data table to be sent according to the data sending mode, the first intermediate data table and a reference field name corresponding to each abnormal field name; S700, sending the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

2. The data transmission method according to claim 1, It is characterized in that S100 also includes the following steps: S110, obtaining a plurality of initial application data tables stored on the server side, first attribute information of each initial application data table, and a target application data table provided by the target client and second attribute information of the target application data table, wherein the first attribute information includes an initial table name corresponding to the initial application data table, a plurality of initial field names, and a field type corresponding to each initial field name, and the second attribute information includes a target table name of the target application data table, a plurality of target field names, and a field type corresponding to each target field name; S120, acquiring a matching degree between the target application data table and each initial application data table according to the second attribute information of the target application data table and the first attribute information of each initial application data table; S130, determining the initial application data table corresponding to the maximum matching degree as the designated application data table corresponding to the target application data table; S140: Filter and obtain a first intermediate data table from the designated application data table according to the target field name corresponding to the target application data table and the initial field name corresponding to the designated application data table.

3. The data transmission method according to claim 2, It is characterized in that S120 also includes the following steps: S121, for any initial application data table, obtaining a first matching score corresponding to the current initial application data table according to the initial table name of the current initial application data table and the target table name; S122, obtaining a second matching score corresponding to the current initial application data table according to a plurality of initial field names corresponding to the current initial application data table and a field type corresponding to each initial field name, and a plurality of target field names and a field type corresponding to each target field name; S123: Acquire a matching degree between the target application data table and the current initial application data table according to the first matching score and the second matching score.

4. The data transmission method according to claim 1, It is characterized in that S400 also includes the following steps: S410, obtaining the abnormality degree corresponding to all abnormal field names according to the reference field name corresponding to each abnormal field name, and the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period; S420, acquiring the importance level corresponding to the first intermediate data table according to the first quantity, the total data volume and the number of data updates; S430: Obtain the overall demand level corresponding to the first intermediate data table according to the abnormality level and the importance level.

5. The data transmission method according to claim 1, It is characterized in that S410 also includes the following steps: S411, for any abnormal field name, according to the number of selections and the number of abnormalities corresponding to the reference field name corresponding to the current abnormal field name within a preset time period, obtain the abnormal sub-degree corresponding to the current abnormal field name; S412, traverse all abnormal field names, and obtain the abnormal sub-degree corresponding to each abnormal field name; S413: The sum of all abnormal sub-degrees is determined as the abnormal degree corresponding to all abnormal field names.

6. The data transmission method according to claim 1, It is characterized in that S500 includes the following steps: S510, if the overall demand level is greater than a preset overall demand level threshold, determining that the data sending mode corresponding to the first intermediate data table is to send overall data; S520: If the overall demand level is less than or equal to a preset overall demand level threshold, determine that the data sending method corresponding to the first intermediate data table is to send abnormality-related data.

7. The data transmission method according to claim 1, It is characterized in that S600 also includes the following steps: S610, if the data sending mode is to send the entire data, determining the first intermediate data table as the data table to be sent; S620, if the data sending mode is to send exception-related data, then obtain the position identifier of each exception field name in the target application data table; S630, determining the application data corresponding to the reference field name corresponding to each abnormal field name in the first intermediate data table as the first data to be sent; S640, acquiring second data to be sent corresponding to each first data to be sent according to each first data to be sent and the corresponding position identifier of the abnormal field name corresponding to each first data to be sent in the target application data table; S650: compose a table of data to be sent according to all the second data to be sent.

8. A data transmission device, It is characterized in that The data sending device comprises: A first data acquisition module is used to acquire a target application data table provided by a target client, a first intermediate data table corresponding to the target application data table stored on a server, the number of selections and the number of exceptions corresponding to each reference field name in the first intermediate data table within a preset time period, a reference field name corresponding to each target field name in the target application data table, and a comparison verification result between the target application data table provided by the target client and the first intermediate data table, wherein the comparison verification result includes consistency and inconsistency, the number of selections refers to the number of times the corresponding reference field name is purchased by the client within the preset time period, and the number of exceptions refers to the number of times the corresponding reference field name is inconsistent with the corresponding application data in the target application data table when participating in data consistency verification within the preset time period; A second data acquisition module is used to acquire, if the comparison and verification result is inconsistent, a first number of comparison and verification results provided by the target client within the preset time period, a total amount of data corresponding to the first intermediate data table, and a number of data updates corresponding to the first intermediate data table within the preset time period; an abnormal field name determination module, for determining, for any target field name in the target application data table, the current target field name as an abnormal field name if the application data corresponding to the current target field name in the target application data table is inconsistent with the application data corresponding to the reference field name corresponding to the current target field name in the first intermediate data table; an overall demand degree acquisition module, for acquiring the overall demand degree of the target client for the first intermediate data table according to the reference field name corresponding to each abnormal field name, the number of selections and the number of abnormalities corresponding to each reference field name within a preset time period, the first quantity, the total data volume and the number of data updates, wherein the overall demand degree is used to represent the priority of sending the first intermediate data table as a whole to the target client; a data transmission mode determination module, configured to determine a data transmission mode corresponding to the first intermediate data table according to the overall demand level and a preset overall demand level threshold, wherein the data transmission mode includes transmitting overall data and transmitting abnormality-related data; A module for acquiring a data table to be sent, configured to acquire the data table to be sent according to the data sending mode, the first intermediate data table and a reference field name corresponding to each abnormal field name; The data sending module is used to send the data table to be sent to the target client, wherein the target client is used to synchronize the target application data table according to the data table to be sent.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium. It is characterized in that The at least one instruction or the at least one program is loaded and executed by the processor to implement the data sending method as described in any one of claims 1-7.

10. An electronic device, It is characterized in that The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.