Data processing and auditing method and system

Through the methods of data acquisition, standardized processing and multiple audit rules, the problems of data audit vulnerabilities and repetitive work in the existing technology are solved, and the efficiency and accuracy of data audits are improved.

CN119025510APending Publication Date: 2024-11-26HEFEI GUQIU DATA TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411049395.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

There are vulnerabilities and repetitive work in the existing data processing audit methods, resulting in waste of audit time and reduced accuracy, and the inability to effectively check data problems.

Method used

Propose a data processing and audit method, including data acquisition, standardized processing, missing data detection, audit rule formulation and multiple audits, and determine data integrity by creating a database as a reference data, and formulate different audits based on the importance of data and acquisition time. rule.

Benefits of technology

Through the application of multiple detection and different audit rules, the efficiency and accuracy of data audits are improved, repetitive work is reduced, and the utilization rate of audit time is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025510A_ABST
    Figure CN119025510A_ABST
Patent Text Reader

Abstract

The invention discloses a data processing and auditing method and system. The data processing and auditing method comprises the following steps: acquiring data; performing data standard processing on the acquired data; making an auditing rule for the data subjected to the standard processing; and auditing the data after standard processing according to an auditing rule. According to the data processing auditing method provided by the invention, the acquired data is subjected to multiple detection in the data processing, whether the data is missing or not can be judged through the detection, and different auditing rules are formulated for the data in the subsequent data processing process, so that the data processing accuracy is improved. Different auditing is carried out on the data under different conditions through different auditing rules, so that the data auditing mode is more efficient, and meanwhile, the auditing time is shortened.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular, to a data processing and auditing method and system thereof. Background Art

[0002] The rapid development of Internet technology has greatly changed the development process of modern industry. In the process of development, more and more artificial technologies have begun to process data automatically. In terms of data processing and auditing, there are often some imprecise methods. For example, the data audit process ends after a data processing audit, which will cause many loopholes in the data processing and audit process, or the same processing and audit is performed many times, but this repetitive work is often just a repetition of work, and the problems in the data cannot be effectively checked, so a lot of processing and auditing time will be wasted, and the accuracy of the processing and auditing data will be reduced.

[0003] Therefore, how to provide a method to improve the audit time of data and the accuracy of audit data has become an urgent problem to be solved in this field. Summary of the invention

[0004] The present application proposes a data processing and auditing method, comprising the following steps: acquiring data; performing standard processing on the acquired data; formulating auditing rules for the data after standard processing; and auditing the data after standard processing according to the auditing rules.

[0005] As mentioned above, the acquired data is a variety of data information of different types.

[0006] As mentioned above, the standardized processing of the acquired data includes the following sub-steps: determining whether there is missing data in the acquired data; if there is missing data, re-acquiring the data; if there is no missing data, formulating audit rules for the standardized processed data.

[0007] As described above, before determining whether the acquired data contains missing data, it further includes pre-creating a database, using the data in the database as reference data, and determining whether the acquired data contains missing data based on the reference data.

[0008] As mentioned above, the formulation of audit rules for the standardized data includes formulating a first audit rule and a second audit rule.

[0009] A data processing and auditing system comprises a data acquisition unit, a standard processing unit, an audit rule formulation unit and an audit unit; the data acquisition unit is used to acquire data; the standard processing unit is used to perform standard processing on the acquired data; the audit rule formulation unit is used to formulate audit rules for the data after standard processing; and the audit unit is used to audit the data after standard processing according to the audit rules.

[0010] As mentioned above, the data acquired by the data acquisition unit is a variety of data information of different types.

[0011] As mentioned above, the standard processing unit performs standard processing on the acquired data, including the following sub-steps: determining whether there is missing data in the acquired data; if there is missing data, re-acquiring the data; if there is no missing data, formulating audit rules for the data after standard processing.

[0012] As described above, before the specification processing unit determines whether the acquired data has missing data, it also includes pre-creating a database, using the data in the database as reference data, and determining whether the acquired data has missing data based on the reference data.

[0013] As described above, the audit rule formulation unit formulates audit rules for the data after standard processing, including formulating a first audit rule and a second audit rule.

[0014] This application has the following beneficial effects:

[0015] The present application provides a data processing audit method and system thereof, which performs multiple tests on the acquired data during data processing. Through such tests, it is possible to determine whether there are data missing, and in the subsequent data processing process, different audit rules are formulated for the data. Different audits are performed on data in different situations through different audit rules, making the data audit method more efficient and reducing the audit time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a flow chart of a data processing and auditing method provided according to an embodiment of the present application;

[0018] Figure 2It is a schematic diagram of the internal structure of a data processing and auditing system provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0020] Embodiment 1

[0021] like Figure 1 As shown, the present application provides a data processing and review method, which specifically includes the following steps:

[0022] Step S110: Acquire data.

[0023] Specifically, the data obtained is a variety of data information that needs to be processed and reviewed, which can be obtained from a database. The database is a pre-created database containing various types of data, including various types of data that need and do not need to be reviewed, and can be set in advance by staff. The data contained in the database is not limited here.

[0024] The data obtained from the database can be data that needs to be processed and reviewed in any scenario. For example, if you want to process and review the information of a gas station convenience store, the data obtained can be the location information of the gas station convenience store, the product placement information in the convenience store, the production date of the product, and other data.

[0025] Step S120: performing data standardization processing on the acquired data.

[0026] In this embodiment, the standardized processing of data specifically includes the following sub-steps:

[0027] Step S1201: Determine whether there is any missing data in the acquired data.

[0028] The obtained multiple data are used as comparison data, and the corresponding data in the database are used as reference data. It is determined based on the reference data whether there is any missing data in the comparison data.

[0029] If there is missing data, then execute step S1202. Otherwise, execute step S130.

[0030] The comparison data is k j =[k j1 ,k j2 ,...k ji ...k jp],k j1 ,k j2 ,...k ji ...k jp Represents any data value in the comparison data, and the reference data is k q =[k q1 ,k q2 ,...k qi ...k qp ],k q1 ,k q2 ,...k qi ...k qp Represents any data value in the reference data. Whether the data is missing is determined by comparing the data similarity between the data and the reference data.

[0031] Furthermore, in the process of determining whether there is data missing in the comparison data, this embodiment introduces a weight parameter, that is, assigning an importance to the comparison data, which is also a factor in determining whether the data is missing. For example, if it is considered that one or more data in the comparison data are not so important, whether the data is missing has little impact on the overall result to be obtained in this embodiment, then the corresponding weight parameter ω is assigned to it in advance. ji , in order to determine the final result. Weight parameter ω ji The value of ω ranges from 0.1 to 1. ji The larger the value is, the more important the data is, and vice versa.

[0032] The data similarity result X between the comparison data and the reference data is specifically expressed as:

[0033]

[0034] When the data similarity result is greater than the specified threshold, it means that the comparison data and the reference data are more similar, and it is considered that there is no data missing in the comparison data, and step S130 is executed. Otherwise, step S1202 is executed.

[0035] Step S1202: Re-acquire data.

[0036] Step S130: Formulate audit rules for the standardized data.

[0037] In the above steps, this embodiment has detected whether the acquired data is missing, but this missing detection is often one-sided. Therefore, in this step, this embodiment formulates different audit rules for the various acquired data based on factors such as the acquisition time and importance of the data.

[0038] Wherein step S130 specifically includes the following sub-steps:

[0039] Step S1301: Determine the importance of the acquired data.

[0040] In this embodiment, the overall importance of the acquired data is determined based on factors such as the acquisition time and importance of the data. If the importance is greater than a specified threshold, it means that the acquired data is more important. If the importance is less than the specified threshold, it means that the acquired data is not so important. Different audit rules are formulated according to the importance of the data.

[0041] The importance R is specifically expressed as:

[0042]

[0043] Among them, C j1 Indicates obtaining data k j Time, C j2 Indicates that the system is acquiring multiple data k j Whether other data was being obtained before. If other data was being obtained, then C j2 =0, otherwise C j2 =1,ω jit Indicates the current data k ji The importance at time t, ω jit-1 Indicates the current data k ji The importance at time t-1, T represents the total number of time points, and p represents the total number of acquired data.

[0044] Step S1302: formulate audit rules according to the importance of the acquired data.

[0045] If the importance ratio is greater than the specified threshold, the first review rule is formulated.

[0046] If the importance ratio is less than the specified threshold, the second review rule will be formulated.

[0047] Specifically, the first audit rule is to perform real-time data audit on the acquired data, and the second audit rule is to perform sample data audit on the acquired data.

[0048] Real-time data review is to review the acquired data in real time after it is acquired. Specifically, it is to review whether there are any failures or jams in the data acquisition process. If the review finds that the above situation exists in the acquisition process, an abnormal prompt will be issued immediately.

[0049] The sampling data audit is to repeatedly audit the acquired data a specified number of times within a specified time after the data is acquired, to check whether there are any failures or jams in the data acquisition process. If the above situation is found in the acquisition process during one or more audits, an abnormal prompt will be issued immediately.

[0050] Step S140: Review the data after standard processing according to the review rules.

[0051] The corresponding data are audited according to real-time data audit and sampling data audit respectively. Once an abnormality is found during the audit, an abnormal prompt will be given immediately.

[0052] Embodiment 2

[0053] like Figure 2 As shown, the present application provides a data processing and auditing system, wherein the system specifically includes: a data acquisition unit 210, a specification processing unit 220, an audit rule formulation unit 230 and an audit unit 240.

[0054] The data acquisition unit 210 is used to acquire data.

[0055] Specifically, the data obtained is a variety of data information that needs to be processed and reviewed, which can be obtained from a database. The database is a pre-created database containing various types of data, including various types of data that need and do not need to be reviewed, and can be set in advance by staff. The data contained in the database is not limited here.

[0056] The data obtained from the database can be data that needs to be processed and reviewed in any scenario. For example, if you want to process and review the information of a gas station convenience store, the data obtained can be the location information of the gas station convenience store, the product placement information in the convenience store, the production date of the product, and other data.

[0057] The standard processing unit 220 is used to perform standard processing on the acquired data.

[0058] In this embodiment, the standard processing unit 220 includes the following submodules: a missing data determination module and a re-acquisition module.

[0059] The missing data judgment module is used to judge whether there is missing data in the acquired data.

[0060] The obtained multiple data are used as comparison data, and the corresponding data in the database are used as reference data. It is determined based on the reference data whether there is any missing data in the comparison data.

[0061] If there is missing data, then execute step S1202. Otherwise, execute step S130.

[0062] The comparison data is k j =[k j1 ,k j2 ,...k ji ...k jp ],kj1 ,k j2 ,...k ji ...k jp Represents any data value in the comparison data, and the reference data is k q =[k q1 ,k q2 ,...k qi ...k qp ],k q1 ,k q2 ,...k qi ...k qp Represents any data value in the reference data. Whether the data is missing is determined by comparing the data similarity between the data and the reference data.

[0063] Furthermore, in the process of determining whether there is data missing in the comparison data, this embodiment introduces a weight parameter, that is, assigning an importance to the comparison data, which is also a factor in determining whether the data is missing. For example, if it is considered that one or more data in the comparison data are not so important, whether the data is missing has little impact on the overall result to be obtained in this embodiment, then the corresponding weight parameter ω is assigned to it in advance. ji , in order to determine the final result. Weight parameter ω ji The value of ω ranges from 0.1 to 1. ji The larger the value is, the more important the data is, and vice versa.

[0064] The data similarity result X between the comparison data and the reference data is specifically expressed as:

[0065]

[0066] When the data similarity result is greater than the specified threshold, it means that the comparison data is more similar to the reference data, and it is considered that there is no data missing in the comparison data, and the audit rule formulation unit 230 is executed. Otherwise, the re-acquisition module is executed.

[0067] The re-acquisition module is used to re-acquire data.

[0068] The audit rule formulation unit 230 is used to formulate audit rules for the data after standard processing.

[0069] In the above steps, this embodiment has detected whether the acquired data is missing, but this missing detection is often one-sided. Therefore, in this step, this embodiment formulates different audit rules for the various acquired data based on factors such as the acquisition time and importance of the data.

[0070] The audit rule formulation unit 230 specifically includes the following submodules: an importance determination module and a formulation module.

[0071] The importance determination module is used to determine the importance of the acquired data.

[0072] In this embodiment, the overall importance of the acquired data is determined based on factors such as the acquisition time and importance of the data. If the importance is greater than a specified threshold, it means that the acquired data is more important. If the importance is less than the specified threshold, it means that the acquired data is not so important. Different audit rules are formulated according to the importance of the data.

[0073] The importance R is specifically expressed as:

[0074]

[0075] Among them, C j1 Indicates obtaining data k j Time, C j2 Indicates that the system is acquiring multiple data k j Whether other data was being obtained before. If other data was being obtained, then C j2 =0, otherwise C j2 =1,ω jit Indicates the current data k ji The importance at time t, ω jit-1 Indicates the current data k ji The importance at time t-1, T represents the total number of time points, and p represents the total number of acquired data.

[0076] The formulation module is used to formulate audit rules based on their importance.

[0077] If the importance ratio is greater than the specified threshold, the first review rule is formulated.

[0078] If the importance ratio is less than the specified threshold, the second review rule will be formulated.

[0079] Specifically, the first audit rule is to perform real-time data audit on the acquired data, and the second audit rule is to perform sample data audit on the acquired data.

[0080] Real-time data review is to review the acquired data in real time after it is acquired. Specifically, it is to review whether there are any failures or jams in the data acquisition process. If the review finds that the above situation exists in the acquisition process, an abnormal prompt will be issued immediately.

[0081] The sampling data audit is to repeatedly audit the acquired data a specified number of times within a specified time after the data is acquired, to check whether there are any failures or jams in the data acquisition process. If the above situation is found in the acquisition process during one or more audits, an abnormal prompt will be issued immediately.

[0082] The review unit 240 is used to review the data after the standard processing according to the review rules.

[0083] The corresponding data are audited according to real-time data audit and sampling data audit respectively. Once an abnormality is found during the audit, an abnormal prompt will be given immediately.

[0084] This application has the following beneficial effects:

[0085] The present application provides a data processing audit method and system thereof, which performs multiple tests on the acquired data during data processing. Through such tests, it is possible to determine whether there are data missing, and in the subsequent data processing process, different audit rules are formulated for the data. Different audits are performed on data in different situations through different audit rules, making the data audit method more efficient and reducing the audit time.

[0086] Although the present application is described with reference to examples, this is for illustrative purposes only and is not intended to limit the present application, and changes, additions and / or deletions to the embodiments may be made without departing from the scope of the present application.

[0087] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing and auditing method, It is characterized in that The following steps are involved: Get data; Perform standardized processing on the acquired data; Formulate audit rules for standardized data; Audit the data after standard processing according to the audit rules.

2. The data processing and auditing method according to claim 1, It is characterized in that The acquired data is a variety of data information of different types.

3. The data processing and auditing method according to claim 1, It is characterized in that The standardized processing of the acquired data includes the following sub-steps: Determine whether the acquired data has missing data; If there is missing data, re-acquire the data; If there is no missing data, audit rules will be formulated for the standardized data.

4. The data processing and auditing method according to claim 3, It is characterized in that Before determining whether the acquired data has missing data, it also includes pre-creating a database, using the data in the database as reference data, and determining whether the acquired data has missing data based on the reference data.

5. The data processing and auditing method according to claim 1, It is characterized in that The formulation of audit rules for standardized data includes the formulation of first audit rules and second audit rules.

6. A data processing and auditing system, It is characterized in that It includes a data acquisition unit, a normative processing unit, an audit rule formulation unit, and an audit unit; The data acquisition unit is used to acquire data; The standard processing unit is used to perform standard processing on the acquired data; The audit rule formulation unit is used to formulate audit rules for data after standard processing; The audit unit is used to audit the data after standard processing according to the audit rules.

7. The data processing and auditing system according to claim 6, It is characterized in that The data acquired by the data acquisition unit is a variety of data information of different types.

8. The data processing and auditing system according to claim 6, It is characterized in that The standard processing unit performs standard processing on the acquired data, including the following sub-steps: Determine whether the acquired data has missing data; If there is missing data, re-acquire the data; If there is no missing data, audit rules will be formulated for the standardized data.

9. The data processing and auditing system according to claim 8, It is characterized in that Before the specification processing unit determines whether there is missing data in the acquired data, it also includes pre-creating a database, using the data in the database as reference data, and determining whether there is missing data in the acquired data based on the reference data.

10. The data processing and auditing system according to claim 6, It is characterized in that The audit rule formulation unit formulates the audit rules for the data after the standard processing, including formulating the first audit rule and the second audit rule.