Tracking and tracing system based on big data analysis

Abstract

The invention discloses a tracking and tracing system based on big data analysis. The system comprises a data quality verification module, an anonymization technology module, a multi-dimensional supervision module and a cooperative ecological module. According to the invention, cleaning and verification of original data are realized through the data quality verification module, wrong, redundant and missing data are removed, consistency verification is carried out on data of different data sources by using a data comparison algorithm, and the accuracy and consistency of the data are ensured; through data encryption and authority control, confidentiality of data in storage and transmission processes is ensured, and data leakage and unauthorized access are prevented; an early warning mechanism and a problem solving scheme are designed, so that related personnel can be timely notified to solve problems, and the timeliness and efficiency of supervision are improved; a cooperation relationship and a data sharing platform are established, cooperation and information sharing among participants are promoted, a preferential policy and a reward mechanism are designed, all parties are stimulated to actively participate in system construction and data sharing, and the power and effect of cooperation are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of big data, and in particular to a tracking and tracing system based on big data analysis. Background Art

[0002] In today's era of rapid digital development, tracking and tracing systems based on big data analysis have emerged. It is a comprehensive system that makes full use of big data technology and various analysis methods that are compatible with it. It aims to track and trace the entire process of products from the initial production link, through the circulation path, to the final arrival at the consumer terminal, or the generation, circulation and use of various types of data in an all-round and in-depth manner.

[0003] The effective operation of the system is highly dependent on the quality of the acquired data, which must have key characteristics such as comprehensiveness, accuracy and timeliness. Only in this way can the tracking and tracing work be carried out smoothly, and reliable and practical traceability information can be output. However, in real application scenarios, it faces many severe challenges.

[0004] On the one hand, due to the fact that some participants are driven by interests or other improper motives, they may engage in bad behaviors such as data falsification and providing false reports. For example, in the context of product traceability, some manufacturers deliberately tamper with data records in the production process and falsely report key information such as the source of raw materials and production time in order to cover up product quality problems or evade responsibility; in terms of data traceability, some data owners may forge data flow tracks for the purpose of unfair competition, thereby destroying the authenticity of data in the entire traceability chain. Once such false data is mixed into the system, it will seriously interfere with the normal tracking and tracing work, causing the final traceability information to lose its due reliability and fail to truly reflect the actual situation of the product or data, thereby greatly weakening the important role that the system should play in ensuring quality and safety and maintaining market order.

[0005] On the other hand, in the process of performing its functional duties, the tracking and tracing system often needs to widely collect and store massive amounts of data involving personal privacy and corporate trade secrets. This process undoubtedly puts the system at the forefront of the risk of privacy leakage. How to effectively protect data security and privacy has become a major challenge that needs to be solved urgently. Specifically, from the collection stage, if the data collection process lacks a strict and standardized authorization mechanism, it may be possible to collect personal sensitive information (such as consumers' detailed location information, health status, etc.) or core business data of enterprises beyond the scope; in the storage stage, if the security protection measures of the storage system are not in place, such as data encryption technology is not up to standard, access rights management is loose, etc., then the stored data is very vulnerable to security threats such as hacker attacks and internal personnel's illegal operations, which will lead to the theft and leakage of private information, causing immeasurable losses to individuals and enterprises, and may also cause a series of serious social problems, such as harassment caused by the violation of personal privacy and disordered market competition caused by the leakage of corporate trade secrets. Summary of the invention

[0006] The purpose of the present invention is to provide a tracking and tracing system based on big data analysis to solve the technical difficulties raised in the above-mentioned background technology that "there may be problems such as data falsification and false reporting in reality, which leads to unreliable traceability information. Secondly, the traceability system needs to collect and store a large amount of personal and corporate data, and there is a risk of privacy leakage."

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A tracking and tracing system based on big data analysis includes a data quality verification module, an anonymization technology module, a multi-dimensional supervision module and a cooperative ecology module. The data quality verification module introduces a data verification algorithm and a credibility assessment model to conduct real-time verification and monitoring of the data relied on by the system, including using a data cleaning algorithm to pre-process the data, exclude erroneous and missing data, and use a data comparison algorithm to verify data consistency to ensure data accuracy; the anonymization technology module uses anonymization technology to protect the privacy data of individuals and enterprises, and anonymizes the identities of individuals and enterprises by replacing real identity information with temporary identifiers. At the same time, encryption technology and permission control mechanisms are used to ensure the storage and transmission security of data in the tracing and tracing system to prevent privacy data leakage ; The multi-dimensional supervision module introduces data mining technology to strengthen the supervision ability of the regulatory authorities on the traceability system. By mining and analyzing the data in the system, it can timely monitor and identify abnormal events, including automatically triggering the early warning mechanism when product quality problems and food safety incidents occur, and notifying relevant regulatory authorities and enterprises so that timely measures can be taken to solve the problems and improve the supervision effect; The cooperative ecology module establishes a cooperative ecology of the traceability system, establishes cooperative relationships with participants in all links, shares data and information, and allows all parties to share data by establishing a data sharing platform. It also provides preferential policies and reward mechanisms to encourage all parties to actively participate in system construction and data sharing, promote data integrity and credibility, increase system coverage and effectiveness, and enhance the value of the entire traceability system.

[0009] The data quality verification module includes a data cleaning and verification unit and a false data detection unit. The data cleaning and verification unit collects original data and uses a data cleaning algorithm to perform preliminary cleaning, including deduplication, filling missing values, and correcting format errors. Then, a data comparison algorithm is used to perform data consistency verification, and data from different data sources are compared to ensure the consistency of data in different links. The false data detection unit uses an anomaly detection algorithm, including statistical outlier detection and cluster analysis, to detect false data, and then establishes a data anomaly model, defines the rules of normal data based on historical data and domain knowledge, and uses a data verification algorithm to monitor and analyze new data to detect whether there is false data.

[0010] The anonymization technology module includes an identity anonymization unit and a data security storage and transmission unit. The identity anonymization unit uses a hash function and an encryption algorithm to desensitize personal and corporate identity information and replaces them with temporary identifiers to ensure the uniqueness of the desensitized data, and uses a matching algorithm and indexing technology to ensure the relevance and query capability of the desensitized data; the data security storage and transmission unit uses encryption technology to encrypt and protect data during storage to ensure the confidentiality of the data, and uses access control lists and role-based permission control technology to set data access permissions to control the read and write permissions of the data.

[0011] The multi-dimensional supervision module includes a data mining and analysis unit and an early warning and problem solving unit. The data mining and analysis unit establishes supervision indicators and models, uses time series analysis algorithms to define warning lines and rules for abnormal events, uses classification and association rule algorithms to mine and analyze data in the system, and identifies abnormal events and trends; the early warning and problem solving unit establishes an early warning mechanism, sets trigger conditions and early warning notification methods, and after the early warning is triggered, notifies relevant personnel, tracks the cause of the problem and solves it in a timely manner, uses problem tracing algorithms to track the root causes of abnormal events and data anomalies, and takes corresponding solutions.

[0012] The cooperative ecological module includes a cooperative relationship establishment and data sharing unit and a preferential policy and reward mechanism unit. The cooperative relationship establishment and data sharing unit allows participants in various links to establish cooperative relationships, sign cooperative agreements, and then establish a data sharing platform, using data integration tools, data standardization and data exchange protocol technologies to ensure that participants share data and information; the preferential policy and reward mechanism unit designs preferential policies for data sharing, including fee reductions and service support, and uses reward algorithms and incentive models to design reward mechanisms, encouraging all parties to actively participate in system construction and data sharing through incentives.

[0013] The output data of the data quality verification module will be used as the input data of the anonymization technology module to ensure that the data before anonymization processing has high quality and consistency. The output data of the anonymization technology module will be used as the input data of the multi-dimensional supervision module and the cooperative ecology module to ensure that the analyzed and shared data can protect privacy and identity information.

[0014] The data quality verification module provides a reliable data foundation for the multi-dimensional supervision module to ensure the effectiveness and accuracy of supervision indicators and models. The multi-dimensional supervision module relies on the data cleaning and consistency verification results of the data quality verification module to analyze and identify abnormal events and trends.

[0015] The output data of the anonymization technology module is used for data mining and analysis of the multi-dimensional supervision module to identify abnormal events and trends. The multi-dimensional supervision module utilizes the data security storage and transmission mechanism of the anonymization technology module to ensure the confidentiality and integrity of the supervision data.

[0016] The data quality verification module provides a reliable data foundation for the cooperative ecological module to ensure the quality and consistency of data sharing. The cooperative ecological module uses the data cleaning and consistency verification results of the data quality verification module to manage the data sharing relationship of partners.

[0017] The identity anonymization and data security storage and transmission mechanism of the anonymization technology module provides data privacy protection for partners, and the cooperative ecological module relies on the identity anonymization and data sharing technology of the anonymization technology module to realize the establishment and operation of the data sharing platform.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) The data quality verification module is used to clean and verify the original data, remove erroneous, redundant, and missing data, and use data comparison algorithms to perform consistency checks on data from different data sources to ensure data accuracy and consistency.

[0020] (2) Ensure the confidentiality of data during storage and transmission through data encryption and permission control to prevent data leakage and unauthorized access.

[0021] (3) Design early warning mechanisms and problem-solving solutions to promptly notify relevant personnel and solve problems, thereby improving the timeliness and efficiency of supervision.

[0022] (4) Establish cooperative relationships and data sharing platforms to promote cooperation and information sharing among participants, design preferential policies and reward mechanisms to encourage all parties to actively participate in system construction and data sharing, and enhance the motivation and effectiveness of cooperation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the structure of a tracking and tracing system based on big data analysis of the present invention;

[0024] Figure 2 This is a relationship diagram of a tracking and tracing system based on big data analysis of the present invention.

[0025] 1-Data quality verification module; 2-Anonymization technology module; 3-Multi-dimensional supervision module; 4-Cooperative ecology module; 5-Data cleaning and verification unit; 6-False data detection unit; 7-Identity anonymization unit; 8-Data security storage and transmission unit; 9-Data mining and analysis unit; 10-Early warning and problem solving unit; 11-Cooperative relationship establishment and data sharing unit; 12-Preferential policies and reward mechanism unit. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0027] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1-2 As shown, a tracking and tracing system based on big data analysis includes a data quality verification module 1, an anonymization technology module 2, a multi-dimensional supervision module 3 and a cooperation ecology module 4.

[0029] The data quality verification module 1 introduces data verification algorithms and credibility assessment models to conduct real-time verification and monitoring of the data that the system relies on, including using a data cleaning algorithm to pre-process the data, eliminate erroneous and missing data, and using a data comparison algorithm to perform data consistency verification to ensure data accuracy. The data quality verification module 1 includes a data cleaning and verification unit 5. The data cleaning and verification unit 5 collects original data and uses a data cleaning algorithm to perform preliminary cleaning, including deduplication, filling missing values, and correcting format errors. Then, a data comparison algorithm is used to perform data consistency verification, compare data from different data sources, and ensure the consistency of data in different links.

[0030] The data quality verification module 1 also includes a false data detection unit 6, which uses an anomaly detection algorithm, including statistical outlier detection and cluster analysis, to detect false data, and then establishes a data anomaly model, defines the rules of normal data based on historical data and domain knowledge, and uses a data verification algorithm to monitor and analyze new data to detect whether there is false data.

[0031] The anonymization technology module 2 uses anonymization technology to protect the privacy data of individuals and enterprises. By replacing the real identity information with a temporary identifier, the anonymization of the personal and corporate identities is achieved. At the same time, by adopting encryption technology and permission control mechanism, the storage and transmission security of data in the traceability system is ensured to prevent the leakage of privacy data. The anonymization technology module 2 includes an identity anonymization unit 7. The identity anonymization unit 7 uses a hash function and an encryption algorithm to desensitize the personal and corporate identity information and replaces it with a temporary identifier to ensure the uniqueness of the desensitized data, and adopts a matching algorithm and indexing technology to ensure the relevance and query capability of the desensitized data.

[0032] The anonymization technology module 2 also includes a data security storage and transmission unit 8, which uses encryption technology to encrypt and protect data during storage to ensure the confidentiality of the data, and uses access control lists and role-based permission control technology to set data access permissions to control the read and write permissions of the data.

[0033] The multi-dimensional supervision module 3 introduces data mining technology to strengthen the supervision ability of the supervision department on the traceability system. By mining and analyzing the data in the system, it can timely monitor and identify abnormal events, including automatically triggering an early warning mechanism when product quality problems and food safety incidents occur, and notifying relevant supervision departments and enterprises so that timely measures can be taken to solve the problems and improve the supervision effect. The multi-dimensional supervision module 3 includes a data mining and analysis unit 9. The data mining and analysis unit 9 establishes supervision indicators and models, uses time series analysis algorithms to define warning lines and rules for abnormal events, and uses classification and association rule algorithms to mine and analyze data in the system to identify abnormal events and trends.

[0034] The multi-dimensional supervision module 3 also includes an early warning and problem solving unit 10, which establishes an early warning mechanism, sets trigger conditions and early warning notification methods, and after triggering the early warning, notifies relevant personnel, tracks the cause of the problem and solves it in a timely manner, uses a problem tracing algorithm to track the root causes of abnormal events and data anomalies, and takes corresponding solutions.

[0035] The cooperative ecological module 4 establishes a cooperative ecology of the traceability system, establishes cooperative relationships with participants in various links, shares data and information, and allows all participants to share data by establishing a data sharing platform. It also provides preferential policies and reward mechanisms to encourage all parties to actively participate in system construction and data sharing, promote data integrity and credibility, increase system coverage and effectiveness, and enhance the value of the entire traceability system. The cooperative ecological module 4 includes a cooperative relationship establishment and data sharing unit 11. The cooperative relationship establishment and data sharing unit 11 allows participants in various links to establish cooperative relationships, sign cooperation agreements, and then establish a data sharing platform, using data integration tools, data standardization, and data exchange protocol technologies to ensure that participants share data and information.

[0036] The cooperation ecology module 4 also includes a preferential policy and reward mechanism unit 12, which designs preferential policies for data sharing, including fee reductions and service support, and uses reward algorithms and incentive models to design reward mechanisms, and encourages all parties to actively participate in system construction and data sharing through incentives.

[0037] The output data of the data quality verification module 1 will be used as the input data of the anonymization technology module 2 to ensure that the data before anonymization processing has high quality and consistency. The output data of the anonymization technology module 2 will be used as the input data of the multi-dimensional supervision module 3 and the cooperative ecology module 4 to ensure that the analyzed and shared data can protect privacy and identity information.

[0038] The data quality verification module 1 provides a reliable data foundation for the multi-dimensional supervision module 3 to ensure the effectiveness and accuracy of the supervision indicators and models. The multi-dimensional supervision module 3 relies on the data cleaning and consistency verification results of the data quality verification module 1 to analyze and identify abnormal events and trends.

[0039] The output data of the anonymization technology module 2 is used for data mining and analysis of the multi-dimensional supervision module 3 to identify abnormal events and trends. The multi-dimensional supervision module 3 uses the data security storage and transmission mechanism of the anonymization technology module 2 to ensure the confidentiality and integrity of the supervision data.

[0040] The data quality verification module 1 provides a reliable data foundation for the cooperation ecology module 4 to ensure the quality and consistency of data sharing. The cooperation ecology module 4 uses the data cleaning and consistency verification results of the data quality verification module 1 to manage the data sharing relationship of partners.

[0041] The identity anonymization and data security storage and transmission mechanism of the anonymization technology module 2 provides data privacy protection for partners, and the cooperation ecology module 4 relies on the identity anonymization and data sharing technology of the anonymization technology module 2 to realize the establishment and operation of the data sharing platform.

[0042] The present invention realizes the cleaning and verification of original data through the data quality verification module 1, removes erroneous, redundant and missing data, uses the data comparison algorithm to perform consistency verification on data from different data sources, and ensures the accuracy and consistency of the data; through data encryption and authority control, ensures the confidentiality of data during storage and transmission, prevents data leakage and unauthorized access; designs early warning mechanisms and problem-solving solutions to promptly notify relevant personnel and solve problems, and improves the timeliness and efficiency of supervision; establishes cooperative relationships and data sharing platforms to promote cooperation and information sharing among participants, designs preferential policies and reward mechanisms to encourage all parties to actively participate in system construction and data sharing, and enhances the motivation and effect of cooperation.

Claims

1. A tracking and tracing system based on big data analysis, characterized by: The system comprises a data quality verification module (1), an anonymization technology module (2), a multi-dimensional supervision module (3) and a cooperative ecology module (4). The data quality verification module (1) introduces a data verification algorithm and a credibility assessment model to conduct real-time verification and monitoring of the data relied on by the system, including using a data cleaning algorithm to pre-process the data, eliminate erroneous and missing data, and use a data comparison algorithm to perform data consistency verification to ensure data accuracy; the anonymization technology module (2) uses anonymization technology to protect the privacy data of individuals and enterprises, and replaces the real identity information with a temporary identifier to achieve the anonymization of the identity of individuals and enterprises. At the same time, by adopting encryption technology and permission control mechanism, the storage and transmission security of data in the traceability system is ensured to prevent the leakage of privacy data; The multi-dimensional supervision module (3) introduces data mining technology to strengthen the supervision ability of the supervision department on the traceability system. By mining and analyzing the data in the system, it can timely monitor and identify abnormal events, including automatically triggering the early warning mechanism when product quality problems and food safety incidents occur, notifying relevant supervision departments and enterprises so that timely measures can be taken to solve the problems and improve the supervision effect; the cooperative ecology module (4) establishes a cooperative ecology of the traceability system, establishes cooperative relationships with participants in various links, shares data and information, and allows all parties to share data by establishing a data sharing platform. It also provides preferential policies and reward mechanisms to encourage all parties to actively participate in system construction and data sharing, promote data integrity and credibility, increase the coverage and effectiveness of the system, and enhance the value of the entire traceability system.

2. The tracking and tracing system based on big data analysis according to claim 1 is characterized by: The data quality verification module (1) comprises a data cleaning and verification unit (5) and a false data detection unit (6). The data cleaning and verification unit (5) collects raw data and uses a data cleaning algorithm to perform preliminary cleaning, including deduplication, filling missing values, and correcting format errors. The data comparison algorithm is then used to perform data consistency verification, comparing data from different data sources to ensure data consistency at different links. The false data detection unit (6) uses an anomaly detection algorithm, including outlier detection and cluster analysis based on statistics, to perform false data detection, and then establishes a data anomaly model, defines normal data rules based on historical data and domain knowledge, and uses a data verification algorithm to monitor and analyze new data to detect whether there is false data.

3. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The anonymization technology module (2) comprises an identity anonymization unit (7) and a data security storage and transmission unit (8). The identity anonymization unit (7) uses a hash function and an encryption algorithm to perform desensitization processing on personal and corporate identity information and replaces them with temporary identifiers to ensure the uniqueness of the desensitized data, and uses a matching algorithm and indexing technology to ensure the relevance and query capability of the desensitized data; the data security storage and transmission unit (8) uses encryption technology to encrypt and protect data during storage to ensure the confidentiality of the data, and uses access control lists and role-based permission control technology to set data access rights to control the read and write permissions of the data.

4. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The multi-dimensional supervision module (3) includes a data mining and analysis unit (9) and an early warning and problem solving unit (10), wherein the data mining and analysis unit (9) establishes supervision indicators and models, uses a time series analysis algorithm to define warning lines and rules for abnormal events, and uses classification and association rule algorithms to mine and analyze data in the system to identify abnormal events and trends; The early warning and problem solving unit (10) establishes an early warning mechanism, sets trigger conditions and early warning notification methods, and after the early warning is triggered, notifies relevant personnel, tracks the cause of the problem and solves it in a timely manner, uses a problem tracing algorithm to track the root causes of abnormal events and data anomalies, and takes corresponding solutions.

5. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The cooperative ecological module (4) includes a cooperative relationship establishment and data sharing unit (11) and a preferential policy and reward mechanism unit (12). The cooperative relationship establishment and data sharing unit (11) allows participants in various links to establish cooperative relationships, sign cooperative agreements, and then establish a data sharing platform, using data integration tools, data standardization and data exchange protocol technologies to ensure that participants share data and information; the preferential policy and reward mechanism unit (12) designs preferential policies for data sharing, including fee reductions and service support, and uses reward algorithms and incentive models to design reward mechanisms, encouraging all parties to actively participate in system construction and data sharing through incentives.

6. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The output data of the data quality verification module (1) will be used as the input data of the anonymization technology module (2) to ensure that the data before anonymization has high quality and consistency. The output data of the anonymization technology module (2) will be used as the input data of the multi-dimensional supervision module (3) and the cooperation ecology module (4) to ensure that the analyzed and shared data can protect privacy and identity information.

7. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The data quality verification module (1) provides a reliable data basis for the multi-dimensional supervision module (3) to ensure the effectiveness and accuracy of the supervision indicators and models. The multi-dimensional supervision module (3) relies on the data cleaning and consistency verification results of the data quality verification module (1) to analyze and identify abnormal events and trends.

8. The tracking and tracing system based on big data analysis according to claim 1 is characterized by: The output data of the anonymization technology module (2) is used for data mining and analysis of the multi-dimensional supervision module (3) to identify abnormal events and trends. The multi-dimensional supervision module (3) uses the data security storage and transmission mechanism of the anonymization technology module (2) to ensure the confidentiality and integrity of the supervision data.

9. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The data quality verification module (1) provides a reliable data basis for the cooperative ecological module (4) to ensure the quality and consistency of data sharing. The cooperative ecological module (4) uses the data cleaning and consistency verification results of the data quality verification module (1) to manage the data sharing relationship of partners.

10. The tracking and tracing system based on big data analysis according to claim 1 is characterized in that: The identity anonymization and data security storage and transmission mechanism of the anonymization technology module (2) provides data privacy protection for partners, and the cooperation ecology module (4) relies on the identity anonymization and data sharing technology of the anonymization technology module (2) to realize the establishment and operation of the data sharing platform.

Images