Method for automatically removing duplication of electronic fence data
An electronic fence and data technology, applied in the field of big data processing, can solve the problems of low utilization rate of storage space, low data value density, high transmission overhead, etc., so as to reduce subsequent transmission overhead, improve data value density, and effectively utilize storage space. Effect
Inactive Publication Date: 2018-12-21
NO 54 INST OF CHINA ELECTRONICS SCI & TECH GRP
View PDF8 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
The present invention accesses the target database through the configuration file in the thread task mode, and realizes automatic judgment on the dormancy, cancellation or execution of the task according to the specified task start time and the current system time; according to the set deduplication period and deduplication time Complete de-duplication tasks at intervals, and automatically count and store the de-duplication-processed data to solve the problems of high transmission overhead, low storage space utilization, and low data value density caused by data duplication, and realize data de-duplication Full automation to effectively improve efficiency
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0026] The technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0027] The invention provides a method for automatic deduplication of electronic fence data, including a schematic diagram of the principle of automatic deduplication of electronic fence data and an algorithm flow of automatic deduplication of electronic fence data.
[0028] figure 1 The schematic diagram of the automatic deduplication algorithm is shown, including the deduplication start time setting module, which is used to set the start time of data deduplication; the time judgment module, which is used to judge the deduplication start time and the current system time, and according to the judgment The result determines whether to sleep, cancel or execute the current deduplication task; the ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an automatic deduplication method of electronic fence data, which relates to the technical field of big data processing. In the automatic deduplication algorithm of the invention, the deduplication task thread first reads the configuration file to obtain the deduplication parameters; Then, according to the derestart time and the current time of the system, the task is automatically configured whether to cancel or execute the task or not. Automatic statistics and storage of data after de-reprocessing during de-duplication; finally, the intelligent unattended task is realized by updating the start time of derestart. By reading the data collected by the electronic fence equipment, 2 field of that device number and the international mobile subscribe identification codeof the mobile phone are obtained, According to the uniqueness of the two, the electronic fence data can be accurately and efficiently de-reprocessed, which solves the problems of large system transmission overhead, low storage space utilization rate and low data value density caused by a large number of repeated data accesses. At the same time, the intelligent and unattended data de-duplication isrealized, and the work efficiency is improved.
Description
technical field [0001] The invention belongs to the technical field of big data processing, and in particular relates to a method for automatically deduplicating electronic fence data. Background technique [0002] Big data processing technology refers to the use of data analysis methods to analyze the collected structured, semi-structured and unstructured massive disordered data to obtain the potential value and knowledge hidden in it, so as to better serve decision makers in government, enterprises and other institutions. Provide effective decision-making support. From the perspective of the big data value chain, big data processing technology covers the key technologies involved in four aspects: data generation, data acquisition, data storage and data analysis. [0003] The electronic fence system can collect mobile phone data of specific areas or designated personnel. Through data analysis, it can grasp the activity track and past records of key personnel in the crowd, ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30H04W4/021
CPCH04W4/021
Inventor 戎凯旋高杰侯风茂张策刘师范
Owner NO 54 INST OF CHINA ELECTRONICS SCI & TECH GRP