Metadata automatic association method based on data standard and technology

Abstract

The invention discloses a method for automatically associating data standards and technical metadata of cargo transportation. The method comprises the following steps: acquiring historical data; extracting historical technical metadata in the data; performing external connection on the historical technical metadata to form a data set; acquiring real-time data, and extracting real-time technical metadata in the real-time data; the real-time technical metadata is associated with the data set. The proportion of the occurrence probability of the real-time technical metadata in the case number data set, the time data set, the personnel name data set, the household registration data set and the travel record data set is calculated, the higher the proportion is, the higher the relevancy is, the real-time technical metadata can be combined with priori knowledge, related clues are obtained from the priori knowledge, and information analysis is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method for automatically associating data standards and technical metadata for cargo transportation. Background Art

[0002] In order to promptly deal with emergencies that occur during the daily supervision of the freight transportation market, the market operation order should be standardized. The case element control platform involves multiple different network environments. The administrative case data comes from the industry network. The data collected in different network environments are aggregated into a unified network through the data exchange system in different network environments. Due to the uncertainty of the data source, it is difficult to combine with prior knowledge and obtain relevant clues from prior knowledge. Summary of the invention

[0003] The technical problem to be solved by the present invention is that due to the uncertainty of the data source, it is difficult to combine with prior knowledge and obtain relevant clues from the prior knowledge.

[0004] To solve the above technical problems, the present invention provides the following technical solutions: a method for automatically associating data standards and technical metadata for cargo transportation, comprising: collecting historical data; extracting historical technical metadata from the data; externally connecting the historical technical metadata to form a data set; acquiring real-time data and extracting real-time technical metadata from the real-time data; and associating the real-time technical metadata with the data set.

[0005] Furthermore, the historical technical metadata includes case data, personnel data and information data: the case data includes case number, time, and location; the personnel data includes personnel name, household registration data, travel records, and vehicle information; the information data includes brand, specification, price category, identification code, traceability code, field name, and sales direction.

[0006] Furthermore, the historical technical metadata is outer-joined to form a data set including: The case number, time, location, person’s name, household registration data, and travel record were used as classification criteria to classify the data and form a data set; The data sets include a case number data set, a time data set, a person name data set, a household registration data set, and a travel record data set.

[0007] Furthermore, obtaining real-time data and extracting real-time technical metadata from the real-time data includes: collecting a number of network data in real time through HOOK.

[0008] Furthermore, associating the real-time technical metadata with the data set includes: matching each item of real-time technical metadata in the real-time data with the case number data set, the time data set, the person's name data set, the household registration data set and the travel record data set respectively, and obtaining the relevant case number data set, time data set, person's name data set, household registration data set and travel record data set.

[0009] Furthermore, the real-time technical metadata is matched twice with the relevant case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset, and the number of occurrences of the real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset is calculated respectively, and the number of occurrences of the real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset is counted, and the proportion of the probability of the real-time technical metadata appearing in the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset is calculated; the information entropy of the real-time technical metadata is calculated, and its mathematical expression is:

[0010] in, represents information entropy, It represents the proportion of the occurrence probability of real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset.

[0011] Beneficial effects of the present invention: The present invention can obtain information related to the case number, time, location, person's name, household registration data and travel record by matching each real-time technical metadata in the real-time data with the case number dataset, time dataset, person's name dataset, household registration dataset and travel record dataset respectively, which is conducive to mining the accomplices and related cases of the case; by calculating the proportion of the probability of the real-time technical metadata appearing in the case number dataset, time dataset, person's name dataset, household registration dataset and travel record dataset, the higher the proportion, the higher the correlation, which can be combined with prior knowledge and obtain relevant clues from the prior knowledge, which is conducive to analyzing the information. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A basic flow chart of a method for automatically associating data standards and technical metadata provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0014] Reference Figure 1 , this embodiment provides a method for automatic association based on data standards and technical metadata.

[0015] S1: Collect historical data including: The historical data collected comes from the original data of industry networks, public security criminal cases and public security intranet.

[0016] Information on industry networks, public security criminal cases, and cases on the public security intranet will include people's names, household registration data, travel records, and vehicle information. People's vehicles will frequently pass through checkpoints, and may pass through the same checkpoint multiple times in a short period of time. Transportation activities generally occur between 2 and 6 a.m. (the low period for checkpoint data). People's household registration data and travel records can also be queried from the public security intranet. The brand, specification, price category, identification code, traceability code, field name, and sales direction will also be recorded in the case.

[0017] S2: Extracting historical technical metadata from the data includes: The historical technical metadata includes case data, personnel data and data: The case data includes case number, time, and location; The personnel data includes personnel name, household registration data, travel records, and vehicle information; The data includes brand, specification, price category, identification code, traceability code, field name and sales direction.

[0018] In one of the embodiments, historical technical metadata in the data is automatically extracted through ICR intelligent information extraction.

[0019] S3: Perform an outer join on the historical technical metadata to form a data set including: The case number, time, location, person’s name, household registration data, and travel record were used as classification criteria to classify the data and form a data set; The data sets include a case number data set, a time data set, a person name data set, a household registration data set, and a travel record data set.

[0020] The case number data set specifically includes the time, location, person’s name, household registration data, travel records, vehicle information, brand, specification, price category, identification code, traceability code, field name, and sales direction data involved in the same case; The time data set specifically includes the cases, locations, names of people, household registration data, travel records, vehicle information, brands, specifications, price categories, identification codes, traceability codes, field names, and sales direction data involved in the same time period; The person name data set includes the time, location, person name, household registration data, travel records, vehicle information, brand, specification, price category, identification code, traceability code, field name, and sales direction data of each case involving the same person. The household registration data set specifically includes data on each case, location, person's name, travel record, vehicle information, brand, specification, price category, identification code, traceability code, field name, and sales direction in the same household registration area; The sales direction data specifically includes the cases, locations, names of persons, travel records, vehicle information, brands, specifications, price categories, identification codes, traceability codes, and field name data of sales to the same area.

[0021] By dividing the historical technical metadata into case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset, it is beneficial to analyze the historical technical metadata from different angles and mine the information associated with the case number, time, location, person name, household registration data and travel records, which is conducive to discovering the accomplices of the case.

[0022] S4: Acquire real-time data and extract real-time technical metadata from the real-time data including; Through HOOK, data from industry networks, public security criminal cases and public security intranets are collected in real time. The authenticity and timeliness of data sources are guaranteed, and real-time technical metadata from real-time data is automatically extracted through ICR intelligent information extraction.

[0023] S5: Associating the real-time technical metadata with the data set includes: Each real-time technical metadata in the real-time data is matched with the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset respectively to obtain the relevant case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset.

[0024] By matching each real-time technical metadata in the real-time data with the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset respectively, we can obtain information related to the case number, time, location, person name, household registration data and travel record, which is conducive to discovering accomplices and related cases.

[0025] Perform secondary matching on the real-time technical metadata with the relevant case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset, and calculate the number of occurrences of the real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset, and count the number of occurrences of the real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset, and calculate the proportion of the probability of the real-time technical metadata appearing in the case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset; The information entropy of real-time technical metadata is calculated as follows:

[0026] in, represents information entropy, It represents the proportion of the occurrence probability of real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset.

[0027] By calculating the proportion of the probability of real-time technical metadata appearing in the case number dataset, time dataset, personnel name dataset, household registration dataset and travel record dataset, the higher the proportion, the higher the relevance. It can be combined with prior knowledge and relevant clues can be obtained from prior knowledge, which is conducive to the analysis of information.

[0028] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0029] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for automatically associating data standards and technical metadata for cargo transportation, characterized in that: include: Collect historical data; extracting historical technical metadata from said data; Outer join the historical technical metadata to form a data set; Acquire real-time data and extract real-time technical metadata from the real-time data; The real-time technical metadata is associated with the data set.

2. The method according to claim 1, characterized in that The historical technical metadata includes case data, personnel data and information data: The case data includes case number, time, location The personnel data includes personnel name, household registration data, travel records, and vehicle information; The information data includes brand, specification, price category, identification code, traceability code, field name and sales direction.

3. The method according to claim 2, characterized in that The historical technical metadata is outer-joined to form a data set including: The case number, time, location, person’s name, household registration data, and travel record were used as classification criteria to classify the data and form a data set; The data sets include a case number data set, a time data set, a person name data set, a household registration data set, and a travel record data set.

4. The method according to claim 3, characterized in that Acquiring real-time data and extracting real-time technical metadata from the real-time data includes: collecting a number of network data in real time through HOOK.

5. The method according to claim 4, characterized in that Associating the real-time technical metadata with the data set includes: Each real-time technical metadata in the real-time data is matched with the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset respectively to obtain the relevant case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset.

6. The method according to claim 5, characterized in that: Perform secondary matching on the real-time technical metadata with the relevant case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset, and calculate the number of occurrences of the real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset, and count the number of occurrences of the real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset, and calculate the proportion of the probability of the real-time technical metadata appearing in the case number dataset, time dataset, person name dataset, household registration dataset, and travel record dataset; The information entropy of real-time technical metadata is calculated as follows: ， in, represents information entropy, It represents the proportion of the occurrence probability of real-time technical metadata in the case number dataset, time dataset, person name dataset, household registration dataset and travel record dataset.

Images