Public security resource data service system

Abstract

The invention discloses a public security resource data service system, which is applied to a public security data resource processing process and mainly comprises a data acquisition and processing module and a data service platform, Wherein the data acquisition and processing module adopts a plug-in type structure, transmits data in a unified data format, and comprises databases and a central processor, a star-shaped structure is formed between the processor and each database, and the processor and each database are in seamless data connection through corresponding database plug-ins; The dataservice platform adopts a diversified hybrid architecture for service based on an object storage mode, and comprises a diversified database and a diversified service interface. According to the invention, a unified intermediate data format is adopted for transmission, the connection between different source ends and target ends is simplified, the data acquisition efficiency is improved, the fusion of information resources is realized, the analyzed high-value data in mass data are highlighted, and great convenience is provided for the public security criminal investigation work.

Description

technical field [0001] The invention relates to the technical field of big data processing, in particular to a resource data service system for the public security industry. Background technique [0002] The public security organs are responsible for maintaining national security and social stability. How to strengthen the police by strengthening the construction of scientific and technological information, further strengthen the integration, sharing, and service of information resources of public security organs across the country, and continuously promote the application of information technology to the depth Development, it is clearly stated that the construction of a public security data resource service system requires the integration of various information resources, and the construction of a multi-level integrated and shared public security resource service platform is an important topic in the development of police work. [0003] In order to achieve the purpose of da...

AI Technical Summary

Problems solved by technology

At present, the market lacks a comprehensive and efficient data synchronization and integration platform for public security resources. The existing technology has the following defects: 1) The types of data sources extracted are limited. The general big data platform is the public security resource library, which uses a relatively single Oracle database, Hadoop ecosystem technology is used in some developed regions, such as Hadoop / HDFS / Hive for data storage, so the existing market can generally connect with Oracle database and HDFS / Hive; however, with the continuous integration of public security big data platforms , the scale of data continues to expand, and Oracle databases can no longer meet the storage requirements of large amounts of data; and the usage scenarios of public security industry data are becoming more and more complex, and many other databases have been introduced, such as the MPP database Greenplum, ELASTICSEARCH or Solr for full-text retrieval, and relational databases Titan or Neo4j and different databases for various enterprise clouds and more

This requires the data to be connected to more data source types during the extraction process. Currently, it is rare to support comprehensive data storage. Generally, it is difficult for JDBC extraction to cover non-JDBC connection databases such as ELASTICSEARCH, Solr, Titan, or Neo4j.

2) The efficiency of extraction is low. The general technology adopts the JDBC method to collect data for ordinary databases. This method adopts a unified way to connect and execute by introducing a database driver; however, the efficiency of JDBC connection to extract data is low, generally about Extract 500,000 pieces of data in one hour, depending on the performance of the database server; but at present, the data gathered by the general public security resource platform often exceeds thousands of data resources, hundreds of billions of data volume, so the existing data extraction performance Cannot meet the requirements of high-performance data extraction and integration

[0005] In terms of data services, since the current service method is mainly provided through the interface, the interface is generally published on the bus in the public security field, and the service method of the bus is relatively fixed, and services with different parameters need to customize and develop an interface It is not flexible enough to implement, and there is no effective way to support service requirements such as data comparison and batch download of data; in addition, because the bottom layer of the interface of the database interface service is actually the invoked database, for millions of data, the existing conventional database For example, Oracle, which is used more by public security, can still support it, but the existing data is basically more than 100 million. The traditional method of calling the database can no longer meet the high-concurrency and short-delay requests of the interface.

Images