Government affair data modeling method, device and equipment, storage medium and program product
Patent Information
- Application Number
- CN202311498661.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-10
- Publication Date
- 2026-09-22
- Estimated Expiration
- 2043-11-10
AI Technical Summary
[0004]但是这样的方式,每次新的业务出现,则需要在数据仓库中增加一层级,在该层级进行数据处理,操作复杂,智能化水平较低
[0044]上述政务数据建模方法、装置、计算机设备、存储介质和计算机程序产品,在新增业务需求时,无需在数据仓库中增加一层级,而仅是基于业务需求对应的实体、实体间的关系以及实体属性对数据主题层中的政务数据进行逻辑建模得到政务数据逻辑处理模型,并配置政务数据逻辑处理模型对应的物化规则;基于所述物化规则将所述政务数据逻辑处理模型进行物化处理,这样实现业务需求,简化数据仓库增加一层的操作,提高智能化水平。
Smart Images

Figure CN117520300B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data processing technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for government data modeling. Background Technology
[0002] In the process of providing government services, relevant government departments often face the challenge of collecting data from different commissions and bureaus, resulting in data that is scattered and not centralized, and data with different characteristics.
[0003] The current approach is to govern data through data aggregation, processing data layer by layer according to the data warehouse hierarchy. For example, each layer is processed based on different business dimensions, so the data at each layer is obtained based on different business needs.
[0004] However, with this approach, each new business requires adding a layer to the data warehouse for data processing, which is complex and has a low level of intelligence. Summary of the Invention
[0005] Therefore, it is necessary to provide a government data modeling method, device, computer equipment, computer-readable storage medium, and computer program product that can simplify operation and improve the level of intelligence in order to address the above-mentioned technical problems.
[0006] Firstly, this application provides a method for modeling government data, the method comprising:
[0007] Obtain the entity corresponding to the business requirement;
[0008] Determine the entity attributes of each entity, and obtain the relationships between entities based on the relationships between the entity attributes;
[0009] Based on the entities, entity attributes, and relationships between entities, a logical modeling process is performed on the government data in the data subject layer to obtain a government data logical processing model, including: determining the model execution location and data storage location of the government data logical processing model based on the processor information and hardware storage information of the service objects of the business requirements.
[0010] Obtain the configured materialization rules corresponding to the government data logical processing model;
[0011] The government data logic processing model is materialized based on the materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0012] In one embodiment, the step of logically modeling the government data in the data subject layer based on the entity, the entity attributes, and the relationships between entities to obtain a government data logical processing model includes:
[0013] Determine the naming rules for each entity and its attributes;
[0014] Determine the entity constraint rules for each of the entities;
[0015] Based on the entity constraint rules, duplicate detection is performed on the government data corresponding to the entities in the data subject layer;
[0016] Based on the results of the repeatability test, retain the government data corresponding to the target data source and delete the government data that does not correspond to the target data source;
[0017] The government data corresponding to the target data source is named according to the naming rules.
[0018] The data tables for storing the named government data are determined, and the table structure of the data tables is configured based on the relationships between the entities of the government data. The index relationships between the data tables are configured based on the relationships between the entities. Based on the configured data tables, table structures and index relationships, the logical processing model of the government data is obtained.
[0019] In one embodiment, before obtaining the configured materialized rules corresponding to the government data logical processing model, the method further includes:
[0020] Based on the business information of the service objects corresponding to the business requirements of the government data logic processing model, the materialized location of the data logic processing model is determined.
[0021] The process of materializing the government data logic processing model based on the materialization rules includes:
[0022] Based on the materialized location of the data logic processing model, the entities corresponding to the government data logic processing model are stored in different partitions.
[0023] In one embodiment, determining the materialized location of the data logic processing model based on the business information of the service object corresponding to the business requirements of the government data logic processing model includes:
[0024] Obtain the storage frequency and storage response time requirements of the service objects corresponding to the business requirements of the government data logical processing model;
[0025] Based on the storage frequency and the storage response time requirements, the materialized location of the government data logical processing model is determined.
[0026] In one embodiment, before logically modeling the government data in the data subject layer to obtain the government data logical processing model, the method further includes:
[0027] Receive initial government data pushed from different data sources and store the initial government data in the data storage layer;
[0028] The initial government data in the data storage layer is cleaned, and the cleaned initial government data is stored in the data detail layer.
[0029] Based on the pre-set data business dimensions, the initial government data in the data detail layer is cleaned for business purposes, and the cleaned initial government data is stored in the corresponding data theme in the data theme layer.
[0030] In one embodiment, the step of performing business cleansing on the initial government data of the data detail layer based on pre-set data business dimensions includes:
[0031] Retrieve pre-defined entity information dimensions;
[0032] Based on the entity information dimension, the initial government data of the data detail layer is reconstructed to include entity information.
[0033] Retrieve pre-set tag information;
[0034] Based on the label information, the initial government data after entity information reconstruction is labeled.
[0035] Secondly, this application also provides a government data modeling device, the device comprising:
[0036] The entity acquisition module is used to acquire the entities corresponding to business requirements;
[0037] The entity attribute acquisition module is used to determine the entity attributes of each entity and obtain the relationship between entities based on the relationship between the entity attributes.
[0038] The logical modeling module is used to perform logical modeling on the government data in the data subject layer based on the entity, the entity attributes and the relationship between entities to obtain the government data logical processing model, including: determining the model execution location and data storage location of the government data logical processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0039] The materialization rule acquisition module is used to acquire the configured materialization rules corresponding to the government data logical processing model;
[0040] The materialization module is used to materialize the government data logic processing model based on the materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.
[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.
[0043] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods in any of the above embodiments.
[0044] The aforementioned government data modeling methods, devices, computer equipment, storage media, and computer program products, when introducing new business requirements, do not require adding a new layer to the data warehouse. Instead, they simply perform logical modeling of the government data in the data subject layer based on the entities corresponding to the business requirements, the relationships between entities, and the entity attributes to obtain a government data logical processing model, and configure the materialization rules corresponding to the government data logical processing model. Based on the materialization rules, the government data logical processing model is materialized, thus realizing business requirements, simplifying the operation of adding a layer to the data warehouse, and improving the level of intelligence. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 This is an application environment diagram of the government data modeling method in one embodiment;
[0047] Figure 2 This is a flowchart illustrating a government data modeling method in one embodiment;
[0048] Figure 3 This is a schematic diagram of the data warehouse structure in one embodiment;
[0049] Figure 4This is a schematic diagram of the entity data reconstruction steps in one embodiment;
[0050] Figure 5 This is a structural block diagram of a government data modeling device in one embodiment;
[0051] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0053] The government data modeling method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server.
[0054] Server 104 receives the business requirements sent by 102, obtains the entities corresponding to the business requirements, determines the entity attributes corresponding to each entity, and then determines the relationships between entities based on the relationships between entity attributes. Based on the entities, entity attributes, and relationships between entities, it performs logical modeling on the government data in the data subject layer to obtain the government data logical processing model; and obtains the materialization rules corresponding to the configured government data logical processing model, and then performs materialization processing on the government data logical processing model based on the materialization rules.
[0055] The aforementioned government data modeling method eliminates the need to add a new layer to the data warehouse when new business requirements arise. Instead, it logically models the government data in the data subject layer based on the entities corresponding to the business requirements, the relationships between entities, and the entity attributes to obtain a government data logical processing model. It then configures the materialization rules corresponding to the government data logical processing model. Based on the materialization rules, the government data logical processing model is materialized, thus fulfilling business requirements, simplifying the operation of adding a layer to the data warehouse, and improving the level of intelligence.
[0056] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0057] In one exemplary embodiment, such as Figure 2 As shown, a method for modeling government data is provided, which can be applied to... Figure 1 The following steps are used as an example of the server in the example, including steps S202 to S210.
[0058] in:
[0059] S202: Obtain the entity corresponding to the business requirement.
[0060] Specifically, business requirements are new requirements, such as new applications or new government services. The corresponding entities are the entities required by the new application at the business level, and the relationships between entities are determined based on the business level to realize the conceptual model.
[0061] These entities are stored in a data warehouse, for example, in the subject layer of the data warehouse. A data warehouse, abbreviated as DW or DWH, is a strategic collection that provides all types of data to support decision-making processes at all levels of an enterprise. Specifically, it combines... Figure 3 As shown, Figure 3 This is a schematic diagram of a data warehouse structure in one embodiment. In this embodiment, the data warehouse includes a data storage layer, a data detail layer, a data theme layer, and a data application layer. The data theme layer in this embodiment includes two themes: one is a people-centric digital archive, which refers to a set of government digital archives centered on individuals, covering information on various dimensions such as basic information, education, business registration, taxation, credit, and employment. The other is an enterprise-centric digital archive, which covers information on various dimensions such as enterprise information, establishment and changes, technological innovation, taxation and finance, enterprise credit, and supervision. The applications in the data application layer provide government services to the outside world. These include administrative services such as permits, confirmations, adjudications, rewards, and penalties provided by governments at all levels, relevant departments and public institutions, and, in accordance with laws and regulations, to social organizations, enterprises, and individuals. Government service items include administrative power items and public service items.
[0062] S204: Determine the entity attributes of each entity, and obtain the relationships between entities based on the relationships between entity attributes.
[0063] Among them, entity attributes are the characteristics of an entity. For example, for a person, their entity attributes include multiple dimensions such as household registration, marriage, childbirth and adoption, education and scientific research, professional qualifications, investment and real estate, social security, taxation, judicial notarization, health care, intellectual property, consumer rights protection, social activities, public utilities, credit records, death and funeral. For a company, its entity attributes include multiple dimensions such as establishment and changes, technological innovation, quality and technology, operation and licensing, production safety, cultural relic protection, taxation and finance, investment approval, qualification certification, business and trade, regulatory penalties, public security and fire protection.
[0064] Relationships based on entity attributes can determine the relationships between entities, thereby further refining the entity structure.
[0065] S206: Based on entities, entity attributes, and relationships between entities, logical modeling is performed on government data in the data subject layer to obtain a government data logical processing model.
[0066] The government data in the data theme layer is based on reconstructed entities and entity tags. It is obtained by reconstructing and tagging initial government data from different data sources. Logical modeling includes configuring the data tables corresponding to the government data in the theme layer, as well as the table structures. Specifically, logical modeling refers to organizing and refining the entities required for business needs, the relationships between entities, adding field attributes to entities, and refining the "one person, one file" information resource database into specific tables based on application service business needs during the logical modeling phase. This also involves enriching the table structures, including primary keys, foreign keys, attribute columns, indexes, constraints, and even views and stored procedures. Similarly, the "one enterprise, one file" enterprise database is refined into specific tables based on application service business needs during the logical modeling phase, with enriched table structures including primary keys, foreign keys, attribute columns, indexes, constraints, and even views and stored procedures. This direct processing of the theme layer data, which stores all the necessary data, avoids modeling failures. Furthermore, storing theme layer data in separate tables avoids redundant business processing and improves subsequent processing efficiency.
[0067] S208: Obtain the configured materialized rules corresponding to the government data logical processing model.
[0068] S210: Materialize the government data logic processing model based on materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0069] Specifically, the materialization rules are pre-set, which include rules for determining how entities, relationships, and attributes in the previous model are mapped to the concrete implementation, as well as rules for determining additional adjustments that need to be made in the concrete implementation due to technical or other reasons.
[0070] The materialization rules are related to the service objects required by the business. The processor information and hardware storage information of the service object determine the materialization result of the model. Specifically, the execution location of the model can be determined based on the processor information of the service object, and the data storage location can be determined based on the hardware storage information of the service object. The processor information includes, but is not limited to, CPU information, and the hardware storage information includes, but is not limited to, storage frequency and storage response time requirements.
[0071] In practical applications, taking one person, one file as an example, the population database is logically modeled using a data modeling system. Design constraints and naming rules are determined, and resources are allocated and materialized based on storage frequency and access response time requirements, guided by the logical modeling rules, data dictionary, and materialization rules in the data modeling module. First, rules are defined. When establishing the logical model, normative constraints are imposed on the structure and attributes. Rule types include: structure coding rules, structure naming rules, structure description rules, attribute coding rules, and attribute naming rules. Materialization rules are set for the logical model of population materialization, mainly including basic information and partitioning information. Basic information mainly includes the location where the materialized physical model is stored (it can be materialized to multiple locations to achieve model reuse); partitioning information selects different partitioning methods based on fields. Taking one enterprise, one file as an example, the enterprise resource database is logically modeled using a data modeling system. Design constraints and naming rules are determined, and resources are allocated and materialized based on storage frequency and access response time requirements, guided by the logical modeling rules, data dictionary, and materialization rules in the data modeling module. First, rules are defined. When establishing the logical model, normative constraints must be imposed on the structure and attributes. Rule types include: structure coding rules, structure naming rules, structure description rules, attribute coding rules, and attribute naming rules. Materialization rules are then set for the enterprise's materialized logical model, mainly including basic information and partitioning information. Basic information primarily includes the location where the materialized physical model is stored (it can be materialized to multiple locations, enabling model reuse); partitioning information selects different partitioning methods based on the fields.
[0072] The aforementioned government data modeling method eliminates the need to add a new layer to the data warehouse when new business requirements arise. Instead, it logically models the government data in the data subject layer based on the entities corresponding to the business requirements, the relationships between entities, and the entity attributes to obtain a government data logical processing model. It then configures the materialization rules corresponding to the government data logical processing model. Based on the materialization rules, the government data logical processing model is materialized, thus fulfilling business requirements, simplifying the operation of adding a layer to the data warehouse, and improving the level of intelligence.
[0073] In one embodiment, a logical processing model for government data is obtained by logically modeling government data in the data subject layer based on entities, entity attributes, and relationships between entities. This includes: determining naming rules for each entity and entity attribute; determining entity constraint rules for each entity; performing duplicate detection on the government data corresponding to entities in the data subject layer based on the entity constraint rules; retaining government data corresponding to the target data source and deleting government data corresponding to non-target data sources based on the duplicate detection results; performing naming processing on the government data corresponding to the target data source based on the naming rules; determining a data table to store the named government data, configuring the table structure of the data table based on the relationships between entities in the government data, configuring the index relationships between the data tables based on the relationships between entities, and obtaining the government data logical processing model based on the configured data tables, table structures, and index relationships.
[0074] Specifically, naming rules are the normative constraints on naming entities and entity attributes, such as structure coding rules, structure naming rules, structure description rules, attribute coding rules, and attribute naming rules, etc. Constraint rules include the source of the entity. Since the data in the subject layer comes from multiple government systems, there may be duplicate data. Therefore, different applications have requirements regarding the source of this data. Thus, duplicate detection is performed on the government data corresponding to the entity based on constraint rules. Based on the results of the duplicate detection, government data corresponding to the target data source is retained, while government data corresponding to non-target data sources is deleted to ensure the legality of the data source, ensuring one source per piece of data.
[0075] After identifying the government data, it is named accordingly. For example, identity information includes basic information, household registration information, temporary residence information for migrant populations, birth records, and death records. Basic information includes ID card number, ID document type, document start date, document end date, ID card issuing authority, name, former name, nationality, ethnicity, gender, country / region of origin code, province / city / county of origin code, education level, marital status, political affiliation, religious belief code, military service status code, and living status. Household registration information includes ID card number, ID document type, name, household registration category, household number, relationship with head of household, household registration address, police station field name, community field name, street field name, household registration issuing authority, household registration issuance date, whether there has been a household registration transfer, date of transfer, reason for transfer, whether there has been a household registration cancellation, date of cancellation, reason for cancellation, guardianship code, etc. The names in the government data are processed based on these naming conventions.
[0076] Then, based on the named government data and the relationships between entities, a government data logical processing model is obtained. This includes organizing and improving all entities in the system, the relationships between entities, and adding field attributes to entities. During the establishment of the government data logical processing model, the data tables for the named government data are determined. This can be done by storing all dimensions of government data in one table, or by storing corresponding dimensions of government data in multiple tables; no specific limitation is made here. The table structure is configured based on the relationships between entities, including primary keys, foreign keys, attribute columns, indexes, constraints, and even views and stored procedures. Furthermore, the index relationships of the data tables for different dimensions of government data are configured based on the relationships between entities to obtain the government data logical processing model. In one optional embodiment, to facilitate data querying, the government data corresponding to a business requirement is stored in one table, so that all data can be queried from a single table, improving data query efficiency. In other optional embodiments, a target dimension is obtained from multiple dimensions of government data based on the relationships between entities as an index dimension. This index dimension can be fixed and unchanging, such as data that will not be updated, such as ID card numbers, to ensure the stability of the index.
[0077] In the above embodiments, government data is processed based on entity constraint rules and entity naming rules to ensure one source of data, thereby improving data security and ensuring the security of application services.
[0078] In one embodiment, configuring materialization rules corresponding to the government data logic processing model includes: determining the materialization location of the data logic processing model based on the business information of the service objects corresponding to the business requirements of the government data logic processing model; and materializing the government data logic processing model based on the materialization rules, including: storing the entities corresponding to the government data logic processing model to different partitions based on the materialization location of the data logic processing model.
[0079] Specifically, materialization refers to determining how entities, relationships, and attributes in the previous model are mapped to the concrete implementation, as well as determining any additional adjustments that need to be made in the concrete implementation due to technical or other reasons.
[0080] The materialized location refers to the storage location of the model and data, as well as the location of the service. Therefore, when the government data logic processing model is "fogified," the entities corresponding to the government data logic processing model are stored in different partitions based on the materialized location of the data logic processing model, and the partition information is selected according to different partitioning methods based on the fields.
[0081] In the above embodiments, multi-source data accumulation and governance are achieved, forming data products with one file per person and one file per enterprise in government affairs. Customized data combination supply is provided for personalized application scenarios of government affairs, releasing data value, realizing flexible support for heterogeneous scenarios, and promoting the application construction of smart government affairs.
[0082] In one embodiment, determining the materialized location of the data logic processing model based on the business information of the service object corresponding to the business requirement of the government data logic processing model includes: obtaining the storage frequency and storage response time requirements of the service object corresponding to the business requirement of the government data logic processing model; and determining the materialized location of the government data logic processing model based on the storage frequency and storage response time requirements.
[0083] The materialized location needs to be related to the storage frequency and storage response time requirements of the service object corresponding to the business requirements of the logical processing model. The data logical processing model can be materialized to multiple locations to achieve reuse. The storage frequency refers to the frequency of data processing by the data logical processing model and the response time to the service. Thus, the data logical processing model is atomized based on the storage frequency and storage response time.
[0084] In one optional embodiment, before logically modeling the government data in the data theme layer to obtain the government data logical processing model, the method further includes: receiving initial government data pushed from different data sources and storing the initial government data in the data storage layer; performing data cleaning on the initial government data in the data storage layer and storing the successfully cleaned initial government data in the data detail layer; performing business cleaning on the initial government data in the data detail layer based on a pre-set data business dimension, and storing the business-cleaned initial government data in the corresponding data theme in the data theme layer.
[0085] The data sources can include different government systems. Different government systems push initial government data to the server. Through data governance processes such as unified data standards, problem data cleaning, data model integration, and one-data-one-source identification, a file can be formed that can comprehensively reflect the multi-dimensional characteristics and attributes of individuals and enterprises, creating a file for each person and each enterprise.
[0086] Combination Figure 3 As shown, the data storage layer is the layer of operational data that transitions from the business system to the core integration layer of the data warehouse. The data structure of the data storage layer is basically consistent with the interface specification, and it stores data in a short cycle.
[0087] The detail layer maintains the same data granularity as the data storage layer, and provides a certain level of data quality assurance through data cleaning and quality inspection.
[0088] The data theme layer is based on detailed data from the data storage layer. It needs to revolve around the core business processes of the enterprise, focusing on core business events and entities within those processes, and establishing data associations within the same theme. Business entities and business models are interrelated; entities refer to inherent aspects of the business, while business models record entities or transactions related to the business or its processes and interactions to achieve business activities and goals. Core business events refer to the key events that occur during the interaction of business entities. The relationship between core events and business entities is reflected in the business processes. Therefore, for the theme layer, applications may span different theme libraries. The data supply between different theme libraries has a significant impact on business applications; therefore, it is crucial to monitor changes in business processes and ensure data integrity through logical models.
[0089] Specifically, for government data in the data detail layer, data with null values, missing values, or logical errors identified through cleaning rules will be filtered out during the cleaning process. Data that meets the quality requirements will be stored in the relevant subject database.
[0090] The application layer stores data that is used for secondary integration or summary of thematic data based on business needs such as relationship graphs, derived tags, and various statistical analyses.
[0091] In this embodiment, except Figure 3 In addition to the data layer shown, auxiliary data layers can be included, such as a rule base, which mainly defines and restricts values stored in table columns or user-defined data types. Rules are independently stored database objects. Rules are independent of the tables or user-defined data types they apply to; that is, deleting or modifying a table or user-defined object will not affect the rules connected to it. Rules and constraints can be used simultaneously. An exception library is used to collect exception information based on relevant rules for different exception types in each system, track the process of exception data issues, and perform decision-making analysis on related exception data. A dimension library is used to store the entities mentioned above. A temporary layer is used to store temporary tables generated by each layer.
[0092] In one embodiment, business cleaning is performed on the initial government data of the data detail layer based on a pre-set data business dimension, including: obtaining a pre-set entity information dimension; reconstructing entity information of the initial government data of the data detail layer based on the entity information dimension; obtaining pre-set tag information; and labeling the initial government data after entity information reconstruction based on the tag information.
[0093] The entity information dimension can include both individual and enterprise dimensions. The individual dimension leverages complete population information, social affairs databases, and public information resources. It reconstructs entity information data by abstracting and summarizing all population business data entities generated and collected by various government departments throughout the entire lifecycle of a person, from birth to death. This includes multiple dimensions such as household registration, marriage, childbirth, adoption, education and research, professional qualifications, investment and real estate, social security, taxation, judicial notarization, health care, intellectual property, consumer rights protection, social activities, public utilities, credit records, and death and funeral services, supporting the construction of a unique file for each individual. The enterprise dimension leverages complete basic legal person information, government and enterprise business databases, and public information resources. It reconstructs entity information data by abstracting and summarizing all enterprise business data entities generated and collected by various government departments throughout the entire lifecycle of an enterprise, from establishment to closure and deregistration. This includes multiple dimensions such as establishment and changes, technological innovation, quality and technology, business licenses, production safety, cultural relic protection, taxation and finance, investment approval, qualification certification, business and trade, regulatory penalties, and public security and fire protection, supporting the construction of a unique file for each enterprise.
[0094] Specifically, the set Figure 4 As shown, Figure 4 This is a schematic diagram of the entity data reconstruction steps in one embodiment. In this embodiment, entity data is reconstructed based on data from the data source. For example, taking a person as an example, it can obtain basic personnel information entity, personnel association information entity, personnel travel information entity, etc. Taking an enterprise as an example, it can obtain establishment and change entity, quality and technology entity, tax and finance entity, etc.
[0095] Labeling, in particular, takes a data-driven approach, combining statistical analysis methods to characterize the initial government data after the reconstruction of entity information.
[0096] Taking individuals as an example, a unique tag database is constructed for each person. From a data perspective, combined with statistical analysis methods, indicators are used to characterize basic population information and public management information, transforming subjective characterization into quantifiable metrics. By integrating the tagging system with classification features based on population data, population tags are characterized more accurately and closely align with the actual needs of business applications.
[0097] Taking enterprises as an example, a unique tag library is constructed for each enterprise. From a data perspective, combined with statistical analysis methods, indicators are used to characterize basic enterprise information and public management information, transforming subjective characterization into quantifiable metrics. By integrating the tag system with classification characteristics based on enterprise data, enterprise tags are more accurate and closely aligned with the actual needs of business applications.
[0098] After constructing these entities and tags, subsequent modeling of government data logic processing models can be based on these entities and tags, thereby constructing exclusive data models for each individual and each enterprise. Based on government service data, multi-source data accumulation and governance can be achieved, forming government data products for each individual and each enterprise. This provides customized data combinations for personalized government application scenarios, releases data value, enables flexible support for heterogeneous scenarios, and promotes the application and construction of smart government.
[0099] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0100] Based on the same inventive concept, this application also provides a government data modeling apparatus for implementing the aforementioned government data modeling method. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more government data modeling apparatus embodiments provided below can be found in the limitations of the government data modeling method described above, and will not be repeated here.
[0101] In one exemplary embodiment, such as Figure 5 As shown, a government data modeling device is provided, including: an entity acquisition module 501, an entity attribute acquisition module 502, a logical modeling module 503, a materialization rule acquisition module 504, and a materialization module 505, wherein:
[0102] The entity acquisition module 501 is used to acquire the entities corresponding to business requirements.
[0103] The entity attribute acquisition module 502 is used to determine the entity attributes of each entity and obtain the relationship between entities based on the relationship between entity attributes.
[0104] The logical modeling module 503 is used to perform logical modeling on the government affairs data in the data theme layer based on entities, entity attributes and relationships between entities to obtain a government affairs data logical processing model. The government affairs data in the data theme layer is obtained by reconstructing and labeling the initial government affairs data obtained from different data sources based on the reconstructed entities and entity labels. The logical modeling includes configuring the data table corresponding to the government affairs data in the theme layer and the table structure of the data table.
[0105] The materialization rule acquisition module 504 is used to acquire the configured materialization rules corresponding to the government data logical processing model;
[0106] The materialization module 505 is used to materialize the government data logic processing model based on materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0107] In one embodiment, the aforementioned logical modeling module 503 is further configured to: determine the naming rules corresponding to each entity and its attributes; determine the entity constraint rules for each entity; perform duplicate detection on the government data corresponding to the entities in the data subject layer based on the entity constraint rules; retain the government data corresponding to the target data source and delete the government data corresponding to non-target data sources based on the duplicate detection results; perform naming processing on the government data corresponding to the target data source based on the naming rules; determine the data table for storing the named government data; configure the table structure of the data table based on the relationships between the entities in the government data; configure the index relationships between the data tables based on the relationships between the entities; and obtain the government data logical processing model based on the configured data table, table structure, and index relationships.
[0108] In one embodiment, the materialization rule acquisition module 504 is further used to determine the materialization location of the data logic processing model based on the business information of the service object corresponding to the business requirements of the government data logic processing model.
[0109] The aforementioned materialization module 505 is also used to store the entities corresponding to the government data logical processing model into different partitions based on the materialization location of the data logical processing model.
[0110] In one embodiment, the materialization module 505 is further configured to obtain the storage frequency and storage response time requirements of the service objects corresponding to the business requirements of the government data logic processing model; and determine the materialization location of the government data logic processing model based on the storage frequency and storage response time requirements.
[0111] In one embodiment, the above-mentioned apparatus further includes a data storage module for receiving initial government data pushed from different data sources and storing the initial government data in the data storage layer; performing data cleaning on the initial government data in the data storage layer and storing the successfully cleaned initial government data in the data detail layer; performing business cleaning on the initial government data in the data detail layer based on a pre-set data business dimension, and storing the business-cleaned initial government data in the corresponding data theme in the data theme layer.
[0112] In one embodiment, the data storage module is further configured to acquire a pre-set entity information dimension; reconstruct the initial government data of the data detail layer based on the entity information dimension; acquire pre-set tag information; and label the initial government data after entity information reconstruction based on the tag information.
[0113] Each module in the aforementioned government data modeling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.
[0114] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores government data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a government data modeling method.
[0115] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0116] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: obtaining entities corresponding to business requirements; determining the entity attributes of each entity and obtaining relationships between entities based on the relationships between entity attributes; performing logical modeling on government data in a data theme layer based on entities, entity attributes, and relationships between entities to obtain a government data logical processing model, wherein the government data in the data theme layer is obtained by reconstructing and labeling initial government data obtained from different data sources based on reconstructed entities and entity tags, and the logical modeling includes configuring data tables corresponding to the government data in the theme layer and the table structure of the data tables; obtaining configured materialization rules corresponding to the government data logical processing model; and performing materialization processing on the government data logical processing model based on the materialization rules, including: determining the model execution location and data storage location of the government data logical processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0117] In one embodiment, when a processor executes a computer program, it performs logical modeling on government data in the data subject layer based on entities, entity attributes, and relationships between entities to obtain a government data logical processing model. This includes: determining naming rules for each entity and entity attribute; determining entity constraint rules for each entity; performing duplicate detection on the government data corresponding to entities in the data subject layer based on the entity constraint rules; retaining government data corresponding to the target data source and deleting government data corresponding to non-target data sources based on the duplicate detection results; performing naming processing on the government data corresponding to the target data source based on the naming rules; determining a data table to store the named government data, configuring the table structure of the data table based on the relationships between entities in the government data, configuring the index relationships between the data tables based on the relationships between entities, and obtaining the government data logical processing model based on the configured data tables, table structures, and index relationships.
[0118] In one embodiment, the configuration and materialization rules corresponding to the government data logic processing model implemented by the processor when executing the computer program include: determining the materialization location of the data logic processing model based on the business information of the service objects corresponding to the business requirements of the government data logic processing model; and the materialization processing of the government data logic processing model based on the materialization rules implemented by the processor when executing the computer program includes: storing the entities corresponding to the government data logic processing model to different partitions based on the materialization location of the data logic processing model.
[0119] In one embodiment, when a processor executes a computer program, it implements business information of the service object corresponding to the business requirement based on the government data logic processing model, and determines the materialized location of the data logic processing model, including: obtaining the storage frequency and storage response time requirements of the service object corresponding to the business requirement of the government data logic processing model; and determining the materialized location of the government data logic processing model based on the storage frequency and storage response time requirements.
[0120] In one embodiment, before the processor executes the computer program to logically model the government data in the data theme layer to obtain the government data logical processing model, the method further includes: receiving initial government data pushed from different data sources and storing the initial government data in the data storage layer; performing data cleaning on the initial government data in the data storage layer and storing the successfully cleaned initial government data in the data detail layer; performing business cleaning on the initial government data in the data detail layer based on a pre-set data business dimension, and storing the business-cleaned initial government data in the corresponding data theme in the data theme layer.
[0121] In one embodiment, when the processor executes a computer program, it performs business cleaning on the initial government data of the data detail layer based on a pre-set data business dimension, including: obtaining a pre-set entity information dimension; reconstructing the entity information of the initial government data of the data detail layer based on the entity information dimension; obtaining pre-set label information; and labeling the reconstructed initial government data based on the label information.
[0122] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: obtaining entities corresponding to business requirements; determining the entity attributes of each entity and obtaining the relationships between entities based on the relationships between entity attributes; performing logical modeling on government data in the data theme layer based on entities, entity attributes, and relationships between entities to obtain a government data logical processing model, wherein the government data in the data theme layer is obtained by reconstructing and labeling initial government data obtained from different data sources based on reconstructed entities and entity labels, and the logical modeling includes configuring data tables corresponding to the government data in the theme layer and the table structure of the data tables; obtaining configured materialization rules corresponding to the government data logical processing model; and performing materialization processing on the government data logical processing model based on the materialization rules, including: determining the model execution location and data storage location of the government data logical processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0123] In one embodiment, when a computer program is executed by a processor, it performs logical modeling on government data in the data subject layer based on entities, entity attributes, and relationships between entities to obtain a government data logical processing model. This includes: determining naming rules for each entity and its attributes; determining entity constraint rules for each entity; performing duplicate detection on the government data corresponding to entities in the data subject layer based on the entity constraint rules; retaining government data corresponding to the target data source and deleting government data corresponding to non-target data sources based on the duplicate detection results; renaming the government data corresponding to the target data source based on the naming rules; determining a data table to store the named government data; configuring the table structure of the data table based on the relationships between entities in the government data; configuring the index relationships between the data tables based on the relationships between entities; and obtaining the government data logical processing model based on the configured data tables, table structures, and index relationships.
[0124] In one embodiment, the configuration implemented by the computer program when executed by the processor corresponds to the materialization rules of the government data logic processing model, including: determining the materialization location of the service object corresponding to the business requirement of the data logic processing model based on the business information of the government data logic processing model; and the materialization processing of the government data logic processing model based on the materialization rules implemented by the computer program when executed by the processor, including: storing the entities corresponding to the government data logic processing model to different partitions based on the materialization location of the data logic processing model.
[0125] In one embodiment, when a computer program is executed by a processor, the service information of the service object corresponding to the business requirement based on the government data logic processing model is used to determine the materialized location of the data logic processing model, which includes: obtaining the storage frequency and storage response time requirements of the service object corresponding to the business requirement of the government data logic processing model; and determining the materialized location of the government data logic processing model based on the storage frequency and storage response time requirements.
[0126] In one embodiment, before the computer program, when executed by the processor, performs logical modeling on the government data in the data theme layer to obtain the government data logical processing model, the method further includes: receiving initial government data pushed from different data sources and storing the initial government data in the data storage layer; performing data cleaning on the initial government data in the data storage layer and storing the successfully cleaned initial government data in the data detail layer; and performing business cleaning on the initial government data in the data detail layer based on pre-set data business dimensions and storing the business-cleaned initial government data in the corresponding data theme in the data theme layer.
[0127] In one embodiment, when a computer program is executed by a processor, it performs business cleaning on the initial government data of the data detail layer based on a pre-set data business dimension, including: obtaining a pre-set entity information dimension; reconstructing entity information of the initial government data of the data detail layer based on the entity information dimension; obtaining pre-set label information; and labeling the reconstructed initial government data based on the label information.
[0128] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: obtaining entities corresponding to business requirements; determining entity attributes of each entity and obtaining relationships between entities based on the relationships between entity attributes; logically modeling government data in a data theme layer based on entities, entity attributes, and relationships between entities to obtain a government data logical processing model, wherein the government data in the data theme layer is obtained by reconstructing and labeling initial government data obtained from different data sources based on reconstructed entities and entity labels, and the logical modeling includes configuring data tables corresponding to the government data in the theme layer and the table structure of the data tables; obtaining configured materialization rules corresponding to the government data logical processing model; and materializing the government data logical processing model based on the materialization rules, including: determining the model execution location and data storage location of the government data logical processing model based on the processor information and hardware storage information of the service object of the business requirements.
[0129] In one embodiment, when a computer program is executed by a processor, it performs logical modeling on government data in the data subject layer based on entities, entity attributes, and relationships between entities to obtain a government data logical processing model. This includes: determining naming rules for each entity and its attributes; determining entity constraint rules for each entity; performing duplicate detection on the government data corresponding to entities in the data subject layer based on the entity constraint rules; retaining government data corresponding to the target data source and deleting government data corresponding to non-target data sources based on the duplicate detection results; renaming the government data corresponding to the target data source based on the naming rules; determining a data table to store the named government data; configuring the table structure of the data table based on the relationships between entities in the government data; configuring the index relationships between the data tables based on the relationships between entities; and obtaining the government data logical processing model based on the configured data tables, table structures, and index relationships.
[0130] In one embodiment, the configuration implemented by the computer program when executed by the processor corresponds to the materialization rules of the government data logic processing model, including: determining the materialization location of the data logic processing model based on the business information of the service objects corresponding to the business requirements of the government data logic processing model; and the materialization processing of the government data logic processing model based on the materialization rules implemented by the computer program when executed by the processor, including: storing the entities corresponding to the government data logic processing model to different partitions based on the materialization location of the data logic processing model.
[0131] In one embodiment, when a computer program is executed by a processor, the service information of the service object corresponding to the business requirement based on the government data logic processing model is used to determine the materialized location of the data logic processing model, which includes: obtaining the storage frequency and storage response time requirements of the service object corresponding to the business requirement of the government data logic processing model; and determining the materialized location of the government data logic processing model based on the storage frequency and storage response time requirements.
[0132] In one embodiment, before the computer program, when executed by the processor, performs logical modeling on the government data in the data theme layer to obtain the government data logical processing model, the method further includes: receiving initial government data pushed from different data sources and storing the initial government data in the data storage layer; performing data cleaning on the initial government data in the data storage layer and storing the successfully cleaned initial government data in the data detail layer; and performing business cleaning on the initial government data in the data detail layer based on pre-set data business dimensions and storing the business-cleaned initial government data in the corresponding data theme in the data theme layer.
[0133] In one embodiment, when a computer program is executed by a processor, it performs business cleaning on the initial government data of the data detail layer based on a pre-set data business dimension, including: obtaining a pre-set entity information dimension; reconstructing entity information of the initial government data of the data detail layer based on the entity information dimension; obtaining pre-set label information; and labeling the reconstructed initial government data based on the label information.
[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and government data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.
[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0136] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0137] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for modeling government data, characterized in that, The method includes: Obtain the entities required at the business level to correspond to new business requirements; Determine the entity attributes of each entity, and obtain the relationships between entities based on the relationships between the entity attributes; Based on the entities, entity attributes, and relationships between entities, a logical modeling process is performed on the government data in the data theme layer to obtain a government data logical processing model. This includes: retaining government data corresponding to the target data source based on the results of duplicate detection, and deleting government data corresponding to sources other than the target data source. The government data in the data theme layer is obtained by reconstructing and labeling initial government data obtained from different data sources based on the reconstructed entities and entity labels. The data theme layer stores all the data required for logical modeling. The logical modeling includes configuring additional data tables corresponding to the government data in the theme layer, as well as the table structure of the data tables. Obtain the configured materialization rules corresponding to the government data logical processing model; The government data logic processing model is materialized based on the materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements, wherein the hardware storage information includes storage frequency and storage response time requirements; The step of logically modeling government data in the data subject layer based on the entities, entity attributes, and relationships between entities to obtain a government data logical processing model includes: Determine the naming rules for each entity and its attributes; Determine the entity constraint rules for each of the entities; Based on the entity constraint rules, duplicate detection is performed on the government data corresponding to the entities in the data subject layer; The government data corresponding to the target data source is named according to the naming rules. The data table storing the named government data is determined, and the table structure of the data table is configured based on the relationship between the entities of the government data. The index relationship between the data tables is configured based on the relationship between the entities. Based on the configured data table, table structure and index relationship, the government data logical processing model is obtained. Before obtaining the configured materialized rules corresponding to the government data logical processing model, the process further includes: Based on the business information of the service objects corresponding to the business requirements of the government data logic processing model, the materialized location of the data logic processing model is determined. The process of materializing the government data logic processing model based on the materialization rules includes: Based on the materialized location of the data logic processing model, the entities corresponding to the government data logic processing model are stored in different partitions respectively; The determination of the materialized location of the data logic processing model based on the service object's business information corresponding to the business requirements of the government data logic processing model includes: Obtain the storage frequency and storage response time requirements of the service objects corresponding to the business requirements of the government data logical processing model; Based on the storage frequency and the storage response time requirements, the materialized location of the government data logical processing model is determined.
2. The method according to claim 1, characterized in that, Before performing logical modeling on the government data in the data subject layer to obtain the government data logical processing model, the following steps are also included: Receive initial government data pushed from different data sources and store the initial government data in the data storage layer; The initial government data in the data storage layer is cleaned, and the cleaned initial government data is stored in the data detail layer. Based on the pre-set data business dimensions, the initial government data in the data detail layer is cleaned for business purposes, and the cleaned initial government data is stored in the corresponding data theme in the data theme layer.
3. The method according to claim 2, characterized in that, The process of performing business cleansing on the initial government data of the data detail layer based on pre-set data business dimensions includes: Retrieve pre-defined entity information dimensions; Based on the entity information dimension, the initial government data of the data detail layer is reconstructed to include entity information. Retrieve pre-set tag information; Based on the label information, the initial government data after entity information reconstruction is labeled.
4. A government data modeling device, characterized in that, The device includes: The entity acquisition module is used to acquire the entities required at the business level for new business requirements. The entity attribute acquisition module is used to determine the entity attributes of each entity and obtain the relationship between entities based on the relationship between the entity attributes. The logical modeling module is used to perform logical modeling on the government data in the data theme layer based on the entities, entity attributes, and relationships between entities to obtain a government data logical processing model. This includes: retaining government data corresponding to the target data source based on the results of duplicate detection, and deleting government data that does not correspond to the target data source. The government data in the data theme layer is obtained by reconstructing and labeling initial government data obtained from different data sources based on the reconstructed entities and entity labels. The data theme layer stores all the data required for logical modeling. The logical modeling includes configuring other data tables corresponding to the government data in the theme layer, as well as the table structure of the data tables. The materialization rule acquisition module is used to acquire the configured materialization rules corresponding to the government data logical processing model; The materialization module is used to materialize the government data logic processing model based on the materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements, wherein the hardware storage information includes storage frequency and storage response time requirements; The logical modeling module is further configured to: determine the naming rules corresponding to each entity and its attributes; determine the entity constraint rules for each entity; perform duplicate detection on the government data corresponding to the entities in the data theme layer based on the entity constraint rules; perform naming processing on the government data corresponding to the target data source based on the naming rules; determine the data table for storing the named government data, configure the table structure of the data table based on the relationships between the entities in the government data, configure the index relationships between the data tables based on the relationships between the entities, and obtain the government data logical processing model based on the configured data table, table structure, and index relationships. The materialization rule acquisition module is also used to determine the materialization location of the data logic processing model based on the business information of the service object corresponding to the business requirements of the government data logic processing model. The materialization module is also used to store the entities corresponding to the government data logic processing model into different partitions based on the materialization location of the data logic processing model. The materialization module is also used to obtain the storage frequency and storage response time requirements of the service objects corresponding to the business requirements of the government data logic processing model. Based on the storage frequency and the storage response time requirements, the materialized location of the government data logical processing model is determined.
5. The apparatus according to claim 4, characterized in that, The device further includes: The data storage module is used to receive initial government data pushed from different data sources and store the initial government data in the data storage layer; perform data cleaning on the initial government data in the data storage layer and store the cleaned initial government data in the data detail layer; perform business cleaning on the initial government data in the data detail layer based on the pre-set data business dimensions and store the business-cleaned initial government data in the corresponding data theme in the data theme layer.
6. The apparatus according to claim 5, characterized in that, The data storage module is also used to acquire pre-set entity information dimensions; reconstruct entity information based on the entity information dimensions of the initial government data in the data detail layer; and acquire pre-set tag information. Based on the label information, the initial government data after entity information reconstruction is labeled.
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it performs the following steps: Obtain the entities required at the business level to correspond to new business requirements; Determine the entity attributes of each entity, and obtain the relationships between entities based on the relationships between the entity attributes; Based on the entities, entity attributes, and relationships between entities, a logical modeling process is performed on the government data in the data theme layer to obtain a government data logical processing model. This includes: retaining government data corresponding to the target data source based on the results of duplicate detection, and deleting government data corresponding to sources other than the target data source. The government data in the data theme layer is obtained by reconstructing and labeling initial government data obtained from different data sources based on the reconstructed entities and entity labels. The data theme layer stores all the data required for logical modeling. The logical modeling includes configuring additional data tables corresponding to the government data in the theme layer, as well as the table structure of the data tables. Obtain the configured materialization rules corresponding to the government data logical processing model; The government data logic processing model is materialized based on the materialization rules, including: determining the model execution location and data storage location of the government data logic processing model based on the processor information and hardware storage information of the service object of the business requirements, wherein the hardware storage information includes storage frequency and storage response time requirements; The step of logically modeling government data in the data subject layer based on the entities, entity attributes, and relationships between entities to obtain a government data logical processing model includes: Determine the naming rules for each entity and its attributes; Determine the entity constraint rules for each of the entities; Based on the entity constraint rules, duplicate detection is performed on the government data corresponding to the entities in the data subject layer; The government data corresponding to the target data source is named according to the naming rules. The data table storing the named government data is determined, and the table structure of the data table is configured based on the relationship between the entities of the government data. The index relationship between the data tables is configured based on the relationship between the entities. Based on the configured data table, table structure and index relationship, the government data logical processing model is obtained. Before obtaining the configured materialized rules corresponding to the government data logical processing model, the process further includes: Based on the business information of the service objects corresponding to the business requirements of the government data logic processing model, the materialized location of the data logic processing model is determined. The process of materializing the government data logic processing model based on the materialization rules includes: Based on the materialized location of the data logic processing model, the entities corresponding to the government data logic processing model are stored in different partitions respectively; The determination of the materialized location of the data logic processing model based on the service object's business information corresponding to the business requirements of the government data logic processing model includes: Obtain the storage frequency and storage response time requirements of the service objects corresponding to the business requirements of the government data logical processing model; Based on the storage frequency and the storage response time requirements, the materialized location of the government data logical processing model is determined.
8. The computer device according to claim 7, characterized in that, Before the processor executes the computer program to perform logical modeling of government data in the data subject layer to obtain the government data logical processing model, the process further includes: Receive initial government data pushed from different data sources and store the initial government data in the data storage layer; The initial government data in the data storage layer is cleaned, and the cleaned initial government data is stored in the data detail layer. Based on the pre-set data business dimensions, the initial government data in the data detail layer is cleaned for business purposes, and the cleaned initial government data is stored in the corresponding data theme in the data theme layer.
9. The computer device according to claim 8, characterized in that, When the processor executes the computer program, it performs business cleaning on the initial government data of the data detail layer based on a pre-set data business dimension, including: Retrieve pre-defined entity information dimensions; Based on the entity information dimension, the initial government data of the data detail layer is reconstructed to include entity information. Retrieve pre-set tag information; Based on the label information, the initial government data after entity information reconstruction is labeled.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.
11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Business data output method, device, equipment, medium and product
CN114880303A