Service data grouping method and device and computer equipment

Abstract

The invention relates to a business data grouping method and device and computer equipment. The method comprises the following steps: acquiring business data; based on a pre-constructed key field atlas, the service data is analyzed to obtain key fields in the service data, and the key field atlas comprises key fields, the relation between the key fields and key field priorities; dividing the key field into a main field and an auxiliary field based on the key field priority; under the condition that the main field is not empty, dividing the business data with the same main field into the same combination; and under the condition that the main field is null and / or at least one main field is different, grouping the service data based on the relationship between the auxiliary field and the key field. By adopting the method, high-quality data can be obtained.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, and computer equipment for grouping business data. Background Art

[0002] Administrative license data sources are complex and have diverse data structures. A single piece of data may not provide comprehensive information. Users need to query different data sources multiple times to obtain all the information they need. This not only increases the query burden on users, but may also lead to inconsistency and omissions in information.

[0003] In the related art, multi-source data is integrated to create a unified database for user query. However, the testing of multi-source fusion data involves multiple systems and data sources, which is prone to data conflict problems. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device and computer equipment for grouping business data in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for grouping service data. The method comprises:

[0006] Get business data;

[0007] Based on a pre-built key field map, the business data is parsed to obtain key fields in the business data, wherein the key field map includes preset key fields, relationships between key fields, and key field priorities;

[0008] Dividing the key fields into primary fields and auxiliary fields based on the key field priorities;

[0009] When the primary field is not empty, the business data with the same primary field are grouped into the same group;

[0010] When the primary field is empty and / or at least one primary field is different, the business data is grouped based on the relationship between the auxiliary field and the key field.

[0011] In one embodiment, the method further comprises:

[0012] An identity is added to the business data, where the identity is used to indicate that when the same identity appears in different groups, the grouping result is wrong.

[0013] In one embodiment, the process of constructing the key field map includes:

[0014] Get multiple grouped business data;

[0015] Analyzing the consistency of the grouped service data to obtain a result of the consistency analysis;

[0016] The key field map is obtained based on the result of the consistency analysis.

[0017] In one embodiment, the method further comprises:

[0018] Generate a hash value of the key field, and determine the consistency of the business data based on the hash value.

[0019] In one embodiment, the method further comprises:

[0020] Query the grouping results of business data based on the search tool, and mark the grouping results that do not meet the preset rules.

[0021] In a second aspect, the present application further provides a device for grouping service data, the device comprising:

[0022] Acquisition module, used to acquire business data;

[0023] A parsing module, used to parse the business data based on a pre-built key field map to obtain key fields in the business data, wherein the key field map includes preset key fields, relationships between key fields, and key field priorities;

[0024] A splitting module, used for dividing the key field into a primary field and an auxiliary field based on the priority of the key field;

[0025] A first grouping module, used for grouping the business data with the same main field into the same group when the main field is not empty;

[0026] The second grouping module is used to group the business data based on the relationship between the auxiliary field and the key field when the main field is empty and / or at least one main field is different.

[0027] In one embodiment, the device further comprises:

[0028] An identity is added to the business data, where the identity is used to indicate that when the same identity appears in different groups, the grouping result is wrong.

[0029] In one embodiment, the process of constructing the key field map includes:

[0030] Get multiple grouped business data;

[0031] Analyzing the consistency of the grouped service data to obtain a result of the consistency analysis;

[0032] The key field map is obtained based on the result of the consistency analysis.

[0033] In one embodiment, the device further comprises:

[0034] Generate a hash value of the key field, and determine the consistency of the business data based on the hash value.

[0035] In one embodiment, the device further comprises:

[0036] Query the grouping results of business data based on the search tool, and mark the grouping results that do not meet the preset rules.

[0037] In a third aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the service data grouping method are implemented.

[0038] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method for grouping service data when executed by a processor.

[0039] In a fifth aspect, the present disclosure further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for grouping service data are implemented.

[0040] The above business data grouping method has at least the following beneficial effects:

[0041] The embodiment scheme provided by the present disclosure can obtain key fields in business data. The key fields can be fields that represent data characteristics of the business data, and are divided into primary fields and auxiliary fields according to the priority of the key fields. When the primary field is not empty, the business data with the same primary field are divided into the same group. If the data is omitted or missing during the transmission process, that is, the primary field is empty, and / or at least one primary field is different, the business data is grouped based on the relationship between the auxiliary field and the key field, which can achieve more comprehensive, accurate and comprehensive data analysis.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] FIG1 is an application environment diagram of a method for grouping business data in one embodiment;

[0045] Figure 2 A schematic diagram of a flow chart of a method for grouping business data in one embodiment;

[0046] Figure 3 is a schematic diagram of service data grouping in one embodiment;

[0047] Figure 4 is a structural block diagram of a device for grouping service data in one embodiment;

[0048] FIG5 is a diagram showing the internal structure of a computer device in one embodiment;

[0049] FIG. 6 is a diagram showing the internal structure of a server in an embodiment. DETAILED DESCRIPTION

[0050] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, product or device. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.

[0052] The disclosed embodiment provides a method for grouping business data, which can be applied in an application environment as shown in FIG. 1 . The terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on a cloud or other network server. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0053] In some embodiments of the present disclosure, Figure 2 As shown, a method for grouping business data is provided, and the method is applied to the server in Figure 1 to process business data as an example for explanation. It can be understood that the method can be applied to the server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In a specific embodiment, the method can include the following steps:

[0054] S202: Acquire business data.

[0055] There are many sources of business data, including but not limited to company departments, industry regulatory agencies, public service platforms, etc. There are large differences in the data structure and format of different data sources, such as table format, document format, and XML format. Business data can include multiple types of features. According to the license type, it is divided into 5 categories: general license, franchise, recognition, approval, and registration. According to the type of license subject, it is divided into 5 categories: enterprise, product, project, individual, and matter. According to the license status, it is divided into 7 categories: undisclosed, valid, invalid, revoked, cancelled, expired, and denied. From the perspective of information integrity, at least one of the six contents of license document number, license number, license matter, license content, start time, and end time will be published. From the perspective of information consistency, one or more of the same information may appear in multiple business data.

[0056] S204: Based on a pre-constructed key field map, the business data is parsed and processed to obtain key fields in the business data, wherein the key field map includes preset key fields, relationships between key fields, and key field priorities.

[0057] Key fields can be fields that are representative in business data and can indicate the role of business data. For example, in administrative licensing data, key fields may include "license document number", "license number", "license matters", "license content", "start time", "end time", etc. The relationship between key fields can be the logical relationship between the fields in the business data. For example, "license document number" and "license number" are generally a one-to-one relationship, while "license matters" and "license content" may be a one-to-many relationship. The grouping results of business data can be determined by the relationship between key fields. Key field priority refers to the order of importance of different fields when processing data. Business data can include multiple key fields, each with different weights, and the priority of key fields is determined based on the weights.

[0058] S206: Divide the key fields into primary fields and auxiliary fields based on the key field priorities.

[0059] The primary field can be a key field with a higher priority in the business data, and can be a core component of the business data, determining the integrity and consistency of the data. For example, in administrative license data, the primary field may include the license document number, license number, etc. The auxiliary field can be a key field with a lower priority in the business data, which can play an auxiliary role in the grouping process and provide comprehensive business data information. For example, in administrative license data, the auxiliary fields may include license matters, license content, start time, end time, etc.

[0060] S208: When the main field is not empty, the business data with the same main field are grouped into the same group.

[0061] S210: When the primary field is empty and / or at least one primary field is different, group the business data based on the relationship between the auxiliary field and the key field.

[0062] If the primary field is not empty and the primary field is the same, this type of business data can be classified into the same group. For example, if the license document number and the license number are the same, they can be considered to be the same administrative license. If only one of the license document number and the license number has a value and is different or empty, they cannot be considered to be the same group. For example, if the license items and license contents are the same, or if the license items and license contents have no values, the consistency can be judged based on the start time and end time.

[0063] In the above-mentioned business data grouping method, key fields in the business data can be obtained. The key fields can be fields that represent the data characteristics of the business data, and are divided into primary fields and auxiliary fields according to the priority of the key fields. When the primary field is not empty, the business data with the same primary field are divided into the same combination. If the data is omitted or missing during the transmission process, that is, the primary field is empty, and / or at least one main field is different, the business data is grouped based on the relationship between the auxiliary field and the key field, which can achieve more comprehensive, accurate and comprehensive data analysis.

[0064] In some embodiments of the present disclosure, the method further includes:

[0065] An identity is added to the business data, where the identity is used to indicate that when the same identity appears in different groups, the grouping result is wrong.

[0066] Add an identity to each piece of business data and clarify the grouping information, such as group A and group B. Then, the data falling into the table will not have multiple groups A and multiple groups B, but only one group data.

[0067] In some embodiments of the present disclosure, the process of constructing the key field map includes:

[0068] Get multiple grouped business data;

[0069] Analyzing the consistency of the grouped service data to obtain a result of the consistency analysis;

[0070] The key field map is obtained based on the result of the consistency analysis.

[0071] The grouped business data can be obtained and divided into 7 groups of data, including 26 business data in total. The 7 groups of data can include a fully matched group: all key fields are the same, a partially matched group: some key fields are the same, and a completely mismatched group: the key fields are completely different. The consistency of the grouped business data can be analyzed based on the model to obtain the results of the consistency analysis. The difference between the output results of each group and the expected results is analyzed and fed back to the cleaning personnel. The above steps are repeated until the output results are consistent with expectations, the model training is successful, and the key field map is obtained. The consistency of the data includes the consistency of the data within the same group and the consistency of the data between different groups.

[0072] In some embodiments of the present disclosure, the method further includes:

[0073] Generate a hash value of the key field, and determine the consistency of the business data based on the hash value.

[0074] Generate a hash value for the key field. You can compare the hash values ​​of different key fields to determine the consistency of business data. If the hash values ​​of two key fields are the same, the two key fields are considered to be consistent. Otherwise, they are considered inconsistent.

[0075] Figure 3 Schematic diagram of service data grouping in an embodiment. There are 8 groups of data in total, which can be divided into 4 groups according to the grouping principle, namely: the first group: sequence numbers 1, 3, 4, 7, the second group: sequence number 2, the third group: sequence numbers 5, 6, and the fourth group: sequence number 8.

[0076] In some embodiments of the present disclosure, the grouping results of the business data are queried based on a search tool, and the grouping results that do not meet the preset rules are marked.

[0077] The preset rules may be which types of business data can be grouped into the same combination and which types of business data cannot be grouped into the same combination. For example, data with the same license document number and license number can be grouped into the same combination, but data with different license document numbers and license numbers cannot be grouped into the same combination. Using search tools, such as SQL statements (query statements), to query the grouping results of business data and mark the grouping results that do not meet the preset rules can improve data quality and consistency and ensure data accuracy.

[0078] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0079] Based on the same inventive concept, the embodiment of the present disclosure also provides a device for grouping business data for implementing the above-mentioned method for grouping business data. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific definition in the embodiment of the device for grouping business data provided below can refer to the definition of the method for grouping business data above, and will not be repeated here.

[0080] The device may include a system (including a distributed system), software (application), module, component, server, client, etc. that uses the method described in the embodiments of this specification and is combined with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiments of the present disclosure is as described in the following embodiments. Since the implementation scheme and method of the device to solve the problem are similar, the implementation of the specific device in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0081] In one embodiment, Figure 4 As shown, a device 400 for grouping service data is provided, and the device may be the aforementioned server, or a module, component, device, unit, etc. integrated in the server. The device 400 may include:

[0082] Acquisition module 402, used to acquire business data;

[0083] The parsing module 404 is used to parse the business data based on a pre-built key field map to obtain key fields in the business data, wherein the key field map includes preset key fields, relationships between key fields, and key field priorities;

[0084] A splitting module 406, configured to divide the key field into a primary field and an auxiliary field based on the key field priority;

[0085] A first grouping module 408, configured to group the business data with the same main field into the same group when the main field is not empty;

[0086] The second grouping module 410 is used to group the business data based on the relationship between the auxiliary field and the key field when the main field is empty and / or at least one main field is different.

[0087] In one embodiment, the device further comprises:

[0088] An identity is added to the business data, where the identity is used to indicate that when the same identity appears in different groups, the grouping result is wrong.

[0089] In one embodiment, the process of constructing the key field map includes:

[0090] Get multiple grouped business data;

[0091] Analyzing the consistency of the grouped service data to obtain a result of the consistency analysis;

[0092] The key field map is obtained based on the result of the consistency analysis.

[0093] In one embodiment, the device further comprises:

[0094] Generate a hash value of the key field, and determine the consistency of the business data based on the hash value.

[0095] In one embodiment, the device further comprises:

[0096] Query the grouping results of business data based on the search tool, and mark the grouping results that do not meet the preset rules.

[0097] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0098] Each module in the above-mentioned grouping device for business data can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0099] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG5 . The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store business data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for grouping business data is implemented.

[0100] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for grouping business data is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0101] Those skilled in the art will understand that the structures shown in FIG. 5 and FIG. 6 are merely block diagrams of partial structures related to the scheme of the present disclosure, and do not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figures, or combine certain components, or have a different arrangement of components.

[0102] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0103] In one embodiment, a computer program product is provided, including a computer program, which implements the method described in any embodiment of the present disclosure when executed by a processor.

[0104] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

[0105] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A method for grouping business data, characterized in that: The method comprises: Get business data; Based on a pre-built key field map, the business data is parsed to obtain key fields in the business data, wherein the key field map includes preset key fields, relationships between key fields, and key field priorities; Dividing the key fields into primary fields and auxiliary fields based on the key field priorities; When the primary field is not empty, the business data with the same primary field are grouped into the same group; When the primary field is empty and / or at least one primary field is different, the business data is grouped based on the relationship between the auxiliary field and the key field.

2. The method according to claim 1, characterized in that The method further comprises: An identity is added to the business data, where the identity is used to indicate that when the same identity appears in different groups, the grouping result is wrong.

3. The method according to claim 1, characterized in that The construction process of the key field map includes: Get multiple grouped business data; Analyzing the consistency of the grouped service data to obtain a result of the consistency analysis; The key field map is obtained based on the result of the consistency analysis.

4. The method according to claim 3, characterized in that The method further comprises: Generate a hash value of the key field, and determine the consistency of the business data based on the hash value.

5. The method according to claim 1, characterized in that: The method further comprises: Query the grouping results of business data based on the search tool, and mark the grouping results that do not meet the preset rules.

6. A device for grouping service data, characterized in that: The device comprises: Acquisition module, used to acquire business data; A parsing module, used to parse the business data based on a pre-built key field map to obtain key fields in the business data, wherein the key field map includes preset key fields, relationships between key fields, and key field priorities; A splitting module, used for dividing the key field into a primary field and an auxiliary field based on the priority of the key field; A first grouping module, used for grouping the business data with the same main field into the same group when the main field is not empty; The second grouping module is used to group the business data based on the relationship between the auxiliary field and the key field when the main field is empty and / or at least one main field is different.

7. The device according to claim 6, characterized in that The device also includes: An identity is added to the business data, where the identity is used to indicate that when the same identity appears in different groups, the grouping result is wrong.

8. The device according to claim 6, characterized in that The construction process of the key field map includes: Get multiple grouped business data; Analyzing the consistency of the grouped service data to obtain a result of the consistency analysis; The key field map is obtained based on the result of the consistency analysis.

9. The device according to claim 8, characterized in that The device also includes: Generate a hash value of the key field, and determine the consistency of the business data based on the hash value.

10. The device according to claim 6, characterized in that The device also includes: Query the grouping results of business data based on the search tool, and mark the grouping results that do not meet the preset rules.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images