Data label management method and data label management system

Abstract

The invention discloses a data label management method and a data label management system, and belongs to the technical field of computers and data management, the method is realized based on a data label management platform, and the realization process comprises the following steps: data source management, which is responsible for docking and integrating data from different sources, it is ensured that the platform can obtain data from multiple types of data sources; object management, which is used for defining and managing entities or objects in the label system, and comprises object creation and association relationship establishment; the task management is used for realizing automation and monitoring; label management including label classification and label configuration; and label query: through label screening, the user can quickly find the required data. A standardized label management process is provided, a complete and integrated solution is provided for creation, management, maintenance and use of labels, and a user is assisted in accurately and efficiently constructing a specific label system.

Description

Technical Field

[0001] The present invention relates to the technical field of computer and data management, and in particular to a data tag management method and a data tag management system. Background Art

[0002] In today's era of information explosion, big data is no longer just a technical term. It has deeply penetrated into every corner of the social economy and has become an important force to promote innovation and development in all walks of life. With the rapid development of platforms such as the Internet, the Internet of Things, and social media, the speed and scale of data generation have reached unprecedented levels, which is not only reflected in the expansion of quantity, but also in the richness and diversity of data types and the depth and breadth of value contained. Therefore, how to efficiently manage and utilize these massive and diverse data resources has become a key issue that needs to be solved urgently. Summary of the invention

[0003] The technical task of the present invention is to address the above shortcomings, provide a data label management method and a data label management system, provide a standardized label management process, and provide a complete and integrated solution for the creation, management, maintenance and use of labels, to assist users in accurately and efficiently building a specific label system.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A data tag management method is implemented based on a data tag management platform, and the implementation process includes:

[0006] Data source management, responsible for connecting and integrating data from different sources to ensure that the platform can obtain data from multiple types of data sources;

[0007] Object management, which is used to define and manage entities or objects in the tag system, including creating objects and establishing relationships;

[0008] Task management, which is used to achieve automation and monitoring, including automatic labeling, task monitoring, logging and other functions;

[0009] Tag management, including tag classification and tag configuration. The tag classification can create different tag classifications (such as description tags, policy tags, etc.) according to the tag objects; the tag configuration includes creating, deleting, modifying tags and corresponding rules;

[0010] Tag query, through tag screening, enables users to quickly find the data they need.

[0011] This method uses a tag management platform to create a standardized tag system through specified tag management rules. Based on tag management rules, it can not only help organizations quickly locate target information and improve data retrieval efficiency, but also promote cross-departmental data sharing and effectively enhance the value of data assets.

[0012] Furthermore, the multiple types of data sources include relational databases (such as MySQL, Oracle), NoSQL databases (such as MongoDB), data warehouses, API interfaces, file systems (CSV, Excel, JSON files), etc.;

[0013] The data source management is specifically implemented as follows:

[0014] Configuration and connection: Provide an interface or configuration file to set the connection parameters of the data source, including host address, port, user name, password, etc.

[0015] Data synchronization: Supports scheduled or real-time data synchronization mechanisms to ensure that the data in the platform is kept up to date with the source;

[0016] Data preprocessing: Before data is imported into the labeling system, necessary cleaning, conversion and format standardization are performed to improve data quality;

[0017] Monitoring and alarm: Monitor the health status of data sources and the data transmission process, and immediately notify the administrator if any abnormality occurs.

[0018] Furthermore, the object management is specifically implemented as follows:

[0019] Create an object: By specifying the data source, define the basic information of each object and the attributes or fields it contains. The basic information includes the object name, description, and associated data source information, and the attributes or fields include attributes such as name, age, and gender;

[0020] Establishing association relationships: Setting association relationships between objects. For example, a user object can be configured with its related marriage and housing subordinate objects to form a specific object relationship graph.

[0021] Permission control: Set object access permissions to ensure data security and compliance. For example, only users with specific roles can view or edit sensitive object information.

[0022] Furthermore, the task management is specifically implemented as follows:

[0023] Task configuration: including creating, configuring, and managing tasks, scheduling the frequency and time of tag task execution, starting or stopping task execution, and viewing and manually executing tasks or individual tags;

[0024] Task monitoring: Track the status of task execution, including running, not executed, etc.; record the logs of tag tasks and single tag execution, and provide failure cause analysis.

[0025] Furthermore, the label management includes label levels of city, district, and street levels; label types include basic labels, combined labels, and warning labels;

[0026] Basic tags are divided into rule tags and non-rule tags. Rule tags require you to configure tag rules and bind tag objects and tag tasks. Tags are automatically added by executing tag tasks. Non-rule tags can be added manually by importing tags.

[0027] Combined tags are based on basic tags. By selecting two or more tags and configuring combination rules (such as AND, OR, and other conditions), a new tag is formed.

[0028] Warning tags are added to basic tags to filter out tags that are about to meet a certain rule.

[0029] Furthermore, the tag query supports combined queries based on tags, including filtering data by the intersection and union of selected tags;

[0030] Provide a low-latency query experience, allowing users to instantly obtain the latest label data;

[0031] The tag query allows users to export query results to CSV, Excel and other formats, or directly share them with other team members on the platform.

[0032] Furthermore, the implementation process of the method includes:

[0033] S1, object configuration;

[0034] S2, label classification;

[0035] S3, label configuration, including basic labels, combined labels, and warning labels;

[0036] S4, labeling, including manual import and automatic labeling;

[0037] S5. Query and use of tags.

[0038] The present invention also claims protection for a data tag management system, comprising:

[0039] The data source management module is responsible for connecting and integrating data from different sources to ensure that the system can obtain data from multiple types of data sources;

[0040] Object management module, used to define and manage entities or objects in the tag system;

[0041] Task management module, which is used to achieve automation and monitoring, including automatic labeling, task monitoring, logging and other functions;

[0042] Tag management module, including tag classification and tag configuration;

[0043] Tag query module, which provides tag screening, allowing users to quickly find the data they need;

[0044] The system implements data label management through the above-mentioned data label management method.

[0045] The present invention also claims a data tag management device, comprising: at least one memory and at least one processor;

[0046] The at least one memory is used to store a machine-readable program;

[0047] The at least one processor is used to call the machine-readable program to implement the above method.

[0048] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which, when executed by a processor, causes the processor to perform the above method.

[0049] Compared with the prior art, the data tag management method and data tag management system of the present invention have the following beneficial effects:

[0050] 1. Through the division of data sources, objects, tag classification, tags and other levels, a more reasonable structured tag system is constructed. Compared with traditional tags, the management of tags is more standardized and clear.

[0051] 2. By configuring labeling rules and labeling tasks, the labeling operation can be fully automated, and the records can be monitored in real time, which is highly maintainable.

[0052] 3. Through functions such as combined tags and warning tags, you can make full use of existing basic tags and expand the tag library more flexibly and efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of the data label system construction process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0055] The embodiment of the present invention provides a data tag management method, which is implemented based on a data tag management platform, and the implementation process includes:

[0056] 1. Data source management,

[0057] Data source management is the foundation of the data label management platform. It is responsible for connecting and integrating data from different sources to ensure that the platform can obtain data from various types of data sources such as relational databases (such as MySQL, Oracle), NoSQL databases (such as MongoDB), data warehouses, API interfaces, file systems (CSV, Excel, JSON files), etc.

[0058] The data source management is specifically implemented as follows:

[0059] Configuration and connection: Provide an interface or configuration file to set the connection parameters of the data source, such as host address, port, user name, password, etc.

[0060] Data synchronization: Supports scheduled or real-time data synchronization mechanisms to ensure that the data in the platform is kept up to date with the source.

[0061] Data preprocessing: Before the data is imported into the labeling system, necessary cleaning, conversion and format standardization are performed to improve data quality.

[0062] Monitoring and alarm: Monitor the health status of data sources and the data transmission process, and immediately notify the administrator if any abnormality occurs.

[0063] 2. Object management,

[0064] Object management is used to define and manage entities or objects in the tag system, allowing users to:

[0065] Create objects: By specifying the data source, you can define the basic information of each object (such as object name, description, associated data source information, etc.) and the attributes or fields it contains (for example, a user object may contain attributes such as name, age, and gender);

[0066] Establishing association relationships: Setting association relationships between objects. For example, a user object can be configured with its related marriage and housing subordinate objects to form a specific object relationship graph.

[0067] Permission control: Set object access permissions to ensure data security and compliance. For example, only users with specific roles can view or edit sensitive object information.

[0068] 3. Task management,

[0069] Task management is used to achieve automation and monitoring, including automatic labeling, task monitoring, logging and other functions; the specific implementation is as follows:

[0070] Task configuration: including creating, configuring, and managing tasks, scheduling the frequency and time of tag task execution, starting or stopping task execution, and viewing and manually executing tasks or individual tags;

[0071] Task monitoring: Track the status of task execution, such as running, not executed, etc.; record the logs of tag tasks and single tag execution, and provide failure cause analysis.

[0072] 4. Tag management,

[0073] Tag management is one of the core functions, involving key parts such as tag classification and configuration.

[0074] The label classification can create different label classifications (such as description labels, policy labels, etc.) according to label objects;

[0075] The tag configuration includes creating, deleting, and modifying tags and corresponding rules.

[0076] The label levels are divided into city level, district level, and street level; the label types are divided into basic labels, combined labels, and warning labels;

[0077] Basic tags are divided into rule tags and non-rule tags. Rule tags require you to configure tag rules and bind tag objects and tag tasks. Tags are automatically added by executing tag tasks. Non-rule tags can be added manually by importing tags.

[0078] Combined tags are based on basic tags. By selecting two or more tags and configuring combination rules (such as AND, OR, and other conditions), a new tag is formed.

[0079] Warning tags are added to basic tags to filter out tags that are about to meet a certain rule.

[0080] 5. Tag query,

[0081] Provides flexible tag filtering capabilities, allowing users to quickly find the data they need:

[0082] Supports combined queries based on tags, such as filtering data by the intersection and union of selected tags;

[0083] Provide a low-latency query experience, allowing users to instantly obtain the latest label data;

[0084] Allow users to export query results to CSV, Excel and other formats, or share them with other team members directly on the platform.

[0085] like Figure 1 As shown, the implementation process of this method includes:

[0086] S1, object configuration;

[0087] S2, label classification;

[0088] S3, label configuration, including basic labels, combined labels, and warning labels;

[0089] S4, labeling, including manual import and automatic labeling;

[0090] S5. Query and use of tags.

[0091] The main development language used in this method is Java, the main development framework is SpringBoot+Vue, the interaction between modules within the platform system uses the RPC remote procedure call protocol, and the database storage used includes Mysql, Redis, etc.

[0092] The data labeling system cleans, integrates, classifies and labels the original data to form a standardized and systematic label set, thus achieving refined management and deep insight into the data. This system can not only help organizations quickly locate target information and improve data retrieval efficiency, but also promote cross-departmental data sharing, break down information silos, and provide strong support for various application scenarios such as personalized recommendations, customer profiling, and risk control. It can effectively enhance the value of data assets and provide decision support and guidance for diversified applications and services.

[0093] An embodiment of the present invention further provides a data tag management system, which implements data tag management through the data tag management method described in the above embodiment.

[0094] The system includes:

[0095] 1. The data source management module is responsible for connecting and integrating data from different sources to ensure that the platform can obtain data from various types of data sources such as relational databases (such as MySQL, Oracle), NoSQL databases (such as MongoDB), data warehouses, API interfaces, file systems (CSV, Excel, JSON files), etc.

[0096] The data source management module is specifically implemented as follows:

[0097] Configuration and connection: Provide an interface or configuration file to set the connection parameters of the data source, such as host address, port, user name, password, etc.

[0098] Data synchronization: Supports scheduled or real-time data synchronization mechanisms to ensure that the data in the platform is kept up to date with the source.

[0099] Data preprocessing: Before the data is imported into the labeling system, necessary cleaning, conversion and format standardization are performed to improve data quality.

[0100] Monitoring and alarm: Monitor the health status of data sources and the data transmission process, and immediately notify the administrator if any abnormality occurs.

[0101] 2. Object management module, which is used to define and manage entities or objects in the tag system, allowing users to:

[0102] Create objects: By specifying the data source, you can define the basic information of each object (such as object name, description, associated data source information, etc.) and the attributes or fields it contains (for example, a user object may contain attributes such as name, age, and gender);

[0103] Establishing association relationships: Setting association relationships between objects. For example, a user object can be configured with its related marriage and housing subordinate objects to form a specific object relationship graph.

[0104] Permission control: Set object access permissions to ensure data security and compliance. For example, only users with specific roles can view or edit sensitive object information.

[0105] 3. Task management module: Management is used to achieve automation and monitoring, including automatic labeling, task monitoring, logging and other functions; the specific implementation is as follows:

[0106] Task configuration: including creating, configuring, and managing tasks, scheduling the frequency and time of tag task execution, starting or stopping task execution, and viewing and manually executing tasks or individual tags;

[0107] Task monitoring: Track the status of task execution, such as running, not executed, etc.; record the logs of tag tasks and single tag execution, and provide failure cause analysis.

[0108] 4. Tag management module. The tag management module is one of the core functions, involving key parts such as tag classification and configuration.

[0109] The label classification can create different label classifications (such as description labels, policy labels, etc.) according to label objects;

[0110] The tag configuration includes creating, deleting, and modifying tags and corresponding rules.

[0111] The label levels are divided into city level, district level, and street level; the label types are divided into basic labels, combined labels, and warning labels;

[0112] Basic tags are divided into rule tags and non-rule tags. Rule tags require you to configure tag rules and bind tag objects and tag tasks. Tags are automatically added by executing tag tasks. Non-rule tags can be added manually by importing tags.

[0113] Combined tags are based on basic tags. By selecting two or more tags and configuring combination rules (such as AND, OR, and other conditions), a new tag is formed.

[0114] Warning tags are added to basic tags to filter out tags that are about to meet a certain rule.

[0115] 5. Tag query module,

[0116] Provides flexible tag filtering capabilities, allowing users to quickly find the data they need:

[0117] Supports combined queries based on tags, such as filtering data by the intersection and union of selected tags;

[0118] Provide a low-latency query experience, allowing users to instantly obtain the latest label data;

[0119] Allow users to export query results to CSV, Excel and other formats, or share them with other team members directly on the platform.

[0120] An embodiment of the present invention further provides a data tag management device, comprising: at least one memory and at least one processor;

[0121] The at least one memory is used to store a machine-readable program;

[0122] The at least one processor is used to call the machine-readable program to implement the data tag management method described in the above embodiment.

[0123] The embodiment of the present invention further provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the processor executes the data tag management method described in the above embodiment. Specifically, a system or device equipped with a storage medium can be provided, on which software program codes for implementing the functions of any of the above embodiments are stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0124] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0125] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0126] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0127] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0128] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A data tag management method, characterized in that: This method is implemented based on the data tag management platform, and the implementation process includes: Data source management, responsible for connecting and integrating data from different sources to ensure that the platform can obtain data from multiple types of data sources; Object management, which is used to define and manage entities or objects in the tag system, including creating objects and establishing relationships; Task management, which is used to achieve automation and monitoring, including automatic labeling, task monitoring, and logging; Tag management, including tag classification and tag configuration. The tag classification creates different tag classifications according to tag objects. The tag configuration includes creating, deleting, modifying tags and corresponding rules. Tag query, through tag screening, enables users to quickly find the data they need.

2. A data tag management method according to claim 1, characterized in that: The multiple types of data sources include relational databases, NoSQL databases, data warehouses, API interfaces, and file systems; The data source management is specifically implemented as follows: Configuration and connection: Provide an interface or configuration file to set the connection parameters of the data source, including host address, port, user name, and password; Data synchronization: Supports scheduled or real-time data synchronization mechanisms to ensure that the data in the platform is kept up to date with the source; Data preprocessing: Before data is imported into the labeling system, necessary cleaning, conversion and format standardization are performed; Monitoring and alarm: Monitor the health status of data sources and the data transmission process, and immediately notify the administrator if any abnormality occurs.

3. A data tag management method according to claim 1, characterized in that: The object management is specifically implemented as follows: Create an object: By specifying the data source, define the basic information of each object and the attributes or fields it contains. The basic information includes the object name, description, and associated data source information. The attributes or fields include name, age, and gender. Establishing association relationships: Setting association relationships between objects to form a specific object relationship graph; Permission control: Set object access permissions to ensure data security and compliance.

4. A data tag management method according to claim 1, characterized in that: The task management is specifically implemented as follows: Task configuration: including creating, configuring, and managing tasks, scheduling the frequency and time of tag task execution, starting or stopping task execution, and viewing and manually executing tasks or individual tags; Task monitoring: Track the status of task execution, including running and not executed; record the logs of tag tasks and single tag execution, and provide failure cause analysis.

5. A data tag management method according to claim 1, characterized in that: The label management includes label levels of city, district, and street levels; label types include basic labels, combined labels, and warning labels; Basic tags are divided into rule tags and non-rule tags. Rule tags require you to configure tag rules and bind tag objects and tag tasks. Tags are automatically added by executing tag tasks. Non-rule tags are added manually by importing tags. Combined tags are based on basic tags. You can select two or more tags and configure combination rules to combine them into a new tag. Warning tags are added to basic tags to filter out tags that are about to meet a certain rule.

6. A data tag management method according to claim 1 or 5, characterized in that: The tag query supports combination queries based on tags, including filtering data by the intersection and union of selected tags; The tag query allows users to export query results to CSV, Excel format, or share them directly with other team members on the platform.

7. A data tag management method according to claim 1, characterized in that: The implementation process of this method includes: S1, object configuration; S2, label classification; S3, label configuration, including basic labels, combined labels, and warning labels; S4, labeling, including manual import and automatic labeling; S5. Query and use of tags.

8. A data tag management system, characterized in that: include: The data source management module is responsible for connecting and integrating data from different sources to ensure that the system can obtain data from multiple types of data sources; Object management module, used to define and manage entities or objects in the tag system; Task management module, used to achieve automation and monitoring, including automatic labeling, task monitoring, and logging; Tag management module, including tag classification and tag configuration; Tag query module, which provides tag screening, allowing users to quickly find the data they need; The system implements data tag management through the data tag management method described in any one of claims 1 to 7.

9. A data tag management device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, cause the processor to execute the method according to any one of claims 1 to 7.

Images