Data management method and device based on tag management

Abstract

The present application relates to the technical field of data information management, and in particular to a data management method and device based on label management, the method comprising: obtaining data to be labeled; inputting the data to be labeled into a target recognition model, and outputting a plurality of entity label sets, wherein the target recognition model is a model obtained by pre-training an initial recognition model with sample data; performing comprehensive statistics on a plurality of entity label sets according to a preset evaluation threshold, and obtaining a target label set; and storing the target label set in a storage location corresponding to the data to be labeled. The problem of low accuracy of data management in the prior art is solved, and the accuracy of data management is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data information management, and in particular to a data management method and device based on tag management. Background Art

[0002] With the continuous development of the Internet, various data are generated all the time. In order to better transmit or utilize these data, many companies are using their own data management systems (including software systems and network systems) to manage various data. However, the data management system needs to use multiple basic software to store data, such as relational database + cache, document database + cache and other solutions. However, the use of multiple basic software to realize data storage and management requires a lot of time when adding, deleting, modifying and checking data.

[0003] In order to solve the time-consuming problem of data management system, the prior art usually adopts the method of labeling data for data storage, but only uses the method of labeling data for data storage. However, in this process, the data labeling operation usually adopts the method of pre-setting labels and matching them with data. This method relies heavily on preset labels. When the preset labels remain unchanged for a long time, the accuracy of the label matching results cannot be guaranteed, thereby affecting the accuracy of data management according to the preset labels.

[0004] Currently, no effective solution has been proposed to address the problem of low accuracy of data management in the prior art. Summary of the invention

[0005] The embodiments of the present application provide a data management method and device based on tag management to at least solve the problem of low accuracy of data management in the prior art.

[0006] According to one aspect of an embodiment of the present application, a data management method based on label management is provided, including: obtaining data to be labeled; inputting the data to be labeled into a target recognition model, and outputting multiple entity label sets, wherein the target recognition model is a model obtained by pre-training an initial recognition model with sample data; performing comprehensive statistics on the multiple entity label sets according to a preset evaluation threshold to obtain a target label set; and storing the target label set in a storage location corresponding to the data to be labeled.

[0007] According to another aspect of an embodiment of the present application, a data management device based on label management is also provided, including: an acquisition unit, used to acquire data to be labeled; an input unit, used to input the data to be labeled into a target recognition model, and output multiple entity label sets, wherein the target recognition model is a model obtained by pre-training an initial recognition model with sample data; a comprehensive statistical unit, used to perform comprehensive statistics on multiple entity label sets according to a preset evaluation threshold to obtain a target label set; and a storage unit, used to store the target label set in a storage location corresponding to the data to be labeled.

[0008] Optionally, the input unit includes: an acquisition subunit, which is used to acquire multiple preset prompt texts corresponding to the data to be labeled, and select a prompt text from the multiple preset prompt texts as the current prompt text; a first control subunit, which is used to control the target recognition model to perform a first recognition on the current prompt text to obtain a reference recognition result, wherein the reference recognition result includes the target role, and multiple label types included in the entity label set corresponding to the current prompt text, wherein the multiple label types include entity domain labels, entity category labels and entity attribute labels; a second control subunit, which is used to control the target recognition model to perform a second recognition on the data to be labeled according to the reference recognition result to obtain the current label set, wherein the multiple entity label sets include the current label set, the current label set includes multiple label types corresponding to multiple entities, and reference labels corresponding to each label type, wherein the multiple label types include entity domain labels, entity category labels and entity attribute labels; a selection subunit, which is used to select a preset prompt text from multiple preset prompt texts other than the current prompt text as the current prompt text.

[0009] Optionally, the above-mentioned data management device based on label management also includes a training unit, which includes: a first acquisition subunit, used to acquire a sample data set and a sample label set corresponding to each sample data, and set multiple reference prompt texts for multiple sample data respectively, wherein each sample data set corresponds to an industry field and multiple reference prompt texts; a training subunit, used to iteratively train the initial recognition model according to the sample data set and the multiple reference prompt texts to obtain a training result; a comparison subunit, used to perform a similarity comparison between the training result and the sample label set to obtain a comparison result; the comparison subunit is also used to determine the recognition model corresponding to the training result as the target recognition model when the comparison result is greater than or equal to a preset threshold.

[0010] Optionally, the above-mentioned comprehensive statistical unit includes: a second acquisition subunit, used to obtain the text quantity of multiple preset prompt texts; a determination subunit, used to determine the quantity of each entity label according to multiple entity label sets, and obtain multiple label quantities, wherein each label quantity is less than or equal to the text quantity; a screening subunit, used to screen multiple target label quantities greater than or equal to a preset quantity from the multiple label quantities, and determine the entity labels corresponding to the multiple target label quantities as target labels, and obtain a target label set.

[0011] Optionally, the acquisition unit includes: a third acquisition subunit, used to acquire reference data and a data type corresponding to the reference data; a preprocessing subunit, used to preprocess the reference data using a preprocessing method corresponding to the data type to obtain data to be labeled.

[0012] Optionally, the above-mentioned preprocessing sub-unit includes: a first preprocessing module, which is used to obtain the data structure and data content of the reference data when the data type is a structured data type, and add comment information to the reference data according to the data content and data structure; perform data extraction on the reference data after adding the comment information to obtain the data to be labeled; a second preprocessing module, which is used to obtain the file type corresponding to the reference data when the data type is an unstructured data type; extract the data to be labeled from the reference data according to the data extraction method corresponding to the file type.

[0013] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the above data management method based on tag management.

[0014] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a memory and a processor, characterized in that the memory stores a computer program, and the processor is configured to execute the above-mentioned data management method based on tag management through the computer program.

[0015] The above-mentioned data management method based on tag management of the present application solves the problem of low accuracy of data management in the prior art and improves the accuracy of data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a schematic diagram of a hardware environment of an optional data management method based on tag management according to an embodiment of the present invention;

[0018] Figure 2 is a flow chart of an optional data management method based on tag management according to an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of an optional data management method based on tag management according to an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of another optional data management method based on tag management according to an embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of another optional data management method based on tag management according to an embodiment of the present invention;

[0022] Figure 6 is a schematic structural diagram of an optional data management device based on tag management according to an embodiment of the present invention;

[0023] Figure 7 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0026] It should be noted that the embodiments in this application and all the required data in the embodiments are obtained and used by legitimate means with the consent of the user and relevant personnel, and the user's information security will be guaranteed during the acquisition and use process.

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] In order to solve the problem of low data management accuracy in the prior art, the present application provides a data management method based on label management (for example, labeling student data in an education system and managing them according to labels such as "education-student-name"; for example, labeling patient data in a medical system and managing them according to labels such as "medical-patient-name"). As an optional implementation method, the above-mentioned data management method based on label management can be applied to, but is not limited to, Figure 1 In the virtual positioning detection system composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 through the network 110. The above network may include but is not limited to: wired network, wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The above terminal device may include but is not limited to at least one of the following: mobile phone (such as Android mobile phone, iOS mobile phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices, mobile Internet device), PAD, desktop computer, smart TV, etc. A client is running on the above terminal device, and an application may be installed in the client, wherein the user can upload the file that needs to be managed to the server 104 through the terminal device 102, and the server 104 obtains the data to be labeled by identifying the file; or the terminal device may identify the file to obtain the data to be labeled, and then transmit the data to be labeled to the server 104, that is, the way in which the server obtains the data to be labeled is not limited in this application.

[0029] The terminal device 102 is also provided with a display, a processor and a memory. The display can be used to display the files and / or data to be labeled. The processor can be used to process the files and / or data to be labeled, for example, encryption, compression, filtering, etc. The memory is used to store the data to be processed and processed. It can be understood that when the terminal device 102 receives the data to be labeled from the user, the data to be labeled is sent to the server 104 through the network 110. Alternatively, when the terminal device 102 receives a data management request, the file or data to be labeled carried in the data management request is sent to the server 104. The server 104 processes the data to be labeled to generate a label corresponding to the data to be labeled for storage and management.

[0030] The server 104 may be a single server, a server cluster consisting of multiple servers, a cloud server, or a server cluster for performing a variety of different tasks. The server includes a database 114 and a processing engine 116. The processing engine 116 is used to process the data to be labeled; the database 114 is used to store the data to be labeled and the target label set.

[0031] According to one aspect of an embodiment of the present invention, the above-mentioned data management system based on tag management can also perform the following steps: the terminal device 102 executes step S102 and sends the data to be labeled to the server 104 through the network 110; the server 104 executes steps S104 to S110 to obtain the data to be labeled; the data to be labeled is input into the target recognition model, and multiple entity tag sets are output, wherein the target recognition model is a model obtained by pre-training the initial recognition model with sample data; a plurality of entity tag sets are comprehensively counted according to a preset evaluation threshold to obtain a target tag set; the target tag set is stored in the storage location corresponding to the data to be labeled. After that, the server 104 executes S112 and sends the target tag set and the storage location of the target tag set to the terminal device 102 through the network 110, so that the terminal device 102 displays the data to be labeled and the target tag set and provides the user with a data editing component, which is used to allow the user to add, delete, modify and query the tags in the target tag set.

[0032] In the above-mentioned embodiment of the present invention, after multiple entity label sets corresponding to the data to be labeled are identified by the target recognition model in this application, a more accurate target label set corresponding to the data to be labeled is further obtained by comprehensive statistics of the multiple entity labels, thereby solving the problem of low data management accuracy in the prior art and improving the accuracy of data management.

[0033] The above is only an example and is not limited in this embodiment.

[0034] As an optional implementation, please refer to Figure 2 , which shows a flow chart of a data management method based on tag management according to an embodiment of the present application. The method comprises the following steps:

[0035] S202, obtaining data to be labeled;

[0036] S204, inputting the data to be labeled into a target recognition model, and outputting a plurality of entity label sets, wherein the target entity label is a model obtained by pre-training an initial recognition model with sample data;

[0037] S206, performing comprehensive statistics on multiple entity tag sets according to a preset evaluation threshold to obtain a target tag set;

[0038] S208, storing the target tag set in a storage location corresponding to the data to be labeled.

[0039] It should be noted that the data to be labeled in the above S202 can be structured data or unstructured data. Structured data can be understood as but not limited to relational database tables; while unstructured data is mainly stored in file objects, such as minio, ftp, etc., and the file formats include various, such as pictures, documents, etc. There are many specific ways to obtain the data to be labeled, for example, the first is to send it directly by the terminal device; the second is that the terminal device sends a file containing the data to be labeled, and the server obtains the data to be labeled by further identifying the data to be labeled; the third is to set a target file storage location on the terminal device for storing files containing the data to be labeled; and the server can detect the number of files in the above target file storage location at regular intervals; whenever there is a file stored in the target file storage location, the server obtains the file therein to identify the data to be labeled, so as to perform further processing and storage, thereby managing the data to be labeled through labels, and finally sends the data to be labeled and the target label set corresponding to the data to be labeled to the terminal device, and the terminal device displays the data to be labeled and the target label set corresponding to the data to be labeled and provides a data editing component, which is used to allow the user to add, delete, modify and query the label data through the labels in the target label set, that is, the user can register a new label, such as "Finance-Customer-Account Number", or modify or delete an existing label, for example, modify "Education-Student-Name" to "Education-Student-Full Name".

[0040] It can be understood that when the second and third data acquisition methods are used to acquire data, after the file containing the data to be labeled is obtained, the specific acquisition methods are different for different types of data (structured data and unstructured data), and the data acquisition method corresponding to the data type can be used to acquire the data to be labeled.

[0041] That is, obtaining the data to be labeled in the above S202 includes:

[0042] S1, obtaining reference data and data types corresponding to the reference data;

[0043] S2, preprocessing the reference data using a preprocessing method corresponding to the data type to obtain the data to be labeled.

[0044] The above S2 specifically includes: when the data type is a structured data type, obtaining the data structure and data content of the reference data, and adding remark information to the reference data according to the data content and data structure; performing data extraction on the reference data after adding the remark information to obtain the data to be labeled; when the data type is an unstructured data type, obtaining the file type corresponding to the reference data; extracting the data to be labeled from the reference data according to the data extraction method corresponding to the file type.

[0045] The target recognition model in S204 is a model obtained by pre-training the initial potential model with sample data. The specific model is, for example, LLM. In addition, it can also be other models. The specific model can be selected and trained as needed. The target recognition model is used to identify multiple entity label sets corresponding to the data to be labeled. The reason why multiple entity label sets can be identified for the same data to be labeled is that the target recognition model recognizes the data to be labeled multiple times, and each recognition is based on different preset prompt texts (also referred to as prompt words). According to each preset prompt text, the data to be labeled can be recognized to identify the corresponding entity label set, such as Figure 3 As shown, each preset prompt text (such as Figure 3 The prompt word 1, prompt word 2, prompt word n) in corresponds to an entity label set (such as Figure 3 Result 1, Result 2, Result n shown in ), each entity tag set includes multiple tags (such as Figure 3 tag1, tag2, tag3, tagn in ); and the preset prompt text includes the identity role set for the target recognition model, the operations that the target recognition model needs to perform, the restrictions on the tags to be recognized (which can be understood as, but is not limited to, outputting the tags in the entity tag set according to the tag restriction requirements) and the restrictions on the output results.

[0046] It should be noted that each identified entity tag set corresponds to at least one entity. If the entity tag set corresponds to multiple entities, each entity tag set includes multiple entity tag subsets, and each entity corresponds to an entity tag subset. That is, the multiple identified entity tag subsets are all attached to a corresponding entity. The relationship between the entity and the tag is as follows: Figure 4 shown.

[0047] The above preset prompt text sets the identity role for the target recognition model and the operations that the target recognition model needs to perform, such as "You are a data governance expert who is good at analyzing domain models with DDD thinking. Now you need to extract entity information based on the text given by the user (these contents may come from pictures and various files)"; the above preset prompt text has restrictions on the labels to be recognized, such as "The output entity information must include the field, industry, entity name and entity attributes, and the entity attributes must correspond to the field, industry, and entity must be a common and widely accepted standard noun. The attribute name should be as concise as possible, and a corresponding base name should be given when necessary. For example, the base names corresponding to home address and work address are both addresses. If it is already the simplest name, there is no need to give a base name. If there are multiple entities, multiple entity arrays are returned. If no entity is recognized, an empty array is returned without explaining the reason" (that is, the restrictions on the labels to be recognized also include recognition examples for the target recognition model). The above preset prompt text has restrictions on the output results, such as "Note: No reasoning process or explanation is required, and a string in JSON format is directly returned. If no answer can be given due to logical problems, an empty array is returned without giving any reason." The specific recognition results are shown in Table 1 below:

[0048]

[0049] Table 1

[0050] The above-mentioned preset evaluation threshold can be understood, but not limited to, as a threshold corresponding to a preset number of tags. Comprehensive statistics can be understood, but not limited to, as a comprehensive compilation of multiple entity tag sets, the number of times multiple tags in multiple entity tag sets appear, and the tags whose number of tags reaches the preset evaluation threshold are taken as valid tags, thereby obtaining multiple tags corresponding to a more accurate identification model to be tagged. Finally, multiple tags (i.e., multiple target tags included in the target tag set) are automatically stored in the storage location corresponding to the data to be tagged, so that users can manage the tagged data by adding, deleting, modifying and querying tags in the terminal device. It can be seen from Table 1 above that the tags that can be identified also include the industry field corresponding to the data to be tagged. When automatically storing, the tags are stored in the corresponding locations according to the industry, field, etc.

[0051] The accuracy of various entity recognition technologies used in the prior art, such as NER (named entity recognition), semantic similarity, keywords and other recognition methods, is not high. The large language model (i.e., the above-mentioned target recognition model) is a very powerful entity recognition model obtained after pre-training with a large amount of sample data. Through the above-mentioned implementation method of the present application, the target recognition model can accurately identify multiple entity label sets, and further through comprehensive statistics of multiple entity label sets, the accuracy of the entity labels corresponding to the data to be labeled is further increased, and finally a target label set with higher accuracy is obtained.

[0052] As an optional implementation, the data to be labeled is input into the target recognition model, and multiple entity label sets are output, including:

[0053] S1, obtaining multiple preset prompt texts corresponding to the data to be labeled, and selecting one prompt text from the multiple preset prompt texts as the current prompt text;

[0054] S2, controlling the target recognition model to perform a first recognition on the current prompt text to obtain a reference recognition result, wherein the reference recognition result includes the target role and multiple label types included in the entity label set corresponding to the current prompt text;

[0055] S3, controlling the target recognition model to perform a second recognition on the unlabeled data according to the reference recognition result to obtain a current label set, wherein the multiple entity label sets include the current label set, and the current label set includes multiple label types corresponding to the multiple entities, and reference labels corresponding to each label type;

[0056] S4, selecting a preset prompt text from a plurality of preset prompt texts other than the current prompt text as the current prompt text.

[0057] It should be noted that large language models (such as GPT, ChatGLM, QWen, etc.) are used to automatically identify the labeled data and generate entity label sets. The core of the use of large language models (i.e., the target recognition model mentioned above) is prompt engineering: assign identity roles (data analysts, data architects, etc.) to the large model in the system role; clarify the industry field to which the data to be processed belongs; constrain the output format of the large model (such as JSON format); provide input and output examples; and provide the text data to be processed in the user role, that is, provide structured data (such as database tables) or unstructured data (such as long text, pictures, etc.).

[0058] The output of the large language model (i.e., the target recognition model mentioned above or the large model referred to below) may be unstable. Although the output format is constrained in the prompt word (such as JSON format), some non-JSON characters will still appear, and even some inference reasons and logical texts in the middle will be output. This will cause redundant output data and make it impossible to process it into JSON data later. Therefore, the output of the large language model needs to be processed (for example, using keywords and regular expressions to judge and process the results) to achieve the purpose of ultimately obtaining JSON format data.

[0059] The output of a large model may contain "hallucinations", that is, the output content seems logical, but in fact it makes no sense, so the results need to be combined with sampling data and submitted to the large model for verification, which can improve the accuracy of the results.

[0060] The prompt word text (i.e. the preset prompt text) that the large model can process has a length limit. In order to process long text scenarios, it is necessary to implement a greedy text segmentation algorithm: segment the long text into paragraphs, accumulate each paragraph one by one, and ensure that the total length does not exceed the set threshold. This can ensure semantic continuity on the one hand, and allow the large language model to process more data at one time on the other.

[0061] Finally, the tags in the target tag set corresponding to the final data to be labeled are associated with the data resources for display on the client, and enter the entity tag library on the other hand, and the target tags in the target tag set are reviewed and confirmed through the review program.

[0062] The multiple label types in S2 and S3 above include: entity domain labels, entity category labels, and entity attribute labels; S1 to S4 above can be understood, but not limited to, as follows: the granularity of each two preset prompt texts in the above multiple preset prompt texts is different, and the method for obtaining the multiple preset prompt texts can be to first obtain the initial preset prompt text as the current prompt text, and after using the target recognition model to recognize the label data according to the current prompt text to obtain the initial recognition result (that is, the entity label set corresponding to the initial preset prompt text in the above multiple entity label sets), the prompt word of the current prompt text is optimized according to the initial recognition result, and the optimized preset prompt text (which can be understood, but not limited to, as the prompt text in the multiple preset prompt texts other than the current prompt text) is input into the target recognition model as the current prompt text, and the entity label set corresponding to the current prompt text is output; the recognition is cyclically performed according to the above steps, and finally multiple entity label sets corresponding to the multiple preset prompt texts are obtained.

[0063] Through the above-mentioned implementation mode of the present application, the output results of the target recognition model are further comprehensively counted to improve the accuracy of the identified labels, thereby improving the accuracy of data management.

[0064] The following takes the target recognition model as the LLM model (large language model) as an example, combined with Figure 5 The above-mentioned data management method based on tag management is generally described, including the following contents:

[0065] S1, first obtain structured / unstructured data (i.e. the data to be labeled mentioned above);

[0066] S2, obtaining the initial preset prompt text corresponding to the data to be labeled, and using the initial preset prompt text as the current prompt text;

[0067] S3, using the LLM model to perform entity label recognition on the label data according to the current prompt text, and obtaining an entity label set corresponding to the current prompt text;

[0068] S4, store the entity tag set into the tag library and optimize the current prompt text (i.e. Figure 5 The optimized prompt words shown in the figure, the specific optimization may include adding fields, entity restrictions, etc.), and the optimized prompt text is used as the current prompt text.

[0069] The above S3 to S4 are executed in a loop. It can be understood that the number of optimizations can be preset, or the standard tag set can be preset; in the case of pre-setting the number of optimizations, the loop stops when the number of optimization prompt words is equal to the preset number of optimizations; in the case of pre-setting the standard tag set, the entity tag set obtained each time can be compared with the preset standard tag set for similarity, and the loop stops when the similarity reaches the preset similarity. After that, multiple entity tag sets can be obtained; the total number of occurrences of each entity tag in the multiple entity tag sets is counted, and the entity tags whose occurrences reach the preset number of times are determined as valid tags, and the entity tags whose occurrences do not reach the preset number of times are determined as invalid tags; then all valid tags obtained are the tags in the target tag set; and all valid tags (including industry tags, field tags, entity type tags, and entity attribute tags) are labeled correspondingly with the data to be labeled, so as to obtain the following: Figure 5The labeled data shown; the labels and labeled data can then be stored in corresponding storage locations (according to the identified industry labels, field labels, entity type labels, and entity attribute labels), and the stored labels and the data corresponding to the labels can be used for subsequent label applications, that is, the labels and data can be queried, classified, analyzed, and the like according to the labels. For example, users can use entity labels to retrieve student data in an education system, such as retrieving student data with "education-student-gender" as "male"; and users can use entity labels to count patient data in a medical system, such as counting the number of patients with "medical-patient-gender" as "female", and so on.

[0070] The above-mentioned data management method based on label management can realize high-accuracy identification of entity labels. Moreover, this application does not simply identify labels, but after identifying labels, associates the identified labels with entities and the industries and fields they describe, that is, entity-to-entity management. In this way, when the same labels appear (such as names and genders related to entities, but some are students, some are teachers, some are patients, and some are doctors), data can be managed more accurately according to labels of different entities, industries or fields. That is, this application can realize cross-domain data management. For example, even if student entities in the education system and patient entities in the medical system have the same attributes, they can be distinguished by different entity labels.

[0071] This application provides a data management method based on tag management, which realizes multi-field and multi-dimensional business data management and automatic tagging identification through entity tags and large language models. This method can be widely used in multiple fields such as education and medical care, and has significant technical effects and broad application prospects.

[0072] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0073] According to another aspect of the embodiments of the present invention, a data management device for implementing the above-mentioned tag-based management is also provided. Figure 6 As shown, the device comprises:

[0074] An acquisition unit 602 is used to acquire data to be labeled;

[0075] An input unit 604 is used to input the data to be labeled into a target recognition model and output a plurality of entity label sets, wherein the target recognition model is a model obtained by pre-training an initial recognition model with sample data;

[0076] A comprehensive statistics unit 606 is used to perform comprehensive statistics on multiple entity tag sets according to a preset evaluation threshold to obtain a target tag set;

[0077] The storage unit 608 is used to store the target tag set in a storage location corresponding to the data to be labeled.

[0078] Optionally, in this embodiment, the embodiments to be implemented by the above-mentioned various unit modules can refer to the above-mentioned various method embodiments, which will not be repeated here.

[0079] According to another aspect of the embodiments of the present invention, there is also provided an electronic device for implementing the above-mentioned data management method based on tag management. The electronic device may be Figure 7 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Figure 7 As shown, the electronic device includes a memory 702 and a processor 704. The memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments through the computer program.

[0080] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0081] Optionally, in this embodiment, the processor may be configured to execute the data management method based on tag management through a computer program.

[0082] Alternatively, a person skilled in the art may understand that: Figure 7 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 7 The structure of the electronic device is not limited. Figure 7 More or fewer components (such as network interfaces, etc.) as shown, or with Figure 7 Different configurations shown.

[0083] Among them, the memory 702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data management method and device based on tag management in the embodiments of the present invention. The processor 704 executes various functional applications and data processing by running the software programs and modules stored in the memory 702, that is, to implement the above-mentioned data management method based on tag management. The memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include a memory remotely located relative to the processor 704, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 702 can be specifically, but not limited to, used to store information such as a target tag set. As an example, such as Figure 7 As shown, the memory 702 may include, but is not limited to, the acquisition unit 602, input unit 604, comprehensive statistics unit 606, and storage unit 608 in the data management device based on tag management. In addition, it may also include, but is not limited to, other module units in the data management device based on tag management, which will not be repeated in this example.

[0084] Optionally, the transmission device 706 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 706 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 706 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0085] In addition, the electronic device further includes: a display 708, and a connection bus 710, which is used to connect various module components in the electronic device.

[0086] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P, Peer To Peer) network, and any form of computing device, such as a server, terminal and other electronic devices, may become a node in the blockchain system by joining the peer-to-peer network.

[0087] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program / instruction, the computer program / instruction comprising a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0088] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0089] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned data management method based on tag management.

[0090] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for executing the data management method based on tag management.

[0091] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0092] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the above methods of various embodiments of the present invention.

[0093] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0095] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0097] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data management method based on tag management, characterized in that: include: Get the data to be labeled; Acquire multiple preset prompt texts corresponding to the data to be labeled, and select one prompt text from the multiple preset prompt texts as the current prompt text, each of the preset prompt texts including the identity role set for the target recognition model, the operation to be performed by the target recognition model, the restriction requirements on the labels to be recognized, and the restriction requirements on the output results of the target recognition model; Controlling a target recognition model to perform a first recognition on the current prompt text to obtain a reference recognition result, wherein the target recognition model is a model obtained by pre-training an initial recognition model with sample data, and the reference recognition result includes a target role and multiple label types included in an entity label set corresponding to the current prompt text; Control the target recognition model to perform a second recognition on the data to be labeled according to the reference recognition result to obtain a current label set, wherein the multiple entity label sets include the current label set, and the current label set includes multiple label types corresponding to multiple entities, and reference labels corresponding to each label type; Selecting a preset prompt text from a plurality of preset prompt texts other than the current prompt text as the current prompt text; Performing comprehensive statistics on the plurality of entity tag sets according to a preset evaluation threshold to obtain a target tag set; The target tag set is stored in a storage location corresponding to the data to be labeled.

2. The method according to claim 1, characterized in that The multiple tag types include: entity domain tags, entity category tags, and entity attribute tags.

3. The method according to claim 1, characterized in that The initial recognition model is trained using sample data, including: Acquire a sample data set and a sample label set, and set multiple reference prompt texts for multiple pieces of the sample data, respectively, wherein each sample data set corresponds to an industry field and multiple reference prompt texts; Iteratively training the initial recognition model according to the sample data set and the plurality of reference prompt texts to obtain a training result; Performing a similarity comparison between the training result and the sample label set to obtain a comparison result; When the comparison result is greater than or equal to a preset threshold, the recognition model corresponding to the training result is determined as the target recognition model.

4. The method according to claim 1, characterized in that Comprehensive statistics are performed on the plurality of entity tag sets according to a preset evaluation threshold to obtain a target tag set, including: Obtaining the number of texts of the plurality of preset prompt texts; Determine the quantity of each entity tag according to the plurality of entity tag sets to obtain a plurality of tag quantities, wherein each tag quantity is less than or equal to the text quantity; A plurality of target tag quantities greater than or equal to a preset quantity are screened from the plurality of tag quantities, and entity tags respectively corresponding to the plurality of target tag quantities are determined as target tags to obtain the target tag set.

5. The method according to claim 1, characterized in that Get the data to be labeled, including: Obtain reference data and the data type corresponding to the reference data; The reference data is preprocessed using a preprocessing method corresponding to the data type to obtain the data to be labeled.

6. The method according to claim 5, characterized in that Preprocessing the reference data using a preprocessing method corresponding to the data type to obtain the data to be labeled, including: In the case where the data type is a structured data type, obtaining a data structure and data content of the reference data, and adding remark information to the reference data according to the data content and data structure; Extracting the reference data after adding the annotation information to obtain the data to be labeled; In the case where the data type is an unstructured data type, obtaining a file type corresponding to the reference data; The data to be labeled is extracted from the reference data according to a data extraction method corresponding to the file type.

7. A data management device based on tag management, characterized in that: include: An acquisition unit, used for acquiring data to be labeled; An input unit is used to obtain a plurality of preset prompt texts corresponding to the data to be labeled, and select a prompt text from the plurality of preset prompt texts as the current prompt text, each of which includes an identity role set for a target recognition model, an operation to be performed by the target recognition model, restriction requirements on labels to be recognized, and restriction requirements on output results of the target recognition model; control the target recognition model to perform a first recognition on the current prompt text to obtain a reference recognition result, wherein the target recognition model is a model obtained by pre-training an initial recognition model with sample data, and the reference recognition result includes a target role, and a plurality of label types included in an entity label set corresponding to the current prompt text; control the target recognition model to perform a second recognition on the data to be labeled according to the reference recognition result to obtain a current label set, wherein the plurality of entity label sets include the current label set, and the current label set includes a plurality of label types corresponding to a plurality of entities, respectively, and a reference label corresponding to each label type; select a preset prompt text from the plurality of preset prompt texts other than the current prompt text as the current prompt text; A comprehensive statistics unit, used for performing comprehensive statistics on the plurality of entity tag sets according to a preset evaluation threshold to obtain a target tag set; A storage unit is used to store the target tag set in a storage location corresponding to the data to be labeled.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

Images