Label management method and device and electronic equipment
By automatically obtaining and executing label management requirements, including label calculation strategies and cycles, the problems of high cost and poor efficiency of label management in the existing technology are solved, and efficient and automated label calculation and index update are achieved.
Patent Information
- Application Number
- CN202411998652.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-02
AI Technical Summary
In the prior art, the tag management cost of data asset entities is high and has poor efficiency, and developers need to manually create and calculate tags.
By obtaining tag management requirements, including the tag to be calculated, tag calculation strategy and calculation cycle, the tag calculation task is automatically determined and the thread is called to execute, the tag calculation result is obtained, and when the tag index update event is triggered, the calculation result is selected for index update.
It reduces the cost of tag management, improves the efficiency of tag management, and realizes automated tag calculation and index update processing.
Smart Images
Figure CN119918795A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the technical fields of big data, intelligent search, etc., and in particular to a tag management method, device and electronic device. Background Art
[0002] Currently, for data asset entities, label developers are required to create labels and determine label values through statistics and analysis. Label management is costly and inefficient. Summary of the invention
[0003] The present disclosure provides a tag management method, device and electronic device.
[0004] According to one aspect of the present disclosure, a tag management method is provided, the method comprising: obtaining tag management requirements; the tag management requirements comprising: tags to be calculated of data asset entities, tag calculation strategies and tag calculation cycles corresponding to the tags; determining tag calculation tasks corresponding to the tags according to the tag calculation strategies and tag calculation cycles corresponding to the tags, and calling a tag calculation thread to execute the tag calculation tasks to obtain tag calculation results; storing the tag calculation results in a shared database; in the case of triggering a tag index update event, selecting a first tag calculation result from the tag calculation results in the shared database; determining a tag index update task according to the first tag calculation result, and calling a tag index update thread to execute the tag index update task to obtain a tag index update result.
[0005] According to another aspect of the present disclosure, a label management device is provided, the device comprising: a first acquisition module, used to acquire label management requirements; the label management requirements include: labels to be calculated of data asset entities, label calculation strategies and label calculation cycles corresponding to the labels; a first determination module, used to determine label calculation tasks corresponding to the labels according to the label calculation strategies and label calculation cycles corresponding to the labels, and to call label calculation threads to execute the label calculation tasks to obtain label calculation results; a first storage module, used to store the label calculation results in a shared database; a selection module, used to select a first label calculation result from the label calculation results in the shared database when a label index update event is triggered; a second determination module, used to determine label index update tasks according to the first label calculation results, and to call label index update threads to execute the label index update tasks to obtain label index update results.
[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the tag management method proposed above in the present disclosure.
[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the tag management method proposed above in the present disclosure.
[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the tag management method proposed above in the present disclosure are implemented.
[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0011] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;
[0012] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;
[0013] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;
[0014] Figure 4 It is a schematic diagram of label calculation in label management;
[0015] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;
[0016] Figure 6 It is a block diagram of an electronic device used to implement the tag management method of the embodiment of the present disclosure. DETAILED DESCRIPTION
[0017] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0018] Currently, for data asset entities, label developers are required to create labels and determine label values through statistics and analysis. Label management is costly and inefficient.
[0019] In view of the above problems, the present disclosure proposes a tag management method, device and electronic device.
[0020] Figure 1 It is a schematic diagram according to the first embodiment of the present disclosure. It should be noted that the tag management method of the embodiment of the present disclosure can be applied to a tag management device, which can be configured in an electronic device so that the electronic device can perform a tag management function.
[0021] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a smart speaker, a server, a server cluster, a tag management system, and other hardware devices with various operating systems, touch screens and / or display screens.
[0022] The tag management device may also be software in an electronic device, such as tag management software, etc. In the following embodiments, the execution subject is taken as an example of a backend platform in a tag management system.
[0023] like Figure 1 As shown, the tag management method may include the following steps:
[0024] Step 101, obtaining tag management requirements; the tag management requirements include: tags to be calculated for the data asset entity, tag calculation strategies corresponding to the tags, and tag calculation cycles.
[0025] In an embodiment of the present disclosure, the backend platform may receive tag management requirements from a frontend application in a tag management system. The frontend application may obtain the tag management requirements by interacting with an object. The tag management requirements obtained by the frontend application may include multiple tags to be calculated for a data asset entity, tag calculation strategies corresponding to the tags, and tag calculation cycles. The frontend application may split the tag management requirements according to the tags to obtain multiple split tag management requirements. Each split tag management requirement includes a tag calculation strategy and a tag calculation cycle corresponding to a tag.
[0026] It should be noted that the tag calculation strategy corresponding to the tag is used to indicate the calculation logic of the tag. In combination with the calculation logic, the tag can be calculated and processed. The tag calculation cycle is used to indicate the interval between two tag calculation processes.
[0027] In the embodiment of the present disclosure, the data asset entity may be a data resource, which is represented by a basic attribute table and a behavior attribute table of the data asset. The basic attribute table includes various basic attributes of the data asset. The behavior attribute table includes various behavior records of the data asset.
[0028] In the disclosed embodiment, a cluster may be provided in the backend platform. A plurality of nodes may be provided in the cluster. Specifically, in step 101, the frontend application may store a plurality of split tag management requirements in a relational database, and each node in the backend platform may poll the relational database to obtain the split tag management requirements stored in the relational database. That is, the node may send a tag management requirement acquisition request to the relational database, and receive a response returned by the relational database. In the case where the split tag management requirement is carried in the response, it is determined that the split tag management requirement is obtained.
[0029] The cluster may be, for example, a quartz cluster. Quartz clustering refers to configuring multiple Quartz scheduling instances to work together to achieve high availability and load balancing of task scheduling. The nodes in the quartz cluster may be quartz nodes. Multiple nodes in the quartz cluster are equal and can obtain label management requirements through polling.
[0030] The number of labels to be calculated in the label management requirement in step 101 is one, and the label management requirement is the above-split label management requirement obtained by the node through polling.
[0031] In the embodiments of the present disclosure, it is also necessary to explain that, as an alternative solution, the tag management requirements may include tags to be calculated for the data asset entity and tag calculation strategies corresponding to the tags, indicating that a tag calculation process needs to be performed in combination with the tag calculation strategy.
[0032] Step 102: determine the tag calculation task corresponding to the tag according to the tag calculation strategy and the tag calculation period corresponding to the tag, and call the tag calculation thread to execute the tag calculation task to obtain the tag calculation result.
[0033] In the embodiment of the present disclosure, the number of tag calculation tasks can be multiple. The process of the back-end platform executing step 102 can be, for example, determining multiple tag calculation tasks corresponding to the tag according to the tag calculation strategy and the tag calculation cycle corresponding to the tag; storing the multiple tag calculation tasks in the relational database; determining at least one tag calculation time point according to the tag calculation cycle; when the tag calculation time point is reached and there is no tag calculation task being executed, obtaining the first tag calculation task among the multiple tag calculation tasks in the relational database according to the tag calculation time point; matching the execution time point of the first tag calculation task with the tag calculation time point; calling the tag calculation thread to execute the first tag calculation task to obtain the tag calculation result.
[0034] Among them, at least one label calculation scheduler may be set in the node in the backend platform. The label calculation scheduler may determine multiple label calculation tasks corresponding to the label according to the label calculation strategy and label calculation cycle corresponding to the label, and store them in the relational database; when the label calculation time point is reached and there is no label calculation task being executed, the first label calculation task is obtained, and the first label calculation task is scheduled to a label calculation thread, so that the label calculation thread can execute the first label calculation task.
[0035] Among them, the label calculation thread can determine the data used for label calculation of the data asset entity based on the label calculation strategy; then construct a query statement, call the query statement to query the database that stores the basic attribute table and behavior attribute table of the data asset entity, and obtain the data used for label calculation; then combine the data and the label calculation strategy to perform label calculation processing.
[0036] Among them, multiple label calculation tasks are set with execution time points; the time interval between the execution time points of two adjacent label calculation tasks is the length of the label calculation cycle.
[0037] The execution time point of the first label calculation task matches the label calculation time point. When the execution time of the previous label calculation task is too long, the first label calculation task whose corresponding execution time point matches the label calculation time point is selected for execution, which can avoid executing the unexecuted label calculation tasks between the first label calculation tasks and avoid executing the expired label calculation tasks at the label calculation time point, thereby ensuring the real-time nature of the label calculation results and improving the label calculation efficiency.
[0038] Step 103: Store the tag calculation results in a shared database.
[0039] In the embodiment of the present disclosure, the process of the back-end platform executing step 103 may, for example, be to determine the number of tag calculation tasks that have been executed in the tag calculation tasks corresponding to the tag; determine the identification of the tag calculation result according to the data asset entity, the tag and the number; store the tag calculation result and the identification of the tag calculation result in a shared database. The tag calculation result, for example, is a specific value of the tag. The shared database may be a database shared by each node in the back-end platform. The relational database may be a database dedicated to each node in the back-end platform. The number of relational databases may be one, and each node may use a part of the area therein.
[0040] Among them, the identifier of the label calculation result can be obtained by concatenating the name, label and number of the data asset entity, so that combined with the identifier of the label calculation result, it can be determined for which data asset entity the label calculation result is for, thereby facilitating the subsequent label index update process and improving the accuracy of subsequent label index updates.
[0041] In the disclosed embodiment, in order to facilitate the management of the label calculation results and improve the efficiency of label management, the management status of the label calculation results can be set. Correspondingly, the backend platform can also perform the following process: obtain the first management status of the label calculation task and the second management status of the label index update task corresponding to the label calculation task; store the first management status, the second management status and the identification of the label calculation result in the relational database.
[0042] The first management state of the tag calculation task is, for example, in progress, to be triggered, execution failed, execution succeeded, etc. The second management state of the tag index update task is, for example, in progress, to be triggered, execution failed, execution succeeded, etc.
[0043] Step 104: When a tag index update event is triggered, a first tag calculation result is selected from the tag calculation results in the shared database.
[0044] In an embodiment of the present disclosure, the backend platform can select a first label calculation result from the label calculation results in the shared database according to the label calculation time of the label calculation result; or, the backend platform can arbitrarily select a label calculation result from the label calculation results in the shared database as the first label calculation result.
[0045] In the embodiment of the present disclosure, before step 104, the back-end platform may also perform the following process: obtaining the label index update cycle; determining at least one label index update time point according to the label index update cycle; determining that a label index update event is triggered when the label index update time point is reached; and determining that a label index update event is not triggered when the label index update time point is not reached.
[0046] Among them, at least one label index update scheduler may be set in the node in the backend platform. The label index update scheduler may determine the time point of triggering the label index update event according to the label index update cycle, and then perform label index event triggering processing.
[0047] In one example, the label index update period can be carried in the label management requirements. In another example, the label index update period can be pre-set in the label index update scheduler of each node of the backend platform. In another example, the label index update period can be provided to the backend platform by the front-end application through other messages.
[0048] Among them, the automatic triggering of the label index update event enables the backend platform to automatically perform label index update processing, thereby improving the accuracy and efficiency of the label index update processing.
[0049] Step 105 , determining a label index update task according to the first label calculation result, and calling a label index update thread to execute the label index update task to obtain a label index update result.
[0050] In the embodiment of the present disclosure, the tag index update result may indicate at least one of the following: update success, update failure, etc.
[0051] The tag management method of the embodiment of the present disclosure obtains tag management requirements; the tag management requirements include: tags to be calculated of data asset entities, tag calculation strategies and tag calculation cycles corresponding to the tags; determining tag calculation tasks corresponding to the tags according to the tag calculation strategies and tag calculation cycles corresponding to the tags, and calling tag calculation threads to execute the tag calculation tasks to obtain tag calculation results; storing the tag calculation results in a shared database; in the case of triggering a tag index update event, selecting a first tag calculation result from the tag calculation results in the shared database; determining a tag index update task according to the first tag calculation result, and calling a tag index update thread to execute the tag index update task to obtain a tag index update result; wherein, tag calculation processing and tag index update processing can be performed in combination with the tag calculation strategies and tag calculation cycles corresponding to the tags in the tag management requirements, thereby reducing tag management costs and improving tag management efficiency.
[0052] In order to further improve the accuracy of label index updates and the efficiency of label index updates, the backend platform can combine the first management state and the second management state corresponding to each label calculation result in the shared database, and select the first label calculation result with the latest label calculation event that has completed label calculation processing and has not been updated from each label calculation result for label index update processing. Figure 2 As shown, Figure 2 is a schematic diagram according to a second embodiment of the present disclosure, Figure 2 The illustrated embodiment may include the following steps:
[0053] Step 201, obtaining tag management requirements; the tag management requirements include: tags to be calculated for the data asset entity, tag calculation strategies corresponding to the tags, and tag calculation cycles.
[0054] Step 202: determine the tag calculation task corresponding to the tag according to the tag calculation strategy and the tag calculation period corresponding to the tag, and call the tag calculation thread to execute the tag calculation task to obtain the tag calculation result.
[0055] Step 203, store the label calculation result and the identification of the label calculation result in a shared database; and store the identification of the label calculation result, the first management state of the label calculation task corresponding to the label calculation result and the second management state of the label index update task in a relational database.
[0056] In the embodiment of the present disclosure, the shared database may be, for example, a large file shared storage system (Large File Shared Storage System). The relational database may be, for example, a MySQL relational database, etc.
[0057] Step 204, when a tag index update event is triggered, for each tag calculation result in the shared database, query the relational database according to the identifier of the tag calculation result to obtain the first management state and the second management state corresponding to the tag calculation result.
[0058] In the disclosed embodiment, the first management state may be the first management state of the tag calculation task corresponding to the tag calculation result, and the second management state may be the second management state of the tag index update task corresponding to the tag calculation task.
[0059] The first management state of the tag calculation task is, for example, in progress, to be triggered, execution failed, execution succeeded, etc. The second management state of the tag index update task is, for example, in progress, to be triggered, execution failed, execution succeeded, etc.
[0060] Step 205 : selecting at least one candidate tag calculation result to be subjected to index update processing from each tag calculation result according to the first management state and the second management state corresponding to each tag calculation result in the shared database.
[0061] In the embodiment of the present disclosure, the process of the back-end platform executing step 205 may, for example, be to determine, for each tag calculation result in the shared database, the tag calculation result as a candidate tag calculation result when the first management status corresponding to the tag calculation result indicates that the tag calculation is completed and the second management status corresponding to the tag calculation result indicates that a tag index update is to be triggered.
[0062] Among them, each tag calculation result used for selecting the candidate tag calculation result here can correspond to the same data asset entity and the same tag. The various tag calculation results can be determined from the shared database in combination with the identifier of the tag calculation result. Among them, the identifier of the tag calculation result can be determined in combination with the data asset entity and the tag. Therefore, the data asset identifier and the tag can be uniquely determined based on the identifier.
[0063] There may be multiple candidate tag calculation results. The first management state and the second management state corresponding to the multiple candidate tag calculation results are the same, and the multiple candidate tag calculation results are the same tag for the same data asset entity. The difference is that the tag calculation time of the multiple candidate tag calculation results is different.
[0064] Among them, when the first management status corresponding to the label calculation result indicates that the label calculation is completed, and the second management status corresponding to the label calculation result indicates that the label index update is to be triggered, the label calculation result is determined as a candidate label calculation result, and the label calculation result that can be used for label index update processing can be accurately selected, thereby further improving the accuracy of the label index update.
[0065] Specifically, the label index update scheduler set in the node in the backend platform can generate a scheduling task when the label index update event is triggered; and call a thread to execute the scheduling task to obtain the execution result of the scheduling task, that is, the first label calculation result obtained by selecting. Here, the scheduling task can execute steps 205 and 206 to obtain the first label calculation result.
[0066] Among them, the scheduling task can determine the label index update task based on the selected first label calculation result; provide the label index update task to another label index update scheduler, so that the label index update scheduler can call a label index update thread to execute the label index update task.
[0067] Step 206: Select a candidate label calculation result with the latest label calculation time from the at least one candidate label calculation result as the first label calculation result.
[0068] Step 207: determine a label index update task according to the first label calculation result, and call a label index update thread to execute the label index update task to obtain a label index update result.
[0069] In the embodiment of the present disclosure, the process of the back-end platform determining the update result of the tag index can be, for example, locking the main pointer and the backup pointer in the shared database; calling the tag index update thread to execute the tag index update task, and writing the association relationship between the first tag calculation result and the tag of the data asset entity in the address pointed to by the backup pointer; performing a master-slave exchange process on the main pointer and the backup pointer to obtain a new main pointer and a new backup pointer; writing the association relationship between the first tag calculation result and the tag of the data asset entity in the address pointed to by the new backup pointer; and performing a lock release process on the main pointer and the backup pointer.
[0070] The backup pointer is used to point to the backup index, and the primary pointer is used to point to the primary index.
[0071] Among them, the lock processing of the shared database and the switching processing of the master and standby pointers make it possible to update the index of only one label on the shared database at the same time, thereby avoiding the confusion caused by concurrent updates of multiple labels, thereby further improving the efficiency of label index updates.
[0072] In the embodiment of the present disclosure, in order to avoid the label index update processing of other candidate label calculation results except the first label calculation result in at least one candidate label calculation result and further improve the label index update processing efficiency, after step 207, the back-end platform can also perform the following process: obtain other candidate label calculation results except the first label calculation result in at least one candidate label calculation result; determine the label index update result as the label index update result after the label index update processing is performed on other candidate label calculation results.
[0073] It should be noted that the details of steps 201 to 203 and step 207 can be found in Figure 1 Steps 101 to 103 and step 105 in the illustrated embodiment will not be described in detail herein.
[0074] The tag management method of the embodiment of the present disclosure obtains tag management requirements; the tag management requirements include: tags to be calculated of data asset entities, tag calculation strategies and tag calculation cycles corresponding to the tags; determining tag calculation tasks corresponding to the tags according to the tag calculation strategies and tag calculation cycles corresponding to the tags, and calling tag calculation threads to execute the tag calculation tasks to obtain tag calculation results; storing the tag calculation results and the identifiers of the tag calculation results in a shared database; and storing the identifiers of the tag calculation results, the first management state of the tag calculation tasks corresponding to the tag calculation results, and the second management state of the tag index update tasks in a relational database; in the case of triggering a tag index update event, querying the relational database according to the identifiers of the tag calculation results for each tag calculation result in the shared database to obtain the first management state and the second management state corresponding to the tag calculation results; According to the first management state and the second management state corresponding to each label calculation result in the shared database, at least one candidate label calculation result to be index updated is selected from each label calculation result; a candidate label calculation result with the latest label calculation time is selected from at least one candidate label calculation result as the first label calculation result; a label index update task is determined according to the first label calculation result, and a label index update thread is called to execute the label index update task to obtain a label index update result; wherein, in combination with the first management state and the second management state corresponding to each label calculation result in the shared database, the first label calculation result with the latest label calculation event in which the label calculation processing is completed and the label index update is not performed is selected from each label calculation result for label index update processing, which can further improve the accuracy of label index update and further improve the efficiency of label index update.
[0075] In order to avoid long-term lag during the execution of the tag index update task and ensure the efficiency of tag management, the backend platform can interrupt the tag index update task that is in execution and in the process of execution. Figure 3 As shown, Figure 3 is a schematic diagram according to a third embodiment of the present disclosure, Figure 3 The illustrated embodiment may include the following steps:
[0076] Step 301, obtaining tag management requirements; the tag management requirements include: tags to be calculated for the data asset entity, tag calculation strategies corresponding to the tags, and tag calculation cycles.
[0077] Step 302: determine the tag calculation task corresponding to the tag according to the tag calculation strategy and the tag calculation period corresponding to the tag, and call the tag calculation thread to execute the tag calculation task to obtain the tag calculation result.
[0078] Step 303: store the tag calculation results in a shared database.
[0079] Step 304: When a tag index update event is triggered, a first tag calculation result is selected from the tag calculation results in the shared database.
[0080] Step 305 , determining a label index update task according to the first label calculation result, and calling a label index update thread to execute the label index update task to obtain a label index update result.
[0081] Step 306, obtaining a timeout processing period.
[0082] Step 307: Determine at least one timeout processing time point according to the timeout processing cycle.
[0083] In an embodiment of the present disclosure, in one example, the timeout processing period can be carried in the tag management requirement. In another example, the timeout processing period can be provided by the front-end application to the back-end platform through other messages.
[0084] Step 308, when the timeout processing time point is reached, obtain the first label index update task; the second management state of the first label index update task is in execution, and the time interval between the start update time point of the first label index update task and the current time point is greater than the time threshold.
[0085] In an embodiment of the present disclosure, at least one timeout processing scheduler may be provided in a node in the backend platform. The timeout processing scheduler may trigger a timeout processing task according to at least one timeout processing time point. Specifically, the timeout processing scheduler may obtain the first label index update task when the timeout processing time point is reached, and schedule a timeout processing thread to interrupt the execution process of the first label index update task.
[0086] Among them, the second management state is in execution, and the time interval between the starting update time point and the current time point of the first label index update task is greater than the duration threshold, indicating that the first label index update task is in the execution process for a long time, then the first label index update task is in a stuck state. In order to ensure that the next label index update task can be executed in time, the timeout processing scheduler can schedule a timeout processing thread to interrupt the first label index update task.
[0087] Step 309: interrupt the first tag index update task, and release the lock on the shared database during the execution of the first tag index update task.
[0088] In the embodiment of the present disclosure, during the execution of the first tag index update task, the shared database needs to be locked to avoid confusion caused by concurrent updates of multiple tags. Correspondingly, after the first tag index update task is interrupted, the shared database can be released.
[0089] It should be noted that steps 306 to 309 can be executed before or after any one of steps 301 to 305, that is, the execution of steps 306 to 309 is not associated with the execution of steps 301 to 305. The execution order can be adjusted according to actual needs.
[0090] It should be noted that the details of steps 301 to 305 can be found in Figure 1 Steps 101 to 105 in the illustrated embodiment will not be described in detail herein.
[0091] The tag management method of the embodiment of the present disclosure obtains tag management requirements; the tag management requirements include: tags to be calculated of data asset entities, tag calculation strategies and tag calculation cycles corresponding to the tags; the tag calculation tasks corresponding to the tags are determined according to the tag calculation strategies and tag calculation cycles corresponding to the tags, and the tag calculation threads are called to execute the tag calculation tasks to obtain tag calculation results; the tag calculation results are stored in a shared database; in the case of triggering a tag index update event, a first tag calculation result is selected from the tag calculation results in the shared database; the tag index update task is determined according to the first tag calculation result, and the tag index update thread is called to execute the tag index update task to obtain the tag index update event. Result; obtain a timeout processing cycle; determine at least one timeout processing time point according to the timeout processing cycle; when the timeout processing time point is reached, obtain the first label index update task; the second management state of the first label index update task is in execution, and the time interval between the starting update time point of the first label index update task and the current time point is greater than the time threshold; interrupt the first label index update task, and release the lock processing for the shared database during the execution of the first label index update task; wherein, interrupting the label index update task that is in execution and in the execution time process can avoid long-term jamming during the execution of the label index update task and ensure label management efficiency.
[0092] The following examples are used to illustrate this. Figure 4 The figure below is a schematic diagram of tag calculation in tag management. Figure 4 The following steps may be included.
[0093] Step 401, the damp-service pod (front-end application) receives the scheduled scheduling task published by the object, or the scheduling task manually executed by the object (tag management requirement).
[0094] Step 402, the damp-service pod forwards the task to the damp-back-ground pod (the Quartz service node, i.e., the node in the back-end platform).
[0095] In step 403, the quartz scheduler (label calculation scheduler) in the damp-background pod determines the scheduled task (label calculation task) based on the jobDetall (entity single label calculation, i.e., label calculation strategy) and Tigger (timing scheduling / immediate execution, determined in combination with the label calculation cycle) in the task, and sends the scheduled task to the single label calculation job (label calculation thread).
[0096] Step 404, the calculation is completed, and the entity tag index update task metadata (the first management state of the tag calculation task and the second management state of the tag index update task) is stored in the MYSQL database.
[0097] Step 405, the quartz scheduler in the damp-back-ground pod returns the task registration & start scheduling result (which damp-back-ground pod will execute the task) to the damp-service pod.
[0098] Step 406, damp-service pod interacts with MYSQL to process and store tag update business data.
[0099] In order to implement the above embodiment, the present disclosure also provides a label management device. Figure 5 As shown, Figure 5 The tag management device 50 may include: a first acquisition module 501 , a first determination module 502 , a first storage module 503 , a selection module 504 and a second determination module 505 .
[0100] Among them, the first acquisition module 501 is used to obtain the tag management requirements; the tag management requirements include: the tags to be calculated of the data asset entity, the tag calculation strategy corresponding to the tags, and the tag calculation cycle; the first determination module 502 is used to determine the tag calculation task corresponding to the tag according to the tag calculation strategy and the tag calculation cycle corresponding to the tag, and call the tag calculation thread to execute the tag calculation task to obtain the tag calculation result; the first storage module 503 is used to store the tag calculation result in a shared database; the selection module 504 is used to select the first tag calculation result from the tag calculation results in the shared database when a tag index update event is triggered; the second determination module 505 is used to determine the tag index update task according to the first tag calculation result, and call the tag index update thread to execute the tag index update task to obtain the tag index update result.
[0101] As a possible implementation of an embodiment of the present disclosure, the number of the label calculation tasks is multiple; the first determination module 502 is specifically used to determine the multiple label calculation tasks corresponding to the label according to the label calculation strategy and label calculation cycle corresponding to the label; store the multiple label calculation tasks in a relational database; determine at least one label calculation time point according to the label calculation cycle; when the label calculation time point is reached and there is no label calculation task being executed, obtain the first label calculation task among the multiple label calculation tasks in the relational database according to the label calculation time point; the execution time point of the first label calculation task matches the label calculation time point; call the label calculation thread to execute the first label calculation task to obtain a label calculation result.
[0102] As a possible implementation method of the embodiment of the present disclosure, the first storage module 503 is specifically used to determine the number of tag calculation tasks that have been executed among the tag calculation tasks corresponding to the tag; determine the identifier of the tag calculation result according to the data asset entity, the tag and the number; and store the tag calculation result and the identifier of the tag calculation result in the shared database.
[0103] As a possible implementation method of the embodiment of the present disclosure, the device also includes: a second acquisition module and a second storage module; the second acquisition module is used to obtain the first management state of the label calculation task and the second management state of the label index update task corresponding to the label calculation task; the second storage module is used to store the first management state, the second management state and the identification of the label calculation result in a relational database.
[0104] As a possible implementation method of the embodiment of the present disclosure, the selection module 504 is specifically used to, for each label calculation result in the shared database, query the relational database according to the identifier of the label calculation result to obtain the first management status and the second management status corresponding to the label calculation result; according to the first management status and the second management status corresponding to each label calculation result in the shared database, select at least one candidate label calculation result to be index updated from each label calculation result; and select a candidate label calculation result with the latest label calculation time from the at least one candidate label calculation result as the first label calculation result.
[0105] As a possible implementation method of an embodiment of the present disclosure, the selection module 504 is specifically used to, for each label calculation result in the shared database, determine the label calculation result as the candidate label calculation result when the first management status corresponding to the label calculation result indicates that the label calculation is completed and the second management status corresponding to the label calculation result indicates that a label index update is to be triggered.
[0106] As a possible implementation method of the embodiment of the present disclosure, the device also includes: a third acquisition module and a third determination module; the third acquisition module is used to obtain other candidate label calculation results other than the first label calculation result in the at least one candidate label calculation result; the third determination module is used to determine the label index update result as the label index update result after label index update processing is performed on the other candidate label calculation results.
[0107] As a possible implementation of the embodiment of the present disclosure, the device also includes: a fourth acquisition module, a fourth determination module and a fifth determination module; the fourth acquisition module is used to acquire the label index update cycle; the fourth determination module is used to determine at least one label index update time point according to the label index update cycle; the fifth determination module is used to determine the triggering of the label index update event when the label index update time point is reached; the fifth determination module is also used to determine that the label index update event is not triggered when the label index update time point is not reached.
[0108] As a possible implementation method of the embodiment of the present disclosure, the second determination module 505 is specifically used to lock the main pointer and the backup pointer in the shared database; call the label index update thread to execute the label index update task, and write the association relationship between the first label calculation result and the label of the data asset entity in the address pointed to by the backup pointer; perform a master-slave exchange process on the main pointer and the backup pointer to obtain a new main pointer and a new backup pointer; write the association relationship between the first label calculation result and the label of the data asset entity in the address pointed to by the new backup pointer; and perform a lock release process on the main pointer and the backup pointer.
[0109] As a possible implementation of the embodiment of the present disclosure, the device also includes: a fifth acquisition module, a sixth determination module, a seventh acquisition module and a processing module; the fifth acquisition module is used to obtain a timeout processing period; the sixth determination module is used to determine at least one timeout processing time point according to the timeout processing period; the seventh acquisition module is used to obtain a first label index update task when the timeout processing time point is reached; the second management state of the first label index update task is in execution, and the time interval between the starting update time point and the current time point of the first label index update task is greater than a time threshold; the processing module is used to interrupt the first label index update task, and release the lock processing for the shared database during the execution of the first label index update task.
[0110] The tag management device of the embodiment of the present disclosure obtains tag management requirements; the tag management requirements include: the tags to be calculated of the data asset entity, the tag calculation strategy corresponding to the tags, and the tag calculation cycle; determines the tag calculation task corresponding to the tag according to the tag calculation strategy and the tag calculation cycle corresponding to the tag, and calls the tag calculation thread to execute the tag calculation task to obtain the tag calculation result; stores the tag calculation result in a shared database; when a tag index update event is triggered, selects the first tag calculation result from the tag calculation results in the shared database; determines the tag index update task according to the first tag calculation result, and calls the tag index update thread to execute the tag index update task to obtain the tag index update result; wherein, the tag calculation processing and the tag index update processing can be performed in combination with the tag calculation strategy and the tag calculation cycle corresponding to the tags in the tag management requirements, thereby reducing the tag management cost and improving the tag management efficiency.
[0111] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, comply with the relevant laws and regulations, and do not violate public order and good morals.
[0112] According to an embodiment of the present disclosure, the present disclosure also provides a tag management system, the system includes: a front-end application and a back-end platform; the front-end application is communicatively connected with the back-end platform; a plurality of nodes are arranged in the candidate platform; the front-end application provides tag management requirements to the back-end platform; the tag management requirements include: tags to be calculated of the data asset entity, the tag calculation strategy corresponding to the tags, and the tag calculation cycle; each node in the back-end platform executes the tag management method as described above based on the tag calculation strategy and the tag calculation cycle corresponding to one of the tags of the data asset entity in the tag management requirements.
[0113] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0114] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0115] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0116] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0117] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the tag management method. For example, in some embodiments, the tag management method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the tag management method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the tag management method in any other appropriate manner (e.g., by means of firmware).
[0118] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0119] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0120] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0121] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0122] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0123] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0124] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0125] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A tag management method, the method comprising: Obtain label management requirements; The tag management requirements include: tags to be calculated for data asset entities, tag calculation strategies corresponding to the tags, and tag calculation cycles; Determine the label calculation task corresponding to the label according to the label calculation strategy and label calculation period corresponding to the label, and call the label calculation thread to execute the label calculation task to obtain the label calculation result; Storing the tag calculation results in a shared database; In the case of triggering a label index update event, selecting a first label calculation result from the label calculation results in the shared database; A label index update task is determined according to the first label calculation result, and a label index update thread is called to execute the label index update task to obtain a label index update result.
2. The method according to claim 1, wherein: The number of the label calculation tasks is multiple; the label calculation tasks corresponding to the labels are determined according to the label calculation strategies and label calculation cycles corresponding to the labels, and the label calculation threads are called to execute the label calculation tasks to obtain label calculation results, including: Determine a plurality of label calculation tasks corresponding to the label according to a label calculation strategy and a label calculation period corresponding to the label; Storing the plurality of tag computing tasks in a relational database; Determining at least one tag calculation time point according to the tag calculation cycle; When the label calculation time point is reached and there is no label calculation task being executed, a first label calculation task among the multiple label calculation tasks in the relational database is acquired according to the label calculation time point; the execution time point of the first label calculation task matches the label calculation time point; The label calculation thread is called to execute the first label calculation task to obtain a label calculation result.
3. The method according to claim 1, wherein: The step of storing the tag calculation result in a shared database includes: Determine the number of tag calculation tasks that have been executed among the tag calculation tasks corresponding to the tag; Determining an identifier of the tag calculation result according to the data asset entity, the tag, and the number; The tag calculation result and the identifier of the tag calculation result are stored in the shared database.
4. The method according to claim 1 or 3, wherein: The method further comprises: Acquire a first management state of the label calculation task and a second management state of the label index update task corresponding to the label calculation task; The first management state, the second management state, and an identifier of the tag calculation result are stored in a relational database.
5. The method according to claim 4, wherein: The selecting a first label calculation result from the label calculation results in the shared database includes: For each tag calculation result in the shared database, query the relational database according to the identifier of the tag calculation result to obtain the first management state and the second management state corresponding to the tag calculation result; Selecting at least one candidate tag calculation result to be subjected to index update processing from each tag calculation result according to a first management state and a second management state corresponding to each tag calculation result in the shared database; A candidate label calculation result with the latest label calculation time is selected from the at least one candidate label calculation result as the first label calculation result.
6. The method according to claim 5, wherein: The selecting at least one candidate tag calculation result to be subjected to index update processing from each tag calculation result according to the first management state and the second management state corresponding to each tag calculation result in the shared database comprises: For each label calculation result in the shared database, when a first management state corresponding to the label calculation result indicates that the label calculation is completed and a second management state corresponding to the label calculation result indicates that a label index update is to be triggered, the label calculation result is determined as the candidate label calculation result.
7. The method according to claim 5, wherein: The method further comprises: Obtaining other candidate label calculation results except the first label calculation result from the at least one candidate label calculation result; The label index update result is determined as the label index update result after label index update processing is performed on the other candidate label calculation results.
8. The method according to claim 1, wherein: The method further comprises: Get the label index update period; Determining at least one label index update time point according to the label index update cycle; When the tag index update time point is reached, determining to trigger a tag index update event; If the tag index update time point has not been reached, it is determined that the tag index update event has not been triggered.
9. The method according to claim 1, wherein: The calling of the label index update thread to execute the label index update task and obtain the label index update result includes: Performing lock processing on the main pointer and the standby pointer in the shared database; Calling a label index update thread to execute the label index update task, and writing an association relationship between the first label calculation result and the label of the data asset entity into the address pointed to by the standby pointer; Performing a master-slave exchange process on the primary pointer and the backup pointer to obtain a new primary pointer and a new backup pointer; Writing the association relationship between the first tag calculation result and the tag of the data asset entity into the address pointed to by the new standby pointer; Lock release processing is performed on the main pointer and the standby pointer.
10. The method according to claim 1, wherein: The method further comprises: Get the timeout processing period; Determining at least one timeout processing time point according to the timeout processing cycle; When the timeout processing time point is reached, a first label index update task is obtained; the second management state of the first label index update task is being executed, and the time interval between the start update time point of the first label index update task and the current time point is greater than the time threshold; The first tag index update task is interrupted, and the lock processing for the shared database during the execution of the first tag index update task is released.
11. A label management device, the device comprising: The first acquisition module is used to acquire label management requirements; The tag management requirements include: tags to be calculated for data asset entities, tag calculation strategies corresponding to the tags, and tag calculation cycles; A first determination module is used to determine a label calculation task corresponding to the label according to a label calculation strategy and a label calculation period corresponding to the label, and to call a label calculation thread to execute the label calculation task to obtain a label calculation result; A first storage module, used for storing the label calculation result in a shared database; A selection module, configured to select a first label calculation result from the label calculation results in the shared database when a label index update event is triggered; The second determining module is used to determine a label index updating task according to the first label calculation result, and call a label index updating thread to execute the label index updating task to obtain a label index updating result.
12. A label management system, the system comprising: A front-end application and a back-end platform; the front-end application is communicatively connected with the back-end platform; The candidate platform is provided with a plurality of nodes; The front-end application provides tag management requirements to the back-end platform; The tag management requirements include: tags to be calculated for data asset entities, tag calculation strategies corresponding to the tags, and tag calculation cycles; Each node in the backend platform executes the tag management method as claimed in any one of claims 1 to 10 based on the tag calculation strategy and tag calculation cycle corresponding to one of the tags of the data asset entity in the tag management requirement.
13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.
14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.
15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.