A method, device and electronic device for synchronizing metadata of unstructured data

Abstract

The embodiments of this specification disclose a metadata synchronization method, device and electronic device for unstructured data. The method includes first analyzing the metadata requests received within the target duration, creating a corresponding metadata synchronization unit, then adjusting the metadata heat count in the metadata synchronization unit by determining the queryable state of each metadata request, and finally counting the current metadata heat count after the target duration to determine the target order of synchronizing the metadata repository, thereby completing the metadata synchronization of the unstructured data. Through the above synchronization method, the synchronization period of each metadata can be continuously adjusted according to the staged metadata heat count, and the synchronization period can be shortened when the metadata heat count is too large, thereby meeting the real-time requirements of metadata synchronization of unstructured data. When the metadata heat count is too small, the synchronization period can be increased, thereby avoiding sending a large amount of useless data at the same time and saving computing resources.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of data communication technology, and in particular, to a metadata synchronization method, device and electronic device for unstructured data. Background Art

[0002] With the development of new technologies such as 5G, cloud computing, big data, and artificial intelligence, the scale of unstructured data has exploded. Unstructured data refers to data without fixed formats, rules, and relationships, including text, audio, video, pictures, and other forms. It is difficult to store and process through traditional relational database management systems. In order to solve the obstacles to data integration, the main method is to build metadata corresponding to unstructured data and store data in the form of metadata. However, the metadata stored will still change in real time due to factors such as request path and update time. Therefore, a metadata synchronization method is needed to maintain and update the metadata of each unstructured data to achieve efficient data management. However, the solutions currently adopted by the industry are either to collect metadata manually or to collect metadata in full through timed triggering. Due to the large amount of data, the real-time requirements of unstructured metadata cannot be met, and a large amount of computing resources are wasted. Summary of the invention

[0003] The embodiments of this specification provide a metadata synchronization method, device and electronic device for unstructured data, and the technical solution thereof is as follows:

[0004] In a first aspect, an embodiment of the present specification provides a method for synchronizing metadata of unstructured data, the method comprising:

[0005] Continuously receiving metadata requests within a target duration, creating metadata synchronization units according to metadata types corresponding to each metadata request, and allocating the metadata requests to each metadata synchronization unit according to the metadata types, each metadata synchronization unit being used to process synchronization information corresponding to a type of metadata;

[0006] For any of the metadata synchronization units, whenever the metadata synchronization unit is assigned a new metadata request, the metadata heat count in each of the metadata synchronization units is adjusted based on the queryable state of each metadata in the metadata repository, and the metadata heat count is used to express the synchronization heat corresponding to each of the metadata;

[0007] After the target time has passed, the current metadata heat counts in each metadata synchronization unit are counted respectively, and based on the target ranking of each current metadata heat count, the metadata synchronization unit is selected to synchronize the metadata repository.

[0008] In a second aspect, a metadata synchronization device for unstructured data is provided, the device comprising:

[0009] A receiving module, configured to continuously receive metadata requests within a target duration, create metadata synchronization units according to metadata types corresponding to each metadata request, and allocate the metadata requests to each metadata synchronization unit according to the metadata types, each metadata synchronization unit being configured to process synchronization information corresponding to a type of metadata;

[0010] An adjustment module, for adjusting the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository for each metadata synchronization unit, each time the metadata synchronization unit is assigned a new metadata request, wherein the metadata heat count is used to express the synchronization heat corresponding to each metadata;

[0011] The synchronization module is used to count the current metadata heat counts in each metadata synchronization unit after the target time has passed, and select the metadata synchronization unit to synchronize the metadata repository based on the target ranking of each current metadata heat count.

[0012] In a third aspect, an electronic device is provided, including a device processor and a memory;

[0013] The device processor is connected to the memory;

[0014] The memory is used to store executable program code;

[0015] The device processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method provided in the first aspect or any possible implementation manner of the first aspect.

[0016] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and the computer-readable storage medium stores instructions, which, when the instructions are executed on a computer or a device processor, cause the computer or the device processor to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0017] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0018] In one or more embodiments of the present specification, the metadata requests received within the target duration can be analyzed first, and a corresponding metadata synchronization unit can be created. Then, the metadata heat count in the metadata synchronization unit can be adjusted by determining the queryable state of each metadata request. Finally, the current metadata heat count after the target duration is counted to determine the target order of synchronizing the metadata repository, thereby completing the metadata synchronization of unstructured data. Through the above synchronization method, the synchronization period of each metadata can be continuously adjusted according to the staged metadata heat count. When the metadata heat count is too large, the synchronization period can be shortened, thereby meeting the real-time requirements of metadata synchronization of unstructured data. When the metadata heat count is too small, the synchronization period can be increased, thereby avoiding sending a large amount of useless data at the same time and saving computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of the system architecture of a metadata synchronization method for unstructured data provided in an embodiment of this specification;

[0021] Figure 2 A flowchart of a metadata synchronization method for unstructured data provided in an embodiment of this specification;

[0022] Figure 3 A schematic diagram of the structure of a metadata synchronization device for unstructured data provided in an embodiment of this specification;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0025] The terms "first", "second", "third", etc. in the description and claims of this specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0026] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present specification. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described with respect to some examples may be combined in other examples.

[0027] See also Figure 1 , Figure 1 A schematic diagram of the system architecture of a metadata synchronization method for unstructured data provided in an embodiment of this specification is shown.

[0028] like Figure 1 As shown, the system architecture of the metadata synchronization method for unstructured data may include at least a terminal 10 , a server 20 and a network 30 .

[0029] The terminal 10 includes but is not limited to electronic devices such as smart phones, desktop computers, tablet computers, laptops, smart speakers, digital assistants, smart wearable devices, etc., and may also be software running on the above electronic devices, such as applications, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc. Optionally, the terminal 10 provides the user with a metadata synchronization service for unstructured data. The terminal 10 may obtain a metadata synchronization instruction for unstructured data from an application program interface, and send a metadata synchronization request for unstructured data to the server 20.

[0030] The server 20 can provide background services for the terminal 10. According to the metadata synchronization request of the unstructured data sent by the terminal 10, the server 20 will obtain a series of metadata synchronization instructions of the unstructured data, and the server 20 transmits the metadata synchronization instructions of the unstructured data to other terminals 10 through the network 30. Specifically, the server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0031] The network 30 is used to provide a medium for a communication link between the terminal 10 and the server 20. The network 30 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0032] In addition, it should be noted that Figure 1 What is shown is only one system provided by the present disclosure. In practical applications, other systems may also be included, for example, more terminals may be included.

[0033] In the embodiments of the present specification, the terminal 10 and the server 20 may be directly or indirectly connected via wired or wireless communication, which is not limited in the present disclosure.

[0034] See next Figure 2 , Figure 2 The overall flow chart of a metadata synchronization method for unstructured data provided in an embodiment of the present specification is shown. The metadata synchronization method for unstructured data can be used in the server 20.

[0035] like Figure 2 As shown, the metadata synchronization method for unstructured data may include at least the following steps:

[0036] Step 201: Continuously receive metadata requests within a target duration, create metadata synchronization units according to metadata types corresponding to each metadata request, and distribute the metadata requests to each metadata synchronization unit according to the metadata types.

[0037] Each of the metadata synchronization units is used to process synchronization information corresponding to a type of metadata.

[0038] In an embodiment of the present specification, the target system continuously receives different metadata requests within a target duration. Among them, the types and quantities of target metadata required to be retrieved in different metadata requests are different. Therefore, after receiving each metadata request, it is necessary to create a corresponding metadata synchronization unit according to each metadata type. At the same time, each time a metadata request is received, different metadata requests may correspond to the same metadata synchronization unit. Therefore, after querying the corresponding metadata synchronization unit according to the metadata type, the previously created metadata synchronization unit can be directly assigned to the corresponding metadata request. If a new metadata request is subsequently received and there is no corresponding metadata synchronization unit according to the metadata type query, a new metadata synchronization unit is created.

[0039] The metadata synchronization unit may include multiple synchronization information such as metadata heat count, synchronization cycle, and last synchronization time corresponding to a type of metadata.

[0040] In one possible implementation, the step of creating a metadata synchronization unit according to the metadata type corresponding to each metadata request includes:

[0041] Determining the metadata type corresponding to each of the metadata requests;

[0042] The request directory path corresponding to each metadata type is queried, and a metadata synchronization unit is created based on each request directory path.

[0043] In the embodiments of the present specification, since each metadata request may correspond to multiple target metadata, each metadata type corresponding to each metadata request may be determined first. Then, each type of metadata may correspond to a metadata record, wherein the metadata record may include information such as the corresponding unstructured data storage unique identifier, object name, directory path, creation time, modification time, and metadata record synchronization time. Therefore, the request directory path corresponding to the metadata record in each metadata type may be queried. Finally, the request directory path is used as the unique identifier, and a metadata synchronization unit is created according to the hierarchical structure of the request directory path.

[0044] Step 202: For any of the metadata synchronization units, whenever the metadata synchronization unit is assigned a new metadata request, adjust the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository.

[0045] The metadata heat count is used to express the synchronization heat corresponding to each metadata.

[0046] In the embodiment of the present specification, for each metadata synchronization unit, each metadata synchronization unit may be assigned a new metadata request or a new metadata synchronization unit may be constructed for the new metadata request each time a request instruction is received. At this time, the queryable state of each metadata may be determined in the pre-set metadata repository. Among them, the metadata repository is a repository for storing metadata information, and continuously expanding the types of metadata and updating metadata in real time. Therefore, when querying different types of metadata, the queryable state may be characterized as queryable or non-queryable. When the queryable state is characterized as queryable, the metadata system can directly feed back the queried metadata information to the target terminal. When the queryable state is characterized as non-queryable, the metadata system may need to obtain the corresponding metadata information from the non-structural storage or the underlying data. Further, since different queryable states represent different time urgency for synchronizing the metadata, and different metadata synchronization time urgency is related to the metadata heat count, the adjustment mode of the corresponding metadata heat count setting is different. As an example, when the queryable state is characterized as queryable, the metadata heat count is reduced, and when the queryable state is characterized as non-queryable, the metadata heat count is increased. Therefore, the metadata heat count in the metadata synchronization unit may be adjusted according to the queryable status of each metadata in the metadata repository.

[0047] Among them, the metadata heat count can be used to express the corresponding metadata synchronization heat after the synchronization request is currently received and the metadata repository is queried.

[0048] In one possible implementation, adjusting the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository includes:

[0049] Determine a metadata record corresponding to each metadata, and determine a queryable status of each metadata record based on the metadata repository;

[0050] When the queryable state is characterized as queryable, obtaining a target metadata record in the metadata repository, and setting a metadata heat count in the metadata synchronization unit to zero;

[0051] When the queryable state is characterized as not being queryable, the attribute information in the unstructured data storage is read, a metadata record corresponding to the attribute information is constructed, and the metadata heat count is incremented.

[0052] In the embodiment of the present specification, when adjusting the metadata heat count by queryable state, the metadata record corresponding to each metadata can be determined first, and the metadata record can include the corresponding unstructured data storage unique identifier, object name, directory path, creation time, modification time and metadata record synchronization time and other information. Then, since metadata records of different types of metadata are set in the metadata repository, and as the metadata is continuously synchronized and updated, the metadata repository will continue to expand. Therefore, it is possible to query whether the corresponding metadata records exist in the metadata repository, that is, the query state. Optionally, when the corresponding metadata record exists in the metadata repository or other situations, the queryable state can be characterized as queryable, and the target metadata record queried in the metadata repository can be directly fed back to the target terminal, and the metadata heat count in the metadata synchronization unit is set to zero at the same time. Optionally, when the corresponding metadata record does not exist in the metadata repository or other situations, the queryable state can be characterized as non-queryable, and the attribute information in the unstructured data storage corresponding to the metadata is read, and the corresponding metadata record is constructed according to the attribute information and fed back to the target terminal, and the metadata heat count in the metadata synchronization unit is incremented by one.

[0053] In one possible implementation, the queryable state is characterized as queryable when the target metadata record exists in the metadata repository and the target metadata record is characterized as valid; the queryable state is characterized as non-queryable when the target metadata record does not exist in the metadata repository or the target metadata record is characterized as invalid.

[0054] In the embodiments of the present specification, in order to improve the real-time performance of directly querying the corresponding metadata through the metadata repository, it can be limited that the queryable state is characterized as queryable only when the two conditions that the target metadata record corresponding to the target metadata exists in the metadata repository and the recording time corresponding to the target metadata record meets the validity requirement are met at the same time. When the metadata repository does not have the target metadata record, or the metadata repository has the target metadata record, but the recording time corresponding to the target metadata record does not meet the validity requirement, the queryable state is characterized as non-queryable.

[0055] In one possible implementation, the target metadata record is characterized as invalid when the metadata request processing time exceeds the sum of the current latest metadata record time and the metadata consistency tolerance period.

[0056] In an embodiment of the present specification, if it is found that the target metadata record exists in the metadata repository, the current latest recording time of the metadata corresponding to the target metadata record in the metadata repository can be determined first. Then, the metadata consistency tolerance period is obtained, that is, the tolerance period exceeding the latest recording time. Finally, it is determined whether the metadata request processing time corresponding to this time has exceeded the sum of the current latest recording time of the metadata and the metadata consistency tolerance period. If it exceeds, the target metadata record is characterized as invalid, and the latest metadata record obtained needs to be fed back to the target terminal. If it does not exceed, the target metadata record is characterized as valid, and the target metadata record queried in the metadata repository can be directly fed back to the target terminal.

[0057] Step 203: After the target duration has elapsed, the current metadata heat counts in each metadata synchronization unit are counted respectively, and based on the target ranking of each current metadata heat count, the metadata synchronization unit is selected to synchronize the metadata repository.

[0058] In an embodiment of the present specification, after all metadata requests have been received continuously for a target period of time, the current metadata heat counts in each created metadata synchronization unit are counted separately. Among them, each current metadata heat count may be 0 or may increase to a certain value after being adjusted by the queryable state corresponding to each metadata request. Then, the target sorting of all current metadata heat counts can be determined based on the weight or the value of the heat count, and the synchronization period or synchronization order of each metadata synchronization unit can be further determined through the target sorting, and finally the metadata repository is synchronized according to the synchronization period or synchronization order. Among them, after the metadata information of the metadata repository is synchronized, the corresponding metadata heat count is set to 0, and the current latest record time is updated.

[0059] In one possible implementation, the selecting the metadata synchronization unit to synchronize the metadata repository based on the target ranking of each current metadata heat count includes:

[0060] Calculating the weight corresponding to each of the current metadata heat counts based on a weight algorithm, and determining the target ranking corresponding to each of the current metadata heat counts based on the order of the weights from large to small;

[0061] The metadata synchronization unit is selected based on the target ranking to synchronize the metadata repository.

[0062] In the embodiments of the present specification, the tolerance weight coefficients corresponding to different metadata can be first determined according to the length of the metadata consistency tolerance period corresponding to each metadata, and then the weights corresponding to each current metadata heat count can be calculated through a weight algorithm. As an example, assuming that the tolerance weight coefficients of A, B, and C are 3, 5, and 2, but the current metadata heat count corresponding to the A metadata is 8, the current metadata heat count corresponding to the B metadata is 4, and the current metadata heat count corresponding to the C metadata is 0, then the weight corresponding to the A metadata is 24, the weight corresponding to the B metadata is 20, and the weight corresponding to the C metadata is 0. Next, the target order is determined to be A, B, and C according to the order of weights from large to small. Finally, the metadata information in the corresponding metadata synchronization unit is synchronized and updated to the metadata repository according to the obtained target order, so that after receiving the metadata request later, it can be directly retrieved in the metadata repository.

[0063] If the metadata consistency tolerance period corresponding to each metadata is the same in duration, the tolerance weight coefficients corresponding to different metadata are the same.

[0064] In one possible implementation, the selecting the metadata synchronization unit based on the target ranking to synchronize the metadata repository includes:

[0065] Adjusting the synchronization period corresponding to each of the metadata synchronization units based on the target sorting;

[0066] The metadata synchronization units are selected respectively according to the adjusted synchronization periods to synchronize the metadata repository.

[0067] In the embodiments of the present specification, when the metadata synchronization unit is selected by target sorting to synchronize the metadata repository, the synchronization period corresponding to the metadata synchronization unit can be adjusted by target sorting first. When the target sorting is not obtained, the synchronization period corresponding to each metadata synchronization unit is pre-set to a fixed period. When the target sorting corresponding to a certain metadata is high, its fixed synchronization period is shortened in the same proportion as the high proportion. Similarly, when the target sorting corresponding to a certain metadata is low, its fixed synchronization period is increased in the same proportion as the low proportion. Different synchronization periods corresponding to different metadata synchronization units are obtained, and the metadata information in each metadata synchronization unit is synchronized with the data repository according to each adjusted synchronization period.

[0068] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0069] See next Figure 3 , Figure 3 The schematic diagram of the structure of a metadata synchronization device for unstructured data provided by an embodiment of the present specification is shown. It should be noted that: Figure 3 The metadata synchronization device for unstructured data shown is used to execute the present application Figure 2 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2 The embodiment shown.

[0070] like Figure 3 As shown, the metadata synchronization device for unstructured data may include at least:

[0071] The receiving module 301 is used to continuously receive metadata requests within a target duration, create metadata synchronization units according to the metadata types corresponding to the metadata requests, and distribute the metadata requests to the metadata synchronization units according to the metadata types, each of which is used to process synchronization information corresponding to a type of metadata;

[0072] An adjustment module 302 is used for adjusting the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository for each metadata synchronization unit, each time the metadata synchronization unit is assigned a new metadata request, wherein the metadata heat count is used to express the synchronization heat corresponding to each metadata;

[0073] The synchronization module 303 is used to count the current metadata heat counts in each metadata synchronization unit after the target time has passed, and select the metadata synchronization unit to synchronize the metadata repository based on the target ranking of each current metadata heat count.

[0074] In one possible implementation, the receiving module 301 is specifically used for:

[0075] Determining the metadata type corresponding to each of the metadata requests;

[0076] The request directory path corresponding to each metadata type is queried, and a metadata synchronization unit is created based on each request directory path.

[0077] In one possible implementation, the adjustment module 302 is specifically configured to:

[0078] Determine a metadata record corresponding to each metadata, and determine a queryable status of each metadata record based on the metadata repository;

[0079] When the queryable state is characterized as queryable, obtaining a target metadata record in the metadata repository, and setting a metadata heat count in the metadata synchronization unit to zero;

[0080] When the queryable state is characterized as not being queryable, the attribute information in the unstructured data storage is read, a metadata record corresponding to the attribute information is constructed, and the metadata heat count is incremented.

[0081] In one possible implementation, the queryable state is characterized as queryable when the target metadata record exists in the metadata repository and the target metadata record is characterized as valid; the queryable state is characterized as non-queryable when the target metadata record does not exist in the metadata repository or the target metadata record is characterized as invalid.

[0082] In one possible implementation, the target metadata record is characterized as invalid when the metadata request processing time exceeds the sum of the current latest metadata record time and the metadata consistency tolerance period.

[0083] In one possible implementation, the synchronization module 303 is specifically used for:

[0084] Calculating the weight corresponding to each of the current metadata heat counts based on a weight algorithm, and determining the target ranking corresponding to each of the current metadata heat counts based on the order of the weights from large to small;

[0085] The metadata synchronization unit is selected based on the target ranking to synchronize the metadata repository.

[0086] In one possible implementation, the synchronization module 303 is further configured to:

[0087] Adjusting the synchronization period corresponding to each of the metadata synchronization units based on the target sorting;

[0088] The metadata synchronization units are selected respectively according to the adjusted synchronization periods to synchronize the metadata repository.

[0089] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented with the help of software and / or hardware. The "unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array (FPGA), an integrated circuit (IC), etc.

[0090] Each processing unit and / or module of the embodiments of the present application may be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or may be implemented by software that executes the functions described in the embodiments of the present application.

[0091] See next Figure 4 , Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification is shown.

[0092] like Figure 4 As shown, the electronic device 400 may include: at least one device processor 401 , at least one network interface 404 , a user interface 403 , a memory 405 , and at least one communication bus 402 .

[0093] The communication bus 402 may be used to realize the connection and communication among the above-mentioned components.

[0094] The user interface 403 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0095] The network interface 404 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0096] Among them, the device processor 401 may include one or more processing cores. The device processor 401 uses various interfaces and lines to connect various parts within the entire electronic device 400, and executes various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Optionally, the device processor 401 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The device processor 401 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the device processor 401, and it can be implemented separately through a chip.

[0097] The memory 405 may include a RAM or a ROM. Optionally, the memory 405 includes a non-transitory computer-readable medium. The memory 405 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 405 may optionally be at least one storage device located away from the aforementioned device processor 401. As Figure 4 As shown, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0098] Specifically, the device processor 401 may be used to call the metadata synchronization application for the unstructured data stored in the memory 405, and specifically perform the following operations:

[0099] Continuously receiving metadata requests within a target duration, creating metadata synchronization units according to metadata types corresponding to each metadata request, and allocating the metadata requests to each metadata synchronization unit according to the metadata types, each metadata synchronization unit being used to process synchronization information corresponding to a type of metadata;

[0100] For any of the metadata synchronization units, whenever the metadata synchronization unit is assigned a new metadata request, the metadata heat count in each of the metadata synchronization units is adjusted based on the queryable state of each metadata in the metadata repository, and the metadata heat count is used to express the synchronization heat corresponding to each of the metadata;

[0101] After the target time has passed, the current metadata heat counts in each metadata synchronization unit are counted respectively, and based on the target ranking of each current metadata heat count, the metadata synchronization unit is selected to synchronize the metadata repository.

[0102] As an optional embodiment of this specification, the step of creating a metadata synchronization unit according to the metadata type corresponding to each metadata request includes:

[0103] Determining the metadata type corresponding to each of the metadata requests;

[0104] The request directory path corresponding to each metadata type is queried, and a metadata synchronization unit is created based on each request directory path.

[0105] As an optional embodiment of the present specification, the step of adjusting the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository includes:

[0106] Determine a metadata record corresponding to each metadata, and determine a queryable status of each metadata record based on the metadata repository;

[0107] When the queryable state is characterized as queryable, obtaining a target metadata record in the metadata repository, and setting a metadata heat count in the metadata synchronization unit to zero;

[0108] When the queryable state is characterized as not being queryable, the attribute information in the unstructured data storage is read, a metadata record corresponding to the attribute information is constructed, and the metadata heat count is incremented.

[0109] As an optional embodiment of the present specification, the queryable state is characterized as queryable when the target metadata record exists in the metadata repository and the target metadata record is characterized as valid; the queryable state is characterized as non-queryable when the target metadata record does not exist in the metadata repository or the target metadata record is characterized as invalid.

[0110] As an option in the embodiment of this specification, the target metadata record is characterized as invalid when the metadata request processing time exceeds the sum of the current latest metadata recording time and the metadata consistency tolerance period.

[0111] As an optional embodiment of the present specification, the step of selecting the metadata synchronization unit to synchronize the metadata repository based on the target ranking of each current metadata heat count includes:

[0112] Calculating the weight corresponding to each of the current metadata heat counts based on a weight algorithm, and determining the target ranking corresponding to each of the current metadata heat counts based on the order of the weights from large to small;

[0113] The metadata synchronization unit is selected based on the target ranking to synchronize the metadata repository.

[0114] As an optional embodiment of this specification, the step of selecting the metadata synchronization unit based on the target sorting to synchronize the metadata repository includes:

[0115] Adjusting the synchronization period corresponding to each of the metadata synchronization units based on the target sorting;

[0116] The metadata synchronization units are selected respectively according to the adjusted synchronization periods to synchronize the metadata repository.

[0117] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0118] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0119] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0120] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0123] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0124] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct related hardware. The program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0125] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A metadata synchronization method for unstructured data, characterized in that: The method comprises: Continuously receiving metadata requests within a target duration, creating metadata synchronization units according to metadata types corresponding to each metadata request, and allocating the metadata requests to each metadata synchronization unit according to the metadata types, each metadata synchronization unit being used to process synchronization information corresponding to a type of metadata; For any of the metadata synchronization units, whenever the metadata synchronization unit is assigned a new metadata request, the metadata heat count in each of the metadata synchronization units is adjusted based on the queryable state of each metadata in the metadata repository, and the metadata heat count is used to express the synchronization heat corresponding to each metadata; After the target time has passed, the current metadata heat counts in each metadata synchronization unit are counted respectively, and based on the target ranking of each current metadata heat count, the metadata synchronization unit is selected to synchronize the metadata repository; The step of adjusting the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository includes: Determine a metadata record corresponding to each metadata, and determine a queryable status of each metadata record based on the metadata repository; When the queryable state is characterized as queryable, obtaining a target metadata record in the metadata repository, and reducing a metadata heat count in the metadata synchronization unit; When the queryable state is characterized as not being queryable, the attribute information in the unstructured data storage is read, a metadata record corresponding to the attribute information is constructed, and a metadata heat count is incremented; The step of selecting the metadata synchronization unit to synchronize the metadata repository based on the target ranking of each of the current metadata heat counts includes: Calculating the weight corresponding to each of the current metadata heat counts based on a weight algorithm, and determining the target ranking corresponding to each of the current metadata heat counts based on the order of the weights from large to small; The metadata synchronization unit is selected based on the target ranking to synchronize the metadata repository.

2. The method according to claim 1, characterized in that The step of creating a metadata synchronization unit according to the metadata type corresponding to each metadata request includes: Determining the metadata type corresponding to each of the metadata requests; The request directory path corresponding to each metadata type is queried, and a metadata synchronization unit is created based on each request directory path.

3. The method according to claim 1, characterized in that The queryable state is characterized as queryable when the target metadata record exists in the metadata repository and the target metadata record is characterized as valid; the queryable state is characterized as non-queryable when the target metadata record does not exist in the metadata repository or the target metadata record is characterized as invalid.

4. The method according to claim 3, characterized in that The target metadata record is characterized as invalid when the metadata request processing time exceeds the sum of the current latest metadata record time and the metadata consistency tolerance period.

5. The method according to claim 1, characterized in that The step of selecting the metadata synchronization unit based on the target sorting to synchronize the metadata repository includes: Adjusting the synchronization period corresponding to each of the metadata synchronization units based on the target sorting; The metadata synchronization units are selected respectively according to the adjusted synchronization periods to synchronize the metadata repository.

6. A metadata synchronization device for unstructured data, characterized in that: The device comprises: A receiving module, configured to continuously receive metadata requests within a target duration, create metadata synchronization units according to metadata types corresponding to each metadata request, and allocate the metadata requests to each metadata synchronization unit according to the metadata types, each metadata synchronization unit being configured to process synchronization information corresponding to a type of metadata; An adjustment module, for adjusting the metadata heat count in each metadata synchronization unit based on the queryable state of each metadata in the metadata repository for each metadata synchronization unit, each time the metadata synchronization unit is assigned a new metadata request, wherein the metadata heat count is used to express the synchronization heat corresponding to each metadata; A synchronization module, for respectively counting the current metadata heat counts in each metadata synchronization unit after the target duration has passed, and selecting the metadata synchronization unit to synchronize the metadata repository based on the target ranking of each current metadata heat count; The adjustment module is specifically used for: Determine a metadata record corresponding to each metadata, and determine a queryable status of each metadata record based on the metadata repository; When the queryable state is characterized as queryable, obtaining a target metadata record in the metadata repository, and reducing a metadata heat count in the metadata synchronization unit; When the queryable state is characterized as not being queryable, the attribute information in the unstructured data storage is read, a metadata record corresponding to the attribute information is constructed, and a metadata heat count is incremented; The synchronization module is specifically used for: Calculating the weight corresponding to each of the current metadata heat counts based on a weight algorithm, and determining the target ranking corresponding to each of the current metadata heat counts based on the order of the weights from large to small; The metadata synchronization unit is selected based on the target ranking to synchronize the metadata repository.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, wherein the computer-readable storage medium has instructions stored therein, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 5.

Images