A data distribution method, device, equipment and readable storage medium

Abstract

The embodiment of the present application discloses a data distribution method, device, equipment and readable storage medium, wherein the method is applied to the management and scheduling system of the first area, and the method includes: obtaining initial data; the initial data is data uploaded to the first area; indexing the initial data to obtain index data corresponding to the initial data; distributing the index data to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data sharding node of the second area; each data sharding node of the second area is used to store the index data, and the index data is used to realize the search for the initial data, and the first area and the second area belong to different areas. Using the embodiment of the present application, network bandwidth can be saved and data distribution efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data distribution method, device, equipment and readable storage medium. Background Art

[0002] As the user scale and data volume of various applications increase, the performance requirements for the search and promotion architecture of applications are also increasing dramatically. In order to improve the stability of the search and promotion architecture of various applications, developers of various applications generally adopt a multi-active disaster recovery architecture built in different locations. When the search and promotion architecture in one location is abnormal, the search and promotion architecture in other locations can still operate normally to ensure the normal operation of the application. However, this method of building a multi-active disaster recovery architecture in different locations consumes a lot of network bandwidth when distributing data across clouds and locations, and the data distribution efficiency is reduced due to the long distance between different cloud servers in different regions. Summary of the invention

[0003] The embodiments of the present application provide a data distribution method, apparatus, device and readable storage medium, which can save network bandwidth and improve data distribution efficiency.

[0004] In a first aspect, the present application provides a data distribution method, which is applied to a management and scheduling system of a first area, comprising:

[0005] Acquire initial data; the initial data is data uploaded to the first area;

[0006] Performing index building processing on the initial data to obtain index data corresponding to the initial data;

[0007] The index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to realize the search for the initial data. The first area and the second area belong to different areas.

[0008] In a second aspect, the present application provides a data distribution device, which is applied to a management and scheduling system of a first area, including:

[0009] A data acquisition unit, used to acquire initial data; the initial data is data uploaded to the first area;

[0010] A data construction unit, used to perform index construction processing on the initial data to obtain index data corresponding to the initial data;

[0011] A data distribution unit is used to distribute the index data to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to realize the search for the initial data. The first area and the second area belong to different areas.

[0012] In a third aspect, the present application provides a computer device, including: a processor, a memory, and a network interface;

[0013] The processor is connected to a memory and a network interface, wherein the network interface is used to provide a data communication function, the memory is used to store computer program codes, and the processor is used to call the computer program codes so that a computer device including the processor executes the data distribution method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the above-mentioned data distribution method.

[0015] In a fifth aspect, the present application provides a computer program product, which includes a computer program / instructions, and the computer program / instructions can implement the above-mentioned data distribution method when executed by a processor.

[0016] In an embodiment of the present application, initial data is obtained; index construction processing is performed on the initial data to obtain index data corresponding to the initial data; the index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to implement a search for the initial data. Since the first area and the second area belong to different areas, by distributing the index data constructed in the first area to the synchronization node of the second area, the data can be distributed to the synchronization node of the second area by consuming the network bandwidth once, thereby completing the data synchronization of each data shard node of the second area, and thus the network bandwidth can be saved. Since each data shard node of the second area only needs to perform local data synchronization from the synchronization node of the second area, the data distribution efficiency is improved compared to synchronizing data from the first area in a different location. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a network architecture diagram of a data distribution system provided in an embodiment of the present application;

[0019] Figure 2 It is a flowchart of a data distribution method provided in an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of the structure of a data distribution system provided in an embodiment of the present application;

[0021] Figure 4 It is a flowchart of another data distribution method provided in an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the structure of a data distribution device provided in an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of the composition structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] The technical solution of the present application can be applied to the scenario of cross-cloud remote data distribution, for example, it can be applied to distribute the data of the first area to the synchronization node of the second area in a remote place through cross-cloud distribution, and the synchronization node of the second area can synchronize the data to each data sharding node of the second area. Since each data sharding node of the second area does not need to synchronize remotely from the first area when synchronizing data, but synchronizes locally at the synchronization node of the second area, it can save network bandwidth and improve the efficiency of data distribution and data synchronization. The technical solution of the embodiment of the present application can be applied to any scenario that requires cross-cloud remote data distribution, for example, it can be applied to various social applications, instant messaging applications or platforms that need to perform data search and promotion, and the embodiment of the present application is not limited to this. The collection, use and processing of initial data, index data, object tags and other data involved in the embodiment of the present application require the consent of the subject of personal information, and need to comply with the relevant laws, regulations and standards of the relevant regions.

[0026] Please refer to Figure 1 , Figure 1 is a network architecture diagram of a data distribution system provided in an embodiment of the present application, such as Figure 1 As shown, the network architecture diagram may include multiple cross-cloud remote regions. For example, a region may refer to a computer room, each of which may be deployed with corresponding servers, and each computer room is located in a different region. For example, the network architecture diagram includes a first region 11 and multiple second regions 12. The server in the first region 11 may distribute data to the servers in each second region 12 to achieve data synchronization, so that the data in each cross-cloud remote region is the same. It is understandable that for each second region 12, the first region 11 may distribute data in the same way.

[0027] The server mentioned in the embodiments of the present application can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, and big data and artificial intelligence platforms.

[0028] For further information, see Figure 2 , Figure 2 is a flow chart of a data distribution method provided in an embodiment of the present application; Figure 2 As shown, the data distribution method can be applied to the management and scheduling system of the first area. The management and scheduling system of the first area can be deployed on the server of the first area. The data distribution method includes but is not limited to the following steps:

[0029] S101, obtaining initial data.

[0030] The area mentioned in the embodiment of the present application may refer to a computer room, and a computer room may refer to a place where servers are stored, and one or more servers may be deployed in a computer room. The computer room may be divided into a main computer room and a remote computer room, and the main computer room is used to process the received data and distribute the processed data to other remote computer rooms. A remote computer room refers to a computer room deployed in other regions, and a remote computer room is used to synchronize data from the main computer room to ensure data consistency between the main computer room and the remote computer room, thereby reducing the server pressure in the main computer room. In addition, when an abnormality occurs in the main computer room, synchronized data can also be obtained from the remote computer room and returned to the corresponding terminal device, thereby avoiding situations such as server crashes caused by abnormalities in the main computer room. The terminal device mentioned in the embodiment of the present application may be an electronic device, including but not limited to mobile phones, tablet computers, laptops, vehicle-mounted devices, etc.

[0031] The first area in the embodiment of the present application may refer to a main computer room, and the second area may refer to a remote computer room. By distributing the data in the first area to the second area, the data in the second area can be made consistent with the data in the first area, thereby achieving data backup in multiple regions. The management and scheduling system of the first area can be deployed on a server in the first area, and can be used to manage data processing in the first area, such as index construction, and manage data distribution in the first area.

[0032] The initial data in the embodiment of the present application may refer to data uploaded to the first area, for example. The initial data may refer to source data, that is, data that has not been processed after being uploaded. For example, the initial data may include but is not limited to data uploaded by the registered account of the target application or data obtained from other databases. The target application may refer to a social application, an instant messaging application, or other applications that require data search and recommendation. The initial data may include but is not limited to the note data published by the registered account of the target application, the browsing behavior data of the object associated with the registered account, the comment data of the object associated with the registered account on the note data published by other registered accounts, and so on. The type of initial data is not limited in the embodiment of the present application. For example, the note data may include but is not limited to data composed of text content, data composed of image content, or data composed of text content and image content, and so on. The browsing behavior data can be used to reflect the object's preferences for various data. For example, the browsing behavior data shows that the browsing frequency of the object for a certain type of data is greater than the frequency threshold, indicating that the object is interested in this type of data. If the browsing frequency of the object for a certain type of data is less than the frequency threshold, it means that the object is not interested in this type of data, and so on.

[0033] In the embodiment of the present application, the management and scheduling system can scan the first area in real time to determine whether there is initial data in the first area. If there is no initial data, it means that the initial data has not been obtained, and no processing is required. If the initial data is found in the first area, the initial data can be processed, and the processed data can be distributed to the second area later, thereby realizing data backup in multiple regions and improving the stability of the target application operation.

[0034] S102, performing index construction processing on the initial data to obtain index data corresponding to the initial data.

[0035] In an embodiment of the present application, after the management and scheduling system of the first area obtains the initial data, it can perform index construction processing on the initial data to obtain index data corresponding to the initial data. Here, the index construction process is used to construct index data corresponding to the initial data, and the index data can be used to find the initial data. For example, the index data may include but is not limited to key information in the initial data, or may be used to indicate the acquisition address of the initial data, or may be used to reflect the type of the initial data, and so on. The storage space occupied by the index data is less than the storage space occupied by the initial data. By constructing the index data of the initial data, the index data can be distributed to the second area, and when the server in the second area subsequently receives a data search request sent by the client, it can return the corresponding data to the client in combination with the index data.

[0036] In one embodiment, if the initial data includes text content, index data can be constructed in the following manner: perform word segmentation processing on the text content in the initial data to obtain multiple words corresponding to the initial data; perform keyword extraction processing on the multiple words corresponding to the initial data to obtain keywords in the initial data; obtain the identifier of the initial data; the identifier of the initial data is used to indicate the initial data; and generate index data corresponding to the initial data based on the keywords in the initial data and the identifier of the initial data.

[0037] Among them, by performing word segmentation processing on the text content in the initial data, the initial data can be divided into multiple words, and by performing keyword extraction processing, the keywords in the initial data can be determined. Further, the identifier of the initial data can be obtained, for example, the identifier of the initial data can be used to reflect the type of the initial data, the storage address of the initial data, or the data ID (Identity, identity identification number) of the initial data. In an optional implementation, for example, if the initial data is note data uploaded by a registered account of the target application, the identifier of the initial data can refer to the note ID, and the note ID can be used to uniquely indicate this note. For example, by performing word segmentation processing and keyword extraction processing on the note data, the keywords in the note data can be obtained, and the association relationship between the keywords and the note ID of the note data can be used as the index data of the note data. By generating the association relationship between the keywords of the initial data and the note ID, when performing a subsequent keyword search, the note ID corresponding to the matching keyword in the index data can be determined, so that the note data can be obtained based on the note ID, and the note data can be returned. For example, based on the note ID, the note data matching the note ID can be queried in the database associated with the target application, so as to return the note data.

[0038] It is understandable that when there are multiple pieces of initial data, by performing keyword extraction processing on multiple words corresponding to each piece of initial data, the keywords contained in multiple pieces of initial data may be the same. In this case, when constructing an index, it is also possible to generate index data corresponding to multiple pieces of initial data based on the data identifiers and keywords of multiple pieces of initial data with the same keywords. Then, when performing a subsequent search, multiple pieces of note data corresponding to the keywords can be searched out by searching for the keywords.

[0039] In an optional implementation, when extracting keywords from the text content of the initial data, the object tag associated with the registered account that uploaded the initial data can be obtained. The object tag can be used to reflect the gender, age group, preferences, and other information of the object that uses the registered account to log in to the target application. The object here refers to the user, that is, the user who logs in to the target application using the registered account. Since the object tags of the objects are different, different objects have different focuses when uploading similar initial data. Therefore, by combining the object tags to perform keyword extraction on the text content in the initial data, keywords that match the object tags can be extracted as keywords in the initial data, thereby improving the accuracy of index construction.

[0040] In one implementation, if the initial data includes image content, the index data can be constructed in the following manner: image recognition processing is performed on the image content in the initial data to obtain text corresponding to the image content, word segmentation processing is performed on the text corresponding to the image content to obtain multiple words corresponding to the image content; keyword extraction processing is performed on the multiple words corresponding to the image content to obtain keywords in the initial data; the identifier of the initial data is obtained; the identifier of the initial data is used to indicate the initial data; based on the keywords in the initial data and the identifier of the initial data, index data corresponding to the initial data is generated. By performing image recognition processing on the image content in the initial data, the text information corresponding to the image content can be determined, thereby determining the keywords, and then realizing the construction of index data.

[0041] Optionally, if the image content included in the initial data does not contain text, the image content can be classified to obtain image classification labels, and the image assignment labels can be used as keywords in the initial data, thereby generating index data corresponding to the initial data based on the keywords in the initial data and the identifiers of the initial data. Since the image content in the initial data does not contain text, for example, the image content is pure picture content (i.e., only contains pictures), such as landscape images or character images, and keywords cannot be extracted from such initial data, the initial data can be classified based on the image content to obtain image classification labels, which can be used to indicate the type of image, and the image classification labels can be determined as keywords, thereby realizing index construction.

[0042] In another optional implementation, the key image information of the image content contained in the initial data can be extracted, and based on the identification of the key image information and the initial data, the index data corresponding to the initial data can be generated. In the subsequent image search scenario, the note ID corresponding to the image matched in the index data can be determined, so that the note data can be obtained based on the note ID, and the note data can be returned. In other words, an index can be constructed based on the key image information and note ID in the initial data, and then a search can be performed based on the input image to return the corresponding data.

[0043] In an optional implementation, if the initial data includes text content and image content, keywords in the text content and key image information in the image content can be extracted respectively, and index data corresponding to the initial data is generated based on the key image information, keywords in the initial data and the identifier of the initial data. In subsequent search scenarios, searches can be performed based on both words and image information, which improves the flexibility of data searches.

[0044] In one embodiment, the index can be constructed in combination with the trigger conditions of index construction. Specifically, the trigger conditions corresponding to the initial data can be obtained; when it is detected that the trigger conditions are met, the initial data is indexed and processed to obtain the index data corresponding to the initial data. The trigger conditions include the current time meeting the trigger cycle, detecting the trigger instruction of the specified object, the number of initial data meeting the target number, or immediate triggering.

[0045] Among them, immediate triggering may refer to triggering index construction when the initial data is obtained, and the specified object may refer to a management object such as a backend administrator of the target application. For example, the triggering condition may be determined according to the type of initial data. If the initial data is initial data with an urgency greater than the target threshold, the triggering condition of the initial data may be determined to be an immediate trigger. If the initial data is initial data with an urgency less than or equal to the target threshold, the triggering condition of the initial data may be determined to be that the current time satisfies the triggering period or the number of initial data satisfies the target number. For example, the urgency of initial data for real-time news may be greater than that of initial data for scenery. The urgency of initial data for natural disasters is greater than that of initial data for food, and so on. Alternatively, the triggering conditions corresponding to each type of initial data may be pre-set, and the corresponding triggering conditions may be selected to trigger index construction later.

[0046] In the embodiment of the present application, the trigger conditions corresponding to the initial data are determined by combining the type of the initial data, and different trigger conditions can be selected for each piece of initial data, so that the index construction for the initial data can be triggered in combination with the trigger conditions, making the index construction method more flexible. In addition, the index is constructed as soon as possible for the initial data with a higher degree of urgency, so that the client can quickly search for the initial data, thereby improving the data search efficiency.

[0047] In a possible implementation, the trigger condition is that the current time meets the trigger period, and the trigger timing of index building can be further determined in combination with the amount of initial data. For example, if the current time meets the trigger period, and the amount of initial data is greater than the quantity threshold, the initial data is indexed to obtain index data corresponding to the initial data. By determining whether to build an index in combination with the trigger period and the amount of initial data, it is possible to avoid occupying memory resources due to the small amount of initial data resulting from building a small amount of indexes.

[0048] Optionally, if the current time meets the trigger cycle and the amount of initial data is equal to the quantity threshold, the initial data is not indexed, or the initial data can be indexed at this time, and the index data corresponding to the initial data is empty, so no subsequent processing is performed. The quantity threshold can be 0, that is, when the current time meets the trigger cycle, the indexing process can be not triggered because the initial data was not obtained in the previous trigger cycle, or the index can be built, but the index data built at this time is empty, so there is no need to perform subsequent processes such as index data distribution.

[0049] S103: Distribute the index data to the synchronization nodes in the second area, so that the synchronization nodes in the second area synchronize the index data to each data shard node in the second area.

[0050] In the embodiment of the present application, after constructing the index data, the management and scheduling system of the first area can distribute the index data to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data sharding node of the second area. Each data sharding node of the second area is used to store the index data, and the index data is used to realize the search for the initial data. The first area and the second area belong to different areas.

[0051] Since the management and scheduling system of the first area distributes the index data to the synchronization node of the second area, each data shard node of the second area can synchronize data from the synchronization node of the second area, that is, each data shard node of the second area only needs to synchronize the index data locally, and does not need to synchronize the index data from a different place, that is, the first area, respectively, which can improve the efficiency of data synchronization. In addition, since each data shard node of the second area needs to consume network bandwidth to synchronize the index data from the first area, data synchronization in this way only needs to consume network bandwidth once to distribute the index data to the synchronization node of the second area, thus saving network bandwidth.

[0052] Optionally, the management and scheduling system may refer to Autobots (a management and scheduling system), for example, the synchronization node of the first area and the synchronization node of the second area may refer to DataNode (a working node of the file system), and DataNode may have the function of a synchronization component, that is, it may distribute data in the synchronization node of the first area to the synchronization node of the second area. Each data shard node of the first area and each data shard node of the second area may refer to a shard node, for example.

[0053] In one embodiment, the first region includes one or more data sharding nodes. The management and scheduling system of the first region may also synchronize the index data corresponding to the generated initial data to each data sharding node of the first region. Each data sharding node of the first region is used to store the index data. By storing the constructed index in each data sharding node of the first region, when the server of the first region subsequently receives a search request, it can determine the index data corresponding to the search request from each data sharding node of the first region, and then return the initial data requested by the search request.

[0054] In one embodiment, the first area may include a synchronization node of the first area. When generating index data, the index data may be stored in the synchronization node of the first area. When distributing the index data to the synchronization node of the second area, the synchronization node of the first area may be called to distribute the index data to the synchronization node of the second area.

[0055] In an optional implementation, the synchronization node in the first area can distribute the index data across clouds through a cross-cloud dedicated line, that is, distribute the synchronization node in the first area to the synchronization node in the second area through the cross-cloud dedicated line. The synchronization node in the first area can also distribute the index data to the synchronization node in the second area in other ways, which is not limited in the embodiments of the present application.

[0056] In one embodiment, the management and scheduling system of the first area can also synchronize the index data in the synchronization node of the first area to each data shard node of the first area. By synchronizing the index data in the synchronization node of the first area to each data shard node of the first area, the index data can be subsequently obtained from each data shard node of the first area for return. In addition, by synchronizing the index data to each data shard node of multiple first areas, the pressure of data storage can be reduced.

[0057] In one embodiment, the number of each data sharding node in the first area is multiple. When synchronizing the index data of the first area to each data sharding node in the first area, synchronization can be performed in the following manner: in the process of synchronizing the index data in the synchronization node of the first area to any data sharding node in the first area, if an operation instruction for the index data is obtained, the index data corresponding to the operation instruction is searched from other data sharding nodes in the first area that have not yet been synchronized. If the index data corresponding to the operation instruction is searched, the initial data corresponding to the index data is returned. If the index data corresponding to the operation instruction is not searched, when the synchronization of any data sharding node in the first area is completed, the index data corresponding to the operation instruction is searched from any data sharding node that has completed synchronization. If the index data corresponding to the operation instruction is searched, the initial data corresponding to the index data is returned.

[0058] That is to say, when synchronizing the index data of the synchronization node of the first area to any data sharding node in the first area, the data sharding node cannot perform online services, that is, it cannot implement search push and other processing. Therefore, when any data sharding node in the first area synchronizes index data from the synchronization node in the first area, if an operation instruction for the index data is obtained, such as a search or push operation instruction, since the data sharding node being synchronized cannot perform online services, it can search from other data sharding nodes in the first area that have not yet been synchronized, and return the searched initial data associated with the index data corresponding to the operation instruction, and the data corresponding to the operation instruction can be returned in time. It can be understood that the index data in other data sharding nodes in the first area that have not yet been synchronized is old data. After the synchronization of any data sharding node in the first area is completed, the index switching of other data sharding nodes is performed in turn. Index switching can refer to updating old index data with new index data.

[0059] Furthermore, if the index data corresponding to the operation instruction is not found, the synchronization of any data shard node in the first region can be waited for, and after the synchronization of any data shard node in the first region is completed, the index data corresponding to the operation instruction can be searched from any data shard node that has completed synchronization, and if the index data corresponding to the operation instruction is found, the initial data corresponding to the index data is returned. Data return can be achieved by waiting for the synchronization of any data shard node in the first region to be completed.

[0060] It is understandable that, since there are multiple data shard nodes in the first area, if an operation instruction for the index data is obtained during the process of synchronizing the index data in the synchronization node of the first area to any data shard node of the first area, and the first area includes synchronized and unsynchronized data shard nodes, the index data corresponding to the operation instruction can be searched from the synchronized data shard nodes in the first area. By searching for data from synchronized data shard nodes, the accuracy of data search can be improved.

[0061] In an embodiment of the present application, after the index data of the synchronization node in the first area is distributed to the synchronization node in the second area, each data shard node in the first area and each data shard node in the second area perform data synchronization from the local synchronization node. Therefore, the data synchronization method of the two areas is the same. Therefore, the processing method of obtaining the operation instruction for the index data during the synchronization process of any data shard node in the second area is the same as the processing method of obtaining the operation instruction for the index data during the synchronization process of any data shard node in the first area, which will not be repeated here.

[0062] In one embodiment, after synchronizing the index data to each data shard node of the first region and each data shard node of the second region, it can be applied to the data search scenario. Since the search methods of the first region and the second region in the search scenario are the same, the data search in the first region is used as an example for explanation: when the search request sent by the search terminal is obtained, the target keyword matching the target term in the search request is queried from each data shard node of the first region; the identifier of the initial data corresponding to the target keyword is obtained, and the initial data corresponding to the target keyword is obtained based on the identifier of the initial data corresponding to the target keyword, and the initial data corresponding to the target keyword is returned to the search terminal.

[0063] Among them, since the index data constructed in the first area is synchronized to each data sharding node of the first area and each data sharding node of the second area, and the search request includes the target word, any area can query the target keyword matching the target word from the respective data sharding node when receiving the search request sent by the search terminal. For example, the target keyword can include the target word, the approximate word of the target word, the full name of the target word or the abbreviation of the target word, etc. Therefore, a piece or note data containing the target keyword can be obtained based on the identifier of the initial data corresponding to the acquired target keyword, such as the note ID, and returned to the search terminal.

[0064] In an embodiment of the present application, by distributing index data across clouds to remote computer rooms, the data in the remote computer rooms can be guaranteed to be consistent with that in the main computer room, and the remote computer rooms can also perform data search operations, thereby reducing the search pressure in the main computer room.

[0065] In one embodiment, after synchronizing the index data to each data sharding node in the first region and each data sharding node in the second region, it can be applied to the scenario of data recommendation. The first region and the second region have the same recommendation method in the recommendation scenario, and data recommendation in the first region is used as an example for explanation: obtaining the historical search records of the push terminal to which data is to be pushed, and determining the historical data of the push terminal based on the historical search records; the historical data is used to reflect the object label of the object associated with the push terminal; obtaining the identifier of the historical data, and matching at least one identifier of reference data from each data sharding node in the first region based on the identifier of the historical data; the index data stored in each data sharding node in the first region includes multiple identifiers of reference data; and returning the reference data indicated by the matched identifier of at least one reference data to the push terminal.

[0066] Among them, the historical search records may include historical search terms, historical search images and other data of the registered account that uses the push terminal to log in to the target application within the historical time period. The historical data of the push terminal can be determined through the historical search records of the registered account within the historical time period, and the historical data can reflect the object label of the object associated with the registered account. Then, the historical note ID of the note data whose search times or click-to-view times in the historical search records are greater than the times threshold can be obtained. Furthermore, since the association relationship between note IDs and note IDs can be reflected in the index data, all note IDs associated with the historical note ID can be determined, so that all associated note IDs can be returned to the push terminal.

[0067] In an embodiment of the present application, by distributing index data across clouds to remote computer rooms, the data in the remote computer rooms can be guaranteed to be consistent with that in the main computer room, and the remote computer rooms can also perform data push operations, thereby reducing the data push pressure in the main computer room.

[0068] See also Figure 3 , Figure 3 is a schematic diagram of the structure of a data distribution system provided in an embodiment of the present application. Figure 3In the example, a first area and two second areas (second area 1 and second area 2, second area 1 and second area 2 and first area are different areas) are used for explanation: the first area may include three parts: management and scheduling system, index link, and online service. The second area may include two parts: index link and online service. Among them, the management and scheduling system of the first area can manage the process of data distribution, such as managing index data construction and index data distribution, etc. The index link of the first area can perform index construction processing, which includes full index construction and incremental index construction, and store the constructed index data on the synchronization node of the first area. Full index means that all initial data are indexed, and incremental index construction means indexing the newly added data. The index link of the first area can synchronize the index data to the index link of the second area, for example, synchronizing the index data to the synchronization node of the second area. Each data shard node of the online service only needs to synchronize the index data from the local synchronization node, for example, each data shard node of the first area only needs to synchronize the index data from the synchronization node of the first area, and each data shard node of the second area only needs to synchronize the index data from the synchronization node of the second area, which can save multiple repeated data synchronization bandwidth. For example, each data shard node in the online service can synchronize index data from the synchronization node in the region. For example, one data shard node stores index data such as "X type lipstick" and note IDs containing the keyword "X type lipstick", and another data shard node stores index data such as "Y type lipstick" and note IDs containing the keyword "Y type lipstick". When the keyword in the search scenario is "lipstick", the notes corresponding to all note IDs containing "lipstick" can be searched in each data node in the region and returned to the searcher. Among them, by synchronizing the index data to the synchronization node in a remote computer room using a cross-cloud dedicated line, unified management of data synchronization can be achieved and the total bandwidth consumption of multiple data can be controlled.

[0069] In an optional implementation, the index link may also include a Dump (memory dump file) module, which is used to store source data, i.e., initial data. The management and scheduling system may pull the source data in the Dump module into the index construction module for index construction. For example, the full amount of initial data (i.e., all initial data) in the Dump module may be pulled to construct index data. Optionally, the index link may also include a message queue (MQ) module, which may pull incremental initial data (i.e., part of the newly added initial data) to construct real-time index data, and synchronize the real-time index data to each data sharding node in this area. Or the message queue module in the first area may pull the incremental initial data of the first area to construct real-time index data, and store the real-time index data in the synchronization node of the first area, thereby distributing the real-time index data to the synchronization node of the second area, and then synchronizing to each data sharding node in the second area, and each data synchronization node in the first area.

[0070] In an embodiment of the present application, initial data is obtained; index construction processing is performed on the initial data to obtain index data corresponding to the initial data; the index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to implement a search for the initial data. Since the first area and the second area belong to different areas, by distributing the index data constructed in the first area to the synchronization node of the second area, the data can be distributed to the synchronization node of the second area by consuming the network bandwidth once, thereby completing the data synchronization of each data shard node of the second area, and thus the network bandwidth can be saved. Since each data shard node of the second area only needs to perform local data synchronization from the synchronization node of the second area, the data distribution efficiency is improved compared to synchronizing data from the first area in a different location.

[0071] For further information, see Figure 4 , Figure 4 is a flow chart of another data distribution method provided by an embodiment of the present application; Figure 4 As shown, the data distribution method can be applied to a data distribution system, and is jointly executed by a server in a first area and a server in a second area in the data distribution system. The server in the first area may include a management scheduling system, an index building module, a synchronization node, and a data sharding node. The server in the second area may include a synchronization node and a data sharding node. The data distribution method includes but is not limited to the following steps:

[0072] S201, a management and scheduling system scans synchronization nodes in a first area.

[0073] In the embodiment of the present application, the management and scheduling system can schedule and control the construction of all index data in the first area, the distribution of index data, and the switching of index data. The switching of index data refers to switching the old index data in each data sharding node to the new index data when new index data exists. The management and scheduling system scanning the synchronization nodes in the first area may refer to scanning the data in the synchronization nodes in the first area. If new data is found in the synchronization nodes in the first area, subsequent index construction can be performed.

[0074] Optionally, before scanning the synchronization nodes in the first area, the management and scheduling system can also register the data source and index information. The data source is used to indicate the source of the data. By registering the data source and index information, the management and scheduling system can trigger the subsequent index construction process when it receives the initial data corresponding to the data source. The index information can be used to describe the type of index data, that is, what kind of index data is output. For example, the type of index data includes but is not limited to keywords and note IDs, or note IDs and note IDs, and so on. Optionally, a scheduler can be set in the management and scheduling system, and the scheduler can scan the data in the synchronization nodes in the first area in real time to determine whether there is new data in the synchronization nodes in the first area, and trigger index construction when new data exists.

[0075] S202: The index building module performs index building processing on the initial data to generate index data.

[0076] Here, when new data such as initial data is scanned, the management and scheduling system can call the index building module to perform index building processing on the initial data to generate index data. Specifically, when the management and scheduling system scans the initial data, it can send trigger information for triggering index building to the index building module. When the index building module obtains the trigger information, it can perform index building processing on the initial data to generate index data.

[0077] S203: The index building module stores the index data in the synchronization node in the first region.

[0078] Here, when the index data is successfully constructed, the index construction module stores the index data in the synchronization node of the first region. For example, when the index data is successfully constructed, the management and scheduling system can call the index construction module to execute the process of storing the index data in the synchronization node of the first region.

[0079] S204: The synchronization node in the first area distributes the index data to the synchronization node in the second area.

[0080] Here, since the index data is stored in the synchronization node of the first region, the synchronization node of the first region can distribute the index data to the synchronization node of the second region to achieve cross-cloud remote synchronization. For example, the management and scheduling system can call the synchronization node of the first region to distribute the index data to the synchronization node of the second region.

[0081] Optionally, the synchronization node in the second region may also call back the management scheduling system to prompt the management scheduling system that the cross-cloud distribution of index data is completed so that subsequent steps can be executed.

[0082] S205: The management and scheduling system sends data synchronization indication information to the synchronization node in the first area.

[0083] Here, since the index data is stored in the synchronization node of the first area, and in subsequent application scenarios, the index data needs to be obtained from each data shard node in the first area, the management and scheduling system can send data synchronization indication information to the synchronization node of the first area, so that the synchronization node of the first area can synchronize the index data to the each data shard node in the first area.

[0084] S206: The management and scheduling system sends data synchronization indication information to the synchronization node in the second area.

[0085] Here, since the index data is synchronized from the synchronization node in the first area to the second area, it is synchronized to the synchronization node in the second area, and in subsequent application scenarios, it is necessary to obtain index data from each data shard node in the second area. Therefore, the management and scheduling system can send data synchronization indication information to the synchronization node in the second area, so that the synchronization node in the second area can synchronize the index data to each data shard node in the second area.

[0086] S207: The synchronization node in the first area synchronizes index data to each data shard node in the first area.

[0087] S208: The synchronization node in the second area synchronizes index data to each data shard node in the second area.

[0088] In an embodiment of the present application, by synchronizing index data from the synchronization node of the first area to each data shard node of the first area, and synchronizing index data from the synchronization node of the second area to each data shard node of the second area, the consistency of data in the first area and data in the second area can be ensured, off-site data backup can be achieved, and data security can be improved.

[0089] In a possible implementation, both the server in the first area and the server in the second area may include an administration module (Admin), and the Admin may be used to perform index switching, for example, the Admin may be used to control when the respective data shard nodes perform index switching. For example, the management and scheduling system may notify the cluster Admin to perform index switching, and the Admin in each area may send new index position indication information to each data shard node in the area, and the index position indication information may include, for example, the data address of the index data stored on the DataNode. Each data shard node reads the index data from the corresponding data address on the local DataNode to the hard disk of the engine node. After the index data in each area is synchronized to each shard node of the local engine, the Admin controls each shard node of the engine to perform index switching one by one, that is, after the previous shard node is switched, the next shard node is indexed, thereby achieving the final index data synchronization. Optionally, when reading the index data to the hard disk of the engine node, for example, the DP module (a data reading module) of each data shard node may read the index data from the corresponding data address on the local DataNode to the hard disk of the engine node.

[0090] Since DataNode can include a lot of data, for example, it can refer to a large directory, which includes many file directories. When each shard node switches, it needs to determine which file directory in the DataNode to read the data from. Therefore, the Admin of the computer room can be used to inform the shard node in the computer room to pull the index data from the DataNode directory to the hard disk of the shard node.

[0091] In the embodiment of the present application, the specific implementation methods of steps S201 to S208 that are not mentioned can refer to the implementation methods of the above-mentioned steps S101 to S103, and will not be repeated here.

[0092] In the embodiment of the present application, in order to solve the distribution of index data in different locations and multiple clouds, and the rolling switching of multi-distributed and multi-sharded distributed systems, i.e., index switching, the search and promotion architecture can be highly stable and bandwidth can be saved during data synchronization. Moreover, compared with the fact that each data shard node in the second area synchronizes index data from the first area for index switching, in the embodiment of the present application, only the local synchronized index data from the second area is needed to perform index switching, which can improve the efficiency of index switching. Moreover, the embodiment of the present application can support the index switching of multiple engine clusters with different clouds in multiple regions. One-time synchronization is adopted, that is, the main computer room synchronizes index data to the remote computer room once, and multiple copies of the remote computer room can be used, which can save network bandwidth. Through the tidal control of the total bandwidth, the synchronization speed and the dedicated line bandwidth cost are balanced, and the synchronization speed of each index data is controlled from a macro level to ensure that the use of network bandwidth does not exceed the dedicated line capacity, that is, it will not occupy too much dedicated line capacity at the same time. Multiple copies of data in the engine cluster are pulled from the computer room to reduce the time-consuming process. At the same time, the latest version of data is synchronized in each computer room, which is convenient for rapid expansion.

[0093] In the embodiment of the present application, by adopting a system with a multi-cloud multi-active disaster recovery architecture in different locations, a universal solution is provided, which is highly stable, low-cost, and has many application scenarios. For distributed cluster applications, the problem of redundant cross-cloud bandwidth is solved by deploying DataNode in each computer room, and the time delay problem caused by direct cross-cloud synchronization is greatly reduced. Scaling is a very important feature of cloud native infrastructure, but if the scaling of data services depends on cross-cloud synchronization of data, the dynamic scaling capability will be greatly discounted. Each computer room has the latest version of the data stored on the DataNode, which ensures the efficiency of the dynamic scaling capability.

[0094] In an embodiment of the present application, initial data is obtained; index construction processing is performed on the initial data to obtain index data corresponding to the initial data; the index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to implement a search for the initial data. Since the first area and the second area belong to different areas, by distributing the index data constructed in the first area to the synchronization node of the second area, the data can be distributed to the synchronization node of the second area by consuming the network bandwidth once, thereby completing the data synchronization of each data shard node of the second area, and thus the network bandwidth can be saved. Since each data shard node of the second area only needs to perform local data synchronization from the synchronization node of the second area, the data distribution efficiency is improved compared to synchronizing data from the first area in a different location.

[0095] The method of the embodiment of the present application is introduced above, and the device of the embodiment of the present application is introduced below.

[0096] See also Figure 5 , Figure 5 : is a schematic diagram of the composition structure of a data distribution device provided in an embodiment of the present application. The above-mentioned data distribution device can be deployed in the management and scheduling system of the first area, and the management and scheduling system of the first area can be deployed on the server of the first area; the data distribution device can be used to execute the corresponding steps in the data distribution method provided in the embodiment of the present application. The data distribution device 50 includes:

[0097] The data acquisition unit 501 is used to acquire initial data; the initial data is data uploaded to the first area;

[0098] A data construction unit 502 is used to perform index construction processing on the initial data to obtain index data corresponding to the initial data;

[0099] The data distribution unit 503 is used to distribute the index data to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to realize the search for the initial data. The first area and the second area belong to different areas.

[0100] Optionally, the data construction unit 502 is further used to:

[0101] The synchronization node storing the index data in the first region;

[0102] The data distribution unit 503 is specifically used for:

[0103] The synchronization node in the first area is called to distribute the index data to the synchronization node in the second area.

[0104] Optionally, the data distribution device 50 further includes a data synchronization unit 504, and the data synchronization unit 504 is used to:

[0105] The index data in the synchronization node of the first area is synchronized to each data shard node of the first area.

[0106] Optionally, the number of each data shard node in the first area is multiple, and the data synchronization unit 504 is specifically used to:

[0107] In the process of synchronizing the index data in the synchronization node of the first area to any data shard node of the first area, if an operation instruction for the index data is obtained, search for the index data corresponding to the operation instruction from other data shard nodes of the first area that have not yet been synchronized;

[0108] If the index data corresponding to the operation instruction is found, the initial data corresponding to the index data is returned;

[0109] If the index data corresponding to the operation instruction is not found, then when the synchronization of any data shard node in the first area is completed, the index data corresponding to the operation instruction is searched from the any data shard node that has completed the synchronization. If the index data corresponding to the operation instruction is found, the initial data corresponding to the index data is returned.

[0110] Optionally, the data construction unit 502 is used to:

[0111] Perform word segmentation processing on the text content in the initial data to obtain multiple words corresponding to the initial data;

[0112] Perform keyword extraction processing on multiple words corresponding to the initial data to obtain keywords in the initial data;

[0113] Acquire an identifier of the initial data; the identifier of the initial data is used to indicate the initial data;

[0114] Based on the keywords in the initial data and the identifier of the initial data, index data corresponding to the initial data is generated.

[0115] Optionally, the data distribution device 50 further includes: a data search unit 505, wherein the data search unit 505 is used to:

[0116] When a search request sent by the search terminal is obtained, querying the target keyword matching the target word in the search request from each data shard node in the first area;

[0117] An identifier of the initial data corresponding to the target keyword is obtained, and based on the identifier of the initial data corresponding to the target keyword, the initial data corresponding to the target keyword is obtained, and the initial data corresponding to the target keyword is returned to the search terminal.

[0118] Optionally, the data distribution device 50 further includes: a data pushing unit 506, wherein the data pushing unit 506 is used to:

[0119] Acquire the historical search record of the push terminal to which data is to be pushed, and determine the historical data of the push terminal based on the historical search record; the historical data is used to reflect the object label of the object associated with the push terminal;

[0120] Obtaining an identifier of the historical data, and matching at least one identifier of reference data from each data sharding node of the first region based on the identifier of the historical data; the index data stored in each data sharding node of the first region includes identifiers of multiple reference data;

[0121] The reference data indicated by the identifier of the matched at least one reference data is returned to the push terminal.

[0122] Optionally, the data acquisition unit 501 is further used to:

[0123] Obtain the trigger condition corresponding to the initial data;

[0124] The data acquisition unit 501 is specifically used for:

[0125] When it is detected that the trigger condition is met, index building processing is performed on the initial data to obtain index data corresponding to the initial data;

[0126] The trigger condition includes one of the following: the current time meets the trigger period, a trigger instruction of a specified object is detected, or the quantity of the initial data meets the target quantity.

[0127] Optionally, the trigger condition is that the current time satisfies the trigger period, and the data acquisition unit 501 is specifically used to:

[0128] If the current time meets the trigger period, and the quantity of the initial data is greater than the quantity threshold, the initial data is indexed and constructed to obtain index data corresponding to the initial data.

[0129] It should be noted that Figure 5 For the contents not mentioned in the corresponding embodiments, please refer to the description of the method embodiments, which will not be repeated here.

[0130] In an embodiment of the present application, initial data is obtained; index construction processing is performed on the initial data to obtain index data corresponding to the initial data; the index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to implement a search for the initial data. Since the first area and the second area belong to different areas, by distributing the index data constructed in the first area to the synchronization node of the second area, the data can be distributed to the synchronization node of the second area by consuming the network bandwidth once, thereby completing the data synchronization of each data shard node of the second area, and thus the network bandwidth can be saved. Since each data shard node of the second area only needs to perform local data synchronization from the synchronization node of the second area, the data distribution efficiency is improved compared to synchronizing data from the first area in a different location.

[0131] See also Figure 6 , Figure 6 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 6As shown, the above-mentioned computer device 60 may include: a processor 601, a network interface 604 and a memory 605. In addition, the above-mentioned computer device 60 may also include: a user interface 603, and at least one communication bus 602. Among them, the communication bus 602 is used to realize the connection and communication between these components. Among them, the user interface 603 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 603 may also include a standard wired interface and a wireless interface. The network interface 604 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 605 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 605 may optionally also be at least one storage device located away from the aforementioned processor 601. As Figure 6 As shown, the memory 605 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0132] exist Figure 6 In the computer device 60 shown, the network interface 604 can provide a network communication function; the user interface 603 is mainly used to provide an input interface for the user; and the processor 601 can be used to call the device control application stored in the memory 605 to achieve:

[0133] Acquire initial data; the initial data is data uploaded to the first area;

[0134] Performing index building processing on the initial data to obtain index data corresponding to the initial data;

[0135] The index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to realize the search for the initial data. The first area and the second area belong to different areas.

[0136] It should be understood that the computer device 60 described in the embodiment of the present application can execute the above Figure 2 and Figure 4 The description of the above data distribution method in the corresponding embodiment can also be performed as described above. Figure 5 The description of the data distribution device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0137] In an embodiment of the present application, initial data is obtained; index construction processing is performed on the initial data to obtain index data corresponding to the initial data; the index data is distributed to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the index data, and the index data is used to implement a search for the initial data. Since the first area and the second area belong to different areas, by distributing the index data constructed in the first area to the synchronization node of the second area, the data can be distributed to the synchronization node of the second area by consuming the network bandwidth once, thereby completing the data synchronization of each data shard node of the second area, and thus the network bandwidth can be saved. Since each data shard node of the second area only needs to perform local data synchronization from the synchronization node of the second area, the data distribution efficiency is improved compared to synchronizing data from the first area in a different location.

[0138] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, which includes program instructions, which, when executed by a computer, causes the computer to execute the method of the aforementioned embodiment, and the computer may be part of the computer device mentioned above. For example, it is the above-mentioned processor 601. As an example, the program instructions may be deployed on a computer device for execution, or deployed on multiple computer devices located at one location for execution, or, executed on multiple computer devices distributed at multiple locations and interconnected by a communication network, and multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain network.

[0139] Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0140] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data distribution method, characterized in that: The method is applied to a management and scheduling system of a first area, and the method comprises: Acquire initial data; the initial data is the full amount of initial data uploaded to the first area; Performing index building processing on the initial data to obtain index data corresponding to the initial data; the index data corresponding to the initial data refers to the full index data of the full amount of initial data; Calling the synchronization node of the first area to distribute the index data to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the full index data, and the full index data stored by each data shard node of the second area is used to search the full index data by local search when performing data search to return the initial data corresponding to the searched index data to the search terminal, the first area and the second area belong to different areas; the synchronization node of the first area and the synchronization node of the second area are two different nodes; Among them, the index data stored in each data sharding node of the second area is used to return the reference data indicated by the identifier of at least one matched reference data to the push terminal to which the data is to be pushed when pushing data, and the index data stored in each data sharding node of the second area includes multiple reference data identifiers, and the identifier of at least one reference data is matched from each data sharding node of the second area based on the identifier of historical data, and the historical data is determined based on the historical search record of the push terminal, and the historical data is used to reflect the object label of the object associated with the push terminal.

2. The method according to claim 1, characterized in that: The method further comprises: The index data is stored in a synchronization node in the first region.

3. The method according to claim 2, characterized in that The method further comprises: Synchronize the index data in the synchronization node of the first area to each data shard node of the first area.

4. The method according to claim 3, characterized in that The number of data shard nodes in the first area is multiple, and synchronizing the index data in the synchronization node of the first area to each data shard node in the first area includes: In the process of synchronizing the index data in the synchronization node of the first area to any data shard node of the first area, if an operation instruction for the index data is obtained, search for the index data corresponding to the operation instruction from other data shard nodes of the first area that have not yet been synchronized; If the index data corresponding to the operation instruction is found, the initial data corresponding to the index data is returned; If the index data corresponding to the operation instruction is not found, then when the synchronization of any data shard node in the first area is completed, the index data corresponding to the operation instruction is searched from any data shard node that has completed the synchronization. If the index data corresponding to the operation instruction is found, the initial data corresponding to the index data is returned.

5. The method according to claim 3, characterized in that: The performing index building processing on the initial data to obtain index data corresponding to the initial data includes: Performing word segmentation processing on the text content in the initial data to obtain a plurality of words corresponding to the initial data; Performing keyword extraction processing on a plurality of words corresponding to the initial data to obtain keywords in the initial data; Acquire an identifier of the initial data; the identifier of the initial data is used to indicate the initial data; Based on the keywords in the initial data and the identifiers of the initial data, index data corresponding to the initial data are generated.

6. The method according to claim 5, characterized in that The method further comprises: When a search request sent by a search terminal is obtained, querying each data shard node in the first area for a target keyword that matches a target word in the search request; An identifier of the initial data corresponding to the target keyword is obtained, and the initial data corresponding to the target keyword is obtained based on the identifier of the initial data corresponding to the target keyword, and the initial data corresponding to the target keyword is returned to the search terminal.

7. The method according to claim 5, characterized in that The method further comprises: Acquire a historical search record of a push terminal to which data is to be pushed, and determine historical data of the push terminal based on the historical search record; the historical data is used to reflect an object tag of an object associated with the push terminal; Obtaining an identifier of the historical data, and matching at least one identifier of reference data from each data sharding node of the first region based on the identifier of the historical data; the index data stored in each data sharding node of the first region includes identifiers of multiple reference data; The reference data indicated by the identifier of the at least one matched reference data is returned to the push terminal.

8. The method according to claim 1, characterized in that The method further comprises: Acquire a trigger condition corresponding to the initial data; When it is detected that the trigger condition is met, the step of performing index building processing on the initial data to obtain index data corresponding to the initial data is executed; The trigger condition includes one of the following: the current time meets the trigger period, a trigger instruction of a specified object is detected, or the quantity of the initial data meets the target quantity.

9. The method according to claim 8, characterized in that The trigger condition is that the current time meets the trigger period, and the method further includes: If the current time satisfies the trigger period and the quantity of the initial data is greater than the quantity threshold, the step of performing index construction processing on the initial data to obtain index data corresponding to the initial data is executed.

10. A data distribution device, characterized in that: The device is applied to a management and scheduling system of a first area, and the device comprises: A data acquisition unit, used to acquire initial data; the initial data is the full amount of initial data uploaded to the first area; A data construction unit, configured to perform index construction processing on the initial data to obtain index data corresponding to the initial data; the index data corresponding to the initial data refers to the full index data of the full amount of initial data; A data distribution unit, used for calling the synchronization node of the first area to distribute the index data to the synchronization node of the second area, so that the synchronization node of the second area synchronizes the index data to each data shard node of the second area; each data shard node of the second area is used to store the full index data, and the full index data stored by each data shard node of the second area is used to search the full index data by local search when performing data search to return the initial data corresponding to the searched index data to the search terminal, the first area and the second area belong to different areas; the synchronization node of the first area and the synchronization node of the second area are two different nodes; Among them, the index data stored in each data sharding node of the second area is used to return the reference data indicated by the identifier of at least one matched reference data to the push terminal to which the data is to be pushed when pushing data, and the index data stored in each data sharding node of the second area includes multiple reference data identifiers, and the identifier of at least one reference data is matched from each data sharding node of the second area based on the identifier of historical data, and the historical data is determined based on the historical search record of the push terminal, and the historical data is used to reflect the object label of the object associated with the push terminal.

11. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store program code, and the processor is used to call the program code so that the computer device executes the method described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images