Feature data acquisition method, service prediction system, and service prediction device

Abstract

The application relates to a feature data acquisition method, a business prediction system, a business prediction device, a computer readable storage medium and a computer program product, and relates to the technical field of computers. The method is applied to a business prediction device, the business prediction device comprises a plurality of cache nodes and a prediction node, and the method comprises the following steps: determining target feature data required by a business prediction request in the case that the business prediction request is received; determining a target cache node in which the target feature data is cached in the plurality of cache nodes; the feature data cached by each cache node is obtained according to relationship information between the feature data, and the relationship information is obtained based on a plurality of historical business prediction requests; the target feature data is acquired from the target cache node; and the prediction node is used for performing business prediction corresponding to the business prediction request based on a loaded business prediction model and the target feature data. By adopting the method, the time consumption for acquiring feature data can be shortened in a high-concurrency scene.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for acquiring feature data, a business forecasting system, a business forecasting device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the continuous development of large model technology, more and more businesses are starting to use large models for prediction. Large model prediction often requires feature data as input. In complex business prediction, the amount of feature data can be very large. Therefore, feature data is usually stored in external data sources. When the large model needs to obtain feature data, it obtains the feature data from the external data source through network I / O (Input / Output).

[0003] However, when the prediction request requires a large amount of feature data, network I / O will quickly reach its limit, resulting in a long time to obtain the feature data. Summary of the Invention

[0004] Therefore, it is necessary to address the aforementioned technical problem of long acquisition time for feature data by providing a feature data acquisition method, business prediction system, business prediction device, computer-readable storage medium, and computer program product that can reduce the time consumption.

[0005] In a first aspect, this application provides a feature data acquisition method applied to a business prediction device, wherein the business prediction device includes cache nodes and prediction nodes, and the number of cache nodes is multiple; the method includes:

[0006] Upon receiving a business forecasting request, determine the target feature data required for the business forecasting request;

[0007] Among multiple cache nodes, a target cache node that caches the target feature data is determined; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is based on multiple historical business prediction requests.

[0008] The target feature data is obtained from the target cache node; the prediction node is used to perform business prediction corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

[0009] In one embodiment, upon receiving a business forecasting request, determining the target feature data required for the business forecasting request includes:

[0010] Upon receiving the business prediction request through the first cache node, the target feature data is determined through the first cache node; the first cache node is any one of the plurality of cache nodes.

[0011] The step of obtaining the target feature data from the target cache node includes:

[0012] If the first cache node is the target cache node, then the target feature data is read from the first cache node and sent to the prediction node.

[0013] If the second cache node is the target cache node, then the business prediction request is forwarded to the second cache node through the first cache node; the second cache node is any one of the plurality of cache nodes other than the first cache node;

[0014] The target feature data is read from the second cache node and sent to the prediction node.

[0015] In one embodiment, each cache node also caches the cache mapping relationship between feature data and cache nodes;

[0016] The step of determining the target cache node that caches the target feature data among multiple cache nodes includes:

[0017] The target cache node is determined from the plurality of cache nodes by using the first cache node and the cache mapping relationship cached on the first cache node;

[0018] The method further includes:

[0019] If none of the cache nodes have cached the target feature data, the first cache node is used to access the data source used to store the feature data, and the target feature data is obtained from the data source; the data source is located outside the business prediction device.

[0020] In one embodiment, before determining the target feature data required for the business forecasting request upon receiving the business forecasting request, the method further includes:

[0021] Obtain multiple historical business prediction requests, and determine the relationship information between the various feature data based on the multiple historical business prediction requests;

[0022] Based on the relationship information, a cache mapping relationship between feature data and cache nodes is constructed, and the cache mapping relationship is distributed to each cache node;

[0023] Through each cache node and the cache mapping relationship, each cache node is used to access the data source for storing the feature data based on the cache mapping relationship, obtain the corresponding feature data from the data source and cache it; the data source is located outside the business prediction device.

[0024] In one embodiment, the service prediction device receives the service prediction request through any one of the cache nodes; the cache node is used to report the received service prediction request to the service prediction device.

[0025] The method further includes:

[0026] Receive the service prediction request reported by the cache node;

[0027] Based on the business prediction request, update the relationship information to obtain the updated relationship information;

[0028] Based on the updated relationship information, the cache mapping relationship is updated to obtain the updated cache mapping relationship;

[0029] The updated cache mapping relationship is then distributed to each cache node.

[0030] In one embodiment, the relationship information between the feature data includes at least the cache priority between the feature data;

[0031] The step of constructing a cache mapping relationship between feature data and cache nodes based on the relationship information includes:

[0032] Based on the caching priority among the various feature data, select the feature data to be cached from the feature data whose corresponding caching priority is greater than or equal to a preset priority threshold;

[0033] Based on the feature data to be cached, a list of feature data to be cached for each cache node is determined, and based on the list of feature data to be cached for each cache node, the cache mapping relationship is obtained.

[0034] In one embodiment, determining the relationship information between the various feature data based on the plurality of historical business prediction requests includes:

[0035] Based on the multiple historical business prediction requests, the access frequency of each feature data is determined;

[0036] Based on the access frequency of each feature data, the cache priority among the feature data is determined.

[0037] In one embodiment, the relationship information between the feature data includes at least the correlation between the feature data;

[0038] The step of constructing a cache mapping relationship between feature data and cache nodes based on the relationship information further includes:

[0039] Based on the correlation between the feature data, multiple sets of feature data to be cached are determined from the feature data; the correlation between the feature data in each set of feature data to be cached is greater than or equal to a preset correlation threshold.

[0040] Based on the multiple sets of feature data to be cached, a list of feature data to be cached for each cache node is determined, and based on the list of feature data to be cached for each cache node, the cache mapping relationship is obtained; the list of feature data to be cached for each cache node corresponds to a set of cache feature data.

[0041] In one embodiment, the method further includes:

[0042] Monitor the space occupancy rate of each cache node;

[0043] If the space occupancy rate of any cache node is greater than or equal to a preset occupancy rate threshold, the cache node will be used as the source node for data migration, and a new cache node will be created as the target node for data migration.

[0044] A synchronization command is issued to the target node so that the target node synchronizes at least one feature data from the source node;

[0045] A deletion command is issued to the source node so that the source node deletes the feature data synchronized by the target node.

[0046] Secondly, this application also provides a business forecasting system, which includes a control node, a cache node, and a forecasting node, wherein the number of cache nodes is multiple.

[0047] The control node is used to obtain the relationship information between various feature data based on multiple historical business prediction requests, and to obtain the feature data cached by each cache node based on the relationship information between the various feature data; each cache node caches at least one feature data.

[0048] The cache node is used to determine the target feature data required by the business prediction request when a business prediction request is received, determine the target cache node that caches the target feature data among multiple cache nodes, and obtain the target feature data from the target cache node.

[0049] The prediction node is used to perform business predictions corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

[0050] Thirdly, this application also provides a business forecasting device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0051] Upon receiving a business forecasting request, determine the target feature data required for the business forecasting request;

[0052] Among multiple cache nodes, a target cache node that caches the target feature data is determined; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is based on multiple historical business prediction requests.

[0053] The target feature data is obtained from the target cache node; the prediction node is used to perform business prediction corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

[0054] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0055] Upon receiving a business forecasting request, determine the target feature data required for the business forecasting request;

[0056] Among multiple cache nodes, a target cache node that caches the target feature data is determined; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is based on multiple historical business prediction requests.

[0057] The target feature data is obtained from the target cache node; the prediction node is used to perform business prediction corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

[0058] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0059] Upon receiving a business forecasting request, determine the target feature data required for the business forecasting request;

[0060] Among multiple cache nodes, a target cache node that caches the target feature data is determined; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is based on multiple historical business prediction requests.

[0061] The target feature data is obtained from the target cache node; the prediction node is used to perform business prediction corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

[0062] The aforementioned feature data acquisition method, business prediction system, business prediction device, computer-readable storage medium, and computer program product first determine the target feature data required for the business prediction request upon receiving a business prediction request; then, they determine a target cache node that caches the target feature data among multiple cache nodes; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is obtained based on multiple historical business prediction requests; finally, they acquire the target feature data from the target cache node; and the prediction node is used to perform business prediction corresponding to the business prediction request based on the onboard business prediction model and the target feature data. In this way, cache nodes are used to cache feature data. By identifying the target cache node among multiple cache nodes that caches the target feature data required for the business prediction request, the target feature data required for the business prediction request can be obtained from the target cache node. Then, based on the business prediction model carried by the prediction node and the obtained target feature data, business prediction corresponding to the business prediction request can be performed. Based on the feature data acquisition method of the above process, by caching the feature data on the cache node that is in the same business prediction device as the business prediction model, the target feature data can be obtained on the cache node that is in the same business prediction device as the business prediction model. Therefore, it no longer relies on network I / O to obtain feature data from external data sources, thereby shortening the time spent obtaining feature data in high-concurrency scenarios. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is an application environment diagram of a feature data acquisition method in one embodiment;

[0065] Figure 2 This is a flowchart illustrating a feature data acquisition method in one embodiment;

[0066] Figure 3 This is a flowchart illustrating the steps of determining the target feature data required for a business prediction request and obtaining the target feature data from the target cache node when a business prediction request is received, in one embodiment.

[0067] Figure 4 This is a flowchart illustrating the steps of constructing a cache mapping relationship between feature data and cache nodes, and caching feature data based on the cache mapping relationship in one embodiment.

[0068] Figure 5 This is a flowchart illustrating the steps of updating cache mapping relationships in one embodiment;

[0069] Figure 6 This is a flowchart illustrating the steps of constructing a cache mapping relationship between feature data and cache nodes based on relational information in one embodiment.

[0070] Figure 7 This is a flowchart illustrating the steps of determining the relationship information between various feature data based on multiple historical business prediction requests in one embodiment.

[0071] Figure 8 This is a flowchart illustrating the steps of constructing a cache mapping relationship between feature data and cache nodes based on relational information in another embodiment.

[0072] Figure 9 This is a flowchart illustrating the steps of rearranging and expanding cache nodes in one embodiment;

[0073] Figure 10 This is an internal structure diagram of a service prediction device in one embodiment. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0075] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0076] The feature data acquisition method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is a business forecasting system integrated into a business forecasting device. This system includes a control node, multiple cache nodes, and forecasting nodes. In specific applications, the business forecasting device is a standalone physical server or terminal, which can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc.

[0077] The control node manages the cache nodes. In practical applications, the control node is used to obtain the relationship information between various feature data based on multiple historical business prediction requests, and then uses this relationship information to obtain the feature data cached by each cache node.

[0078] The cache nodes are used to cache feature data. In specific applications, each cache node caches at least one piece of feature data.

[0079] The prediction node is equipped with a business prediction model, which is used to make business predictions corresponding to the business prediction request based on the business prediction model and target feature data. In specific applications, the business prediction model is a neural network model.

[0080] Specifically, upon receiving a business prediction request, the business prediction system determines the target feature data required for the business prediction request; the business prediction system identifies a target cache node that caches the target feature data among multiple cache nodes; the business prediction system retrieves the target feature data from the target cache node; and the prediction node is used to perform business predictions corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

[0081] In one exemplary embodiment, such as Figure 2 As shown, a feature data acquisition method is provided, which can be applied to devices equipped with, for example, features such as Figure 1 The following steps are used as an example to illustrate the business forecasting system shown in the diagram:

[0082] Step S202: Upon receiving a business forecasting request, determine the target feature data required for the business forecasting request.

[0083] Among them, the business forecasting request is a request from a user to the business forecasting device to perform business forecasting.

[0084] Business forecasting requires feature data as input, and the target feature data is the feature data required to realize the business forecasting corresponding to the business forecasting request.

[0085] Specifically, the business prediction device receives a business prediction request sent by a user; when a business prediction request is received, the business prediction device parses the business prediction request, determines the feature data required to realize the business prediction corresponding to the business prediction request, and obtains the target feature data required for the business prediction request.

[0086] In practical applications, the number of target feature data is at least one.

[0087] Step S204: Determine the target cache node that caches the target feature data among multiple cache nodes.

[0088] Among them, see Figure 1 The business forecasting system in the business forecasting device includes multiple cache nodes, each of which caches at least one feature data. In practical applications, the feature data cached by each cache node can be completely different or partially the same.

[0089] The feature data cached by each cache node is obtained based on the relationship information between the feature data, and the relationship information is obtained based on multiple historical business prediction requests.

[0090] For specific applications, please refer to Figure 1 The business forecasting system in the business forecasting device also includes a control node. The relationship information between various feature data is obtained by parsing multiple historical business forecasting requests through the control node.

[0091] Specifically, the business prediction device determines the cache node containing the target feature data among multiple cache nodes based on the cache mapping relationship between feature data and cache nodes.

[0092] The cache mapping relationship is used to characterize the mapping relationship between each cache node and the feature data cached on that cache node.

[0093] In practical applications, the number of target cache nodes is at least one.

[0094] For example, see Figure 1 The target feature data required for the business prediction request can be entirely cached on cache node 1, so cache node 1 is the target cache node for the business prediction request; the target feature data required for the business prediction request can also be partially cached on cache node 1 and partially cached on cache node 2, so both cache node 1 and cache node 2 are target cache nodes for the business prediction request.

[0095] Step S206: Obtain target feature data from the target cache node.

[0096] Specifically, the business prediction device obtains all target feature data required for the business prediction request from each target cache node, and sends all target feature data to the prediction node. The prediction node performs business prediction corresponding to the business prediction request based on the business prediction model it carries and the received target feature data.

[0097] Among them, see Figure 1 The business forecasting system in the business forecasting device also includes forecasting nodes, which are equipped with pre-trained business forecasting models. The forecasting nodes are used to make business forecasts corresponding to business forecasting requests based on the onboard business forecasting models and target feature data.

[0098] In the above-described feature data acquisition method, firstly, upon receiving a business prediction request, the business prediction device determines the target feature data required by the business prediction request; then, the business prediction device identifies a target cache node among multiple cache nodes that caches the target feature data; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is obtained based on multiple historical business prediction requests; finally, the business prediction device obtains the target feature data from the target cache node; the prediction node is used to perform business prediction corresponding to the business prediction request based on the business prediction model it carries and the target feature data. In this way, cache nodes are used to cache feature data. By identifying the target cache node among multiple cache nodes that caches the target feature data required for the business prediction request, the target feature data required for the business prediction request can be obtained from the target cache node. Then, based on the business prediction model carried by the prediction node and the obtained target feature data, business prediction corresponding to the business prediction request can be performed. Based on the feature data acquisition method of the above process, by caching the feature data on the cache node that is in the same business prediction device as the business prediction model, the target feature data can be obtained on the cache node that is in the same business prediction device as the business prediction model. Therefore, it no longer relies on network I / O to obtain feature data from external data sources, thereby shortening the time spent obtaining feature data in high-concurrency scenarios.

[0099] In one exemplary embodiment, such as Figure 3 As shown, step S202, upon receiving a business prediction request, determines the target feature data required for the business prediction request, and step S204, obtains the target feature data from the target cache node. Specifically, this includes the following steps:

[0100] Step S302: Upon receiving a business prediction request through the first cache node, the target feature data is determined through the first cache node.

[0101] Step S304: If the first cache node is the target cache node, then the target feature data is read from the first cache node and sent to the prediction node.

[0102] Step S306: If the second cache node is the target cache node, then the business prediction request is forwarded to the second cache node through the first cache node.

[0103] Step S308: Read the target feature data from the second cache node and send the target feature data to the prediction node.

[0104] The first cache node is any one of the multiple cache nodes; the second cache node is any one of the multiple cache nodes other than the first cache node.

[0105] Among them, in such Figure 1 In the business forecasting system shown, any cache node is used to receive business forecasting requests.

[0106] Specifically, the business prediction device receives a business prediction request sent by a user through any one of the cache nodes; when a business prediction request is received on any one of the cache nodes, the business prediction device designates that cache node as the first cache node.

[0107] Then, the first cache node parses the business prediction request, obtains the target feature data required by the business prediction request, and determines the cache node that caches the target feature data among multiple cache nodes, thus obtaining the target cache node.

[0108] If the first cache node is also the target cache node, meaning that the first cache node caches the target feature data, then the first cache node directly reads the target feature data it has cached and sends the read target feature data to the prediction node.

[0109] If the target feature data is cached on any cache node other than the first cache node, then the first cache node designates that cache node as the second cache node, and the second cache node is designated as the target cache node. Then refer to... Figure 1 The first cache node forwards the business prediction request to the second cache node. After receiving the forwarded business prediction request, the second cache node parses the business prediction request, obtains the target feature data required by the business prediction request, and then directly reads the target feature data that it has cached, and sends the read target feature data to the prediction node.

[0110] In this embodiment, by receiving and parsing the business prediction request through the cache node, the target feature data required by the business prediction request can be determined; through the cache node, the target cache node containing the target feature data can be identified among multiple cache nodes; by reading the feature data from the cache node and forwarding the business prediction request, the target feature data can be read from multiple cache nodes; based on the above process, the target feature data can be obtained from the cache node located in the same business prediction device as the business prediction model, thus eliminating the need to rely on network I / O to obtain feature data from external data sources, thereby shortening the time spent obtaining feature data in high-concurrency scenarios.

[0111] In one exemplary embodiment, each cache node also caches the cache mapping relationship between feature data and cache nodes.

[0112] The cache mapping relationship is used to characterize the mapping relationship between each cache node and the feature data cached on that cache node; in specific applications, the cache mapping relationship is determined based on the relationship information between each feature data.

[0113] Step S204 above, determining the target cache node containing the target feature data among multiple cache nodes, specifically includes the following: determining the target cache node among multiple cache nodes through the first cache node and the cache mapping relationship cached on the first cache node;

[0114] The feature data acquisition method provided in this application also includes the following for acquiring target feature data from external data sources: if no target feature data is cached on any of the cache nodes, the target feature data is acquired by accessing the data source used to store each feature data through the first cache node.

[0115] The data source is located outside the business forecasting device; in specific applications, the data source includes, but is not limited to, a Redis database.

[0116] Specifically, the first cache node determines the cache node that caches the target feature data among multiple cache nodes based on its own cache mapping relationship, and obtains the target cache node; if the first cache node determines that none of the cache nodes cache the target feature data based on its own cache mapping relationship, then the first cache node accesses the data source outside the business prediction device that is used to store the feature data, and obtains the target feature data from the data source.

[0117] For example, suppose the target feature data includes feature data 1, feature data 2, and feature data 3; suppose there are multiple cache nodes including cache node 1, cache node 2, and cache node 3, where the first cache node is cache node 1, and cache node 1 finds that feature data 1 is cached on cache node 1 and feature data 2 is cached on cache node 2 based on the cache mapping relationship, while cache node 1, cache node 2, and cache node 3 do not cache feature data 3; then, on the one hand, cache node 1 reads feature data 1 from itself; on the other hand, cache node 1 forwards the business prediction request to cache node 2, which reads feature data 2 from itself; and on yet another hand, cache node 1 accesses an external data source and reads feature data 3 from the external data source.

[0118] In this embodiment, when none of the cache nodes have cached the target feature data, the target feature data can be obtained from the external data source by accessing the external data source through the first cache node.

[0119] In an exemplary embodiment, when the first cache node forwards a business prediction request to the second cache node, it adds a forwarding identifier to the business prediction request so that the second cache node knows that the received business prediction request is a forwarded business prediction request. If the received business prediction request is a forwarded business prediction request, the second cache node will not trigger access to the external data source.

[0120] In one exemplary embodiment, such as Figure 4 As shown, in step S202 above, before determining the target feature data required by the business prediction request upon receiving it, the following steps are further included: constructing a cache mapping relationship between feature data and cache nodes, and caching the feature data based on the cache mapping relationship.

[0121] Step S402: Obtain multiple historical business prediction requests, and determine the relationship information between the various feature data based on the multiple historical business prediction requests.

[0122] Step S404: Based on the relationship information, construct the cache mapping relationship between feature data and cache nodes, and distribute the cache mapping relationship to each cache node.

[0123] Step S406: Through each cache node and cache mapping relationship, each cache node is used to access the data source for storing each feature data based on the cache mapping relationship, obtain the corresponding feature data from the data source and cache it.

[0124] The data source is located outside the business forecasting equipment.

[0125] The information relating to each feature data includes, but is not limited to, the cache priority and correlation between each feature data.

[0126] See Figure 1 The business forecasting system in the business forecasting device also includes a control node, which is used to manage the cache node. Specifically, the control node acquires multiple historical business forecasting requests, parses each historical business forecasting request to obtain the target feature data required for each historical business forecasting request, and then performs statistical analysis on the target feature data required for each historical business forecasting request to obtain the relationship information between the feature data.

[0127] Next, the control node constructs a cache mapping relationship between the feature data and the cache nodes based on the relationship information between each feature data, that is, it determines which feature data each cache node needs to cache, and sends the cache mapping relationship to each cache node.

[0128] After receiving the cache mapping relationship, each cache node accesses the data source outside the business prediction device that is used to store various feature data, retrieves the feature data that it needs to cache from the data source, and caches it.

[0129] In this embodiment, by parsing multiple historical business prediction requests by the control node, the cache mapping relationship between feature data and cache nodes can be obtained, thereby determining the feature data that each cache node needs to cache. By distributing the cache mapping relationship by the control node, each cache node can also be cached accurately.

[0130] In one exemplary embodiment, the business prediction device receives a business prediction request through any one of the cache nodes; the cache node is used to report the received business prediction request to the business prediction device.

[0131] like Figure 5 As shown, the feature data acquisition method provided in this application further includes the following steps for updating the cache mapping relationship:

[0132] Step S502: Receive the business prediction request reported by the cache node.

[0133] Step S504: Based on the business forecast request, update the relationship information to obtain the updated relationship information.

[0134] Step S506: Based on the updated relationship information, update the cache mapping relationship to obtain the updated cache mapping relationship.

[0135] Step S508: Distribute the updated cache mapping relationship to each cache node.

[0136] Specifically, the cache node that receives the business prediction request reports the business prediction request to the control node in the business prediction device; the control node receives the business prediction request and updates the relationship information between each feature data based on the business prediction request to obtain the updated relationship information; then, based on the updated relationship information, the control node updates the cache mapping relationship between the feature data and the cache node to obtain the updated cache mapping relationship, and sends the updated cache mapping relationship to each cache node.

[0137] After receiving the updated cache mapping relationship, each cache node determines whether the feature data it needs to cache has changed. If it has changed, it performs the corresponding operation. For example, if it needs to add new feature data to be cached, the cache node accesses the external data source, obtains the corresponding feature data, and caches it. If it needs to delete the cached feature data, the cache node deletes the corresponding feature data.

[0138] In this embodiment, by reporting the received business prediction requests by the cache nodes, the control nodes can update the relationship information between each feature data, thereby updating the cache mapping relationship between the feature data and the cache nodes, and thus optimizing and adjusting the distribution of the cached feature data on each cache node.

[0139] In one exemplary embodiment, the relationship information between the feature data includes at least the cache priority between the feature data.

[0140] Among them, cache priority is used to characterize the caching order of each feature data by the business prediction device; for example, the business prediction device prioritizes caching feature data with higher cache priority.

[0141] In practical applications, cache priority can be a priority index such as priority level or priority score.

[0142] like Figure 6 As shown, in step S404 above, based on relational information, the cache mapping relationship between feature data and cache nodes is constructed, specifically including the following steps:

[0143] Step S602: Based on the caching priority among the feature data, select the feature data to be cached from the feature data whose corresponding caching priority is greater than or equal to the preset priority threshold.

[0144] Step S604: Based on the feature data to be cached, determine the list of feature data to be cached for each cache node, and obtain the cache mapping relationship based on the list of feature data to be cached for each cache node.

[0145] In this application, feature data exists in the form of data item-data value. The list of feature data to be cached for each cache node includes the data items of the feature data that the cache node needs to cache.

[0146] Specifically, the control node identifies the feature data whose corresponding cache priority is greater than or equal to a preset priority threshold as the feature data to be cached; then, based on each feature data to be cached, the control node determines the list of feature data to be cached for each cache node; next, the control node combines the list of feature data to be cached for each cache node to obtain the cache mapping relationship between the feature data and the cache node.

[0147] In practical applications, after receiving the cache mapping relationship, the cache node accesses the external data source based on the list of feature data to be cached corresponding to itself in the cache mapping relationship, retrieves the data values ​​corresponding to each data item in the list of feature data to be cached from the external data source, and caches them.

[0148] In this embodiment, the control node can determine the cache mapping relationship based on the priority order of each feature data, thereby enabling the feature data that is prioritized for caching to be cached in the cache node.

[0149] In one exemplary embodiment, such as Figure 7 As shown, in step S402 above, the relationship information between various feature data is determined based on multiple historical business prediction requests, specifically including the following steps:

[0150] Step S702: Determine the access frequency of each feature data based on multiple historical business prediction requests.

[0151] Step S704: Determine the cache priority among the feature data based on the access frequency of each feature data.

[0152] Specifically, the control node parses each historical business prediction request to obtain the target feature data required for each historical business prediction request. Then, it performs statistics on the target feature data required for each historical business prediction request to obtain the access frequency of each feature data. Finally, it determines the cache priority among the feature data based on the access frequency of each feature data.

[0153] It is easy to understand that in this application, access frequency and cache priority are positively correlated, that is, the higher the access frequency of feature data, the higher its cache priority.

[0154] In this embodiment, the control node determines the cache priority based on the access frequency, giving priority to caching feature data with higher access frequency in the cache node, which can further avoid excessive network I / O pressure caused by frequent access to external data sources.

[0155] In one exemplary embodiment, the relationship information between the feature data includes at least the correlation between the feature data.

[0156] Among them, the correlation between the target feature data required for the same business prediction request is often quite large.

[0157] like Figure 8 As shown, in step S404 above, based on relational information, the cache mapping relationship between feature data and cache nodes is constructed, specifically including the following steps:

[0158] Step S802: Based on the correlation between the feature data, determine multiple sets of feature data to be cached from the feature data.

[0159] Step S804: Based on multiple sets of feature data to be cached, determine the list of feature data to be cached for each cache node, and obtain the cache mapping relationship based on the list of feature data to be cached for each cache node.

[0160] Among them, the correlation between each feature data in each set of feature data to be cached is greater than or equal to a preset correlation threshold.

[0161] Each cache node's list of features to be cached corresponds to a cached feature data set.

[0162] Specifically, the control node parses each historical service prediction request to obtain the target feature data required for each request. Then, the control node sets a high correlation between the target feature data within the same historical service prediction request. Next, the control node stores feature data with correlation greater than or equal to a preset correlation threshold into a set of feature data to be cached, resulting in multiple sets of feature data to be cached. Then, the control node assigns each set of feature data to a cache node, thus obtaining a list of feature data to be cached for each cache node. Finally, the control node combines the lists of feature data to be cached from all cache nodes to obtain the cache mapping relationship between feature data and cache nodes.

[0163] In practical applications, after receiving the cache mapping relationship, the cache node accesses the external data source based on the list of feature data to be cached corresponding to itself in the cache mapping relationship, retrieves the data values ​​corresponding to each data item in the list of feature data to be cached from the external data source, and caches them.

[0164] In this embodiment, the control node can determine the cache mapping relationship based on the correlation between each feature data, so as to cache the target feature data required by the same business prediction request on the same cache node as much as possible, so that a business prediction request can obtain the corresponding target feature data on a cache node as much as possible.

[0165] In one exemplary embodiment, such as Figure 9 As shown, the feature data acquisition method provided in this application also includes the following steps for rearranging and expanding cache nodes:

[0166] Step S902: Monitor the space occupancy rate of each cache node.

[0167] Step S904: If the space occupancy rate of any cache node is greater than or equal to the preset occupancy rate threshold, the cache node is used as the source node for data migration, and a new cache node is created as the target node for data migration.

[0168] Step S906: Send a synchronization command to the target node so that the target node can synchronize at least one feature data from the source node.

[0169] Step S908: Send a deletion command to the source node so that the source node deletes the feature data synchronized by the target node.

[0170] Specifically, the control node of the business forecasting system in the business forecasting device monitors the space occupancy rate of each cache node in real time. If the space occupancy rate of any cache node is greater than or equal to a preset occupancy rate threshold, the control node performs data migration for that cache node. The specific process is as follows:

[0171] The control node first uses the cache node as the source node for data migration, and then creates a new cache node in the business forecasting system as the target node for data migration.

[0172] Then, the control node sends a synchronization command to the target node. The target node responds to the synchronization command by synchronizing at least one feature data from the source node and sends a response message back to the control node after successful synchronization.

[0173] Next, after receiving the response information from the target node, the control node sends a deletion command to the source node. The source node responds to the deletion command and deletes the feature data synchronized by the target node.

[0174] In this embodiment, the control node can rearrange and expand the cache nodes by monitoring the space occupancy rate of the cache nodes and performing data migration operations on the cache nodes, thereby making the feature data evenly distributed on each cache node.

[0175] Based on the same inventive concept, this application also provides a business prediction system for implementing the feature data acquisition method described above. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more business prediction system embodiments provided below can be found in the limitations of the feature data acquisition method described above, and will not be repeated here.

[0176] In one exemplary embodiment, such as Figure 1 As shown, a business forecasting system is provided, including: a control node, a cache node, and a forecasting node. The number of cache nodes is multiple. The business forecasting system is mounted on a business forecasting device, wherein:

[0177] The control node is used to obtain the relationship information between various feature data based on multiple historical business prediction requests, and to obtain the feature data cached by each cache node based on the relationship information between various feature data; each cache node caches at least one feature data.

[0178] A cache node is used to determine the target feature data required by a business prediction request when a business prediction request is received, identify the target cache node that caches the target feature data among multiple cache nodes, and retrieve the target feature data from the target cache node.

[0179] Prediction nodes are used to make business predictions corresponding to business prediction requests based on the business prediction model they are equipped with and the target feature data.

[0180] In one exemplary embodiment, the first cache node is further configured to determine target feature data upon receiving a business prediction request; the first cache node is any one of a plurality of cache nodes.

[0181] The first cache node is also used to read target feature data from the first cache node and send the target feature data to the prediction node if the first cache node is the target cache node; if the second cache node is the target cache node, the business prediction request is forwarded to the second cache node through the first cache node; the second cache node is any one of the multiple cache nodes other than the first cache node.

[0182] The second cache node is used to read the target feature data from the second cache node and send the target feature data to the prediction node.

[0183] In one exemplary embodiment, each cache node also caches the cache mapping relationship between feature data and cache nodes.

[0184] The first cache node is also used to determine the target cache node among multiple cache nodes through the cache mapping relationship of the cache.

[0185] The first cache node is also used to retrieve the target feature data from the data source that stores the feature data if none of the cache nodes have cached the target feature data; the data source is located outside the business prediction device.

[0186] In one exemplary embodiment, the control node is further configured to acquire multiple historical business prediction requests, determine the relationship information between each feature data based on the multiple historical business prediction requests, construct a cache mapping relationship between the feature data and the cache nodes based on the relationship information, and distribute the cache mapping relationship to each cache node.

[0187] The cache node is also used to access the data source for storing various feature data through the cache mapping relationship, retrieve the corresponding feature data from the data source and cache it; the data source is located outside the business prediction device.

[0188] In one exemplary embodiment, the service prediction device receives a service prediction request through any one of the cache nodes.

[0189] The cache node is used to report the received business forecasting requests to the business forecasting device.

[0190] The control node is also used to receive business prediction requests reported by cache nodes; based on the business prediction requests, update the relationship information to obtain the updated relationship information; based on the updated relationship information, update the cache mapping relationship to obtain the updated cache mapping relationship; and distribute the updated cache mapping relationship to each cache node.

[0191] In one exemplary embodiment, the relationship information between the feature data includes at least the cache priority between the feature data.

[0192] The control node is also used to filter out cacheable feature data from each feature data based on the cache priority between each feature data, and to determine the cacheable feature data list for each cache node based on the cacheable feature data, and to obtain the cache mapping relationship based on the cacheable feature data list for each cache node.

[0193] In one exemplary embodiment, the control node is further configured to determine the access frequency of each feature data based on multiple historical business prediction requests; and to determine the cache priority among the feature data based on the access frequency of each feature data.

[0194] In one exemplary embodiment, the relationship information between the feature data includes at least the correlation between the feature data.

[0195] The control node is also used to determine multiple sets of feature data to be cached from each feature data based on the correlation between each feature data; the correlation between each feature data in each set of feature data to be cached is greater than or equal to a preset correlation threshold; based on the multiple sets of feature data to be cached, the list of feature data to be cached for each cache node is determined, and based on the list of feature data to be cached for each cache node, the cache mapping relationship is obtained; the list of feature data to be cached for each cache node corresponds to a set of cached feature data.

[0196] In one exemplary embodiment, the control node is further configured to monitor the space occupancy rate of each cache node. If the space occupancy rate of any cache node is greater than or equal to a preset occupancy rate threshold, the cache node is used as the source node for data migration, and a new cache node is created as the target node for data migration.

[0197] The control node is also used to send synchronization commands to the target node.

[0198] The target node is used to synchronize at least one feature data from the source node in response to a synchronization command.

[0199] The control node is also used to send deletion commands to the source node.

[0200] The source node is used to delete the feature data synchronized by the target node in response to a delete command.

[0201] Each module in the aforementioned business forecasting system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the business forecasting device in hardware form or independent of it, or stored in the memory of the business forecasting device in software form, so that the processor can call and execute the corresponding operations of each module.

[0202] To more clearly illustrate the feature data acquisition method and business prediction system provided in the embodiments of this application, a specific embodiment is described below. However, it should be understood that the embodiments of this application are not limited thereto. In one exemplary embodiment, this application also provides a high-performance large model service feature data acquisition method, specifically including the following:

[0203] This embodiment applies to a business forecasting system based on a large-scale forecasting model architecture. (See also...) Figure 1 In this embodiment, a control node and multiple cache nodes are added to the large prediction model, and all nodes of the business prediction system are deployed on the same business prediction device.

[0204] The control node, on the one hand, requests the data source to obtain the feature data storage status of the data source, and on the other hand, receives prediction requests sent by users from the cache nodes. It then analyzes which feature data or combinations of feature data each cache node needs to synchronize and cache from the data source, converts this into a cache mapping relationship, and sends it to the cache nodes. Furthermore, the control node monitors the memory usage of the cache nodes in real time. When memory usage becomes excessive, it automatically expands the cache nodes and rearranges the cached feature data of each cache node to keep the memory usage of each cache node within a reasonable range.

[0205] The cache node receives the cache mapping relationship from the control node, initiates a request to the data source, and synchronizes and caches feature data or combinations of feature data. When a cache node receives a prediction request from a user, it queries the cache mapping relationship to determine if the feature data or combination of feature data required for the prediction request exists on this cache node. If it does, it directly reads the required feature data or combination of feature data for the prediction request; otherwise, it further queries the cache mapping relationship to determine which cache node contains the required feature data or combination of feature data for the prediction request and forwards the prediction request to that cache node. In addition, the cache node also reports the prediction request to the control node, allowing the control node to optimize and adjust the cache mapping relationship.

[0206] In practical applications, the control node determines the cache mapping relationship based on the access frequency of feature data.

[0207] In practical applications, the control node also determines the cache mapping relationship based on the correlation between feature data (whether they are feature data required by the same prediction request), so as to allow the prediction request to find all the required feature data or combination of feature data on a cache node as much as possible.

[0208] The high-performance large-model service feature data acquisition method proposed in this embodiment is applicable to fields such as logistics that require large models to improve business performance. This embodiment distributes massive feature data or combinations of feature data evenly across various cache nodes, avoiding network I / O bottlenecks caused by queries to middleware such as Redis. This significantly improves the query performance of feature data, thereby enhancing the performance of the large-model service.

[0209] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0210] In one exemplary embodiment, a business forecasting device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the business forecasting device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database caches feature data. The I / O interfaces allow the processor to exchange information with external devices. The communication interface allows communication with external terminals via a network connection. The business forecasting device carries a business forecasting system; when the computer program is executed by the processor, it implements a feature data acquisition method applicable to the business forecasting system.

[0211] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the business forecasting device to which the present application is applied. A specific business forecasting device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0212] In one exemplary embodiment, a business forecasting device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0213] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0214] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0215] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0216] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0217] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for acquiring feature data, characterized in that, The method, applied to a business forecasting device including cache nodes and forecasting nodes, wherein the number of cache nodes is multiple, includes: Upon receiving a business forecasting request, determine the target feature data required for the business forecasting request; Among multiple cache nodes, a target cache node that caches the target feature data is determined; each cache node caches at least one feature data, and the feature data cached by each cache node is obtained based on the relationship information between the feature data, which is based on multiple historical business prediction requests. The target feature data is obtained from the target cache node; the prediction node is used to perform business prediction corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

2. The method according to claim 1, characterized in that, Upon receiving a business forecasting request, determining the target feature data required for the business forecasting request includes: Upon receiving the business prediction request through the first cache node, the target feature data is determined through the first cache node; the first cache node is any one of the plurality of cache nodes. The step of obtaining the target feature data from the target cache node includes: If the first cache node is the target cache node, then the target feature data is read from the first cache node and sent to the prediction node. If the second cache node is the target cache node, then the business prediction request is forwarded to the second cache node through the first cache node; the second cache node is any one of the plurality of cache nodes other than the first cache node; The target feature data is read from the second cache node and sent to the prediction node.

3. The method according to claim 2, characterized in that, Each cache node also caches the cache mapping relationship between feature data and cache nodes; The step of determining the target cache node that caches the target feature data among multiple cache nodes includes: The target cache node is determined from the plurality of cache nodes by using the first cache node and the cache mapping relationship cached on the first cache node; The method further includes: If none of the cache nodes have cached the target feature data, the first cache node is used to access the data source used to store the feature data, and the target feature data is obtained from the data source; the data source is located outside the business prediction device.

4. The method according to claim 1, characterized in that, Before determining the target feature data required for the business forecasting request upon receiving it, the process further includes: Obtain multiple historical business prediction requests, and determine the relationship information between the various feature data based on the multiple historical business prediction requests; Based on the relationship information, a cache mapping relationship between feature data and cache nodes is constructed, and the cache mapping relationship is distributed to each cache node; Through each cache node and the cache mapping relationship, the data source used to store the feature data is accessed, the corresponding feature data is obtained from the data source and cached; the data source is located outside the business prediction device.

5. The method according to claim 4, characterized in that, The business prediction device receives the business prediction request through any one of the cache nodes; the cache node is used to report the received business prediction request to the business prediction device. The method further includes: Receive the service prediction request reported by the cache node; Based on the business prediction request, update the relationship information to obtain the updated relationship information; Based on the updated relationship information, the cache mapping relationship is updated to obtain the updated cache mapping relationship; The updated cache mapping relationship is then distributed to each cache node.

6. The method according to claim 4, characterized in that, The relationship information between the various feature data includes at least the cache priority between the various feature data; The step of constructing a cache mapping relationship between feature data and cache nodes based on the relationship information includes: Based on the caching priority among the various feature data, select the feature data to be cached from the feature data whose corresponding caching priority is greater than or equal to a preset priority threshold; Based on the feature data to be cached, a list of feature data to be cached for each cache node is determined, and based on the list of feature data to be cached for each cache node, the cache mapping relationship is obtained.

7. The method according to claim 6, characterized in that, The process of determining the relationship information between various feature data based on the multiple historical business prediction requests includes: Based on the multiple historical business prediction requests, the access frequency of each feature data is determined; Based on the access frequency of each feature data, the cache priority among the feature data is determined.

8. The method according to claim 4, characterized in that, The relationship information between the various feature data includes at least the correlation between the various feature data; The step of constructing a cache mapping relationship between feature data and cache nodes based on the relationship information further includes: Based on the correlation between the feature data, multiple sets of feature data to be cached are determined from the feature data; the correlation between the feature data in each set of feature data to be cached is greater than or equal to a preset correlation threshold. Based on the multiple sets of feature data to be cached, a list of feature data to be cached for each cache node is determined, and based on the list of feature data to be cached for each cache node, the cache mapping relationship is obtained; the list of feature data to be cached for each cache node corresponds to a set of cache feature data.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Monitor the space occupancy rate of each cache node; If the space occupancy rate of any cache node is greater than or equal to a preset occupancy rate threshold, the cache node will be used as the source node for data migration, and a new cache node will be created as the target node for data migration. A synchronization command is issued to the target node so that the target node synchronizes at least one feature data from the source node; A deletion command is issued to the source node so that the source node deletes the feature data synchronized by the target node.

10. A business forecasting system, characterized in that, The business forecasting system includes control nodes, cache nodes, and forecasting nodes, and the number of cache nodes is multiple. The control node is used to obtain the relationship information between various feature data based on multiple historical business prediction requests, and to obtain the feature data cached by each cache node based on the relationship information between the various feature data; each cache node caches at least one feature data. The cache node is used to determine the target feature data required by the business prediction request when a business prediction request is received, determine the target cache node that caches the target feature data among multiple cache nodes, and obtain the target feature data from the target cache node. The prediction node is used to perform business predictions corresponding to the business prediction request based on the business prediction model it carries and the target feature data.

11. A business forecasting device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images