Multi-stage model-based reasoning system and method, readable medium and electronic equipment

Abstract

The invention relates to the technical field of computers, and discloses an inference system and method based on a multi-stage model, a readable medium and electronic equipment. The system comprises an intermediate cache module and a model reasoning module, wherein the model reasoning module comprises a feature construction sub-module and a result reasoning sub-module. When a reasoning request is received, an intermediate result can be queried in an intermediate cache module based on a cache identifier in the reasoning request, and a result reasoning module can directly perform reasoning based on the intermediate result under the condition of successful query to obtain a reasoning result corresponding to the reasoning request; and under the condition that the query fails, the feature construction sub-module constructs an intermediate result based on the original features in the reasoning request, and the intermediate cache module caches the intermediate result based on the cache identifier. The system provides a unified deployment platform of feature processing and model calculation, additional overhead can be avoided in different stages of the whole process, and most reasoning requests can multiplex cached intermediate results, so that the overhead of the system is further reduced.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more specifically, to a reasoning system based on a multi-stage model, a reasoning method based on a multi-stage model, a computer-readable medium, and an electronic device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims, and no description herein is admitted to be prior art by inclusion in this section.

[0003] Model inference systems usually use model parameters trained with sample data to predict and classify unknown data based on inference calculations. They are widely used in recommendation, recognition, generation and other fields.

[0004] In practical applications, it is usually necessary to perform feature query, extraction, and assembly processing on unknown data, and then provide the obtained features to the model reasoning system for calculation to achieve the reasoning task. Therefore, in this solution, feature construction is required every time processing is performed, and the complete features are provided to the model reasoning system. However, due to different data sources, the data format, language, and logic are usually not unified, which leads to a large amount of memory duplication during the processing process, wasting computing resources and weakening service throughput. Summary of the invention

[0005] In this context, embodiments of the present disclosure are intended to provide a multi-stage model-based reasoning system, a multi-stage model-based reasoning method, a computer-readable medium, and an electronic device.

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an inference system based on a multi-stage model, which may include an intermediate cache module and a model inference module, the model inference module including a feature construction submodule and a result inference submodule; the intermediate cache module is used to query the intermediate result corresponding to the cache identifier in response to the inference request; the inference request includes the original feature, and the cache identifier corresponding to the original feature; the result inference submodule is used to perform the inference calculation corresponding to the inference request based on the intermediate result to obtain the inference result when the query of the intermediate result based on the cache identifier is successful; the feature construction submodule is used to construct the intermediate result corresponding to the inference request based on the original feature when the query of the intermediate result based on the cache identifier fails; and the result inference submodule is further used to perform the inference calculation corresponding to the inference request based on the intermediate result to obtain the inference result; the intermediate cache module is further used to cache the intermediate result based on the cache identifier corresponding to the original feature.

[0007] Optionally, the intermediate cache module is further configured to determine that querying the intermediate result based on the cache identifier is successful when the intermediate result is queried based on the cache identifier and the intermediate result is in a valid state.

[0008] Optionally, the valid state includes that the creation time of the intermediate result is within the cache validity period, and the cache validity period is set according to the update frequency of the original feature; or, the valid state includes that the feature version of the intermediate result is consistent with the feature version of the original feature.

[0009] Optionally, the intermediate cache module is further configured to delete the intermediate result whose most recently queried interval duration is greater than or equal to the reserved interval duration when the remaining cache space is less than or equal to the reserved cache space.

[0010] Optionally, the feature construction submodule is specifically used to perform feature query and extraction in the original features based on the inference request to obtain the inference features when the query of the intermediate result based on the cache identifier fails, and to perform feature independent matrix calculation on the inference features to obtain the intermediate result corresponding to the inference request.

[0011] Optionally, the inference request is a user recommendation request, and the original features include original user features and original recommendation candidate features; a feature construction submodule is specifically used to construct the user intermediate result corresponding to the inference request based on the original user features, and to construct the recommendation candidate intermediate result corresponding to the inference request based on the original recommendation candidate features when the query of the intermediate result based on the cache identifier fails; a result inference submodule is specifically used to cross-calculate the user intermediate result and the recommendation candidate intermediate result to obtain the inference result; and an intermediate cache module is specifically used to cache the user intermediate result based on the cache identifier corresponding to the original user feature, and to cache the recommendation candidate intermediate result based on the cache identifier corresponding to the original recommendation candidate feature.

[0012] Optionally, the result reasoning submodule is used to perform cross calculations on the user intermediate results and the recommendation candidate intermediate results in the intermediate results to obtain reasoning results when the query of the intermediate results based on the cache identifier is successful.

[0013] According to a second aspect of an embodiment of the present disclosure, a multi-stage model-based reasoning method is provided. The method can be implemented based on the multi-stage model-based reasoning system of the aforementioned first aspect. The method can include: in response to a reasoning request, querying an intermediate result corresponding to a cache identifier; the reasoning request includes original features, and a cache identifier corresponding to the original features; in the case where the query of the intermediate result based on the cache identifier is successful, performing the reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result; in the case where the query of the intermediate result based on the cache identifier fails, constructing the intermediate result corresponding to the reasoning request based on the original features; and, performing the reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result; caching the intermediate result based on the cache identifier corresponding to the original feature.

[0014] Optionally, querying the intermediate result corresponding to the cache identifier includes: when the intermediate result is queried based on the cache identifier and the intermediate result is in a valid state, determining that querying the intermediate result based on the cache identifier is successful.

[0015] Optionally, the valid state includes that the creation time of the intermediate result is within the cache validity period, and the cache validity period is set according to the update frequency of the original feature; or, the valid state includes that the feature version of the intermediate result is consistent with the feature version of the original feature.

[0016] Optionally, the method further includes: when the remaining cache space is less than or equal to the reserved cache space, deleting the intermediate result of the most recent query whose interval duration is greater than or equal to the reserved interval duration.

[0017] Optionally, in the event that querying the intermediate result based on the cache identifier fails, constructing the intermediate result corresponding to the reasoning request based on the original features, including: in the event that querying the intermediate result based on the cache identifier fails, performing feature query and extraction in the original features based on the reasoning request to obtain the reasoning feature; performing feature-independent matrix calculation on the reasoning feature to obtain the intermediate result corresponding to the reasoning request.

[0018] Optionally, the inference request is a user recommendation request, and the original features include original user features and original recommendation candidate features; in the case where the query of the intermediate result based on the cache identifier fails, constructing the intermediate result corresponding to the inference request based on the original features, including: in the case where the query of the intermediate result based on the cache identifier fails, constructing the user intermediate result corresponding to the inference request based on the original user features, and constructing the recommendation candidate intermediate result corresponding to the inference request based on the original recommendation candidate features; performing inference calculation corresponding to the inference request based on the intermediate result to obtain the inference result, including: performing cross calculation on the user intermediate result and the recommendation candidate intermediate result to obtain the inference result; and caching the intermediate result based on the cache identifier corresponding to the original feature includes: caching the user intermediate result based on the cache identifier corresponding to the original user feature, and caching the recommendation candidate intermediate result based on the cache identifier corresponding to the original recommendation candidate feature.

[0019] Optionally, when the query of the intermediate result based on the cache identifier is successful, the inference calculation corresponding to the inference request is performed based on the intermediate result to obtain the inference result, including: when the query of the intermediate result based on the cache identifier is successful, the user intermediate result and the recommended candidate intermediate result in the intermediate result are cross-calculated to obtain the inference result.

[0020] According to a third aspect of the embodiments of the present disclosure, there is provided a computer-readable medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned reasoning methods based on the multi-stage model is implemented.

[0021] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned multi-stage model-based reasoning methods by executing the executable instructions.

[0022] According to the reasoning system based on the multi-stage model of the embodiment of the present disclosure, the system includes an intermediate cache module and a model reasoning module, wherein the model reasoning module includes a feature construction submodule and a result reasoning submodule. When receiving a reasoning request, the intermediate result can be queried in the intermediate cache module based on the cache identifier in the reasoning request. If the query is successful, the result reasoning module can directly reason based on the intermediate result to obtain the reasoning result corresponding to the reasoning request; if the query fails, the feature construction submodule constructs the intermediate result based on the original feature in the reasoning request. On the one hand, the result reasoning submodule reasoning based on the intermediate result to obtain the reasoning result corresponding to the reasoning request, and on the other hand, the intermediate cache module caches the intermediate result based on the cache identifier. The system provides a unified deployment platform for feature processing and model calculation, which can realize feature pointer operations at different stages of the whole process, avoid additional overhead such as serialization, deserialization, network transmission, and split different reasoning task stages, so that most reasoning requests can reuse the cached intermediate results, without the need for feature query, extraction and assembly, and some preliminary reasoning calculations, shortening the amount of calculation and time required for a single request, and further reducing system overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0024] Figure 1 A schematic diagram of the structure of a multi-stage model-based reasoning system according to an embodiment of the present disclosure is shown.

[0025] Figure 2 A schematic diagram of the application of a multi-stage model-based reasoning system in user recommendation according to an embodiment of the present disclosure is shown.

[0026] Figure 3 A schematic diagram of a multi-stage model-based reasoning method according to an embodiment of the present disclosure is shown.

[0027] Figure 4 A schematic diagram of a computer readable medium illustrating an embodiment of the present disclosure is shown.

[0028] Figure 5 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0031] According to an embodiment of the present disclosure, a reasoning method based on a multi-stage model, a reasoning system based on a multi-stage model, a computer-readable medium, and an electronic device are provided.

[0032] In this document, any number of elements in the drawings is used for illustration rather than limitation, and any naming is used only for distinction and does not have any limiting meaning.

[0033] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0034] In related technologies, after the model inference system performs parameter training on sample data, it performs inference calculations on unknown data to obtain corresponding prediction results. In order to facilitate the understanding of the model inference system, it is usually necessary to perform feature processing on the unknown data, such as feature query, processing, and assembly, and provide the processed feature data to the model inference system. At this time, feature processing needs to be performed outside the model inference system before each inference calculation, which wastes computing resources. When splicing, a large number of features need to be copied to increase overhead, thereby affecting the throughput of the model system. The sources of unknown data are different, which may lead to inconsistencies in data logic, language, and format, making it difficult to unify the processing logic, and increasing the cost of feature extraction and calling within the system.

[0035] In response to the above problems, the embodiments of the present disclosure provide a reasoning system based on a multi-stage model, which integrates feature processing services, intermediate result services, and reasoning computing services, and can perform feature construction processing under unified logic, language, and format, cache and pointer call of intermediate results, and perform reasoning calculations based on this. The system reduces the cost of feature extraction and calling, can reuse intermediate results multiple times, optimize the utilization of computing resources, reduce system overhead, and improve throughput.

[0036] After introducing the basic principles of the present invention, various non-limiting embodiments of the present invention are described in detail below.

[0037] Exemplary application scenarios

[0038] It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principle of the present invention, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0039] The multi-stage model-based reasoning method of the disclosed embodiment can be applied to a variety of application scenarios involving model execution reasoning tasks.

[0040] In one application scenario, the reasoning task may be to recommend items (candidates) to users, where items may be commodities, music, information, or other users, etc. The original features of the user and the item are input into the reasoning system through the reasoning request, and then the intermediate results can be constructed based on the original features for further calculation to obtain the recommendation results, and the intermediate results can be cached. In subsequent other reasoning requests, when the user is involved, the intermediate results of the user in the cache can be reused for calculation, and when the item is involved, the intermediate results of the item in the cache can be reused for calculation.

[0041] In another application scenario, other reasoning tasks may be involved, such as text generation, recognition and fusion, speech generation, recognition and fusion, image generation, recognition and fusion, etc., and the embodiments of the present disclosure do not impose specific restrictions on this.

[0042] Exemplary Systems

[0043] In combination with the above application scenarios, Figure 1 A multi-stage model-based reasoning method according to an exemplary embodiment of the present disclosure is described.

[0044] like Figure 1 The structure diagram of the reasoning system 100 based on the multi-stage model (hereinafter referred to as the reasoning system 100) according to the exemplary embodiment of the present disclosure is shown as follows. Figure 1 As shown, the reasoning system 100 may include an intermediate cache module 110 and a model reasoning module 120 , and the model reasoning module 120 may include a feature construction submodule 121 and a result reasoning submodule 122 .

[0045] In the disclosed embodiment, the reasoning system 100 can be generally divided into an intermediate cache module 110 and a model reasoning module 120. The intermediate cache module 110 can cache the intermediate results during the execution of the reasoning task so that the intermediate results can be directly reused in other subsequent reasoning tasks, omitting the preceding calculation steps. The model reasoning module 120 is divided into a feature construction submodule 121 and a result reasoning submodule 122 based on the execution process of the reasoning task. Feature processing and reasoning calculation can be implemented within an integrated system, avoiding the access pressure of external systems under multi-system docking.

[0046] Among them, the feature construction submodule 121 and the result reasoning submodule 122 are divided based on the split of the reasoning task stage, so the specific functional division is related to the complexity of the reasoning task and the system computing power support. When the calculation of the reasoning task is simple and the process is short, the feature construction submodule 121 can be responsible for feature engineering processing of the original features, and the result reasoning submodule 122 performs the reasoning calculation; when the calculation of the reasoning task is complex and the process is long, the feature construction submodule 121 can be responsible for performing preliminary reasoning calculations on the basis of feature engineering processing of the original features, and the result reasoning submodule 122 performs subsequent reasoning calculations. The stage division of the reasoning task makes the execution of the reasoning task more flexible.

[0047] In the disclosed embodiment, the inference system 100 deploys feature processing and inference calculation in the same process, avoiding cross-network transmission of features, serialization, deserialization, and frequent and large-scale memory application and release scenarios, and can achieve intermediate result caching and pointer transmission, which can effectively improve system performance. In practical applications, the inference system 100 can process the original features into the Tensor format required for forward reasoning during feature processing, avoiding the need to convert the input from Features format to Example format and from Example format to Tensor format in the related art. Multiple conversions in the process.

[0048] Furthermore, the data sources of the original features are different, and may come from offline data, real-time data, and online request data, etc. Different data sources usually correspond to different processing logics. For example, offline data can be processed on the spark platform, real-time data can be processed on the flink platform, and online request data can be processed in the C++ language environment, etc. At this time, due to differences in platforms or languages, multi-language code development is usually required, which may further lead to inconsistent processing logic. The reasoning system 100 can provide a calling interface for multiple platforms and languages ​​by defining a universal data protocol, and encapsulate the internal core to shield the calling details of different platforms.

[0049] For example, when the inference system 100 is implemented in C++, it can provide a C++ interface to support the processing of online request data, provide a Java interface to support the processing of offline data and real-time data, and extend the Python interface to support the reception of diversified data sources. On this basis, loading call files can also be provided by compilation, such as SO library files can be deployed in the C++ environment, and JAR packages can be deployed in the spark platform and the flink platform, so as to facilitate cross-platform integrated operation.

[0050] The above description is for example only. Those skilled in the art may select the development language of the reasoning system 100 according to actual conditions, and provide a method of unifying different formats, languages ​​and logics of data sources within the platform for feature processing and reasoning calculations. The embodiments of the present disclosure do not impose specific restrictions on this.

[0051] In the reasoning system 100, the intermediate cache module 110 is used to query the intermediate result corresponding to the cache identifier in response to the reasoning request; the reasoning request includes the original feature and the cache identifier corresponding to the original feature.

[0052] In the disclosed embodiment, the inference system 100 may receive an inference request, which is used to instruct the inference system 100 to start and execute a corresponding inference task. Each inference request may include an original feature and a cache identifier corresponding to the original feature. The cache identifier is used to indicate an intermediate result corresponding to the original feature in the cache. The intermediate result is generated after the inference system 100 performs feature processing on the original feature during the execution of the inference task, and may also include some intermediate stages of preliminary reasoning calculations, and its form is related to the reasoning task being performed. The inference system 100 can cache the intermediate results through the cache identifier for retrieval and reuse.

[0053] In the reasoning system 100, the result reasoning submodule 122 is used to perform the reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result when the query of the intermediate result based on the cache identifier is successful.

[0054] In the embodiment of the present disclosure, when the query of the intermediate result based on the cache identifier is successful, the reasoning system 100 has obtained the result generated by the original feature in the intermediate stage of the process of executing the reasoning task, and therefore the preceding steps of the intermediate stage can be omitted to directly perform the subsequent reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result.

[0055] In the reasoning system 100, the feature construction submodule 121 is used to construct an intermediate result corresponding to the reasoning request based on the original feature when the query of the intermediate result based on the cache identifier fails; and the result reasoning submodule 122 is also used to perform reasoning calculations corresponding to the reasoning request based on the intermediate result to obtain the reasoning result.

[0056] In the embodiment of the present disclosure, when the query of the intermediate result based on the cache identifier fails, the reasoning system 100 has not yet obtained the result generated by the original feature in the intermediate stage of the process of executing the reasoning task, so the feature construction submodule 121 can process the original feature to construct the intermediate result corresponding to the reasoning request. On the basis of the stage division of the reasoning task, it can be considered that the feature processing before the intermediate stage, and also including the preliminary reasoning calculation, are divided into the feature construction submodule 121 for execution, and the subsequent reasoning calculation after the intermediate stage is divided into the result reasoning submodule 122 for execution. On this basis, the intermediate result output by the feature construction submodule 121 can be further executed by the result reasoning submodule 122 to perform the subsequent reasoning calculation corresponding to the reasoning request, thereby outputting the reasoning result corresponding to the reasoning request.

[0057] In the reasoning system 100 , the intermediate cache module 110 is further used to cache the intermediate results based on the cache identifiers corresponding to the original features.

[0058] In the disclosed embodiment, after the feature construction submodule 121 constructs the intermediate result based on the original feature, the intermediate result can be cached based on the cache identifier corresponding to the original feature, so that the intermediate result can be queried based on the cache identifier.

[0059] In an optional embodiment of the present disclosure, the intermediate cache module 110 is further configured to determine that querying the intermediate result based on the cache identifier is successful when the intermediate result is queried based on the cache identifier and the intermediate result is in a valid state.

[0060] In the disclosed embodiment, a cache invalidation mechanism may be further set, and the intermediate result may be in a valid state or in an invalid state in the cache. When a query based on a cache identifier hits, if the intermediate result is in an invalid state, it can be considered that the query of the intermediate result based on the cache identifier has failed; when the hit intermediate result is in a valid state, it can be determined that the query of the intermediate result based on the cache identifier has succeeded. The cache invalidation mechanism can be set according to the actual storage structure, computing power limitations, business needs, etc., and is affected by factors such as write-read methods and storage space size. The disclosed embodiment does not impose specific restrictions on this.

[0061] In the disclosed embodiment, the intermediate results can be directly queried and called during the valid state to avoid invalid feature external query, parsing and extraction. In order to improve the cache performance of the intermediate results, different query and cache strategies can be adopted, such as synchronous cache query, thread pool asynchronous cache query, bucket cache, etc., which are not specifically limited in the disclosed embodiment.

[0062] In an optional embodiment of the present disclosure, the valid state includes that the creation time of the intermediate result is within the cache validity period, and the cache validity period is set according to the update frequency of the original feature.

[0063] In the disclosed embodiment, the original feature may be in dynamic update, and its update frequency is different based on different data sources, feature types, etc. In general, the original feature is immutable within a period of time when it is not updated, so its intermediate result is also immutable within a corresponding period of time, and after the original feature is updated, its intermediate result needs to be updated accordingly. Therefore, the cache validity period of the intermediate result can be set according to the update frequency of the original feature, and it can be assumed that the original feature has not reached the time point of the next update during the cache validity period, so the corresponding intermediate result creation time can be determined to be in a valid state within the cache validity period, otherwise it is in an invalid state. For example, when the original feature is offline data, the update frequency can be hourly or daily, and the cache validity period can be set to hourly or daily accordingly; when the original feature is real-time data, the update frequency can be second-level or minute-level, and the cache validity period can be set to second-level or minute-level accordingly, so that the intermediate result in the cache becomes invalid before the next update of the original feature. In general, the cache validity period can be less than or equal to the update interval duration, and the cache validity period can be set according to the next update time of the original feature indicated by the update frequency when the intermediate result is written to the cache. The cache validity period should not be too short to avoid triggering the intermediate results to become invalid when the original feature does not trigger the next update, which will reduce the cache hit rate.

[0064] In an optional embodiment of the present disclosure, the valid state includes that the feature version of the intermediate result is consistent with the feature version of the original feature.

[0065] In the disclosed embodiment, each time the original feature is updated, it is considered that its version has changed. Therefore, the consistency calculation can be performed on the feature version of the original feature and the feature version of the intermediate result to determine whether the intermediate result is in a valid state. When the versions are consistent, it can be determined that the original feature has not yet been updated to the next time and the intermediate result is still in a valid state; otherwise, it is determined that the original feature has been updated and the intermediate result is in an invalid state. The feature version can be indicated in the cache identifier and cached together with the intermediate result. By comparing feature versions, the cache hit rate can be guaranteed, update delays can be avoided, and the reasoning performance can be effectively improved.

[0066] In an optional embodiment of the present disclosure, the intermediate cache module 110 is further configured to delete the intermediate result whose most recently queried interval duration is greater than or equal to the reserved interval duration when the remaining cache space is less than or equal to the reserved cache space.

[0067] In the disclosed embodiment, the cache space size can also be managed, and the reserved cache space can be set based on the total cache space size. The access history of the cached intermediate results that are queried and hit is recorded. When the remaining cache space is less than or equal to the reserved cache space, the intermediate results whose interval duration of the most recent query is greater than or equal to the reserved interval duration are deleted according to the historical record to release their cache space. Among them, the size of the reserved cache space can be zero, or a value greater than zero and less than the total cache space. The reserved interval duration can be used to evaluate the probability of the intermediate result being queried. For example, when the interval duration of the most recent query of the intermediate result is greater than or equal to the reserved interval duration, it can be considered that the probability of the reasoning task related to the intermediate result being started in a short time is low, and the intermediate result can be deleted to optimize the cache space utilization. The reserved interval duration can be set according to business needs, and the disclosed embodiment does not limit the specific size of the specific reserved interval duration.

[0068] In an optional embodiment of the present disclosure, the feature construction submodule 121 is specifically used to perform feature query and extraction in the original features based on the inference request to obtain the inference feature when the query of the intermediate result based on the cache identifier fails, and to perform feature independent matrix calculation on the inference feature to obtain the intermediate result corresponding to the inference request.

[0069] In the embodiment of the present disclosure, when the cache identifier query intermediate result fails, the feature construction submodule 121 can construct the intermediate result based on the original feature, and the process stage to be executed by the feature construction submodule 121 is different according to the different reasoning task execution processes. For example, it can be to perform feature query and extraction in the original features to obtain the reasoning features, and perform feature independent matrix calculation on the reasoning features, and obtain the intermediate result corresponding to the reasoning request in the case of preliminary reasoning calculation.

[0070] In the disclosed embodiment, the extraction efficiency of the inference feature can be improved by multi-threaded concurrent feature extraction. At this time, the problem of multi-threaded data synchronization can be avoided by hash bucketing, and a multi-threaded lock-free solution can be implemented. Specifically, the cache identifier of the original feature can be hashed, and then the remainder operation can be performed according to the number of concurrent threads. Each remainder result corresponds to a thread, and each thread corresponds to a cache pool. In this way, in a high-concurrency scenario, the same set of original features will be processed in the same thread, which can effectively avoid the problem of multi-threaded synchronization. When the cache hit rate is low, the multi-threaded lock-free solution can effectively improve read and write performance and reduce time consumption.

[0071] Furthermore, in order to avoid the problem of uneven computing load among threads caused by uneven data distribution, a load balancing layer can be set to dynamically adjust the amount of data processed by the thread based on the remainder result to reduce the deviation of the computing amount among threads and effectively improve the efficiency of feature extraction.

[0072] like Figure 2 FIG. 2 is a schematic diagram showing an application of a multi-stage model-based reasoning system 200 (hereinafter referred to as reasoning system 200 ) in user recommendation according to an exemplary embodiment of the present disclosure, as shown in FIG. Figure 2 As shown, the reasoning system 200 may include an intermediate cache module 210 and a model reasoning module 220 , and the model reasoning module 220 may include a feature construction submodule 221 and a result reasoning submodule 222 .

[0073] In the reasoning system 200, the intermediate cache module 210 is used to query the intermediate result corresponding to the cache identifier in response to the reasoning request; the reasoning request includes the original feature and the cache identifier corresponding to the original feature.

[0074] In the embodiment of the present disclosure, the inference system 200 and the intermediate cache module 210, the model inference module 220, the feature construction submodule 221 and the result inference submodule 222 can correspond to the aforementioned Figure 1 To avoid repetition, the related description will not be repeated here.

[0075] The inference request is a user recommendation request, and the original features include original user features and original recommendation candidate features.

[0076] In the embodiment of the present disclosure, the reasoning request may be a user recommendation request, that is, requesting the reasoning system 200 to output the user's preference probability for each candidate item based on the input original features including the original user features and the original recommended candidate item features, so as to perform a reasoning task of recommending candidate items to the user based on the preference probability. In the reasoning system 200, feature query and extraction may be performed on the original user features and the original recommended candidate item features to obtain relevant information of the user's recommendation requirements, and reasoning calculation may be performed to complete the reasoning task.

[0077] In the inference system 200, the feature construction submodule 221 is specifically used to construct the user intermediate result corresponding to the inference request based on the original user feature and to construct the recommendation candidate intermediate result corresponding to the inference request based on the original recommendation candidate feature when the query of the intermediate result based on the cache identifier fails.

[0078] In the disclosed embodiment, the feature construction submodule 221 may query the intermediate result based on the cache identifier, and may query the user intermediate result corresponding to the cache identifier of the original user feature, and query the recommended candidate intermediate result corresponding to the cache identifier of the original recommended candidate feature. In the event of a query failure, the user intermediate result corresponding to the inference request may be constructed based on the original user feature, and the recommended candidate intermediate result corresponding to the inference request may be constructed based on the original recommended candidate feature.

[0079] For example, in the original user features, the user's gender, age, interest tags and other data related to the user's interest preferences can be obtained through feature query and extraction as inference features, and then the inference features are subjected to feature independent matrix calculation to obtain the vector representation of the user as the user intermediate result. The user intermediate result can represent the prediction of the user's interest preferences.

[0080] In the original recommendation candidate features, the data related to the candidate content, such as pictures, texts, and voices, can be obtained as inference features through feature query and extraction, and then the inference features are subjected to feature independent matrix calculation to obtain the vector representation of the recommendation candidate as the recommendation candidate intermediate result. The recommendation candidate intermediate result can represent the prediction of the candidate content.

[0081] In the disclosed embodiment, a multilayer perceptron (MLP) can be used to perform feature-independent matrix calculation on the inference features to obtain the corresponding feature representation as an intermediate result. When multiple original features are input, each original feature can be calculated using an independent multilayer perceptron to obtain respective intermediate results for separate caching and management.

[0082] It should be noted that the user intermediate results and the recommendation candidate intermediate results may both fail to query, or one fails to query and the other succeeds. In the case where one query fails and the other succeeds, the feature construction submodule 221 can construct the intermediate result of the one that fails to query, and directly obtain the pointer of the cached intermediate result for the other to call.

[0083] In the disclosed embodiment, the original features can also be cached in the process for future use during feature query and extraction. A recommendation request usually consists of a user original feature and a batch of original recommendation candidate features. Different cache strategies and query strategies can be set based on different business needs and feature scales. For example, in actual deployment, a solution combining external query and local cache can be adopted, or a solution of separate local cache can be adopted. For example, for original features with high business importance and large feature scale, synchronous non-cached query can be used, and the query result is synchronously waited after the query is initiated. The query performance is low but the data accuracy can be guaranteed; for original features with low business importance and large feature scale, synchronous cache query can be used, and the query is first queried in the cache of the process, and it is used when it hits and has not expired. After hitting but expired, a query is initiated and the query result is waited for to be updated to the cache. If it does not hit, a query is initiated and the query result is waited for to be updated to the cache. The query performance is high and the data accuracy can be guaranteed; or, for original features with small feature scale and high cache hit rate, The feature can adopt asynchronous cache query, first query in the cache of the process, use it when it hits and has not expired, initiate asynchronous query after it hits but expires, update to the cache after the query result is returned, if it does not hit, return empty result and initiate asynchronous query task, update to the cache after the query result is returned, the query performance is high but it may cause feature missing when the cache does not hit; or, for the original features with a small feature scale, a data file in a specified format can be produced in an offline environment, and imported into the cache of the inference system 200 in batches, and queried directly in the cache, if it hits, the queried feature is returned, otherwise, an empty result is returned.

[0084] In the reasoning system 200, the result reasoning submodule 222 is specifically used to perform cross calculations on the user intermediate results and the recommendation candidate intermediate results to obtain the reasoning results.

[0085] In the embodiment of the present disclosure, on the basis of obtaining the user intermediate results and the recommendation candidate intermediate results, the result reasoning submodule 222 can perform cross calculation on the user intermediate results and the recommendation candidate intermediate results. Since the user intermediate results represent the user's interest preference and the recommendation candidate intermediate results represent the content of the recommendation candidate, the user's preference probability for each recommendation candidate can be obtained as the reasoning result through cross calculation.

[0086] In the inference system 200, the intermediate cache module 210 is specifically used to cache user intermediate results based on the cache identifier corresponding to the original user feature, and cache the recommendation candidate intermediate results based on the cache identifier corresponding to the original recommendation candidate feature.

[0087] In the embodiment of the present disclosure, when the query of the intermediate result based on the cache identifier fails in the preceding execution, the user intermediate result constructed by the feature construction submodule 221 can be cached based on the cache identifier corresponding to the original user feature. Similarly, the recommendation candidate intermediate result constructed by the feature construction submodule 221 can be cached based on the cache identifier corresponding to the original recommendation candidate feature, so that subsequent reasoning tasks can perform query calls.

[0088] In the reasoning system 200, the result reasoning submodule 222 is used to perform cross calculations on the user intermediate results and the recommendation candidate intermediate results in the intermediate results to obtain the reasoning results when the query of the intermediate results based on the cache identifier is successful.

[0089] In the disclosed embodiment, when the query of the user intermediate result and the recommendation candidate intermediate result based on the cache identifier is successful, the user's preference probability for each recommendation candidate can be directly obtained as the inference result through cross calculation.

[0090] The inference system 200 provided in the embodiment of the present disclosure can be deployed in an inference computing scenario. Under the throughput of hundreds of thousands of QPS (Queries Per Second), the average time consumed in inference task calculation can be shortened from 60 milliseconds to 20 milliseconds, which can meet the high real-time and low-latency business requirements. The single-machine throughput capacity is increased by 2 times, saving a lot of overhead resources.

[0091] According to the reasoning system based on the multi-stage model of the embodiment of the present disclosure, the system includes an intermediate cache module and a model reasoning module, wherein the model reasoning module includes a feature construction submodule and a result reasoning submodule. When receiving a reasoning request, the intermediate result can be queried in the intermediate cache module based on the cache identifier in the reasoning request. If the query is successful, the result reasoning module can directly reason based on the intermediate result to obtain the reasoning result corresponding to the reasoning request; if the query fails, the feature construction submodule constructs the intermediate result based on the original feature in the reasoning request. On the one hand, the result reasoning submodule reasoning based on the intermediate result to obtain the reasoning result corresponding to the reasoning request, and on the other hand, the intermediate cache module caches the intermediate result based on the cache identifier. The system provides a unified deployment platform for feature processing and model calculation, which can realize feature pointer operations at different stages of the whole process, avoid additional overhead such as serialization, deserialization, network transmission, and split different reasoning task stages, so that most reasoning requests can reuse the cached intermediate results, without the need for feature query, extraction and assembly, and some preliminary reasoning calculations, shortening the amount of calculation and time required for a single request, and further reducing system overhead.

[0092] It should be noted that although several modules or units of the reasoning system based on the multi-stage model are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0093] Exemplary Methods

[0094] After introducing the reasoning system based on the multi-stage model according to the exemplary embodiment of the present disclosure, next, reference is made to Figure 3 A multi-stage model-based reasoning method according to an exemplary embodiment of the present disclosure is described.

[0095] It should be noted that other specific details of each implementation of the multi-stage model-based reasoning method of the embodiment of the present disclosure have been described in detail in the functional modules of the above-mentioned multi-stage model-based reasoning system, and will not be repeated here.

[0096] Figure 3 The multi-stage model-based reasoning method 300 of the exemplary embodiment of the present disclosure is shown. The method can be based on the aforementioned Figure 1 to Figure 2 For any of the above-mentioned multi-stage model-based reasoning system implementations, the method may include the following steps 301 to 304.

[0097] Step 301: In response to an inference request, query the intermediate result corresponding to the cache identifier; the inference request includes the original feature and the cache identifier corresponding to the original feature. If the query of the intermediate result based on the cache identifier is successful, execute step 302; if the query of the intermediate result based on the cache identifier fails, execute step 303.

[0098] Step 302: Execute the inference calculation corresponding to the inference request based on the intermediate result to obtain the inference result.

[0099] Step 303: construct an intermediate result corresponding to the inference request based on the original features. Execute steps 302 and 304 respectively.

[0100] Step 304: cache the intermediate results based on the cache identifiers corresponding to the original features.

[0101] In an optional embodiment of the present disclosure, step 301 may include determining that querying the intermediate result based on the cache identifier is successful when the intermediate result is queried based on the cache identifier and the intermediate result is in a valid state.

[0102] In an optional embodiment of the present disclosure, the valid state includes that the creation time of the intermediate result is within the cache validity period, and the cache validity period is set according to the update frequency of the original feature; or, the valid state includes that the feature version of the intermediate result is consistent with the feature version of the original feature.

[0103] In an optional embodiment of the present disclosure, the method may further include: when the remaining cache space is less than or equal to the reserved cache space, deleting the intermediate results of the most recent query whose interval duration is greater than or equal to the reserved interval duration.

[0104] In an optional embodiment of the present disclosure, step 303 may include, when querying the intermediate result based on the cache identifier fails, performing feature query and extraction in the original features based on the inference request to obtain the inference feature; performing feature independent matrix calculation on the inference feature to obtain the intermediate result corresponding to the inference request.

[0105] In an optional embodiment of the present disclosure, the inference request is a user recommendation request, and the original features include original user features and original recommendation candidate item features; step 303 may include, in the case where querying the intermediate result based on the cache identifier fails, constructing the user intermediate result corresponding to the inference request based on the original user feature, and constructing the recommendation candidate item intermediate result corresponding to the inference request based on the original recommendation candidate item feature; step 302 may include cross-calculating the user intermediate result and the recommendation candidate item intermediate result to obtain the inference result; and step 304 may include caching the user intermediate result based on the cache identifier corresponding to the original user feature, and caching the recommendation candidate item intermediate result based on the cache identifier corresponding to the original recommendation candidate item feature.

[0106] In an optional embodiment of the present disclosure, when the query of the intermediate result based on the cache identifier is successful, step 302 may include cross-calculating the user intermediate result and the recommendation candidate intermediate result in the intermediate result to obtain an inference result.

[0107] According to the reasoning method based on the multi-stage model of the embodiment of the present disclosure, the method is implemented by the reasoning system of the multi-stage model, the method responds to the reasoning request, queries the intermediate result corresponding to the cache identifier; the reasoning request includes the original feature and the cache identifier corresponding to the original feature; in the case of successful query of the intermediate result based on the cache identifier, the reasoning calculation corresponding to the reasoning request is performed based on the intermediate result to obtain the reasoning result; in the case of failure to query the intermediate result based on the cache identifier, the intermediate result corresponding to the reasoning request is constructed based on the original feature; the reasoning calculation corresponding to the reasoning request is performed based on the intermediate result to obtain the reasoning result; and the intermediate result is cached based on the cache identifier corresponding to the original feature. On the one hand, reasoning is performed based on the intermediate result to obtain the reasoning result corresponding to the reasoning request, and on the other hand, the intermediate result is cached based on the cache identifier. The method is implemented on a unified platform where feature processing and model calculation are deployed, and feature pointer operations can be implemented at different stages of the whole process, avoiding additional overhead such as serialization, deserialization, and network transmission, and splitting different reasoning task stages, so that most reasoning requests can reuse the cached intermediate results, without the need for feature query, extraction and assembly, and some preliminary reasoning calculations, shortening the calculation amount and time required for a single request, and further reducing system overhead.

[0108] Exemplary computer readable media

[0109] The following is a description of a computer-readable medium according to an exemplary embodiment of the present disclosure.

[0110] In this exemplary embodiment, reference Figure 4 As shown, a program product 400 for implementing the above method according to an exemplary embodiment of the present disclosure is described, such as a portable compact disk read-only memory (CD-ROM) and including program code, and can be run on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a computer-readable medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus, or a device.

[0111] The program product 400 may be any combination of one or more readable media. The readable medium may be a readable signal medium or a computer readable medium. The computer readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer readable media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0112] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than computer readable media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0113] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RE, etc., or any suitable combination of the foregoing.

[0114] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (FAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0115] Exemplary Electronic Devices

[0116] refer to Figure 5 An electronic device according to an exemplary embodiment of the present disclosure is described.

[0117] Figure 5 The electronic device 500 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0118] like Figure 5 As shown, the electronic device 500 is in the form of a general computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including the storage unit 520 and the processing unit 510), and a display unit 540.

[0119] The storage unit stores program codes, which can be executed by the processing unit 510, so that the processing unit 510 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 510 can perform the following steps: Figure 3 The method steps shown, etc.

[0120] The storage unit 520 may include a volatile storage unit, such as a random access storage unit (RAM) 521 and / or a cache storage unit 522 , and may further include a read-only storage unit (ROM) 523 .

[0121] The storage unit 520 may also include a program / utility 524 having a set (at least one) of program modules 525, such program modules 525 including but not limited to: an operating system, one or more server applications, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0122] The bus 530 may include a data bus, an address bus, and a control bus.

[0123] The electronic device 500 may also communicate with one or more external devices 600 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and such communication may be performed through an input / output (I / O) interface 550. The electronic device 500 also includes a display unit 540, which is connected to the input / output (I / O) interface 550 for display. In addition, the electronic device 500 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 500 through a bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0124] It should be noted that, although several modules or submodules of the device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0125] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0126] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A reasoning system based on a multi-stage model, characterized in that: The system includes an intermediate cache module and a model reasoning module, and the model reasoning module includes a feature construction submodule and a result reasoning submodule; The intermediate cache module is used to query the intermediate result corresponding to the cache identifier in response to the reasoning request; the reasoning request includes the original feature and the cache identifier corresponding to the original feature; The result reasoning submodule is used to perform the reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result when the query of the intermediate result based on the cache identifier is successful; The feature construction submodule is used to construct the intermediate result corresponding to the reasoning request based on the original feature when the query of the intermediate result based on the cache identifier fails; and the result reasoning submodule is also used to perform the reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result; The intermediate cache module is further used to cache the intermediate result based on the cache identifier corresponding to the original feature.

2. The system according to claim 1, characterized in that The intermediate cache module is further configured to determine that querying the intermediate result based on the cache identifier is successful when the intermediate result is queried based on the cache identifier and the intermediate result is in a valid state.

3. The system according to claim 2, characterized in that The valid state includes that the creation time of the intermediate result is within the cache validity period, and the cache validity period is set according to the update frequency of the original feature; Or, the valid state includes that the feature version of the intermediate result is consistent with the feature version of the original feature.

4. The system according to claim 1, characterized in that The intermediate cache module is further used to delete the intermediate result whose interval duration of the most recent query is greater than or equal to the reserved interval duration when the remaining cache space is less than or equal to the reserved cache space.

5. The system according to claim 1, characterized in that The feature construction submodule is specifically used to perform feature query and extraction in the original features based on the inference request to obtain inference features when the query of the intermediate result based on the cache identifier fails, and to perform feature independent matrix calculation on the inference features to obtain the intermediate result corresponding to the inference request.

6. The system according to claim 1, characterized in that The inference request is a user recommendation request, and the original features include original user features and original recommendation candidate features; The feature construction submodule is specifically used to construct the user intermediate result corresponding to the reasoning request based on the original user feature and to construct the recommendation candidate item intermediate result corresponding to the reasoning request based on the original recommendation candidate item feature when the query of the intermediate result based on the cache identifier fails; The result reasoning submodule is specifically used to perform cross calculation on the user intermediate result and the recommendation candidate intermediate result to obtain the reasoning result; And, the intermediate cache module is specifically used to cache the user intermediate result based on the cache identifier corresponding to the original user feature, and cache the recommendation candidate intermediate result based on the cache identifier corresponding to the original recommendation candidate feature.

7. The system according to claim 6, characterized in that The result inference submodule is used to perform cross calculation on the user intermediate result and the recommendation candidate intermediate result in the intermediate result to obtain an inference result when the query of the intermediate result based on the cache identifier is successful.

8. A reasoning method based on a multi-stage model, characterized in that: The method is implemented based on the multi-stage model-based reasoning system described in any one of claims 1 to 7, and the method comprises: In response to an inference request, querying an intermediate result corresponding to a cache identifier; the inference request includes an original feature and a cache identifier corresponding to the original feature; In the case where the query of the intermediate result based on the cache identifier is successful, performing the inference calculation corresponding to the inference request based on the intermediate result to obtain the inference result; In the event that querying the intermediate result based on the cache identifier fails, constructing the intermediate result corresponding to the reasoning request based on the original feature; and performing reasoning calculation corresponding to the reasoning request based on the intermediate result to obtain the reasoning result; and caching the intermediate result based on the cache identifier corresponding to the original feature.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-stage model-based reasoning method according to claim 8 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to perform the multi-stage model-based reasoning method described in claim 8 by executing the executable instructions.

Images