Data Processing Method, Device, and Storage Medium

By using preset configuration files to configure feature extraction function names in the model service system, the mixed problems of model access service and feature processing service are solved, flexible configuration and sharing of features are realized, development and maintenance costs are reduced, and system efficiency and scalability are improved.

CN114020318BActive Publication Date: 2025-07-18BEIJING SHAREIT INFORMATION TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202111105789.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-22
Publication Date
2025-07-18
Estimated Expiration
2041-09-22

AI Technical Summary

Technical Problem

In the existing model service system, the model access service and the model's service to process features are mixed together, resulting in high development and maintenance costs, and there are problems of duplicate feature acquisition and low efficiency when requesting multi-model service.

Method used

By configuring feature extraction function names through preset configuration files, stripping model access services and feature processing services, realizing flexible configuration and sharing of features, reducing code modification, and improving code reusability and scalability.

Benefits of technology

It reduces the development and maintenance costs, reduces the memory footprint and low efficiency caused by feature duplication acquisition, and improves the flexibility and scalability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114020318B_ABST
    Figure CN114020318B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a data processing method, apparatus, and storage medium. The method includes: in response to receiving a model service request, obtaining, through a preset configuration file, features required for the model requested by the model service request; wherein, the preset configuration file is configured with feature extraction function names corresponding to the features, and the features are used as inputs to the model; and obtaining an output result of the model after processing the features, so as to respond to the model service request. By this method, the development and maintenance costs can be reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of big data processing, and in particular, to a data processing method, apparatus, and storage medium. Background Art

[0002] With the improvement of social informatization and intelligence levels, it has gradually become a common means in the big data industry to train business models using big data systems and use the trained business models to achieve intelligent processing of big data services.

[0003] Generally, model services can be provided after the model is deployed. In the common model service systems in the industry today, there is a model access service to obtain various features and then assemble and transmit them to the model service. However, the model access service code and the model code are often mixed together, resulting in high development and maintenance costs. Summary of the Invention

[0004] The present disclosure provides a data processing method, apparatus, and storage medium.

[0005] According to a first aspect of the embodiments of the present disclosure, a data processing method is provided, including:

[0006] In response to receiving a model service request, obtaining, through a preset configuration file, features required for the model requested by the model service request; wherein, feature extraction function names corresponding to the features are configured in the preset configuration file, and the features are used as inputs to the model;

[0007] Obtaining an output result of the model after processing the features to respond to the model service request.

[0008] In some embodiments, the model service request includes service requests for multiple models, and feature extraction function names corresponding to each feature in the union of the features required for the multiple models are configured in the preset configuration file;

[0009] The step of, in response to receiving a model service request, obtaining, through a preset configuration file, features required for the model requested by the model service request includes:

[0010] In response to receiving service requests for multiple models, loading functions corresponding to the feature extraction function names in the preset configuration file to obtain the union of the features required for the multiple models;

[0011] Determining, from the union of the features, features required for each model requested by the model service request.

[0012] In some embodiments, a feature mapping method corresponding to the model is further configured in the preset configuration file;

[0013] Obtaining the features required by the model requested by the model service request through a preset configuration file further includes:

[0014] After determining the features required by each model requested by the model service request from the union of the features, perform mapping processing on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

[0015] In some embodiments, the method further includes:

[0016] Determine the features that the model requested by the model service request needs to load;

[0017] Based on the features that the model needs to load, determine the union of the features required by the model;

[0018] At least write the feature extraction function names corresponding to each feature in the union of the features into a file to generate the preset configuration file.

[0019] In some embodiments, the configuration file further includes at least one of the following:

[0020] The name of the feature;

[0021] The data type of the feature.

[0022] According to the second aspect of the embodiments of the present disclosure, a data processing device is provided, including:

[0023] An acquisition module configured to, in response to receiving a model service request, obtain the features required by the model requested by the model service request through a preset configuration file; wherein, the feature extraction function names corresponding to the features are configured in the preset configuration file, and the features are used as inputs to the model;

[0024] A response model configured to obtain the output result of the model after processing the features to respond to the model service request.

[0025] In some embodiments, the model service request includes service requests for multiple models, and the feature extraction function names corresponding to each feature in the union of the features required to be extracted by the multiple models are configured in the preset configuration file;

[0026] The acquisition module is further configured to, in response to receiving service requests for multiple models, load the functions corresponding to the feature extraction function names in the preset configuration file to obtain the union of the features required by the multiple models; and determine the features required by each model requested by the model service request from the union of the features.

[0027] In some embodiments, the feature mapping method corresponding to the model is further configured in the preset configuration file;

[0028] The obtaining module is further configured to determine, from the union of the features, the features required by each of the model service requests for the requested model, and then perform mapping processing on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

[0029] In some embodiments, the apparatus further includes:

[0030] A first determination module, configured to determine the features to be loaded by the model requested by the model service request;

[0031] A second determination module, configured to determine the union of the features required by the model based on the features to be loaded by the model;

[0032] A generation module, configured to at least write the feature extraction function names corresponding to the features in the union of the features into a file to generate the preset configuration file.

[0033] According to a third aspect of the embodiments of the present disclosure, there is provided a data processing apparatus, including:

[0034] A processor;

[0035] A memory for storing instructions executable by the processor;

[0036] Wherein, the processor is configured to execute the data processing method as described in the first aspect above.

[0037] According to a fourth aspect of the embodiments of the present disclosure, there is provided a storage medium, including:

[0038] When the instructions in the storage medium are executed by the processor of the computer, the computer is enabled to execute the data processing method as described in the first aspect above.

[0039] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0040] In the embodiments of the present disclosure, by separating the service of connecting the model to the service and the service of the model processing the features by configuring the feature extraction function names through the preset configuration file, and by configuring the corresponding feature extraction function names through the preset configuration file, it is convenient to change the configuration of the features required by the model later without modifying the code, which has the advantages of strong flexibility and convenient expansion, and thus can reduce the development and maintenance costs.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Description of the Drawings

[0042] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0043] Figure 1 is a flowchart of a data processing method shown in an embodiment of the present disclosure Figure 1 。

[0044] Figure 2 is a flowchart of a data processing method shown in an embodiment of the present disclosure Figure 2 。

[0045] Figure 3 is an example diagram of a flowchart of a data processing method shown in an embodiment of the present disclosure.

[0046] Figure 4 is a diagram of a data processing device shown according to an exemplary embodiment.

[0047] Figure 5 is a block diagram of a server shown in an embodiment of the present disclosure. Detailed implementation manners

[0048] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0049] Figure 1 is a flowchart of a data processing method shown in an embodiment of the present disclosure Figure 1 ,as Figure 1 shown, the data processing method includes the following steps:

[0050] S11. In response to receiving a model service request, obtain the features required by the model requested by the model service request through a preset configuration file; wherein, the name of the feature extraction function corresponding to the feature is configured in the preset configuration file, and the feature is used as input to the model;

[0051] S12. Obtain the output result of the model after processing the feature to respond to the model service request.

[0052] In the embodiments of the present disclosure, the data processing method can be applied to a server, and a model can be deployed in the server to provide model services. In addition, the model can also be deployed on another server. After the server of the present disclosure receives a model service request, it obtains the model output result returned from the server on which the model is deployed elsewhere, so as to respond to the model service request.

[0053] Taking the application of model services to a recommendation system as an example, most recommendation systems make recommendations based on implicit feedback. For example, video recommendations are made based on user behaviors such as attention, sharing, liking, and full playback. In this regard, the data processing method of the present disclosure can be applied to request and obtain the output results of various models such as an attention model, a sharing model, a liking model, or a full playback model, so as to comprehensively determine the videos suitable for recommendation to users.

[0054] Taking the application of model services to a voice assistant as an example, a voice assistant usually needs to comprehensively determine the reply message after performing emotion recognition, intent recognition, semantic matching, etc. on the user input voice. Each module of emotion recognition, intent recognition, or semantic matching may adopt different machine learning algorithms, and each corresponds to a model. In this regard, the data processing method of the present disclosure can be applied to request and obtain the output results of models such as an emotion recognition model, an intent recognition model, and semantic matching, so as to comprehensively determine the reply message.

[0055] Taking the application of model services to product classification as an example, the pictures of products can be analyzed to obtain the categories of the products in the pictures. In this regard, the data processing method of the present disclosure can be applied to request and obtain the output result of, for example, a product classification model, so as to place the picture in the corresponding picture repository of the category.

[0056] In step S11, the server can receive a model service request, and the model service request specifies the requested model. Among them, the requested model can be one model or multiple models. After receiving the request for the model, the server will obtain the features required by the model requested by the model service request through the feature extraction function name configured in the preset configuration file.

[0057] In the embodiments of the present disclosure, the feature extraction function name is used to indicate the corresponding feature extraction function, and the server can obtain the corresponding features based on this feature extraction function. The features are used as inputs to the model. For example, the features are historical behavior data such as a user's attention, sharing, liking, and full playback of a video; for another example, the features are voice data input by the user, or picture data of a network screenshot, or the user's feature data such as gender, age, or nationality. The features required by different models may be different, and some of the features may also be the same in different models.

[0058] It should be noted that in the embodiments of the present disclosure, in addition to configuring the feature extraction function name corresponding to the feature in the preset configuration file, in one embodiment, the preset configuration file further includes at least one of the following:

[0059] The name of the feature;

[0060] The data type of the feature.

[0061] Among them, the names of the features are, for example, information such as username, gender, types of the user's historical behavior, occurrence time of the user's historical behavior, picture data or voice data, etc., which are used to identify the types of the features. The data type of the feature can be information used to indicate the data storage format. For example, it is specified that the feature is stored in an integer type (int), a floating-point type (float), or a long integer type (long). The data type of the feature can also be information used to indicate the storage format of pictures or audio. For example, it is specified that pictures are stored in the JPEG format or voices are stored in the MP3 format, etc. Among them, both pictures and voices belong to the features of the present disclosure.

[0062] It should be noted that in the embodiments of the present disclosure, if the name of the feature or the data type of the feature is configured in the preset configuration file, the name of the feature or the data type of the feature corresponds one-to-one with the function name of feature extraction. When both the name of the feature and the data type of the feature are configured in the preset configuration file, there is a one-to-one correspondence among the name of the feature, the data type of the feature, and the function name of feature extraction. Among them, the name of the feature, the data type of the feature, and the function name of feature extraction can be manually configured by the R & D personnel according to the features required by the model. The R & D personnel know in advance the features required by each model, and thus can write the above information into the configuration file in advance.

[0063] In the embodiments of the present disclosure, the preset configuration file can be a file saved in the xml format or the json format, and the embodiments of the present disclosure do not make any restrictions on this.

[0064] In step S12, after the server obtains the features required by the model based on the function name of feature extraction in the preset configuration file, it can obtain the output result after the model processes the features based on the features. This output result is the response result of the model service request.

[0065] Further, after the server obtains the output result after the model processes the features, for example, obtains the attention probability of the user output after the attention model processes the features, the sharing probability output by the sharing model, the like probability output by the like model, or the complete playback probability of the user output by the complete playback model, it can, based on the method of multi-model score fusion, for example, adopt a weighted method to comprehensively calculate the probability values output by each model to obtain a total recommendation score value, so as to make a recommendation. The present disclosure does not make specific limitations on the subsequent operations performed by the server based on the output result of the model.

[0066] In the related art, configuration files can also be introduced into the model service system. However, the configuration files usually configure the acquisition address of the model, the model name, etc. to call the corresponding model. In the model service system, the model access service for obtaining the required features of the model and the service for processing the features by the model are mixed together. Therefore, when the features required by the model change, the code needs to be adjusted again, and the development and maintenance costs are both very high.

[0067] In the embodiments of the present disclosure, the model access service and the service for processing the features by the model are separated by presetting the configuration file to configure the feature extraction function name. Moreover, by presetting the configuration file to configure the corresponding feature extraction function name, it is convenient to change the configuration of the features required by the model later without modifying the code, which has the advantages of strong flexibility and convenient expansion. Therefore, it can reduce the development and maintenance costs.

[0068] In the embodiments of the present disclosure, as described above, the model requested by the model service request can be one or more. For the case where the models requested by the model service request are multiple, in one embodiment, the feature extraction function names corresponding to the features required by each model can be configured respectively. The configuration of the feature extraction function names in the preset configuration file is configured according to the model. Therefore, when different models require the same features, the feature extraction function names of the same feature are repeatedly configured, and thus there may be a situation of repeated feature extraction.

[0069] In another embodiment, the method further includes:

[0070] Determine the features to be loaded by the model requested by the model service request;

[0071] Based on the features to be loaded by the model, determine the union of the features required by the model;

[0072] At least write the feature extraction function names corresponding to the features in the union of the features into a file to generate the preset configuration file.

[0073] In this embodiment, the configuration of the feature extraction function names in the preset configuration file is not configured according to the model, but according to the features. In this regard, after the server determines the features to be loaded by the model requested by the model service request, for example, after the user manually inputs the features required by each model, the server automatically determines the union of the features required by the model, and thus writes the feature extraction function names corresponding to the features in the union of the features into a file to generate the preset configuration file, which has the characteristic of high automation.

[0074] Since in this embodiment, the feature extraction function names corresponding to the features in the union of the features are configured in the preset configuration file, feature sharing can be achieved based on this preset configuration file.

[0075] Figure 2 is a data processing method flow shown in an embodiment of the present disclosure Figure 2 , such as Figure 2 shown, the data processing method includes the following steps:

[0076] S21. In response to receiving a service request for multiple models, load the function corresponding to the feature extraction function name in the preset configuration file to obtain the union of the features required by the multiple models; wherein, the preset configuration file is configured with the feature extraction function name corresponding to each feature in the union of the features required to be extracted by the multiple models;

[0077] S22. Determine, from the union of the features, the features required by each model requested by the model service request;

[0078] S23. Obtain the output results of each model after processing the features to respond to the model service request.

[0079] In this embodiment, the preset configuration file is configured with the feature extraction function name corresponding to each feature in the union of the features required to be extracted by the multiple models. Then, when there are identical features required by different models, by configuring the feature extraction function corresponding to the feature in the union of the features through the preset configuration file, the union of the features required by the multiple models can be obtained in step S21. Based on the union of the features required by the multiple models, the features required by each model can be determined in step S22.

[0080] It should be noted that, in the embodiment of the present disclosure, the preset configuration file may include the information of the models corresponding to each feature. Therefore, after the server obtains the union of the features required by the multiple models, it can determine the features required by each model based on this information in the preset configuration file.

[0081] In the related model service system, since the model access service is developed separately for each model, the interfaces of different model access services may also be different. However, the features obtained based on the model access service may overlap. Therefore, for the request of multi-model service, the above method has the problems of repeated feature extraction, occupying memory and low efficiency.

[0082] In response to this, the present disclosure configures the feature extraction function name corresponding to each feature in the union of the features required to be extracted by the multiple models in the preset configuration file, so that the feature acquisition can be shared, which can reduce the unnecessary memory occupation and efficiency impact caused by repeated feature acquisition. By directly sharing the feature extraction function name through the preset configuration file, the code reusability is increased.

[0083] In one embodiment, the preset configuration file is further configured with a feature mapping method corresponding to the model;

[0084] Obtaining the features required by the model requested by the model service request through a preset configuration file further includes:

[0085] After determining the features required by each model service request from the union of the features, perform mapping processing on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

[0086] In this embodiment, the preset configuration file is also configured with a feature mapping method corresponding to the model. The feature mapping method refers to the method of further processing the features. Since different models may have different further processing methods for the features, the present disclosure can perform different mapping processing on the features based on different models through the feature mapping method corresponding to the model configured in the preset configuration file, including different mapping processing of the same feature by different models.

[0087] It should be noted that in the embodiments of the present disclosure, the feature mapping method includes a feature encoding method. For example, the originally obtained feature data of int type is converted into feature data of float type to suit the processing of the model. In addition, the feature mapping method also includes obtaining features associated with the features further based on the feature extraction function name. For example, the features obtained by the server based on the feature extraction function name are all the features pointing to the same user, and different models have different requirements for the features of this user. For example, one model needs the time information of the user's web browsing, and another model needs the content information corresponding to the user's web browsing. Therefore, the features required by the model can be obtained based on the configured feature mapping method, where the feature mapping method can be a mapping from the user name to the web browsing time, or a mapping from the user name to the web browsing content.

[0088] In the related model service system, since the model access service is developed separately for the model to obtain features, and the model access service has included the mapping of the features when obtaining the features, the above method is not applicable to the sharing of features. In the present disclosure, after different models share the features, the features truly required by the models are obtained based on different mapping methods. This loose coupling method can improve the flexibility of feature sharing.

[0089] Figure 3 is a flowchart example of a data processing method shown in the embodiments of the present disclosure. As Figure 3 shown, the data processing method includes the following steps:

[0090] S31. Describe information such as the name, type, and acquisition function of the features that each model needs to extract through a configuration file;

[0091] In this embodiment, the configuration file is the aforementioned preset configuration file, and the acquisition function is the name of the feature extraction function. The name, data type, and feature extraction function name of the features required by the model can be configured in the preset configuration file.

[0092] This feature can be a feature that does not change over time, such as summarized statistical information such as username and user name (abbreviated as feature F1). In addition, it can also include some features that change over time, such as the time information of the user's Internet access (F2), the Internet Protocol (IP) address of the user's login (F3), the address obtained by using a geostationary orbit (GEO) satellite to locate the user's device (F4), and the statistical information of the content browsed by the user (F5), etc.

[0093] Suppose the model service request includes service requests for multiple models. For example, model S1 requests F1, F2, and F3, and model S2 requests F2, F3, F4, and F5. Then, models S1 and S2 have overlapping features F2. In this regard, the present disclosure takes the union of all features, which is F1, F2, F3, F4, F5.

[0094] Since different features correspond to different feature extraction functions, the present disclosure can configure the feature extraction function name corresponding to each feature in the union of features in the configuration file.

[0095] S32. Load all supported feature acquisition functions;

[0096] In this embodiment, loading all supported feature acquisition functions means loading the functions corresponding to the feature extraction function names in the preset configuration file to obtain the union of the features required by multiple models.

[0097] S33. For the model service request, sequentially obtain the required features according to the configuration file;

[0098] In this embodiment, after obtaining the union of features, the features required by each model can be sequentially determined from the union of features according to the features required by each model configured in the preset configuration file.

[0099] S34. Convert the original feature values into encoded feature values according to the corresponding relationship;

[0100] In this embodiment, encoding and converting the features according to the corresponding relationship means performing mapping processing on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

[0101] S35. Add the encoded feature values to different fields of the model request according to the feature type.

[0102] In this embodiment, according to the feature type, the encoded features are added to different fields of the model request, that is, the features are input into the model to obtain the output result after the model processes the features.

[0103] It can be understood that in the embodiments of the present disclosure, the service of connecting the model to the service and the service of the model processing the features are separated by configuring the feature extraction function name through a configuration file, and by configuring the corresponding feature extraction function name through the configuration file, it is convenient to change the configuration of the features required by the model later without modifying the code, which has the advantages of strong flexibility and convenient expansion, thus reducing the development and maintenance costs. In addition, since the union of the features required by multiple models configured in the preset configuration file contains the feature extraction function name corresponding to each feature, the feature acquisition can be shared, which can reduce the unnecessary memory occupation and efficiency problems caused by repeated feature acquisition, and increase the code reusability.

[0104] Figure 4 It is a diagram of a data processing device shown according to an exemplary embodiment. Refer to Figure 4 , the data processing device includes:

[0105] An acquisition module 101, configured to, in response to receiving a model service request, obtain the features required by the model requested by the model service request through a preset configuration file; wherein, the preset configuration file is configured with the feature extraction function name corresponding to the feature, and the feature is used to be input into the model;

[0106] A response model 102, configured to obtain the output result after the model processes the feature to respond to the model service request.

[0107] In some embodiments, the model service request includes service requests for multiple models, and the preset configuration file is configured with the feature extraction function name corresponding to each feature in the union of the features required by multiple models;

[0108] The acquisition module 101 is further configured to, in response to receiving a service request for multiple models, load the functions corresponding to the feature extraction function names in the preset configuration file to obtain the union of the features required by multiple models; and determine the features required by each model requested by the model service request from the union of the features.

[0109] In some embodiments, the preset configuration file is further configured with a feature mapping method corresponding to the model;

[0110] The obtaining module 101 is further configured to, after determining the features required by each of the model service requests from the union of the features, perform mapping processing on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

[0111] In some embodiments, the apparatus further includes:

[0112] A first determination module 103, configured to determine the features to be loaded by the model requested by the model service request;

[0113] A second determination module 104, configured to determine the union of the features required by the model based on the features to be loaded by the model;

[0114] A generation module 105, configured to at least write the feature extraction function names corresponding to the features in the union of the features into a file to generate the preset configuration file.

[0115] Regarding the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0116] Figure 5 It is a block diagram of a server device 900 shown according to an exemplary embodiment. Referring to Figure 5 , the device 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by a memory 932 for storing instructions executable by the processing component 922, such as application programs. The application programs stored in the memory 932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute instructions to perform the above power distribution method.

[0117] The device 900 may further include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0118] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as the memory 932 including instructions, and the above instructions can be executed by the processing component 922 of the device 900 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0119] A non - transitory computer - readable storage medium, when the instructions in the storage medium are executed by a processor of a computer, enables the computer to execute a data - processing method, and the method includes:

[0120] In response to receiving a model service request, obtaining features required for the model requested by the model service request through a preset configuration file; wherein, a feature extraction function name corresponding to the feature is configured in the preset configuration file, and the feature is used as input to the model;

[0121] Obtaining an output result of the model after processing the features to respond to the model service request.

[0122] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include well - known knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0123] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: In response to receiving a model service request, obtaining the features required by the model requested by the model service request through a preset configuration file; wherein, the model service request includes service requests for multiple models, and in the preset configuration file, the function name corresponding to each feature in the union of the features required to be extracted by the multiple models is configured, and the features are used as inputs to the model; wherein, the union does not include the same features; Obtaining the output result after the model processes the features to respond to the model service request; Wherein, the step of in response to receiving a model service request, obtaining the features required by the model requested by the model service request through a preset configuration file includes: In response to receiving service requests for multiple models, loading the functions corresponding to the feature extraction function names in the preset configuration file to obtain the union of the features required by the multiple models; Determining, from the union of the features, the features required by each model requested by the model service request.

2. The method according to claim 1, wherein The preset configuration file is further configured with a feature mapping method corresponding to the model; The step of obtaining the features required by the model requested by the model service request through a preset configuration file further includes: After determining the features required by each model requested by the model service request from the union of the features, performing a mapping process on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

3. The method according to claim 1, characterized in that, The method further includes: Determining the features that need to be loaded by the model requested by the model service request; Based on the features that need to be loaded by the model, determining the union of the features required by the model; At least writing the function names corresponding to each feature in the union of the features into a file to generate the preset configuration file.

4. The method according to claim 1, wherein The preset configuration file further includes at least one of the following: The name of the feature; The data type of the feature.

5. A data processing device, characterized in that, The apparatus includes: An obtaining module configured to, in response to receiving a model service request, obtain the features required by the model requested by the model service request through a preset configuration file; wherein, the model service request includes service requests for multiple models, and in the preset configuration file, the function name corresponding to each feature in the union of the features required to be extracted by the multiple models is configured, and the features are used as inputs to the model; wherein, the union does not include the same features; A response model configured to obtain the output result after the model processes the features to respond to the model service request; The obtaining module is further configured to, in response to receiving service requests for multiple models, load the functions corresponding to the feature extraction function names in the preset configuration file to obtain the union of the features required by the multiple models; and determining, from the union of the features, the features required by each model requested by the model service request.

6. The device according to claim 5, characterized in that, The preset configuration file is further configured with a feature mapping method corresponding to the model; The obtaining module is further configured to, after determining the features required by each model requested by the model service request from the union of the features, perform a mapping process on the features according to the feature mapping method configured in the preset configuration file to obtain the mapped features.

7. A data processing device, characterized in that, Includes: A processor; A memory for storing processor-executable instructions; Wherein, the processor is configured to execute the data processing method according to any one of claims 1 to 4.

8. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by a processor of a computer, the computer is enabled to execute the data processing method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Object feature processing and information pushing method, device and equipment

    CN111382347A