Model screening method and device, computer device and storage medium

Abstract

The application relates to a model screening method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a target training set, a target verification set, a target sample set, a model list and an engine list corresponding to a target service scenario; training each candidate model in the model list to obtain each target model and a corresponding model loss; establishing an initial engine library based on the target model and the target sample set, adjusting each initial engine library based on each engine algorithm in the engine list to obtain a target engine library corresponding to each model-engine combination; verifying the target engine library based on the target verification set and the target model to obtain an engine loss corresponding to each target engine library; obtaining a target loss corresponding to each model-engine combination based on the model loss and the engine loss corresponding to the model-engine combination; and determining a target combination based on each target loss. The method can improve the accuracy of data retrieval.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model screening method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of deep learning technology, data retrieval technology has emerged, which can quickly search for data that meets the corresponding search criteria from massive amounts of data. Traditional data retrieval technology first trains a deep learning model using a training sample set, then uses the trained model to build an index, and finally inputs the sample to be queried into the trained model to obtain the corresponding feature vector. The feature vector is then used to retrieve data that meets the corresponding search criteria from the index.

[0003] In current data retrieval technologies, the construction of indexes is based on blindly building pre-trained deep learning models, which can easily lead to low accuracy in data retrieval. Summary of the Invention

[0004] Therefore, it is necessary to provide a model recommendation method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of data retrieval in response to the above-mentioned technical problems.

[0005] This application provides a model selection method. The method includes:

[0006] Obtain the target training set, target validation set, and target sample set corresponding to the target business scenario; obtain the model list and engine list corresponding to the target business scenario.

[0007] Based on each candidate model in the target training set training model list, the target model and model loss corresponding to each candidate model are obtained;

[0008] An engine library is built based on the target model and the target sample set to obtain the initial engine library corresponding to each target model. Each initial engine library is adjusted based on the engine algorithm in the engine list to obtain the target engine library corresponding to each model engine combination. The model engine combination includes the matched target model and engine algorithm.

[0009] Based on the target validation set and the target model, the corresponding target engine library is validated, and the engine loss corresponding to each target engine library is obtained;

[0010] Based on the model loss and engine loss corresponding to the same model engine combination, the target loss corresponding to each model engine combination is obtained;

[0011] Based on the target loss, a target combination is determined from various model engine combinations; the target combination is used for sample retrieval corresponding to the target business scenario.

[0012] This application also provides a model screening device. The device includes:

[0013] The data acquisition module is used to acquire the target training set, target validation set, and target sample set corresponding to the target business scenario, as well as the model list and engine list corresponding to the target business scenario.

[0014] The model training module is used to train each candidate model in the model list based on the target training set, and to obtain the target model and model loss for each candidate model.

[0015] The target engine library determination module is used to build an engine library based on the target model and the target sample set, obtain the initial engine library corresponding to each target model, and adjust each initial engine library based on each engine algorithm in the engine list to obtain the target engine library corresponding to each model engine combination; the model engine combination includes the matched target model and engine algorithm.

[0016] The engine loss determination module is used to validate the corresponding target engine library based on the target validation set and the target model, and obtain the engine loss corresponding to each target engine library.

[0017] The target loss determination module is used to obtain the target loss corresponding to each model engine combination based on the model loss and engine loss corresponding to the same model engine combination.

[0018] The target combination determination module is used to determine the target combination from various model engine combinations based on the target loss; the target combination is used for sample retrieval corresponding to the target business scenario.

[0019] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described model screening method.

[0020] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described model screening method.

[0021] A computer program product includes a computer program that, when executed by a processor, implements the steps of the model screening method described above.

[0022] The aforementioned model selection method, apparatus, computer equipment, storage medium, and computer program products acquire a target training set, target validation set, target sample set, model list, and engine list corresponding to the target business scenario. The model list represents selectable model algorithms for the target business scenario. Training each candidate model in the model list yields each target model and its corresponding model loss. The model loss for each target model provides a reliable basis for model selection, thereby improving the accuracy of model selection. An initial engine library is established for each target model based on the target model and target sample set. The initial engine library is adjusted based on the engine algorithms in the engine list to obtain target engine libraries corresponding to each model-engine combination. Using engine algorithms to process the target engine library significantly improves the retrieval speed within the engine library, while different engine algorithms have varying degrees of impact on retrieval accuracy. The corresponding target engine libraries are validated using the target validation set and each target model, yielding the engine loss for each target engine library. The engine loss for each target engine library measures the impact of the corresponding engine algorithm on retrieval accuracy and provides a reliable basis for determining the optimal model-engine combination, thereby improving the accuracy of model selection. By fusing the model loss and engine loss corresponding to the same set of model engine combinations, the target loss for each model engine combination is obtained. The target loss combines the model loss and engine loss, reflecting the retrieval accuracy of the model engine combination from the perspectives of the model and engine algorithms, respectively. Based on the target loss corresponding to each model engine combination, the target combination is determined. This allows us to identify the model engine combination with the best retrieval performance from the model list and engine list. Applying the target combination to sample retrieval in the target business scenario can improve retrieval accuracy. Attached Figure Description

[0023] Figure 1 This is a diagram illustrating the application environment of the model selection method in one embodiment;

[0024] Figure 2 This is a flowchart illustrating a model selection method in one embodiment;

[0025] Figure 3 This is a flowchart illustrating the process of determining engine loss in one embodiment;

[0026] Figure 4 This is a schematic diagram of a model selection system in one embodiment;

[0027] Figure 5 This is a structural block diagram of the model screening device in one embodiment;

[0028] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0029] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0031] The model selection method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart TVs, smart in-vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers. Terminal 102 and server 104 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0032] Both the terminal and the server can be used independently to execute the model screening method provided in the embodiments of this application.

[0033] For example, the terminal obtains the target training set, target validation set, and target sample set corresponding to the target business scenario, and obtains the model list and engine list corresponding to the target business scenario. The terminal trains each candidate model in the model list based on the target training set, obtaining the target model and model loss for each candidate model. The terminal builds an engine library based on the target model and target sample set, obtaining the initial engine library corresponding to each target model. The terminal adjusts each initial engine library based on each engine algorithm in the engine list, obtaining the target engine library corresponding to each model-engine combination. The model-engine combination includes the matching target model and engine algorithm. The terminal validates the corresponding target engine library based on the target validation set and target model, obtaining the engine loss corresponding to each target engine library. The terminal obtains the target loss corresponding to each model-engine combination based on the model loss and engine loss corresponding to the same model-engine combination. Based on the target loss, the terminal determines the target combination from the various model-engine combinations. The target combination is used for sample retrieval corresponding to the target business scenario.

[0034] Terminals and servers can also be used in conjunction to execute the model screening method provided in the embodiments of this application.

[0035] For example, the terminal sends a business identifier corresponding to the target business scenario to the server. The server, based on the business identifier, obtains the target training set, target validation set, and target sample set corresponding to the target business scenario, and acquires the model list and engine list corresponding to the target business scenario. The server trains each candidate model in the model list based on the target training set, obtaining the target model and model loss for each candidate model. The server builds an engine library based on the target model and target sample set, obtaining the initial engine library corresponding to each target model. The server adjusts each initial engine library based on the engine algorithms in the engine list, obtaining the target engine library corresponding to each model-engine combination. Each model-engine combination includes the matching target model and engine algorithm. The server validates the corresponding target engine library based on the target validation set and target model, obtaining the engine loss for each target engine library. The server obtains the target loss for each model-engine combination based on the model loss and engine loss corresponding to the same model-engine combination. Based on the target loss, the server determines a target combination from each model-engine combination. This target combination is used for sample retrieval corresponding to the target business scenario. The server sends the target combination to the terminal, which can display the target combination or respond to query tasks under the target business scenario based on the target combination.

[0036] In one embodiment, such as Figure 2 As shown, a model selection method is provided. Taking the application of this method to a computer device as an example, the computer device can be a terminal or a server. The method can be implemented independently by the terminal or server, or through interaction between the terminal and the server. The model selection method includes the following steps:

[0037] Step S202: Obtain the target training set, target validation set, and target sample set corresponding to the target business scenario, and obtain the model list and engine list corresponding to the target business scenario.

[0038] Here, "business scenario" refers to a scenario that requires providing a specific search service, such as a scenario that requires providing text search services; a scenario that requires providing image search services; a scenario that requires providing dialogue intent search services; and so on. "Target business scenario" refers to any scenario that requires providing a specific search service.

[0039] The target training set refers to the training set corresponding to the target business scenario. Each training sample in the target training set is used to train each candidate model in the model list corresponding to the target business scenario.

[0040] The target validation set refers to the validation set corresponding to the target business scenario. Each validation sample in the target validation set is used to verify the retrieval accuracy of the target engine library.

[0041] The target sample set refers to the sample set corresponding to the target business scenario. Each target sample in the target sample set is used to build the engine library.

[0042] A sample refers to the retrieval object corresponding to the target business scenario. For example, when the target business scenario is a text retrieval service, the sample can be a piece of text; when the target business scenario is an image retrieval service, the sample can be an image; when the target business scenario is a dialogue intent retrieval service, the sample can be a dialogue; and so on.

[0043] A model list refers to a list of multiple candidate models. A candidate model is a model that has not undergone training and is used to extract features from samples in a target business scenario. The input data for a candidate model is the training samples in the target training set, and the output data is the feature information corresponding to the training samples. Different candidate models correspond to different model algorithms. For example, model algorithms can be random forest algorithms, decision tree algorithms, and logistic regression algorithms, etc.

[0044] An engine list refers to a list of multiple engine algorithms. An engine algorithm is an algorithm that processes the engine library to speed up the retrieval process. For example, engine algorithms can be PCA (Principal Component Analysis) dimensionality reduction, inverted sorting, and other algorithms.

[0045] For example, a computer device obtains information such as a target training set, target validation set, target sample set, model list, and engine list corresponding to a target business scenario. Based on the target training set, target validation set, and target sample set corresponding to the target business scenario, it filters the various model-engine combinations obtained from the model list and engine list to obtain a target combination, thereby achieving model filtering.

[0046] Step S204: Based on the candidate models in the target training set training model list, obtain the target model and model loss corresponding to each candidate model.

[0047] The target model refers to the model obtained after the candidate model has been trained, which is used to extract the target features corresponding to the samples. The model loss refers to the loss value obtained by the loss function in the last calculation when the model training is completed.

[0048] For example, the computer device trains each candidate model in the candidate model list based on each training sample in the target training set. After the model training is completed, the target model and model loss corresponding to each candidate model are obtained.

[0049] Step S206: Establish an engine library based on the target model and the target sample set to obtain the initial engine library corresponding to each target model. Adjust each initial engine library based on each engine algorithm in the engine list to obtain the target engine library corresponding to each model engine combination. The model engine combination includes the matched target model and engine algorithm.

[0050] The initial engine library refers to the engine library composed of the target features corresponding to each target sample in the target sample set. The target engine library is the engine library obtained after processing the initial engine library with the engine algorithm. Adjusting the engine library using the engine algorithm can improve retrieval speed. The model-engine combination refers to a pair of target models and engine algorithms. The target model in the model-engine combination builds the initial engine library based on the target sample set, and the engine algorithm in the model-engine combination processes the initial engine library built by the target model to obtain the target engine library corresponding to the model-engine combination.

[0051] For example, a computer device inputs a target sample library into a single target model to obtain an initial engine library corresponding to that target model. Using the same method, initial engine libraries corresponding to other target models are determined, resulting in initial engine libraries for each target model. Based on the engine algorithms in the engine list, data processing is performed on the initial engine library corresponding to the target model to obtain a target engine library corresponding to a model engine combination composed of the aforementioned engine algorithms and the target model. Using the same method, target engine libraries corresponding to other model engine combinations are determined, resulting in target engine libraries for each model engine combination.

[0052] Step S208: Based on the target validation set and the target model, validate the corresponding target engine library to obtain the engine loss corresponding to each target engine library.

[0053] Engine loss refers to the loss of retrieval accuracy when searching in the target engine library obtained after engine algorithm processing, compared to searching in the initial engine library. It is used to characterize the loss of retrieval accuracy when using engine algorithm to improve retrieval speed.

[0054] For example, the computer device inputs each validation sample from the target validation set into the target model for feature extraction, obtaining the target features corresponding to each validation sample. Based on the target features corresponding to each validation sample, searches are performed in the initial engine library corresponding to a single target model and in each target engine library corresponding to the target model. Based on the results obtained from the searches in the initial engine library and each target engine library, the engine loss corresponding to each target engine library is obtained. Using the same method, the engine loss of each target engine library corresponding to each other target model is determined, thus obtaining the engine loss corresponding to each target engine library.

[0055] For example, when the model list corresponds to target model A, target model B, and target model C, and the engine list corresponds to engine algorithm a and engine algorithm b, target models A, B, and C respectively obtain initial engine libraries A, B, and C based on the target sample set. Adjusting the initial engine library A based on engine algorithm a yields target engine library Aa corresponding to the model-engine combination composed of target model A and engine algorithm a. Adjusting the initial engine library A based on engine algorithm b yields target engine library Ab corresponding to the model-engine combination composed of target model A and engine algorithm b. Using the same method, target engine libraries Ba, Bb, Ca, and Cb corresponding to the model-engine combination composed of target model C and engine algorithm a are obtained. Based on target model A, the target features corresponding to each validation sample in the target validation set are obtained. Based on each target feature, searches are performed in the initial engine library A, the target engine library Aa, and the target engine library Ab corresponding to target model A, respectively, to obtain the search results A, Aa, and Ab. Based on the search results A and Aa, the engine loss Aa corresponding to target engine library Aa is obtained. Based on the search results A and Ab, the engine loss Ab corresponding to target engine library Ab is obtained. Using the same method, the engine loss Ba, Bb, Ca, and Cb corresponding to target engine libraries Ba, Bb, Ca, and Cb are obtained.

[0056] Step S210: Based on the model loss and engine loss corresponding to the same model engine combination, obtain the target loss corresponding to each model engine combination.

[0057] The target loss refers to the total loss of the model engine combination obtained by fusing the model loss and engine loss corresponding to the same model engine combination, which is used to characterize the overall retrieval accuracy of the model engine combination.

[0058] For example, the computer device fuses the model loss corresponding to the target model in the same model engine combination with the engine loss of the target engine library corresponding to the model engine combination to obtain the target loss corresponding to the model engine combination. Based on the same method, the target losses corresponding to other model engine combinations are calculated to obtain the target losses corresponding to each model engine combination.

[0059] Step S212: Based on the target loss, determine the target combination from the various model engine combinations; the target combination is used for sample retrieval corresponding to the target business scenario.

[0060] The target combination refers to the model engine combination with the highest retrieval accuracy determined from various model engine combinations, which is used to respond to sample retrieval tasks in the target business scenario.

[0061] For example, the computer device determines the model engine combination with the smallest target loss value as the target combination based on the target loss corresponding to each model engine combination.

[0062] In one embodiment, after obtaining the target combination corresponding to the target business scenario, the computer device responds to the sample retrieval task under the target business scenario based on the target combination. The sample to be retrieved is input into the target model in the target combination, the target model extracts the target features of the target sample, and performs a retrieval in the target engine library corresponding to the target combination based on the target features to obtain the retrieval result corresponding to the sample.

[0063] In the above model selection method, the target training set, target validation set, target sample set, model list, and engine list corresponding to the target business scenario are obtained. The model list consists of selectable model algorithms for the target business scenario. Training each candidate model in the model list yields each target model and its corresponding model loss. The model loss for each target model provides a reliable basis for model selection, thereby improving the accuracy of model selection. An initial engine library is established for each target model based on the target model and target sample set. The initial engine library is adjusted based on the engine algorithms in the engine list to obtain target engine libraries for each model-engine combination. Using engine algorithms to process the target engine library can significantly improve the retrieval speed within the engine library, while different engine algorithms have varying degrees of impact on retrieval accuracy. The corresponding target engine libraries are validated using the target validation set and each target model to obtain the engine loss for each target engine library. The engine loss for each target engine library measures the impact of the corresponding engine algorithm on retrieval accuracy and provides a reliable basis for determining the optimal model-engine combination, thereby improving the accuracy of model selection. The model loss and engine loss corresponding to the same set of model-engine combinations are fused to obtain the target loss for each model-engine combination. The target loss integrates model loss and engine loss, reflecting the retrieval accuracy of the model-engine combination from the perspectives of both the model and engine algorithms. Based on the target loss corresponding to each model-engine combination, the target combination is determined. This allows identifying the best-performing model-engine combination from both the model and engine lists. Applying this target combination to sample retrieval within the target business scenario can improve retrieval accuracy.

[0064] In one embodiment, the target training set, target validation set, and target sample set corresponding to the target business scenario are obtained, as well as the model list and engine list corresponding to the target business scenario, including:

[0065] Obtain the configuration file corresponding to the target business scenario, parse the configuration file to obtain the target business scenario and the corresponding data layer parameters, model layer parameters, and engine layer parameters; based on the data layer parameters, obtain the target training set, target validation set, and target sample set corresponding to the target business scenario; based on the model layer parameters, obtain the model list corresponding to the target business scenario; based on the engine layer parameters, obtain the engine list corresponding to the target business scenario.

[0066] In this context, the configuration file refers to a file defined by algorithm engineers to configure parameters such as data layer parameters, model layer parameters, and engine layer parameters corresponding to the target business scenario. Data layer parameters configure the data information corresponding to the target business scenario. For example, data layer parameters may include the target training set path, target validation set directory, target sample set path, test set path, and dataset format. Model layer parameters configure the model information corresponding to the target business scenario. For example, model layer parameters may include the candidate model path, candidate model type, training batch size, validation batch size, and training epochs. Engine layer parameters configure the engine information corresponding to the target business scenario. For example, engine layer parameters may include the engine algorithm type, engine algorithm path, and engine backend components.

[0067] For example, the computer device obtains a configuration file, parses the configuration file to obtain the target business scenario, and the corresponding data layer parameters, model layer parameters, and engine layer parameters. Based on the data layer parameters, the computer device obtains the target training set, target validation set, and target sample set corresponding to the target business scenario; based on the model layer parameters, it obtains the model list corresponding to the target business scenario; and based on the engine layer parameters, it obtains the engine list corresponding to the target business scenario.

[0068] In the above embodiments, the computer device obtains the target business scenario, as well as parameter information such as data layer parameters, model layer parameters, and engine layer parameters corresponding to the target business scenario, based on the configuration file. Based on this parameter information, it obtains the dataset information, model list, and engine list required to determine the target combination. If it is necessary to determine the target combination corresponding to other business scenarios, the configuration file corresponding to those other business scenarios is obtained, which improves the flexibility and efficiency of model selection.

[0069] In one embodiment, an engine library is built based on the target model and the target sample set, resulting in an initial engine library corresponding to each target model, including:

[0070] Based on each target model, feature extraction is performed on each target sample in the target sample set to obtain the target features corresponding to each target sample under each target model; an initial engine library is established based on the target features of each target sample corresponding to the same target model to obtain the initial engine library corresponding to each target model.

[0071] Among them, target features refer to the feature information obtained by the target model from the feature extraction of the sample, which is used to retrieve the corresponding search results in the index.

[0072] For example, the computer device inputs each target sample in the target sample set into a single target model for feature extraction, and obtains the target features corresponding to each target sample. The target features corresponding to each target sample form the initial engine library corresponding to the target model. Based on the same method, the initial engine libraries corresponding to each other target model are determined, and the initial engine libraries corresponding to each target model are obtained.

[0073] In the above embodiments, the computer device extracts target features corresponding to each target sample in the target sample set based on the target model, and establishes an initial engine library corresponding to the target model based on the target features corresponding to each target sample. Subsequent sample retrieval can be performed in the initial engine library, or the initial index library can be processed using engine algorithms to obtain the target engine library, thereby improving the retrieval speed in the engine library.

[0074] In one embodiment, such as Figure 3 As shown, based on the target validation set and the target model, the corresponding target engine libraries are validated, and the engine loss corresponding to each target engine library is obtained, including:

[0075] Step S302: Based on each target model, feature extraction is performed on each verification sample in the target verification set to obtain the target features corresponding to each verification sample under each target model.

[0076] Step S304: Based on the target features of each validation sample corresponding to the same target model, search in the initial engine library and the target engine library corresponding to the target model respectively to obtain the retrieval quality scores corresponding to the initial engine library and the target engine library respectively.

[0077] Step S306: Based on the retrieval quality scores corresponding to the same target model, generate the engine loss of the target engine library corresponding to the target model, and obtain the engine loss corresponding to each target engine library.

[0078] The retrieval quality score refers to the accuracy of the retrieval results obtained based on the target validation set in the engine library, and is used to characterize the retrieval accuracy of different engine libraries.

[0079] For example, the computer device inputs each verification sample in the target verification set into a single target model for feature extraction to obtain the target features corresponding to each verification sample. Based on the same method, the target features corresponding to each verification sample under other target models are obtained, thus obtaining the target features corresponding to each verification sample under each target model.

[0080] The computer equipment searches the initial engine library corresponding to the target model based on the target features of each validation sample corresponding to the same target model. The search results for each validation sample are evaluated, resulting in an evaluation result for each validation sample. For example, evaluation metrics such as NDCG (Normalized Discounted Cumulative Gain) and top-1 accuracy can be used to assess the accuracy of the search results. Based on the evaluation results for each validation sample, a search quality score is obtained for the initial engine library. Then, the computer equipment searches the individual target engine library corresponding to the target model, evaluating the search results for each validation sample, resulting in an evaluation result for each validation sample. Based on the search results for each validation sample, a search quality score is obtained for the target engine library. Using the same method, the search quality scores for other target engine libraries corresponding to the same target model are determined, resulting in a search quality score for each target engine library. Finally, using the same method, the search quality scores for the initial engine libraries and individual target engine libraries corresponding to other target models are obtained.

[0081] Computer equipment determines the engine loss for a target engine library based on the retrieval quality scores of the initial and target engine libraries corresponding to the same target model. For example, the engine loss can be calculated as the ratio of the target engine library's retrieval quality score to the initial engine library's retrieval quality score; or as the difference between the target engine library's retrieval quality score and the initial engine library's retrieval quality score; and so on. Using the same method, the engine losses for other target engine libraries corresponding to the same target model are determined, resulting in the engine losses for each target engine library corresponding to the same target model. Similarly, using the same method, the engine losses for each target engine library corresponding to each other target model are determined, resulting in the engine losses for each target engine library corresponding to each target model.

[0082] In one embodiment, the engine loss can be calculated using the following formula:

[0083]

[0084] Among them, L engineLet q be the engine loss corresponding to the target engine library, q be the validation samples in the target validation set, C be the target sample set, and Rank be the rank. {engine} To evaluate the search results of the validation samples in the target engine library, the resulting evaluation result is Rank. {origin} To evaluate the search results of the validation samples in the initial engine library, the evaluation results are as follows: {∑ q C*Rank {engine}} represents the retrieval quality score corresponding to the target engine library, {∑ q C*Rank {origin}} represents the search quality score corresponding to the initial engine library.

[0085] In the above embodiments, target features corresponding to each validation sample in the target validation set are obtained based on the target model. Retrieval is then performed in both the initial engine library and the target engine library based on each target feature, resulting in retrieval quality scores for the initial and target engine libraries, respectively. The target engine library is obtained by adjusting the initial engine library using an engine algorithm. Retrieving data in the target engine library is faster than retrieving data in the initial engine library, while adjusting the initial engine library using an engine algorithm may lead to a decrease in retrieval accuracy. Since the retrieval quality score reflects the accuracy of the retrieval results, using the retrieval quality scores corresponding to the initial and target engine libraries to calculate the engine loss for the target engine library ensures the reliability and accuracy of the engine loss, thereby improving the accuracy of model selection.

[0086] In one embodiment, based on the model loss and engine loss corresponding to the same model engine combination, the target loss corresponding to each model engine combination is obtained, including:

[0087] Obtain the loss weight corresponding to the target business scenario; the loss weight is determined based on the scenario retrieval conditions corresponding to the target business scenario; adjust the model loss and engine loss corresponding to the same model engine combination based on the loss weight to obtain the target loss corresponding to each model engine combination.

[0088] Here, loss weight refers to the numerical value used to adjust the proportion of engine loss and model loss in the target loss when calculating the target loss. Scenario retrieval conditions refer to the relative importance of retrieval accuracy and retrieval speed within the target business scenario, i.e., the requirements of the target business scenario for retrieval accuracy and speed.

[0089] For example, the computer device obtains the loss weights corresponding to the target business scenario, and adjusts the model loss and engine loss corresponding to the same model engine combination based on the loss weights to obtain the target loss corresponding to the model engine combination. For example, multiplying the loss weights by the engine loss and adding the product to the model loss yields the target loss corresponding to the model engine combination; multiplying the loss weights by the engine loss and multiplying the difference between the preset value and the loss weights, then adding the corresponding products yields the target loss; multiplying the loss weights by the engine loss and adding the product to the model loss yields a first intermediate loss, the sum of the engine loss and the model loss is calculated to obtain a second intermediate loss, and the ratio of the first intermediate loss to the second intermediate loss is used as the target loss; and so on. Based on the same method, the target losses corresponding to other model engine combinations are calculated to obtain the target losses corresponding to each model engine combination.

[0090] In one embodiment, the target loss can be calculated using the following formula:

[0091] L=(1-α)L ce +αL engine

[0092] Where L is the target loss, L ce For model loss, L engine Let α be the engine loss and α be the loss weight. For example, in the target business scenario, when the importance of retrieval accuracy and retrieval speed are equal, α can be set to 0.5. When the importance of retrieval accuracy is greater than the importance of retrieval speed, α can be increased to increase the proportion of engine loss. When the importance of retrieval accuracy is less than the importance of retrieval speed, α can be decreased to decrease the proportion of engine loss. It can be understood that the engine algorithms in the target combination obtained based on different loss weights have different impacts on retrieval accuracy and retrieval speed improvement effects. For example, when α is less than 0.5, the engine algorithms in the target combination have a greater impact on retrieval accuracy and a better effect on improving retrieval speed; when α is greater than 0.5, the engine algorithms in the target combination have a smaller impact on retrieval accuracy and a poorer effect on improving retrieval speed.

[0093] In the above embodiments, the loss weight is determined based on the retrieval conditions corresponding to the target business scenario, reflecting the different requirements of different business scenarios for retrieval accuracy and retrieval speed. Calculating the target loss corresponding to each model engine combination based on the loss weight can improve the model selection accuracy, so that the final target combination can better meet the needs of the target business scenario, thereby improving the retrieval accuracy.

[0094] In one embodiment, the model selection method further includes:

[0095] Obtain the target prediction set corresponding to the target business scenario; based on the target model corresponding to the target combination, extract features from each prediction sample in the target prediction set to obtain the target features corresponding to each prediction sample; based on the target features corresponding to each prediction sample, search in the target engine library corresponding to the target combination to obtain the search results corresponding to each prediction sample.

[0096] The target prediction set refers to the prediction set corresponding to the target business scenario, and the prediction set contains at least one prediction sample. A prediction sample is a sample that needs to be retrieved through a combination of targets. The retrieval result is the result obtained by retrieving the prediction sample through the combination of targets. For example, when the target business scenario requires providing similar image retrieval services, the prediction sample is the image to be retrieved, and the retrieval result is the similar images corresponding to the image to be retrieved; when the target business scenario requires providing dialogue intent retrieval services, the prediction sample is the dialogue to be retrieved, and the retrieval result is the dialogue intent corresponding to the dialogue to be retrieved; and so on.

[0097] For example, a computer device acquires a target prediction set corresponding to a target business scenario, and the target model in the target combination extracts the target features of each prediction sample in the target prediction set. Based on the target features corresponding to a single prediction sample, a search is performed in the target engine library obtained based on the target combination to obtain the search results corresponding to the prediction sample. Using the same method, the search results corresponding to each other prediction sample are obtained, thus obtaining the search results corresponding to each prediction sample separately.

[0098] In one embodiment, the computer device calculates the similarity between the target feature corresponding to the predicted sample and the target feature corresponding to each target sample in the target engine library, and takes the target sample corresponding to the target feature with the highest similarity in the target engine library as the retrieval result corresponding to the predicted sample.

[0099] In the above embodiments, since the target combination is more in line with the requirements of the target business scenario for retrieval accuracy and retrieval speed than other model engine combinations, the retrieval of each prediction sample in the target prediction set based on the target combination corresponding to the target business scenario can improve the retrieval accuracy and ensure the retrieval speed.

[0100] In one embodiment, the model selection method is applied to a model selection system, which includes a data layer, a model layer, an engine layer, and an evaluation layer. The data layer is used to acquire the target training set, target validation set, target sample set, and target prediction set corresponding to the target business scenario. The model layer is used to train each candidate model in the model list to obtain the corresponding target model and model loss. The model layer is also used to construct an initial engine library. The engine layer is used to adjust the initial engine library based on the engine list to obtain the corresponding target engine library. The engine layer is also used to calculate the engine loss corresponding to the target engine library. The evaluation layer is used to obtain the corresponding target loss based on the model loss and engine loss corresponding to the same model-engine combination, and to determine the target combination based on each target loss.

[0101] The model selection system can exist on a single computer device or be distributed across multiple computer devices. It is used to provide model selection functions for different business scenarios and output the target combination corresponding to the target business scenario.

[0102] For example, the model selection system parses the configuration file to obtain data layer parameters, model layer parameters, engine layer parameters, and evaluation layer parameters, and sends each parameter to the data layer, model layer, engine layer, and evaluation layer respectively. The data layer, based on the data layer parameters, obtains the target training set, target validation set, and target sample set. The model layer, based on the model layer parameters, obtains a list of models and trains each candidate model in the list based on the target training set input from the data layer, obtaining the target model and model loss for each candidate model. The model layer, based on the target sample set input from the data layer, obtains the initial engine library corresponding to each target model. The engine layer, based on the engine layer parameters, obtains an engine list and adjusts each initial engine library based on the algorithms of each engine in the engine list, obtaining the target engine libraries corresponding to each model-engine combination. The model layer and engine layer validate each model-engine combination based on the target validation set, obtaining the engine loss corresponding to each target engine library. The evaluation layer determines the loss weights based on the evaluation layer parameters, and adjusts the model loss and engine loss input to the model layer and engine layer respectively based on the loss weights to obtain the target loss corresponding to each model-engine combination. Then, based on the target loss corresponding to each model-engine combination, the target combination is determined.

[0103] In the above embodiments, the computer device parses the configuration file to obtain data layer parameters, model layer parameters, engine layer parameters, and evaluation layer parameters, and sends each parameter to the data layer, model layer, engine layer, and evaluation layer respectively. The data layer, model layer, engine layer, and evaluation layer obtain corresponding data information based on the corresponding parameters. The model layer, based on the model list and the target training set, trains each candidate model in the model list to obtain each target model and its corresponding model loss. The model loss corresponding to each target model can provide a reliable basis for model selection, thereby improving the accuracy of model selection. The model layer establishes an initial engine library corresponding to each target model based on each target model and the target sample set. The engine layer adjusts the initial engine library based on each engine algorithm in the engine list to obtain the target engine library corresponding to each model-engine combination. Using engine algorithms to process the target engine library can greatly improve the retrieval speed in the engine library. The model layer and engine layer validate each model-engine combination based on the target validation set to obtain the engine loss corresponding to each target engine library. The engine loss corresponding to each target engine library can measure the impact of the corresponding engine algorithm on retrieval accuracy and provide a reliable basis for determining the optimal model-engine combination, thereby improving the accuracy of model selection. The evaluator determines the loss weight based on the evaluation layer parameters and adjusts the model loss and engine loss input to the model layer and engine layer respectively based on the loss weight to obtain the target loss corresponding to each model-engine combination. The target loss integrates the model loss and engine loss, reflecting the retrieval accuracy of the model-engine combination from the perspectives of both the model and engine algorithms. Based on the target loss corresponding to each model-engine combination, the model layer determines the target combination. It can identify the model-engine combination with the best retrieval performance from the model list and engine list. Applying the target combination to sample retrieval in the target business scenario can improve retrieval accuracy.

[0104] In one specific embodiment, the model screening method of this application can be applied to a model screening system.

[0105] like Figure 4 As shown, the model selection method includes the following steps:

[0106] 1. Parse the configuration file

[0107] The configuration parser (Config component) in the model selection system parses the configuration file input by the algorithm personnel and sends the parsed parameters of the data layer, model layer, engine layer and evaluation layer to the data layer, model layer and engine layer and the evaluator of the model selection system, respectively.

[0108] 2. Obtain the dataset

[0109] The model selection system stores data information corresponding to different business scenarios in the data terminal, which may or may not be included in the model selection system. The model selection system filters and labels the data in the data terminal before inputting it into the comprehensive sample library. Filtering and labeling refers to filtering each data entry and assigning corresponding label information to each filtered data entry. The DataReader component in the data layer, based on the corresponding parameters of the data layer, retrieves the datasets corresponding to the target business scenarios defined in the configuration file from the comprehensive sample library and inputs the datasets into the InputExample component. The InputExample component converts each data entry in the dataset into feature information and inputs it into the Feature component. The Feature component performs feature processing operations on the feature information input from the InputExample and outputs the results. The DataSet component processes the feature information input from the Feature component into batches and inputs the feature information into the model layer in batches.

[0110] 3. Model Training

[0111] The abstract model component (Model) in the model layer obtains a list of models based on the model layer parameters. The training component (Trainer) obtains the target training set input from the data layer and trains each candidate model in the model list in the abstract model component based on the target training set, obtaining the target model and model loss corresponding to each candidate model. The model loss corresponding to each candidate model is then input into the evaluator component (Evaluator).

[0112] 4. Build the engine library and determine engine losses.

[0113] The training component in the model layer acquires the target sample set input from the data layer, and builds an initial engine library based on the target model and the target sample set, resulting in the initial engine library corresponding to each target model. The engine wrapper component in the engine layer selects the appropriate engine backend component as the backend based on the engine backend components defined in the corresponding parameters of the engine layer. The engine wrapper component can calculate the engine loss, provide CRUD (Create, Read, Update, Delete) functions for the index library, and respond to training and prediction tasks. Engine backend components are components that provide retrieval methods; for example, engine backend components can be Faiss (Facebook AI Similarity Search, a similarity vector retrieval library) and NMSLIB (Non-Metric Space Library). The engine layer obtains a list of engines based on its corresponding parameters. For example, engine algorithms can be PCA (Principal Component Analysis) dimensionality reduction, inverted sorting, etc. The engine wrapper component adjusts each initial engine library based on each engine algorithm in the engine list, resulting in the target engine library corresponding to each model-engine combination. The model layer obtains the target validation set from the data layer input. Based on the target validation set and the target model, it validates the corresponding target engine library, obtaining the engine loss for each target engine library. For example, evaluation metrics such as NDCG and top-1 accuracy can be used to assess retrieval accuracy, and then the engine loss can be calculated.

[0114] 5. Determine the optimal model and offline engine library

[0115] The evaluator determines evaluation metrics based on the corresponding parameters of the evaluation layer. It then fuses the model loss and engine loss corresponding to the same model-engine combination based on these evaluation metrics. Dynamic programming is performed based on the fusion result to determine the optimal model (the target model in the target combination) and the offline engine library (the target engine library corresponding to the target combination). The training component outputs the optimal model, and the engine wrapper component outputs the target engine library.

[0116] The prediction component (Predictor) in the model layer loads the best model and the offline engine library, and makes predictions on the input model to filter the prediction samples of the system. The prediction component inputs the prediction samples into the best model to obtain the target features corresponding to the prediction samples, calculates the similarity between the target features and each target feature in the offline engine library, and takes the target sample corresponding to the target feature with the highest similarity in the offline engine library as the retrieval result corresponding to the prediction sample.

[0117] The above embodiments establish a modular, easily expandable, and easily configurable integrated deep learning framework. Corresponding algorithm modules are integrated at different processing stages, and configuration files are used to automatically connect various deep learning components, making operation convenient and significantly reducing the time cost of deep learning model migration and iteration. When selecting models, the evaluator considers the algorithmic loss caused by the engine acceleration algorithm, interacts with the model layer and engine layer, and performs dynamic programming based on the model loss input from the model layer and the engine loss input from the engine layer. Finally, it outputs the best model and offline engine library, improving the accuracy of model selection. Furthermore, searching based on the best model and offline engine library can greatly improve the accuracy of retrieval.

[0118] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0119] Based on the same inventive concept, this application also provides a model screening device for implementing the model screening method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more model screening device embodiments provided below can be found in the limitations of the model screening method described above, and will not be repeated here.

[0120] In one embodiment, such as Figure 5 As shown, a model selection device is provided, including: a data acquisition module 502, a model training module 504, a target engine library determination module 506, an engine loss determination module 508, a target loss determination module 510, and a target combination determination module 512, wherein:

[0121] The data acquisition module 502 is used to acquire the target training set, target validation set, and target sample set corresponding to the target business scenario, and to acquire the model list and engine list corresponding to the target business scenario.

[0122] The model training module 504 is used to train each candidate model in the model list based on the target training set, and obtain the target model and model loss corresponding to each candidate model.

[0123] The target engine library determination module 506 is used to build an engine library based on the target model and the target sample set, obtain the initial engine library corresponding to each target model, and adjust each initial engine library based on each engine algorithm in the engine list to obtain the target engine library corresponding to each model engine combination; the model engine combination includes the matched target model and engine algorithm.

[0124] The engine loss determination module 508 is used to verify the corresponding target engine library based on the target validation set and the target model, and obtain the engine loss corresponding to each target engine library.

[0125] The target loss determination module 510 is used to obtain the target loss corresponding to each model engine combination based on the model loss and engine loss corresponding to the same model engine combination.

[0126] The target combination determination module 512 is used to determine the target combination from various model engine combinations based on the target loss; the target combination is used for sample retrieval corresponding to the target business scenario.

[0127] The aforementioned model selection device acquires a target training set, a target validation set, a target sample set, a model list, and an engine list corresponding to the target business scenario. The model list contains selectable model algorithms for the target business scenario. Training each candidate model in the model list yields each target model and its corresponding model loss. The model loss for each target model provides a reliable basis for model selection, thereby improving the accuracy of model selection. An initial engine library is established for each target model based on the target model and target sample set. The initial engine library is adjusted based on the engine algorithms in the engine list to obtain target engine libraries for each model-engine combination. Using engine algorithms to process the target engine library significantly improves the retrieval speed within the engine library, while different engine algorithms have varying degrees of impact on retrieval accuracy. The corresponding target engine libraries are validated using the target validation set and each target model to obtain the engine loss for each target engine library. The engine loss for each target engine library measures the impact of the corresponding engine algorithm on retrieval accuracy and provides a reliable basis for determining the optimal model-engine combination, thereby improving the accuracy of model selection. The model loss and engine loss corresponding to the same set of model-engine combinations are fused to obtain the target loss for each model-engine combination. The target loss integrates model loss and engine loss, reflecting the retrieval accuracy of the model-engine combination from the perspectives of both the model and engine algorithms. Based on the target loss corresponding to each model-engine combination, the target combination is determined. This allows identifying the best-performing model-engine combination from both the model and engine lists. Applying this target combination to sample retrieval within the target business scenario can improve retrieval accuracy.

[0128] In one embodiment, the data acquisition module 502 is further configured to:

[0129] Obtain the configuration file corresponding to the target business scenario, parse the configuration file to obtain the target business scenario and the corresponding data layer parameters, model layer parameters, and engine layer parameters; based on the data layer parameters, obtain the target training set, target validation set, and target sample set corresponding to the target business scenario; based on the model layer parameters, obtain the model list corresponding to the target business scenario; based on the engine layer parameters, obtain the engine list corresponding to the target business scenario.

[0130] In one embodiment, the target engine library determination module 506 is further configured to:

[0131] Based on each target model, feature extraction is performed on each target sample in the target sample set to obtain the target features corresponding to each target sample under each target model; an initial engine library is established based on the target features of each target sample corresponding to the same target model to obtain the initial engine library corresponding to each target model.

[0132] In one embodiment, the engine loss determination module 508 is further configured to:

[0133] Based on each target model, features are extracted from each validation sample in the target validation set to obtain the target features corresponding to each validation sample under each target model. Based on the target features of each validation sample corresponding to the same target model, searches are performed in the initial engine library and the target engine library corresponding to the target model to obtain the retrieval quality scores corresponding to the initial engine library and the target engine library respectively. Based on the retrieval quality scores corresponding to the same target model, the engine loss of the target engine library corresponding to the target model is generated to obtain the engine loss corresponding to each target engine library.

[0134] In one embodiment, the target loss determination module 510 is further configured to:

[0135] Obtain the loss weight corresponding to the target business scenario; the loss weight is determined based on the scenario retrieval conditions corresponding to the target business scenario; adjust the model loss and engine loss corresponding to the same model engine combination based on the loss weight to obtain the target loss corresponding to each model engine combination.

[0136] In one embodiment, the model selection method is applied to a model selection system, which includes a data layer, a model layer, an engine layer, and an evaluation layer. The data layer is used to obtain the target training set, target validation set, target sample set, and target prediction set corresponding to the target business scenario. The model layer is used to train each candidate model in the model list to obtain the corresponding target model and model loss. The model layer is also used to build an initial engine library. The engine layer is used to adjust the initial engine library based on the engine list to obtain the corresponding target engine library. The engine layer is also used to calculate the engine loss corresponding to the target engine library. The evaluation layer is used to obtain the corresponding target loss based on the model loss and engine loss corresponding to the same model-engine combination, and to determine the target combination based on each target loss.

[0137] Each module in the aforementioned model screening device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0138] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as target training sets, model lists, and engine lists. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a model selection method.

[0139] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a model selection method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0140] Those skilled in the art will understand that Figure 6 , 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0142] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0143] In one embodiment, a computer program product or computer program is provided, the computer product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above-described method embodiments.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model selection method, characterized in that, The method includes: Obtain the target training set, target validation set, and target sample set corresponding to the target business scenario; obtain the model list and engine list corresponding to the target business scenario. Train each candidate model in the model list based on the target training set to obtain the target model and model loss corresponding to each candidate model. The model loss refers to the loss value obtained by the loss function in the last calculation when the model training is completed. An engine library is established based on the target model and the target sample set to obtain an initial engine library corresponding to each target model. Each initial engine library is adjusted based on each engine algorithm in the engine list to obtain a target engine library corresponding to each model engine combination. The model engine combination includes a matching target model and an engine algorithm. Based on the target validation set and target model, the corresponding target engine library is validated to obtain the engine loss corresponding to each target engine library. This includes: extracting features from each validation sample in the target validation set based on each target model to obtain the target features corresponding to each validation sample under each target model; searching in the initial engine library and the target engine library corresponding to the target model based on the target features of each validation sample corresponding to the same target model to obtain the search quality scores corresponding to the initial engine library and the target engine library respectively; generating the engine loss of the target engine library corresponding to the target model based on the search quality scores corresponding to the same target model to obtain the engine loss corresponding to each target engine library. The engine loss refers to the search accuracy lost when searching in the target engine library obtained after engine algorithm processing compared to searching in the initial engine library. Based on the model loss and engine loss corresponding to the same model engine combination, the target loss corresponding to each model engine combination is obtained, including: obtaining the loss weight corresponding to the target business scenario; the loss weight is determined based on the scenario retrieval conditions corresponding to the target business scenario; adjusting the model loss and engine loss corresponding to the same model engine combination based on the loss weight to obtain the target loss corresponding to each model engine combination, wherein the target loss refers to the total loss corresponding to the model engine combination obtained by fusing the model loss and engine loss corresponding to the same model engine combination, and the loss weight refers to the value used to adjust the proportion of engine loss and model loss in the target loss when calculating the target loss; Based on the target loss, a target combination is determined from the various model engine combinations; the target combination is used for sample retrieval corresponding to the target business scenario.

2. The method according to claim 1, characterized in that, The steps of obtaining the target training set, target validation set, and target sample set corresponding to the target business scenario, and obtaining the model list and engine list corresponding to the target business scenario, include: Obtain the configuration file corresponding to the target business scenario, and parse the configuration file to obtain the target business scenario and the data layer parameters, model layer parameters and engine layer parameters corresponding to the target business scenario; Based on the data layer parameters, obtain the target training set, target validation set, and target sample set corresponding to the target business scenario; Based on the model layer parameters, obtain the model list corresponding to the target business scenario; Based on the engine layer parameters, obtain the engine list corresponding to the target business scenario.

3. The method according to claim 1, characterized in that, The step of establishing an engine library based on the target model and the target sample set, resulting in an initial engine library corresponding to each target model, includes: Based on each target model, feature extraction is performed on each target sample in the target sample set to obtain the target features corresponding to each target sample under each target model. An initial engine library is established based on the target features of each target sample corresponding to the same target model, thus obtaining the initial engine library corresponding to each target model.

4. The method according to claim 1, characterized in that, The retrieval quality score refers to the accuracy of the retrieval results obtained based on the target validation set in the engine library, and is used to characterize the retrieval accuracy corresponding to different engine libraries.

5. The method according to claim 1, characterized in that, The scenario retrieval conditions refer to the relative importance of retrieval accuracy and retrieval speed in the target business scenario.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the target prediction set corresponding to the target business scenario; Based on the target model corresponding to the target combination, feature extraction is performed on each prediction sample in the target prediction set to obtain the target features corresponding to each prediction sample. Based on the target features corresponding to each predicted sample, a search is performed in the target engine library corresponding to the target combination to obtain the search results corresponding to each predicted sample.

7. The method according to any one of claims 1 to 6, characterized in that, It is applied to a model selection system, which includes a data layer, a model layer, an engine layer, and an evaluation layer; The data layer is used to obtain the target training set, target validation set, target sample set, and target prediction set corresponding to the target business scenario; The model layer is used to train each candidate model in the model list to obtain the corresponding target model and model loss. The model layer is also used to build the initial engine library. The engine layer is used to adjust the initial engine library based on the engine list to obtain the corresponding target engine library. The engine layer is also used to calculate the engine loss corresponding to the target engine library. The evaluation layer is used to obtain the corresponding target loss based on the model loss and engine loss corresponding to the same model engine combination, and to determine the target combination based on each target loss.

8. A model screening device, characterized in that, The device includes: The data acquisition module is used to acquire the target training set, target validation set and target sample set corresponding to the target business scenario, and to acquire the model list and engine list corresponding to the target business scenario; The model training module is used to train each candidate model in the model list based on the target training set, and to obtain the target model and model loss corresponding to each candidate model. The model loss refers to the loss value obtained by the loss function in the last calculation when the model training is completed. The target engine library determination module is used to establish an engine library based on the target model and the target sample set, obtain an initial engine library corresponding to each target model, and adjust each initial engine library based on each engine algorithm in the engine list to obtain a target engine library corresponding to each model engine combination; the model engine combination includes a matching target model and an engine algorithm. The engine loss determination module is used to verify the corresponding target engine library based on the target validation set and the target model, and obtain the engine loss corresponding to each target engine library. This includes: extracting features from each validation sample in the target validation set based on each target model to obtain the target features corresponding to each validation sample under each target model; searching in the initial engine library and the target engine library corresponding to the target model based on the target features of each validation sample corresponding to the same target model to obtain the search quality scores corresponding to the initial engine library and the target engine library respectively; and generating the engine loss of the target engine library corresponding to the target model based on the search quality scores corresponding to the same target model, thus obtaining the engine loss corresponding to each target engine library. The engine loss refers to the loss of search accuracy when searching in the target engine library obtained after engine algorithm processing, compared to searching in the initial engine library. The target loss determination module is used to obtain the target loss corresponding to each model engine combination based on the model loss and engine loss corresponding to the same model engine combination. This includes: obtaining the loss weight corresponding to the target business scenario; the loss weight is determined based on the scenario retrieval conditions corresponding to the target business scenario; adjusting the model loss and engine loss corresponding to the same model engine combination based on the loss weight to obtain the target loss corresponding to each model engine combination; the target loss refers to the total loss corresponding to the model engine combination obtained by fusing the model loss and engine loss corresponding to the same model engine combination; and the loss weight refers to the value used to adjust the proportion of engine loss and model loss in the target loss when calculating the target loss. The target combination determination module determines a target combination from the various model engine combinations based on the target loss; the target combination is used for sample retrieval corresponding to the target business scenario.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images