Data processing method and device, equipment and medium

Abstract

The invention relates to a data processing method and device, equipment and a medium, and the data processing method comprises the steps: receiving a first processing request sent by a client; the first processing request is sent to a second-level model, so that the second-level model determines the processing capacity of the first processing request, and the second-level model is a machine learning model used for processing the processing request sent by the client; when the second-level model determines that the first processing request is incapable of being processed, the first processing request is sent to the first-level model, meanwhile, target data corresponding to the first processing request are obtained from a preset memory, and the preset memory is used for storing the target data corresponding to the first processing request in a centralized mode; the data set corresponding to the first-level model is larger than the data set corresponding to the second-level model; and inputting the target data into the primary model, so that the primary model processes the target data and outputs a processing result. Therefore, the data transmission loss in the data processing process is reduced, and meanwhile, the data processing efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, equipment and medium. Background Art

[0002] Deep learning is a new research direction in the field of machine learning. It refers to a machine learning technology that learns and infers specific tasks based on artificial neural network models. The main advantage of this technology is that it can automatically learn and extract key features from raw data, thereby reducing dependence on professional knowledge in related fields, and its application scenarios cover multiple fields such as image, speech, and natural language processing. A deep learning model usually refers to a learning model composed of a series of interconnected neural network models such as recurrent neural network models (RNN) and convolutional neural networks (CNN). Deep learning models play a vital role in executing deep learning tasks.

[0003] When an existing trained deep learning model performs deep learning tasks such as natural language tasks, the processed data corresponding to the deep learning task is transmitted to each learning model of the deep learning model respectively, resulting in repeated downloading of the processed data corresponding to the deep learning task.

[0004] Therefore, the existing trained deep learning models have problems of large data transmission loss and low processing efficiency when processing deep learning tasks. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a data processing method, device, equipment and medium, wherein the data processing method is applied to a deep learning model, the deep learning model includes a primary model and a secondary model, and the primary model and the secondary model are connected in series.

[0006] A first aspect of an embodiment of the present disclosure provides a data processing method, the method comprising:

[0007] Receive a first processing request sent by a client;

[0008] Sending the first processing request to the secondary model so that the secondary model determines a processing capability for the first processing request, the secondary model being a machine learning model for processing the processing request sent by the client;

[0009] When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model, and the target data corresponding to the first processing request is obtained from the preset memory at the same time, the preset memory is used to centrally store the target data corresponding to the first processing request, and the data set corresponding to the primary model is larger than the data set corresponding to the secondary model;

[0010] The target data is input into the primary model, so that the primary model processes the target data and outputs the processing results.

[0011] A second aspect of an embodiment of the present disclosure provides a data processing device, the device comprising:

[0012] A first receiving module, used for receiving a first processing request sent by a client;

[0013] A first sending module, used to send the first processing request to the secondary model, so that the secondary model determines the processing capability of the first processing request, the secondary model is a machine learning model used to process the processing request sent by the client;

[0014] A second sending module is used to send the first processing request to the primary model when the secondary model determines that it is unable to process the first processing request, and at the same time obtain the target data corresponding to the first processing request from the preset memory, the preset memory is used to centrally store the target data corresponding to the first processing request, and the data set corresponding to the primary model is larger than the data set corresponding to the secondary model;

[0015] The request processing module is used to input the target data into the primary model so that the primary model processes the target data and outputs the processing results.

[0016] A third aspect of an embodiment of the present disclosure provides an electronic device, the device comprising:

[0017] Memory;

[0018] Processor; and

[0019] A computer program, wherein the computer program is stored in a memory and is configured to be executed by a processor to implement the data processing method as described in the first aspect above.

[0020] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data processing method described in the first aspect above is implemented.

[0021] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0022] The data processing method, apparatus, device and medium provided by the embodiments of the present disclosure can receive a first processing request sent by a client. After receiving the first processing request, the first processing request is sent to the secondary model so that the secondary model determines the processing capability of the first processing request. When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model, and the target data corresponding to the first processing request is obtained from the preset memory at the same time, wherein the preset memory is used to centrally store the target data corresponding to the first processing request, the data set corresponding to the primary model is larger than the data set corresponding to the secondary model, the target data is input to the primary model, so that the primary model processes the target data and outputs the processing result, thereby, only when it is determined that the secondary model or the primary model is capable of processing the first processing request, the target data corresponding to the first processing request is sent to the module so that the module processes the first processing request, thereby reducing the data transmission loss in the data processing process and improving the data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0025] Figure 1 is a flow chart of a data processing method provided by an embodiment of the present disclosure;

[0026] Figure 2 is a flow chart of another data processing method provided by an embodiment of the present disclosure;

[0027] Figure 3 It is a structural schematic diagram of a deep learning model data processing process provided by an embodiment of the present disclosure;

[0028] Figure 4 It is a structural schematic diagram of a standard unit in a deep learning model data processing process provided by an embodiment of the present disclosure;

[0029] Figure 5 is a structural schematic diagram of a data processing device provided by an embodiment of the present disclosure;

[0030] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0033] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0034] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0035] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0036] Typically, when an existing trained deep learning model performs a deep learning task, such as a natural language task, the processed data corresponding to the deep learning task is transmitted to each learning model of the deep learning model, resulting in repeated downloading of the processed data corresponding to the deep learning task. Therefore, when the existing trained deep learning model processes deep learning tasks, there are problems of large data transmission loss and low processing efficiency. To address this problem, an embodiment of the present disclosure provides a data processing method, which is described below in conjunction with a specific embodiment.

[0037] Figure 1It is a flowchart of a data processing method provided by an embodiment of the present disclosure. The method can be executed by a data processing device. The data processing device can be implemented in software and / or hardware. The data processing device can be configured in an electronic device, such as a server or a terminal or a server cluster.

[0038] The data processing method provided by the embodiment of the present disclosure is applied to a deep learning model, wherein the deep learning model includes a primary model and a secondary model, the primary model and the secondary model are connected in series, and in terms of response speed, the secondary model is greater than the primary model, and in terms of parameter quantity, data set or processing power, the primary model is greater than the secondary model. The deep learning model may also include multiple models, for example, the deep learning model also includes a three-level model, in terms of response speed, the primary model is greater than the three-level model, and in terms of parameter quantity, data set or processing power, the three-level model is greater than the primary model.

[0039] In the embodiments of the present disclosure, the three-level model, the first-level model and the second-level model may be the same machine learning model or different machine learning models. For example, the three-level model, the first-level model and the second-level model may adopt the same basic structure or different basic structures.

[0040] Exemplarily, the three-level model can use the T5-Base model as the basic structure, including 12 layers and 200M parameters; the first-level model and the second-level model can choose the T5-Small model as the basic structure, including 6 layers and 60M parameters.

[0041] Among them, the three-level model, the first-level model and the second-level model are models that meet different levels of complexity of the same type of tasks, such as text tasks, image processing tasks, audio processing tasks, etc., but the corresponding tasks have different levels of complexity.

[0042] like Figure 1 As shown, the data processing method may include the following steps:

[0043] S110: Receive a first processing request sent by a client.

[0044] In an embodiment of the present disclosure, when responding to a data processing instruction from a client, the electronic device receives a first processing request sent by the client.

[0045] The first processing request may be any data processing request, wherein the data processing request may be a natural language processing request.

[0046] Furthermore, the first processing request includes a first processing request instruction, target data corresponding to the first processing request, and the like.

[0047] The type of the first processing request may vary depending on the deep learning model, and the type of the first processing request may include text-related tasks, image processing tasks, audio processing tasks, etc.

[0048] In some embodiments of the present disclosure, the electronic device is provided with a dedicated preset memory, which is used to store the target data corresponding to the first processing request. When the target data corresponding to the first processing request is stored in the preset memory, the target data is organized in parallel to facilitate the calculation requirements of subsequent models at different levels.

[0049] S120: Send the first processing request to the secondary model, so that the secondary model determines the processing capability of the first processing request.

[0050] In the embodiment of the present disclosure, after receiving the first processing request, the electronic device sends the first processing request to the secondary model, so that the secondary model determines the processing capability of the first processing request.

[0051] The first processing request sent to the secondary model includes a first processing request instruction, wherein the first processing request instruction includes a unique identifier corresponding to the first processing request and address information of target data corresponding to the first processing request.

[0052] The address information of the target data corresponding to the first processing request is used to obtain the target data corresponding to the first processing request from the centralized memory.

[0053] In the embodiment of the present disclosure, sending the first processing request to the secondary model can be understood as sending the first processing request instruction in the first processing request to the secondary model, and the target data corresponding to the first processing request is directly stored in the preset memory.

[0054] Optionally, a first judgment logic for the first processing request is provided in the secondary model, and based on the first judgment logic, it is judged whether the secondary model has the ability to process the first processing request.

[0055] Specifically, the electronic device performs judgment through a preset first judgment logic and sends the judgment result to the electronic device.

[0056] S130. When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model, and the target data corresponding to the first processing request is obtained from the preset memory at the same time. The preset memory is used to centrally store the target data corresponding to the first processing request. The data set corresponding to the primary model is larger than the data set corresponding to the secondary model.

[0057] In an embodiment of the present disclosure, when the secondary model determines that the electronic device is unable to process the first processing request, the electronic device sends the first processing request to the primary model and simultaneously obtains target data corresponding to the first processing request from a preset memory.

[0058] In the embodiment of the present disclosure, the data set corresponding to the primary model is larger than the data set corresponding to the secondary model, which can be understood as the processing capability of the primary model is greater than that of the secondary model.

[0059] Specifically, when the secondary model of the electronic device determines that it is unable to process the first processing request, the electronic device sends the first processing request to the primary model, and searches and obtains the target data corresponding to the first processing request from the preset memory according to the unique identifier corresponding to the first processing request.

[0060] S140, inputting the target data into the primary model, so that the primary model processes the target data and outputs the processing result.

[0061] In the disclosed embodiment, after acquiring the target data, the electronic device inputs the target data into the primary model, so that the primary model processes the target data and outputs the processing result.

[0062] At the same time, after the first-level model outputs the processing result, the electronic device feeds back the obtained processing result to the user's client.

[0063] In an embodiment of the present disclosure, it is possible to receive a first processing request sent by a client. After receiving the first processing request, the first processing request is sent to the secondary model so that the secondary model determines its processing capability for the first processing request. When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model, and target data corresponding to the first processing request is obtained from a preset memory at the same time, wherein the preset memory is used to centrally store the target data corresponding to the first processing request, the data set corresponding to the primary model is larger than the data set corresponding to the secondary model, the target data is input into the primary model, so that the primary model processes the target data and outputs the processing result. Thus, only when it is determined that the secondary model or the primary model is capable of processing the first processing request, the target data corresponding to the first processing request is sent to the module so that the module processes the first processing request, thereby reducing the data transmission loss during the data processing process and improving the data processing efficiency.

[0064] Based on the above-mentioned embodiments of the present disclosure, after S110, the data processing method may further include: determining the data type corresponding to the first processing request; when the data type corresponding to the first processing request is text, uploading the first processing request and the target data corresponding to the first processing request to the secondary model, and the secondary model processes the target data to obtain a first processing result; and determining whether to send the first processing request to the primary model based on the accuracy of the first processing result.

[0065] In an embodiment of the present disclosure, after receiving a first processing request sent by a client, the electronic device determines the data type corresponding to the first processing request. When the data type corresponding to the first processing request is text, the electronic device uploads the first processing request and the target data corresponding to the first processing request to the secondary model, and the secondary model processes the target data to obtain a first processing result. The accuracy of the first processing result is determined according to the first accuracy judgment logic corresponding to the secondary model, and based on the accuracy of the first processing result, it is determined whether to send the first processing request to the primary model.

[0066] Optionally, the data type corresponding to the first processing request may include at least one of text, audio, and picture.

[0067] In the disclosed embodiment, when the data type corresponding to the first processing request is text, since the text data is relatively small, the target data can be directly uploaded to the secondary model, and the secondary model processes the target data to obtain the first processing result.

[0068] Specifically, a first accuracy judgment logic is pre-set in the secondary model. After the secondary model obtains the first processing result, the electronic device can judge the accuracy of the first processing result through the first accuracy judgment logic, and then determine whether to send the first processing request to the primary model based on the accuracy of the first processing result.

[0069] In the disclosed embodiment, by judging the data type corresponding to the first processing request, when the data type corresponding to the first processing request is text, the target data can be directly uploaded to the secondary model and processed by the secondary model, thereby further improving the efficiency of data processing.

[0070] Furthermore, determining whether to send the first processing request to the primary model based on the accuracy of the first processing result may specifically include: when the accuracy of the first processing result is greater than or equal to a preset accuracy threshold, feeding back the first processing result to the client; when the accuracy of the first processing result is less than the preset accuracy threshold, sending the first processing request to the primary model.

[0071] In some embodiments of the present disclosure, the electronic device is able to feed back the first processing result to the client when the accuracy of the first processing result is greater than or equal to a preset accuracy threshold, and send the first processing request to the primary model when the accuracy of the first processing result is less than the preset accuracy threshold.

[0072] In other embodiments of the present disclosure, the electronic device is able to feed back the first processing result to the client when the accuracy of the first processing result is greater than or equal to a preset accuracy threshold. When the accuracy of the first processing result is less than the preset accuracy threshold, the first processing result is first fed back to the client for reference by the client user, and the first processing request is sent to the primary model. After the first-level model processes the first processing result, the first processing result is updated based on the processing result of the first-level model, so as to feed back a more detailed processing result to the client.

[0073] In the disclosed embodiment, whether to send the first processing request to the primary model can be determined by the accuracy of the first processing result, thereby ensuring that the processing result fed back to the client satisfies a preset accuracy threshold, thereby improving the accuracy of the processing result fed back to the client.

[0074] On the basis of the above-mentioned embodiments of the present disclosure, after the first processing result is fed back to the client, the data processing method may further include: receiving a second processing request for the first processing result sent by the client; sending the second processing request and the target data to the primary model, and the primary model processes the target data to obtain a second processing result; determining the accuracy of the second processing result according to the second accuracy judgment logic corresponding to the primary model, and determining whether to feed back the second processing result to the client based on the accuracy of the second processing result.

[0075] In the embodiment of the present disclosure, the second processing request may be understood as a processing request made by the client based on the first processing result after receiving the first processing result.

[0076] The second processing request may be a processing request issued to obtain a more detailed processing result based on the first processing result, or may be an extended processing request issued for the first processing request based on the first processing result.

[0077] In some embodiments of the present disclosure, the electronic device can feed back the second processing result to the client when the accuracy of the second processing result is greater than or equal to the preset accuracy threshold, and send the second processing request to the upper-level model of the first-level model when the accuracy of the second processing result is less than the preset accuracy threshold. The upper-level model of the first-level model can be a model with a higher parameter quantity or data set or processing capability than the first-level model, such as the third-level model in the above embodiment.

[0078] In other embodiments of the present disclosure, the electronic device is able to feed back the second processing result to the client when the accuracy of the second processing result is greater than or equal to a preset accuracy threshold. When the accuracy of the second processing result is less than the preset accuracy threshold, the second processing result is first fed back to the client for reference by the client user, and the second processing request is simultaneously sent to the upper-level model of the first-level model, such as the third-level model, and then after processing by the third-level model, the second processing result is updated based on the processing result of the third-level model, so as to feed back a more detailed processing result to the client.

[0079] In the embodiment of the present disclosure, by receiving a second processing request for a first processing result sent by a client, the second processing request and target data can be sent to the primary model, and the primary model processes the target data to obtain a second processing result, and then determines whether to feed back the second processing result to the client based on the accuracy of the second processing result. In this way, the data processing needs of the client user can be met and the client user experience is further improved.

[0080] In some embodiments of the present disclosure, the deep learning model also includes a three-level model, and the data set corresponding to the three-level model is larger than the one-level model. Figure 2 is a flow chart of another data processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the data processing method may include the following steps:

[0081] S210: Receive a first processing request sent by a client.

[0082] S220: Send the first processing request to the secondary model, so that the secondary model determines the processing capability of the first processing request.

[0083] In the embodiment of the present disclosure, the specific implementation of steps S210 - S220 is similar to that of steps S110 - S120 and will not be described in detail here.

[0084] When the secondary model determines that it is incapable of processing the first processing request, step S230 is executed, and when the secondary model determines that it is capable of processing the first processing request, step S260 is executed.

[0085] S230. When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model so that the primary model determines the processing capability of the first processing request.

[0086] In the embodiment of the present disclosure, the specific implementation of step S230 is similar to that of step S130 and will not be described in detail here.

[0087] S240. When the first-level model determines that it is unable to process the first processing request, the first processing request is sent to the third-level model, and target data corresponding to the first processing request is obtained from a preset memory.

[0088] In the embodiment of the present disclosure, when the primary model determines that it is capable of processing the first processing request, S270 is executed.

[0089] S250, input the target data into the three-level model, so that the three-level model processes the target data and outputs the processing results.

[0090] In the embodiment of the present disclosure, the specific implementation of steps S240-S250 is similar to that of steps S130-S140, and will not be described in detail here.

[0091] S260. When the secondary model determines that it is capable of processing the first processing request, it obtains target data corresponding to the first processing request, and inputs the target data into the secondary model, so that the secondary model processes the target data and outputs the processing result.

[0092] Specifically, when the secondary model of the electronic device determines that it is capable of processing the first processing request, it obtains the target data corresponding to the first processing request from the centralized memory, inputs the target data into the secondary model, and the secondary model executes the first processing request task, thereby enabling the secondary model to output the processing result.

[0093] S270. When the primary model determines that it is capable of processing the first processing request, the target data corresponding to the first processing request is obtained, and the target data is input into the primary model, so that the primary model processes the target data and outputs the processing result.

[0094] Specifically, when the first-level model determines that the electronic device is capable of processing the first processing request, it obtains the target data corresponding to the first processing request from the centralized memory, inputs the target data into the first-level model, and the first-level model executes the first processing request task, thereby enabling the first-level model to output the processing result.

[0095] In the embodiment of the present disclosure, after receiving the first processing request sent by the client, the first processing request can be sent to the secondary model. When the secondary model is capable of processing the first processing request, the target data corresponding to the first processing request will be obtained. Similarly, when the secondary model is unable to process the first processing request, it will be further sent to the primary model to determine whether it is capable of processing the first processing request. When the primary model is capable of processing the first processing request, the target data corresponding to the first processing request will be obtained, and so on. As a result, the data transmission loss in the data processing process is reduced, and the data processing efficiency is improved.

[0096] The following is a specific explanation using the example of a deep learning model including a secondary model, a primary model, and a tertiary model. Of course, the deep learning model can also include multiple other N-level models, which is not limited here.

[0097] Figure 3 is a structural diagram of a deep learning model data processing process provided by an embodiment of the present disclosure, such as Figure 3 As shown, it includes a centralized memory, which is arranged in an electronic device, wherein the centralized memory is the preset memory in the above-mentioned embodiment of the present disclosure, and is used to store target data corresponding to a processing request. The client is a user end that issues a processing request. The capabilities of the three-level model, the first-level model and the second-level model for processing tasks decrease step by step, that is, the processing capability of the three-level model is stronger than that of the first-level model, and the processing capability of the first-level model is stronger than that of the second-level model. The processing request can be a first processing request or a second processing request.

[0098] In the embodiment of the present disclosure, a detailed description is given by taking the first processing request as an example. When the first processing request sent by the client is received, the target data corresponding to the first processing request is organized and processed in parallel and then stored in the centralized processor. At the same time, the first processing request instruction in the first processing request is sent to the secondary model. When the secondary model determines that it is capable of processing the first processing request, the electronic device obtains the target data from the centralized storage and sends it to the secondary model, and the secondary model performs task processing and feeds back the processing result to the client; when the secondary model determines that it is incapable of processing the first processing request, the first processing request instruction in the first processing request is sent to the primary model. When the primary model determines that it is capable of processing the first processing request, the electronic device obtains the target data from the centralized storage and sends it to the primary model, and the primary model performs task processing and feeds back the processing result to the client; when the primary model determines that it is incapable of processing the first processing request, the first processing request instruction in the first processing request is sent to the tertiary model, and the target data is obtained from the centralized storage and sent to the tertiary model, and the tertiary model performs task processing and feeds back the processing result to the client.

[0099] Furthermore, when data processing is performed through a deep learning model, the deep learning model can be abstracted to obtain a target standard unit.

[0100] In the embodiment of the present disclosure, the data processing method may also include: extracting model parameters corresponding to the primary model and the secondary model respectively; determining the target model parameters from the model parameters; writing the target model parameters into a preset model to generate a standard unit; performing target processing on the standard unit to obtain a deep learning model.

[0101] The target model parameters may be model parameters common to the primary model and the secondary model.

[0102] The standard unit can also be understood as the unified model structure information in the primary model and the secondary model except for the number of layers and the parameter information of each layer.

[0103] Specifically, the electronic device can respectively extract the model parameters in the primary model and the secondary model, such as the number of layers of the primary model and the secondary model, the parameters corresponding to each layer, the position information of each layer in the model, etc., and then determine the same model parameters in the primary model and the secondary model from the extracted model parameters, determine them as target model parameters, and write the target model parameters into a preset model to generate a standard unit, where the preset model is a pre-set machine learning model.

[0104] Furthermore, target processing is performed on the standard units to obtain a deep learning model, which may specifically include: performing value assignment processing on the standard units to obtain a target model of a preset structure; and freely combining the target models to obtain a deep learning model.

[0105] The assignment process can be understood as instantiation processing based on the standard unit according to design parameters and the like.

[0106] In the embodiments of the present disclosure, standard units can be generated based on the primary model and the secondary model, and free combinations of levels can be performed based on the standard units, so as to optimize the deep learning model in terms of the number of layers and improve the rate of the deep learning data processing process.

[0107] In some other embodiments of the present disclosure, the data processing method may also include: extracting model parameters corresponding to the three-level model, the first-level model and the second-level model respectively; determining the target model parameters from the model parameters; writing the target model parameters into a preset model to generate a standard unit; performing target processing on the standard unit to obtain a deep learning model. The specific implementation method is similar to the above-mentioned implementation method of obtaining a deep learning model through the first-level model and the second-level model, and is not limited here.

[0108] Figure 4 is a schematic diagram of a structure of a standard unit in a deep learning model data processing process provided by an embodiment of the present disclosure, such as Figure 4 As shown, the target standard unit includes a standard data input interface, a result return interface and a control interface, wherein the data input interface is used for data input when the N-level model performs task processing; the result return interface is used for the N-level model to return the output result, and return the output result to the electronic device; the control interface is used to control the upload of the first processing request instruction, and then control the first processing request instruction to be uploaded to the target-level N-level model.

[0109] In the embodiments of the present disclosure, when using the deep learning model for data processing, the electronic device can abstract the deep learning model to generate a target standard unit, thereby reducing the complexity of the data processing task using the deep learning model and improving the data processing rate.

[0110] Figure 5 The data processing device in the embodiment of the present disclosure may be arranged in an electronic device, which may be a server, a terminal or a server cluster, which is not limited here.

[0111] like Figure 5 As shown, the data processing device 500 may include a first receiving module 510 , a first sending module 520 , a second sending module 530 and a request processing module 540 .

[0112] The first receiving module 510 may be configured to receive a first processing request sent by a client.

[0113] The first sending module 520 may be configured to send the first processing request to the secondary model, so that the secondary model determines a processing capability for the first processing request.

[0114] The second sending module 530 can be used to send the first processing request to the first model when the second-level model determines that it is unable to process the first processing request, and at the same time obtain the target data corresponding to the first processing request from the preset memory, the preset memory is used to centrally store the target data corresponding to the first processing request, and the data set corresponding to the first model is larger than the data set corresponding to the second-level model.

[0115] The request processing module 540 can be used to input the target data into the primary model, so that the primary model processes the target data and outputs the processing results.

[0116] In the embodiment of the present disclosure, it is possible to receive a first processing request sent by a client. After receiving the first processing request, the first processing request is sent to the secondary model so that the secondary model determines the processing capability of the first processing request. When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model, and at the same time, target data corresponding to the first processing request is obtained from a preset memory, and the target data is input into the primary model so that the primary model processes the target data and outputs the processing result. Thus, only when it is determined that the secondary model or the primary model is capable of processing the first processing request, the target data corresponding to the first processing request is sent to the module so that the module processes the first processing request, thereby reducing the data transmission loss in the data processing process and improving the data processing efficiency.

[0117] In some embodiments of the present disclosure, the data processing device 500 may further include a type judgment module 550 , a first processing module 560 and a first determination module 570 .

[0118] The type determination module 550 may be configured to determine the data type corresponding to the first processing request.

[0119] The first processing module 560 can be used to upload the first processing request and the target data corresponding to the first processing request to the secondary model when the data type corresponding to the first processing request is text, and the secondary model processes the target data to obtain a first processing result.

[0120] The first determination module 570 can be used to determine the accuracy of the first processing result according to the first accuracy judgment logic corresponding to the secondary model, and determine whether to send the first processing request to the primary model based on the accuracy of the first processing result.

[0121] In some embodiments of the present disclosure, the first determination module 570 can be specifically used to feed back the first processing result to the client when the accuracy of the first processing result is greater than or equal to a preset accuracy threshold; when the accuracy of the first processing result is less than the preset accuracy threshold, send the first processing request to the primary model.

[0122] In some embodiments of the present disclosure, the data processing device 500 may further include a second receiving module 580 , a second processing module 590 and a second determining module 5010 .

[0123] The second receiving module 580 may be configured to receive a second processing request for the first processing result sent by the client after feeding back the first processing result to the client.

[0124] The second processing module 590 can be used to send the second processing request and the target data to the primary model, and the primary model processes the target data to obtain a second processing result.

[0125] The second determination module 5010 can be used to determine the accuracy of the second processing result according to the second accuracy judgment logic corresponding to the primary model, and determine whether to feed back the second processing result to the client based on the accuracy of the second processing result.

[0126] In some embodiments of the present disclosure, the data processing device 500 may further include a first acquisition module 5020 and a third processing module 5030 .

[0127] The first acquisition module 5020 can be used to acquire target data corresponding to the first processing request when the secondary model determines that it is capable of processing the first processing request.

[0128] The third processing module 5030 can be used to input the target data into the secondary model, so that the secondary model processes the target data and outputs the processing results.

[0129] In some embodiments of the present disclosure, a three-level model is included, and the data set corresponding to the three-level model is larger than that of the one-level model. The data processing device 500 may also include a second acquisition module 5040 and a fourth processing module 5050.

[0130] The second acquisition module 5040 can be used to send the first processing request to the third-level model and acquire target data corresponding to the first processing request when the first-level model determines that it is unable to process the first processing request.

[0131] The fourth processing module 5050 can be used to input the target data into the three-level model, so that the three-level model processes the target data and outputs the processing results.

[0132] In some embodiments of the present disclosure, the data processing device 500 may further include a parameter extraction module 5060 , a third determination module 5070 , a unit generation module 5080 and a target processing module 5090 .

[0133] The parameter extraction module 5060 can be used to extract the model parameters corresponding to the primary model and the secondary model respectively.

[0134] The third determination module 5070 may be configured to determine target model parameters from the model parameters.

[0135] The unit generation module 5080 can be used to write the target model parameters into the preset model generation standard unit.

[0136] The target processing module 5090 can be used to perform target processing on the standard unit to obtain a deep learning model.

[0137] It should be noted that Figure 5 The data processing device 500 shown can execute each step in the above data processing method embodiment and realize each process and effect in the above data processing method embodiment, which will not be described in detail here.

[0138] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0139] In the disclosed embodiment, Figure 6 The electronic device shown may be a server or a terminal or a server cluster, wherein the terminal may specifically include a mobile phone, a computer or a tablet computer, a vehicle-mounted terminal, or any device that can be used for the data processing method, etc., and is not limited here.

[0140] like Figure 6As shown, the electronic device may include a processor 610 and a memory 620 storing computer program instructions.

[0141] Specifically, the processor 610 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0142] The memory 620 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 620 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 620 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 620 is a non-volatile solid-state memory. In a particular embodiment, the memory 620 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0143] The processor 610 reads and executes the computer program instructions stored in the memory 620 to perform the steps of the data processing method provided in the embodiment of the present disclosure.

[0144] In one example, the electronic device may further include a transceiver 630 and a bus 640. Figure 6 As shown, the processor 610, the memory 620 and the transceiver 630 are connected via a bus 640 and communicate with each other.

[0145] The bus 640 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 640 may include one or more buses.

[0146] The embodiments of the present disclosure further provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the data processing method provided by the embodiments of the present disclosure.

[0147] The above-mentioned storage medium may, for example, include a memory 620 of computer program instructions, and the above-mentioned instructions may be executed by the processor 610 of the electronic device to complete the data processing method provided in the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0148] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0149] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that: The method comprises: Receive a first processing request sent by a client; Sending the first processing request to a secondary model so that the secondary model determines a processing capability for the first processing request, the secondary model being a machine learning model for processing the processing request sent by the client; When the secondary model determines that it is unable to process the first processing request, the first processing request is sent to the primary model, and the target data corresponding to the first processing request is obtained from a preset memory, the preset memory is used to centrally store the target data corresponding to the first processing request, and the data set corresponding to the primary model is larger than the data set corresponding to the secondary model; The target data is input into the primary model so that the primary model processes the target data and outputs a processing result.

2. The method according to claim 1, characterized in that After receiving the first processing request sent by the client, the method further includes: Determining the data type corresponding to the first processing request; When the data type corresponding to the first processing request is text, uploading the first processing request and the target data corresponding to the first processing request to the secondary model, and the secondary model processes the target data to obtain a first processing result; The accuracy of the first processing result is determined according to a first accuracy judgment logic corresponding to the secondary model, and based on the accuracy of the first processing result, it is determined whether to send the first processing request to the primary model.

3. The method according to claim 2, characterized in that The determining whether to send the first processing request to the primary model based on the accuracy of the first processing result includes: When the accuracy of the first processing result is greater than or equal to a preset accuracy threshold, feeding back the first processing result to the client; When the accuracy of the first processing result is less than the preset accuracy threshold, the first processing request is sent to the primary model.

4. The method according to claim 3, characterized in that After feeding back the first processing result to the client, the method further includes: receiving a second processing request sent by the client for the first processing result; Sending the second processing request and the target data to the primary model, and having the primary model process the target data to obtain a second processing result; The accuracy of the second processing result is determined according to the second accuracy judgment logic corresponding to the primary model, and based on the accuracy of the second processing result, it is determined whether to feed back the second processing result to the client.

5. The method according to claim 1, characterized in that The method further comprises: When the secondary model determines that it is capable of processing the first processing request, acquiring target data corresponding to the first processing request; The target data is input into the secondary model so that the secondary model processes the target data and outputs a processing result.

6. The method according to claim 1, characterized in that A three-level model is included, wherein the data set corresponding to the three-level model is larger than that of the first-level model, and the method further includes: When the first-level model determines that it is unable to process the first processing request, sending the first processing request to the third-level model and acquiring target data corresponding to the first processing request; The target data is input into the three-level model so that the three-level model processes the target data and outputs a processing result.

7. The method according to claim 1, characterized in that The method further comprises: Extracting model parameters corresponding to the primary model and the secondary model respectively; determining target model parameters from the model parameters; Writing the target model parameters into a preset model generation standard unit; The standard unit is subjected to target processing to obtain a deep learning model.

8. A data processing device, characterized in that: The device comprises: A first receiving module, used for receiving a first processing request sent by a client; A first sending module, used for sending the first processing request to a secondary model, so that the secondary model determines a processing capability of the first processing request, wherein the secondary model is a machine learning model for processing the processing request sent by the client; a second sending module, configured to send the first processing request to the first model when the second model determines that it is unable to process the first processing request, and simultaneously obtain target data corresponding to the first processing request from a preset memory, wherein the preset memory is used to centrally store the target data corresponding to the first processing request, and the data set corresponding to the first model is larger than the data set corresponding to the second model; The request processing module is used to input the target data into the primary model, so that the primary model processes the target data and outputs the processing result.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images