Data processing method and electronic equipment

Abstract

The embodiment of the invention provides a data processing method and electronic equipment. The data processing method comprises the steps that reasoning request information is obtained; predicting the reasoning request information to obtain a predicted length of a reasoning result corresponding to the reasoning request information; the predicted length and reasoning request information are input into a target model, the target model can apply for memory resources based on the predicted length, and the memory resources are used for storing output information of the target model in the reasoning process.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data processing method and electronic device. Background Art

[0002] In many existing models, during the inference process, each inference request requests the maximum amount of memory. Although this avoids inference failures due to insufficient memory, it wastes memory resources. Summary of the Invention

[0003] The present invention provides a data processing method, including:

[0004] Obtaining inference request information;

[0005] Predicting the inference request information to obtain a predicted length of an inference result corresponding to the inference request information;

[0006] The prediction length and the inference request information are input into the target model. The target model can apply for memory resources based on the prediction length. The memory resources are used to store the output information of the target model during the inference process.

[0007] In one embodiment, predicting the inference request information to obtain a predicted length of an inference result corresponding to the inference request information includes:

[0008] Determining user preference information, wherein the user preference information includes a user preference for a length of an inference result;

[0009] The inference request information is predicted according to the user preference information to obtain the predicted length.

[0010] In one embodiment, it further includes:

[0011] Determine a length upper limit parameter of the corresponding inference result based on the predicted length, wherein the length upper limit parameter is used to limit the length of the target model output result, and the length value corresponding to the length upper limit parameter is less than the predicted length;

[0012] The upper length limit parameter is input into the target model.

[0013] In one embodiment, determining a length upper limit parameter of the corresponding inference result based on the predicted length includes at least one of the following:

[0014] Determine the length upper limit parameter based on the predicted length and a preset ratio; or

[0015] The length upper limit parameter is determined based on the predicted length and a preset adjustment value.

[0016] In one embodiment, predicting the inference request information includes at least one of the following:

[0017] Predicting the inference request information based on a pre-built prediction model; or

[0018] The inference request information is predicted based on a pre-built correspondence table between inference request information and prediction length.

[0019] In one embodiment, the prediction model is constructed by at least one of the following methods:

[0020] Training the initial model based on the historical reasoning request information and the length values of the historical reasoning results as training data to obtain the prediction model; or

[0021] The initial model is trained based on historical reasoning request information, length values of historical reasoning results and user preference information as training data to obtain the prediction model, where the user preference information includes the user's preference for the length of the reasoning result.

[0022] In one embodiment, the method further comprises:

[0023] Determining the actual length of the inference result output by the target model;

[0024] The inference request information, the predicted length, and the actual length are matched and stored in a database, and the data in the database is used to update the prediction model or the corresponding relationship table.

[0025] In one embodiment, the method further comprises:

[0026] Obtaining a target length of a target inference result, where the target inference result is an inference result output by the target model corresponding to the inference request information;

[0027] The inference request information and the target length are matched and stored in a database, and the data in the database is used to update the prediction model or the corresponding relationship table.

[0028] In one embodiment, the method further comprises:

[0029] In the case where the prediction length cannot be predicted, the inference request information is input into the target model, so that the target model applies for memory resources based on a preset strategy, and the preset strategy includes at least one of the following: applying for minimum memory resources or memory resources of a specified size each time.

[0030] Another embodiment of the present application also provides an electronic device, including:

[0031] The processor obtains inference request information, predicts the inference request information, obtains the predicted length of the inference result corresponding to the inference request information, and inputs the predicted length and the inference request information into the target model. The target model can apply for memory resources based on the predicted length, and the memory resources are used to store the output information of the target model during the inference process.

[0032] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0033] The technical solution of the present application is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the related technologies, the following briefly introduces the drawings required for use in the specific embodiments or the related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is a flow chart of the data processing method in an embodiment of the present application.

[0036] Figure 2 This is a flow chart of a data processing method in another embodiment of the present application.

[0037] Figure 3 This is an application flow chart of the data processing method in an embodiment of the present application.

[0038] Figure 4 This is a structural block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0039] Below, specific embodiments of the present application are described in detail with reference to the accompanying drawings, but are not intended to limit the present application.

[0040] It should be understood that various modifications may be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope of the present disclosure will occur to those skilled in the art.

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0042] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0043] It should also be understood that although the present application has been described with reference to certain specific examples, those skilled in the art will be able to implement many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0044] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0045] Specific embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant detail. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but rather serve merely as a basis and representative basis for the claims to teach those skilled in the art to variously employ the present disclosure with substantially any suitable detailed structure.

[0046] This description may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," each of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0047] Below, the embodiments of the present application are described in detail with reference to the accompanying drawings.

[0048] like Figure 1 As shown, the embodiment of the present application provides a data processing method, including:

[0049] S1: Get reasoning request information;

[0050] S2: predicting the inference request information to obtain a predicted length of an inference result corresponding to the inference request information;

[0051] S3: Input the prediction length and inference request information into the target model. The target model can apply for memory resources based on the prediction length. The memory resources are used to store the output information of the target model during the inference process.

[0052] The method in this embodiment is applied to a device that sends an inference request, such as a local device or a terminal device. It can also be applied to a device where the target model is located, such as a cloud server. It can also be applied to a third device, that is, a third device other than the device that sends the inference request and the device where the target model is located, such as a proxy device. The specific details are uncertain. In this embodiment, the data processing method is applied to a device where the target model is located as an example for explanation. The target model can be located in a local device or in a cloud server, and there is no limitation on this. The inference request information in this embodiment is generated and output by an application in the device. For example, the application generates inference request information in response to an operation request input by the user, or a voice request, etc. The inference request information is sent to the corresponding target model. The target model can be, but is not limited to, any inference model, including a large language model, etc. In this embodiment, the target model is described as a large language model, but it is not a limitation to this application.

[0053] When the application / client outputs the inference request information, the system obtains the inference request information, predicts the inference request information, and determines the predicted value of the length of the inference result corresponding to the inference request information, that is, the predicted length. Taking the inference result as a paragraph of text or a sentence as an example, its predicted length can represent the number of characters contained in the inference result, and the number of characters is positively correlated with the required memory resource size; or the predicted length can represent the number of tokens contained in the inference result, and the number of tokens is positively correlated with the required memory resource size. After the system obtains the predicted length, it will send it together with the inference request information to the target model, so that the inference engine in the target model obtains the inference request information and the predicted length, and matches the application memory resources for the model to process the inference request information based on the predicted length, ensuring that the target model can efficiently process the inference request information, obtain the inference result, and not waste memory resources, while avoiding the phenomenon of GPU delaying the output of the inference result due to insufficient video memory. The target model is a machine learning model that can identify natural language and / or other inputs (such as audio and video, images, tables, etc.) input to the target model, and perform comprehensive language processing tasks such as semantic analysis and question answering, thereby generating output related to the input and / or responding to the input. The target model learns the characteristics and laws of natural language by training a large amount of diverse data, so that it can understand and generate natural language. It usually has model parameters of hundreds of millions to hundreds of billions of levels (model parameters are variables that control the behavior of the target model) and can capture complex relationships and patterns in natural language. The target model can be a generative model (generative models), a generative language model (generative language models, GLMs). For example, it can specifically include large language models (large language models, LLMs), GPT (Generative Pre-trained Transformer), etc. The model involved in the embodiment of the present application can be a general large model, or it can be an expert large model obtained after fine-tuning based on demand, and the embodiment of the present application is not limited to this. During the inference process, after the target model infers and outputs one or more tokens based on the input inference request information, the input information and output information are used again as input information for subsequent token inference. During this process, the inference process is executed by the GPU or NPU, and the tokens obtained each time are first stored in the memory space corresponding to the GPU or NPU. Until the entire inference process is completed, the final inference result is obtained based on all the inferred tokens, and the corresponding tokens stored in the memory space are cleared. Therefore, the above embodiment enables the target model to apply for the memory space corresponding to the output for storing the output information during the inference process.The target model includes the model data itself, as well as the corresponding program for outputting the model reasoning service externally. For example, the application for memory space is executed by the corresponding program of the target model for outputting the model reasoning service externally.

[0054] In one embodiment, since the inference request information is sent directly to the target model, in order to obtain the inference request information in a timely manner, a proxy service is added between the inference request sending end and the target model in this embodiment. The sending end sends the thrust request information to the proxy service, and then the proxy service sends the inference request information to the target model. The system obtains the inference request information through the proxy service, and then predicts the inference request information to obtain the predicted length, and finally feeds the predicted length together with the inference request information back to the proxy service, so that the proxy service sends the predicted length and inference request information to the target model.

[0055] The proxy service can be configured in the device where the inference request information is sent, in the device where the target model is located, or in the third device mentioned above. The specific location is not fixed. In this embodiment, the proxy service is configured in the device where the target model is located as an example. In this embodiment, the target model is located in the cloud server, and the proxy service is also configured in the cloud server. This is only for illustrative purposes and does not limit this application.

[0056] When predicting inference request information, there are different methods for implementing the prediction, which are described below with reference to different embodiments:

[0057] In the first embodiment, Figure 2 As shown, the predicting of the inference request information to obtain the predicted length of the inference result corresponding to the inference request information includes:

[0058] S4: Determine user preference information, where the user preference information includes the user's preference for the length of the inference result;

[0059] S5: Predict the inference request information according to the user preference information to obtain the predicted length.

[0060] In this embodiment, the system determines user preferences. This information may include user preferences for the length of the inference results, such as whether the user prefers detailed inference results or concise, allowing immediate access to the core content of the inference results. In other words, the user prefers long or short sentences. Furthermore, user preferences may include information about which information the user prefers detailed explanations or concise explanations. Furthermore, user preferences may include information about sentence structures, language styles, and text layout styles, as well as content that requires detailed explanations and content that does not require detailed explanations. For example, a user may prefer explanations of technical terms and examples in inference results, or may prefer direct output of weather forecasts and clothing and travel recommendations. By understanding this user preference information, the system can combine this information with inference request information to predict the length of the inference result, obtaining a predicted value for the length of the inference result that matches the user's needs.

[0061] For example, based on the inference request information, the system first determines the content to be inferred. Specifically, this may include, but is not limited to, semantic analysis and keyword recognition. Based on the determined content, the system may further determine its type and match it with the obtained user preference information to determine whether it has a corresponding user preference. Alternatively, the system may directly match the determined content with the obtained user preference information to determine whether it has a corresponding user preference. If so, a prediction may be made based on the corresponding user preference and the inference request information to determine the predicted length of the inference result. The inference result may contain information that matches the user preference, such as a long sentence type with rich information or a short sentence type with less information.

[0062] In the second embodiment, the method further includes:

[0063] S6: determining a length upper limit parameter of the corresponding inference result based on the predicted length, wherein the length upper limit parameter is used to limit the length of the target model output result, and the length value corresponding to the length upper limit parameter is less than the predicted length;

[0064] S7: Input the upper length limit parameter into the target model.

[0065] In this embodiment, the system first predicts the length of the inference result based on the inference request information. This prediction process can be performed solely in conjunction with the inference request information, or it can be performed in conjunction with the inference request information and the aforementioned user preference information. After obtaining the predicted length, the system determines the corresponding inference result length upper limit parameter based on the predicted length. Different predicted lengths correspond to different length upper limit parameters, that is, each predicted length has a corresponding length upper limit parameter. Each length upper limit parameter is smaller than the corresponding predicted length. The length upper limit parameter can be experimentally determined or calculated based on the corresponding predicted length, the specific details of which vary. Each length upper limit parameter is used to limit the length of the target model output result so that the memory resources requested by the target model engine based on the predicted length can meet the memory resources required by the target model during the processing of the inference request information and the generation of the inference result. In other words, the length of the inference result is limited by the length upper limit parameter, so that when the engine requests memory resources based on a predicted length greater than the length upper limit parameter, the memory resources will inevitably meet the memory requirements of the inference result output with a length equal to the length upper limit parameter. The difference between the length corresponding to the length upper limit parameter and the predicted length is within a preset range. Since the inference result corresponding to the predicted length is a relatively good result for the inference request information, setting the length upper limit parameter can ensure that the actual inference result length of the target model is within the requested memory resources while ensuring the output effect.

[0066] Furthermore, determining a length upper limit parameter of the corresponding inference result based on the predicted length includes at least one of the following:

[0067] S8: Determine the length upper limit parameter based on the predicted length and a preset ratio; or

[0068] S9: Determine the length upper limit parameter based on the predicted length and a preset adjustment value.

[0069] For example, in this embodiment, when the system determines the corresponding upper length limit parameter based on the predicted length, it does so by calculation based on the predicted length. The calculation method is not unique. For example, in one embodiment, a preset ratio value, such as 1 / 9 or 1 / 8, or a percentage, such as 90% or 80%, may be used. The predicted length is then multiplied by the preset ratio to obtain the upper length limit parameter corresponding to the predicted length. In another embodiment, an adjustment value is preset, which can be a fixed value. When calculating the upper length limit parameter, the difference between the predicted length and the adjustment value is calculated to determine the upper length limit parameter based on the difference. Alternatively, there may be multiple adjustment values, each corresponding to a predicted length range, with different adjustment values corresponding to different predicted length ranges. For example, a longer predicted length corresponds to a larger adjustment value than a shorter predicted length. After obtaining the predicted length, the system first determines the predicted length range within which the predicted length falls. The adjustment value corresponding to the predicted length range is then determined as a target adjustment value. The difference between the predicted length and the target adjustment value is then calculated to obtain the upper length limit parameter corresponding to the predicted length.

[0070] In each of the above embodiments, when performing prediction based on the inference request information, it may be performed by a processor in the system or by using a pre-trained prediction model. For example, the prediction of the inference request information includes at least one of the following:

[0071] S10: Predicting the inference request information based on a pre-built prediction model; or

[0072] S11: Predicting the inference request information based on a pre-built correspondence table between inference request information and prediction length.

[0073] In Example 1, to improve the efficiency and accuracy of length prediction, the system can construct an initial model and then train it based on the acquired training data. This results in a prediction model capable of predicting the length of the inference result based on the inference request information, that is, a prediction model capable of calculating the predicted length. After the prediction model is trained, the system can input the inference request information to be predicted into the prediction model, which then outputs the predicted length.

[0074] In the second embodiment, the system can pre-construct a correspondence table between inference request information and predicted lengths. The relationship table stores the predicted lengths of different inference request information and corresponding inference results. The data in the relationship table can be obtained by the system based on its own analysis and collation of historical data, or it can be data input by the user after collection. The specific details are not fixed. After obtaining the correspondence table, the system can use the inference request information to match and search the correspondence table to determine the predicted length corresponding to the inference request information. The predicted length in the correspondence table can be a length that is not combined with user preferences, or a predicted length determined in combination with user preferences. Specifically, different types of predicted lengths can be selected according to actual needs to construct the correspondence table.

[0075] Furthermore, the prediction model is constructed by at least one of the following methods:

[0076] S12: Based on the historical reasoning request information and the length value of the historical reasoning result as training data, the initial model is trained to obtain the prediction model; or

[0077] S13: Based on historical reasoning request information, length values of historical reasoning results and user preference information as training data, the initial model is trained to obtain the prediction model, where the user preference information includes the user's preference for the length of the reasoning result.

[0078] For example, the training data may include a large amount of historical reasoning request information and the actual length of the corresponding reasoning results. The historical reasoning request information and the corresponding reasoning results may be recorded by the system, or collected by the user and input into the system. The reasoning results may be output by the target model, or output by the target model and other different reasoning models. In another embodiment, the above-mentioned user preference information may also be introduced into the training data. For example, the reasoning results of each historical request information may be adjusted according to the user preference so that the length of each reasoning result matches the user preference. Then, the initial model is trained based on the historical reasoning request information and the adjusted reasoning results as training data to obtain a prediction model.

[0079] In one embodiment, the method further comprises:

[0080] S14: Determine the actual length of the inference result output by the target model;

[0081] S15: Match and store the inference request information, predicted length and actual length in a database, where the data in the database is used to update the prediction model or the corresponding relationship table.

[0082] In this embodiment, in order to enable the above-mentioned prediction model to always be in a learning state and improve its prediction ability, the system will determine the actual length of the inference result corresponding to each inference request information, that is, determine the actual length of the inference result output by the target model for each inference request information. Then, the predicted length of the inference request information and the corresponding inference result, and the actual length of the inference result are matched and stored in the database. Afterwards, the prediction model can be retrained based on the data stored in the database on a regular basis to update its parameters and improve its prediction ability, or it can be updated after each prediction is completed, the specific details are not determined. The data in the database can also be regularly optimized according to time information, filtering out data with a long storage time, or performing deduplication processing. For example, if there are multiple groups of data for the same inference request information, corresponding to different times, deduplication processing can be performed on them, and only one copy of the data can be retained to avoid the data in the database from forming class imbalance data and causing interference to the prediction model.

[0083] In another embodiment, the method further comprises:

[0084] S16: Obtain a target length of a target inference result, where the target inference result is an inference result output by the target model corresponding to the inference request information;

[0085] S17: Match and store the inference request information and the target length in a database, where the data in the database is used to update the prediction model or the corresponding relationship table.

[0086] In this embodiment, the system obtains the inference results output by the target model for each inference request information. In this embodiment, the inference results output by the target model for each inference request are defined as target inference results. The system collects each inference request information input into the target model and the target inference result output by the target model, and determines the target length of each target inference result. The inference request information and the target length of the corresponding target inference result are then matched and stored in a database. The data stored in the database is used to update the prediction model and the correspondence table. The update can be a scheduled update, such as a predetermined period. At the end of each period, the prediction model and the correspondence table are updated using the data in the database. The length of the period is variable and can be determined according to actual conditions. Alternatively, when the prediction model or the correspondence table fails to predict and cannot provide a predicted length, or when the predicted length given differs significantly from the length of the actual inference result, the data in the database can be used to update the prediction model or the correspondence table.

[0087] Furthermore, the method further comprises:

[0088] S18: If the prediction length cannot be predicted, the inference request information is input into the target model, so that the target model applies for memory resources based on a preset strategy, and the preset strategy includes at least one of the following: applying for minimum memory resources or memory resources of a specified size each time.

[0089] The failure to predict the predicted length described in this embodiment includes that the prediction model fails to predict the predicted length, or the predicted length is obviously unreasonable, such as the amount of data contained in the inference request information is large, and the length of the inference result output by the target model is too short, so it is known that the predicted length is abnormal and the prediction fails. Or the target length that matches the inference request information cannot be queried based on the correspondence table, etc. For the above-mentioned prediction failure, the system will input the inference request information into the target model, so that the target model applies for memory resources based on a preset strategy, that is, when the target model responds to the lack of a predicted length in the obtained information, it directly applies for memory resources based on a preset strategy. The preset strategy may include applying for memory resources based on the inference request information, such as applying for memory resources based on the amount of data in the inference request information, or applying for minimum memory resources or memory resources of a specified size each time for the target model to use for reasoning. After each use of the memory resources applied for a single time, the model will apply for memory resources of the same size again for continued use by the model.

[0090] In addition, if the prediction model or correspondence table cannot provide inference request information of the predicted length, the system will obtain the inference result corresponding to the inference request information output by the target model and determine the length of the inference result. The inference request information, inference result and inference result length are then matched and stored in the database, and an importance flag is set so that subsequent updates to the prediction model and correspondence table can focus on this type of data and improve the corresponding prediction capabilities.

[0091] In practical applications, such as Figure 3 As shown, for example, taking the target model as a large language model, the inference request information is sent by the client, received by the proxy service, and then forwarded to the large language model. The system will obtain the inference request information based on the proxy service, and input it into the prediction model to obtain the predicted length, which corresponds to the predicted number of tokens in the figure. The predicted length is then sent to the large language model together with the inference request information. The large language model will calculate the required memory size based on the number of tokens received, apply for memory of the corresponding size, complete the inference, and output the inference result. The inference result will be forwarded to the client through the proxy service. The system obtains the inference result through the proxy service, and saves the inference request information, the predicted length, and the actual length of the inference result, that is, in the saved history record and history record module in the corresponding figure, for subsequent updates of the prediction model.

[0092] like Figure 4As shown, another embodiment of the present application also provides an electronic device, including:

[0093] The processor obtains inference request information, predicts the inference request information, obtains the predicted length of the inference result corresponding to the inference request information, and inputs the predicted length and the inference request information into the target model. The target model can apply for memory resources based on the predicted length, and the memory resources are used to store the output information of the target model during the inference process.

[0094] In one embodiment, predicting the inference request information to obtain a predicted length of an inference result corresponding to the inference request information includes:

[0095] Determining user preference information, wherein the user preference information includes a user preference for a length of an inference result;

[0096] The inference request information is predicted according to the user preference information to obtain the predicted length.

[0097] In one embodiment, the processor is further configured to:

[0098] Determine a length upper limit parameter of the corresponding inference result based on the predicted length, wherein the length upper limit parameter is used to limit the length of the target model output result, and the length value corresponding to the length upper limit parameter is less than the predicted length;

[0099] The upper length limit parameter is input into the target model.

[0100] In one embodiment, determining a length upper limit parameter of the corresponding inference result based on the predicted length includes at least one of the following:

[0101] Determine the length upper limit parameter based on the predicted length and a preset ratio; or

[0102] The length upper limit parameter is determined based on the predicted length and a preset adjustment value.

[0103] In one embodiment, predicting the inference request information includes at least one of the following:

[0104] Predicting the inference request information based on a pre-built prediction model; or

[0105] The inference request information is predicted based on a pre-built correspondence table between inference request information and prediction length.

[0106] In one embodiment, the prediction model is constructed by at least one of the following methods:

[0107] Training the initial model based on the historical reasoning request information and the length values of the historical reasoning results as training data to obtain the prediction model; or

[0108] The initial model is trained based on historical reasoning request information, length values of historical reasoning results and user preference information as training data to obtain the prediction model, where the user preference information includes the user's preference for the length of the reasoning result.

[0109] In one embodiment, the processor is further configured to:

[0110] Determining the actual length of the inference result output by the target model;

[0111] The inference request information, the predicted length, and the actual length are matched and stored in a database, and the data in the database is used to update the prediction model or the corresponding relationship table.

[0112] In one embodiment, the processor is further configured to:

[0113] Obtaining a target length of a target inference result, where the target inference result is an inference result output by the target model corresponding to the inference request information;

[0114] The inference request information and the target length are matched and stored in a database, and the data in the database is used to update the prediction model or the corresponding relationship table.

[0115] In one embodiment, the processor is further configured to:

[0116] In the case where the prediction length cannot be predicted, the inference request information is input into the target model, so that the target model applies for memory resources based on a preset strategy, and the preset strategy includes at least one of the following: applying for minimum memory resources or memory resources of a specified size each time.

[0117] Furthermore, an embodiment of the present application further provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method described above. It should be understood that each solution in this embodiment has the corresponding technical effects of the above method embodiment, and will not be further described here.

[0118] Furthermore, an embodiment of the present application also provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions, which, when executed, enable at least one processor to perform a data processing method such as in the embodiment described above.

[0119] It should be noted that the computer storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory medium (RAM), a read-only memory medium (ROM), an erasable programmable read-only memory medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program configured for use by or in conjunction with an instruction execution system, system, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, antenna, optical cable, RF, or any suitable combination thereof.

[0120] In addition, it will be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0121] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0122] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

Claims

1. A data processing method, comprising: Obtaining inference request information; Predicting the inference request information to obtain a predicted length of an inference result corresponding to the inference request information; The prediction length and the inference request information are input into the target model. The target model can apply for memory resources based on the prediction length. The memory resources are used to store the output information of the target model during the inference process.

2. The data processing method according to claim 1, wherein predicting the inference request information to obtain a predicted length of an inference result corresponding to the inference request information comprises: Determining user preference information, wherein the user preference information includes a user preference for a length of an inference result; The inference request information is predicted according to the user preference information to obtain the predicted length.

3. The data processing method according to claim 1, further comprising: Determine a length upper limit parameter of the corresponding inference result based on the predicted length, wherein the length upper limit parameter is used to limit the length of the target model output result, and the length value corresponding to the length upper limit parameter is less than the predicted length; The upper length limit parameter is input into the target model.

4. The data processing method according to claim 3, wherein determining the upper limit parameter of the length of the corresponding inference result based on the predicted length comprises at least one of the following: Determine the length upper limit parameter based on the predicted length and a preset ratio; or The length upper limit parameter is determined based on the predicted length and a preset adjustment value.

5. The data processing method according to claim 1, wherein the predicting the inference request information comprises at least one of the following: Predicting the inference request information based on a pre-built prediction model; or The inference request information is predicted based on a pre-built correspondence table between inference request information and prediction length.

6. The data processing method according to claim 5, wherein the prediction model is constructed by at least one of the following methods: Training the initial model based on the historical reasoning request information and the length values of the historical reasoning results as training data to obtain the prediction model; or The initial model is trained based on historical reasoning request information, length values of historical reasoning results and user preference information as training data to obtain the prediction model, where the user preference information includes the user's preference for the length of the reasoning result.

7. The data processing method according to claim 5, further comprising: Determining the actual length of the inference result output by the target model; The inference request information, the predicted length, and the actual length are matched and stored in a database, and the data in the database is used to update the prediction model or the corresponding relationship table.

8. The data processing method according to claim 5, further comprising: Obtaining a target length of a target inference result, where the target inference result is an inference result output by the target model corresponding to the inference request information; The inference request information and the target length are matched and stored in a database, and the data in the database is used to update the prediction model or the corresponding relationship table.

9. The data processing method according to claim 1, further comprising: In the case where the prediction length cannot be predicted, the inference request information is input into the target model, so that the target model applies for memory resources based on a preset strategy, and the preset strategy includes at least one of the following: applying for minimum memory resources or memory resources of a specified size each time.

10. An electronic device comprising: The processor obtains inference request information, predicts the inference request information, obtains the predicted length of the inference result corresponding to the inference request information, and inputs the predicted length and the inference request information into the target model. The target model can apply for memory resources based on the predicted length, and the memory resources are used to store the output information of the target model during the inference process.

Images