Content generation method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a content generation method and device, electronic equipment and a computer readable storage medium, the method comprises the steps that target content of a to-be-generated task is generated based on a network model, the network model generates the target content through multi-step reasoning, and the multi-step reasoning process comprises the steps that the target content of the to-be-generated task is generated based on the network model of the first sparseness, and the target content of the to-be-generated task is obtained; obtaining a first output result of the first target step reasoning; under the condition that dense calculation conditions are met, a second output result of next-step reasoning of the first target step reasoning is obtained based on the network model of second sparseness and the first output result, and the second sparseness is smaller than the first sparseness, so that the reasoning speed can be increased, and the reasoning efficiency can be improved. And error accumulation and performance loss can be reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a content generation method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] The scale of parameters contained in network models is getting larger and larger. For example, large language models used for natural language processing tasks have brought challenges to the deployment and application of network models. It is very important to further reduce the storage space and resource consumption of network models during operation. In related technologies, weight compression methods can be used. For example, unimportant weights in the parameters of the network model are discarded to achieve sparsity. Although the inference generation of content based on the sparse network model can reduce the amount of weight data loaded, when the sparsity rate is high, it will reduce the performance of the network model, which may lead to unavailability and reduce the accuracy of the inference generated content. Summary of the invention

[0003] The present disclosure provides a content generation method and device, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present disclosure provides a content generation method, the content generation method comprising:

[0005] Based on the network model, the target content of the task to be generated is generated, wherein the network model generates the target content through multi-step reasoning, and the multi-step reasoning process includes:

[0006] Based on the network model of the first sparsity, obtaining a first output result of the first target step reasoning;

[0007] When dense computing conditions are met, a second output result of the next step reasoning of the first target step reasoning is obtained based on the network model of the second sparsity and the first output result, wherein the second sparsity is less than the first sparsity.

[0008] In a second aspect, the present disclosure provides a content generation device, the content generation device comprising:

[0009] A generation module is used to generate target content of the task to be generated based on the network model, wherein the network model generates the target content through multi-step reasoning. For the multi-step reasoning process, the generation module is also used to:

[0010] Based on the network model of the first sparsity, obtaining a first output result of the first target step reasoning;

[0011] When dense computing conditions are met, a second output result of the next step reasoning of the first target step reasoning is obtained based on the network model of the second sparsity and the first output result, wherein the second sparsity is less than the first sparsity.

[0012] In a third aspect, the present disclosure provides an electronic device, comprising: multiple processing cores; and an on-chip network configured to exchange data between the multiple processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the above-mentioned content generation method.

[0013] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned content generation method when executed by a processor / processing core.

[0014] The embodiments provided by the present disclosure generate target content of the task to be generated based on a network model, and in the multi-step reasoning process of generating the target content, the first output result of the first target step reasoning can be obtained based on the network model with a first sparsity. In this way, through sparse reasoning calculation, the calculation amount of the network model can be reduced and the reasoning speed can be improved. In addition, when the dense computing conditions are met, the second output result of the next step reasoning of the first target step reasoning can be obtained based on the network model with a second sparsity smaller than the first sparsity and the first output result. In this way, after the sparse reasoning, the use of a smaller sparsity reasoning method can reduce the performance loss caused by the error accumulation of the network model under sparse reasoning, thereby improving the performance of the network model and the generation quality of the target content.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 A flowchart of a content generation method provided by an embodiment of the present disclosure;

[0018] Figure 2An example diagram of multi-step reasoning within a cycle in the content generation method in an embodiment of the present disclosure;

[0019] Figure 3 A block diagram of a content generation device provided by an embodiment of the present disclosure;

[0020] Figure 4 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0022] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0023] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0024] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0025] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0026] The scale of parameters contained in network models is getting larger and larger. For example, large language models used for natural language processing tasks have brought challenges to the deployment and application of network models. It is very important to further reduce the storage space and resource consumption of network models during operation. In related technologies, weight compression methods can be used. For example, unimportant weights in the parameters of the network model are discarded to achieve sparsity. Although the inference generation of content based on the sparse network model can reduce the amount of weight data loaded, when the sparsity rate is high, it will reduce the performance of the network model, which may lead to unavailability and reduce the accuracy of the inference generated content.

[0027] According to the content generation method of the embodiment of the present disclosure, for the multi-step reasoning process of generating target content based on the network model, a network model with a larger first sparsity can be used for multiple first target step reasoning to obtain a first output result, reduce the weight loading and calculation amount in the reasoning, and improve the reasoning speed. When the dense computing conditions are met, the second output result of the next step of reasoning can be obtained based on the network model with a smaller second sparsity and the first output result. In this way, after multiple sparse reasonings, a denser reasoning method can be used again, which can reduce the performance degradation caused by the error accumulation of the network model under sparse reasoning and improve the accuracy and performance of the network model reasoning.

[0028] The content generation method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory. Alternatively, the method can be executed by a server.

[0029] Figure 1 A flowchart of a content generation method provided by an embodiment of the present disclosure. Figure 1 , the method comprising:

[0030] S101: Generate target content of a task to be generated based on a network model, wherein the network model generates the target content through multi-step reasoning.

[0031] For example, if the task to be generated is to generate an article with the theme of going home, the network model can generate an article based on the requirements of the task to be generated. When generating the article, the network model can generate it step by step through multi-step reasoning.

[0032] In a possible embodiment, the network model is, for example, a large language model. When the large language model is inferring, the result output is serial and autoregressive, that is, the result of the decoding output (also called a token) will be fed back to the input layer, and then decoded to output the next token until the termination condition is met. In this case, the batch size (batch size) of the large language model input is 1, and the batch size represents the number of tokens processed simultaneously each time. In the decoding process of a single token, the vector representation of the key and value of the token inferred in the previous historical step in each layer of the network model can be saved to provide for the next step of reasoning, avoiding recalculation and reducing the amount of calculation. The vector representation of the key and value can also be called a latent vector representation (kv cache), which is the main carrier of short-term memory and stores the information of each token in a certain context.

[0033] It should be noted that in the embodiments of the present disclosure, there is no limitation on the network model, and the network model can be used to perform any of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. The output result of the network model reasoning includes any of images, speech, text, and videos.

[0034] S102: The multi-step reasoning process includes: obtaining a first output result of a first target step reasoning based on a network model with a first sparsity; obtaining a second output result of a next step reasoning of the first target step reasoning based on a network model with a second sparsity and the first output result when dense computing conditions are met, wherein the second sparsity is less than the first sparsity.

[0035] In the disclosed embodiment, sparsity can represent the proportion of weights that are not loaded or discarded in the network model. Sparsity is inversely correlated with the weight loading amount or calculation amount. The larger the sparsity, the fewer weights are loaded or involved in the calculation in the network model. In the network model, the weight loading amount is usually one of the main factors affecting the reasoning speed. Therefore, less weight loading can usually speed up the reasoning speed of the network model.

[0036] However, the use of sparsity may affect the reasoning performance of the network model. When the sparsity is very high, the error is large and gradually accumulates each time the reasoning is performed, which may also cause the network model to be unavailable and unable to obtain the expected output. Therefore, in the embodiment of the present disclosure, the first sparsity is a preset value greater than 0, the second sparsity is greater than or equal to 0, and the second sparsity is greater than the first sparsity. In the multi-step reasoning process of the network model, sparse reasoning can be performed based on the network model with the first sparsity to obtain the first output result, and then reasoning can be performed based on the network model with the second sparsity to obtain the second output result. In a possible embodiment, the second sparsity can be set to 0. When the second sparsity is 0, the network model is expressed as For a complete dense model, when the first sparsity is greater than 0, the network model is represented as a sparse model. In this way, after a certain number of steps of sparse reasoning based on the sparse model, the dense model is used for dense reasoning, and the latent vector representation of the token generated by the sparse model reasoning can be corrected, thereby realizing self-correction of the network model, reducing the error accumulation caused by sparse reasoning, and improving accuracy. Then, in the next reasoning after obtaining the second output result, it is determined whether the dense computing conditions are met. If not, the network model based on the first sparsity continues to be reasoned. When the dense computing conditions are met, the network model based on the second sparsity is reasoned again until the reasoning end conditions are met to obtain the final generated target content.

[0037] In the disclosed embodiment, the target content of the task to be generated is generated based on the network model, and in the multi-step reasoning process of the network model, the first output result of the first target step reasoning is obtained based on the network model with a first sparsity, and when the dense computing condition is met, the second output result of the next step reasoning of the first target step reasoning is obtained based on the network model with a second sparsity and the first output result, and the second sparsity is less than the first sparsity. In this way, in the multi-step reasoning process of the network model, not only can the reasoning speed of the network model be improved and the consumption and demand of reasoning computing resources be reduced through sparse reasoning, but also when the dense computing condition is met, reasoning can be performed based on the network model with a smaller sparsity, so as to achieve self-correction to reduce error accumulation, alleviate the performance loss of the network model, and improve accuracy.

[0038] In addition, in a possible embodiment, the model data of the network model is stored in an off-chip memory. Then, when executing a multi-step reasoning process, it also includes: obtaining the model data of the network model from the off-chip memory to execute a multi-step reasoning process based on the model data of the network model.

[0039] The content generation method in the disclosed embodiment can be executed by a processor within the chip of an electronic device, and the model data of the network model may occupy a large storage space. Storing the model data in the on-chip memory may affect the content generation efficiency. Therefore, the model data may be stored in an off-chip memory. When generating target content based on the network model, the model data of the network model may be obtained from the off-chip memory, wherein the off-chip memory means being located outside the chip, and may be other locations on the motherboard or a separate storage device, and usually has a larger capacity.

[0040] The content generation method according to the embodiment of the present disclosure is described below.

[0041] In the embodiment of the present disclosure, several possible embodiments are provided for the case where the dense computing condition is met in the above step S102:

[0042] In a possible embodiment, the situation where the dense computing condition is satisfied includes: when reaching the x*Kth reasoning step, it is determined that the dense computing condition is satisfied, where K represents the number of reasoning steps in a cycle, x represents the number of cycles, and x and K are both positive integers.

[0043] In the embodiment of the present disclosure, when reaching the x*Kth step of reasoning, it is determined that the dense computing condition is satisfied. It can be understood that within a cycle K, the first K-1 times are reasoned based on the network model of the first sparsity, and the Kth time is reasoned based on the network model of the second sparsity. Taking the second sparsity as 0 and the first sparsity as r as an example, that is, there is one dense reasoning in every K feedforward reasonings, the reasoning speedup ratio of the network model can be expressed as:

[0044]

[0045] Among them, the total weight loading of K-1 sparse inference iterations is (K-1)(1-r)*network model parameter quantity, and the total weight loading of 1 dense iteration is 1*network model parameter quantity. The sum of the two is the total weight loading within a cycle K. In the related technology, the total weight loading of K inferences is usually K*network model parameter quantity. Therefore, in the embodiments of the present disclosure, the inference speed of the network model is improved compared with the related technology.

[0046] Regarding the method for determining the values ​​of K and r, the present disclosure also provides possible implementation examples, specifically including: determining a first candidate value of K and a second candidate value of the first sparsity; determining a candidate value combination corresponding to K and the first sparsity based on the first candidate value and the second candidate value; performing a performance test on the network model based on the candidate value combination, determining a target value combination from the candidate value combination, and obtaining the first target value of K and the second target value of the first sparsity based on the target value combination.

[0047] For example, the first candidate value of the period K and the second candidate value of the first sparsity r can be preset according to experience and needs. Based on the test set, the performance of the network model under different combinations of K and r can be tested by network search, so as to comprehensively determine the first target value of K and the second target value of r according to needs, wherein the performance test includes, for example, the acceleration ratio and accuracy of the network model. Grid search is a hyperparameter optimization method for determining the best hyperparameter combination for a machine learning model. This method evaluates the model by systematically traversing all possible combinations of predefined hyperparameters to find the optimal parameter settings. For example, a target value combination that is not less than the maximum acceleration ratio under the set performance, or a target value combination that is not less than the maximum performance under the set acceleration ratio, etc. can be selected.

[0048] In a possible embodiment, the situation where the dense computing condition is satisfied includes: for multi-step reasoning in any cycle, when a preset number of reasoning steps in any cycle is reached, it is determined that the dense computing condition is satisfied.

[0049] In the disclosed embodiment, within a cycle, the dense computing condition may not be limited to being satisfied only at the x*Kth reasoning step, and the number and order may not be limited. For example, within a cycle K, the dense computing condition may be satisfied at multiple preset reasoning steps, that is, reasoning is performed based on a network model with a smaller second sparsity. For example, if K takes a value of 5, it can be determined that the first, second and third reasoning steps within the cycle are based on the network model with the first sparsity, and the fourth and fifth reasoning steps are based on the network model with the second sparsity. For another example, the first, third and fourth reasoning steps within the cycle are based on the network model with the first sparsity, and the second and fifth reasoning steps are based on the network model with the second sparsity.

[0050] In addition, in the embodiments of the present disclosure, the preset number of inference steps in each cycle can be the same or different, and can also change dynamically. The period K, the first sparsity and the preset number of inference steps, that is, the arrangement of dense calculations and sparse calculations in one cycle, can be modeled as an optimization problem. The corresponding target optimization function includes the performance and accuracy of the network model, and then based on the grid search method, the values ​​and settings that meet the performance and accuracy requirements are selected.

[0051] Furthermore, in the embodiment of the present disclosure, in order to further improve the performance and reasoning speed of the network model, the value of the period K, the preset number of reasoning steps within the period K, and the first sparsity and the second sparsity can also be dynamically adjusted.

[0052] A possible embodiment includes: determining performance information of the network model according to the first output result; adjusting the value of K and / or adjusting the preset number of inference steps within the cycle according to the performance information.

[0053] Among them, the performance information of the network model, such as accuracy, inference speed, etc., is not limited in the embodiments of the present disclosure.

[0054] For example, based on the first output result, it is determined that the accuracy of the network model is low, which means that the accumulated error of sparse reasoning may be large. Therefore, the value of K can be lowered, so that the number of steps to meet the dense computing conditions will be faster and more frequent, that is, the number of inferences performed by the network model based on the second sparsity can be increased, thereby improving the accuracy. For example, when it is determined that the accuracy of the network model is low, the number of preset inference steps within a cycle can be increased, thereby increasing the number of inferences performed by the network model based on the second sparsity, which can alleviate the errors caused by sparse reasoning and improve the accuracy of the network model.

[0055] A possible embodiment includes: obtaining available resource information currently corresponding to the network model; adjusting the value of K and / or adjusting the preset number of inference steps within the cycle according to the available resource information.

[0056] Among them, the available resource information may include transmission bandwidth resource information, storage resource information, computing resource information, etc., which is not limited in the embodiments of the present disclosure. It can be understood that the lower the sparsity, the more parameters are loaded, and the more resource information is required to be consumed. Therefore, in the embodiments of the present disclosure, the value of K and the preset number of inference steps within the cycle can be dynamically adjusted according to the current available resource information, which can improve resource utilization and availability.

[0057] For example, if the current bandwidth resource information is limited, that is, the available bandwidth resource information is small, the value of K can be increased, that is, the number of times the network model based on the second sparsity performs inference can be reduced, so that the network model can perform inference normally.

[0058] In addition, in the embodiments of the present disclosure, the performance information and available resource information can be comprehensively considered to adjust the value of K and / or the preset number of inference steps within the cycle, and the values ​​of the first sparsity and / or the second sparsity can be adjusted based on the performance information and / or available resource information. Of course, adjustments can also be made based on other factors, which are not limited in the embodiments of the present disclosure.

[0059] In one possible implementation, performance information of the network model is determined based on the first output result; the step of adjusting the value of K based on the performance information, and / or adjusting the preset number of inference steps within the cycle, may be a step in a training phase before using the network model for inference; or, in the inference phase using the network model, a step that is dynamically performed based on the accuracy of the model determined in a certain manner.

[0060] It should be noted that in the embodiments of the present disclosure, reasoning is performed based on the network model of the first sparsity or the second sparsity to obtain output results. Reasoning based on a sparsity greater than 0 can be called sparse reasoning, and in the embodiments of the present disclosure, sparse reasoning can include dynamic sparsity or static sparsity. Dynamic sparsity means that the loading (or discarding) of parameters such as the weights of the network model is controlled by factors such as the input and resource information of the network model, and the selection of parameters to be loaded changes dynamically. Static sparsity means that the selection of parameters to be loaded in the network model is fixed, that is, it is not controlled by input or other factors. The embodiments of the present disclosure are applicable to different scenarios of dynamic sparsity and static sparsity, which are not limited in the embodiments of the present disclosure.

[0061] In this way, in the embodiment of the present disclosure, during the reasoning process of the network model, dynamic adjustments can be made according to actual conditions, which can further improve the balance between the reasoning speed and performance of the network model and improve the overall reasoning effect.

[0062] In an embodiment of the present disclosure, for the above-mentioned step S102, based on the network model with the second sparsity and the first output result, a second output result of the next reasoning of the first target step reasoning is obtained, including: based on the network model with the second sparsity, according to the first output result, feedforward reasoning is performed to update the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result; according to the updated latent vector representation of the first output result, the second output result of the next reasoning of the first target step reasoning is obtained.

[0063] In a possible embodiment, the cycle K takes a value greater than or equal to 3, and when the x*Kth reasoning step is reached, the first output result corresponding to the network model based on the first sparsity includes at least two; then the network model based on the second sparsity performs feedforward reasoning based on the first output result, and updates the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result, including: the network model based on the second sparsity performs parallel feedforward reasoning based on at least two first output results, with K as the batch size, and updates the latent vector representations corresponding to the at least two first output results, wherein the latent vector representation corresponding to the first output result is obtained by the network model based on the first sparsity during reasoning.

[0064] For example, taking the case where the dense computing condition is determined to be met when reaching the x*Kth step of reasoning, with K as the cycle, within a cycle K, the first K-1 times are feedforward inferences based on the network model of the first sparsity to obtain the first output results corresponding to the first K-1 times, and the latent vector representation of each first output result. Since the reasoning is based on the network model of the first sparsity, the accuracy of the latent vector representation of the first K-1 times is also relatively low. During the Kth reasoning, the first output results of the first K-1 times within the cycle and the initial input corresponding to the cycle (which can also be understood as the output result of the last step in the previous cycle) are input together into the network model of the second sparsity, and a parallel feedforward inference with a batch size of K is performed. The latent vector representations of multiple first output results with relatively low accuracy within the cycle can be updated in parallel, and based on the updated latent vector representation, the second output result of the next reasoning step is obtained, and then the reasoning of the next cycle can be started again until the reasoning end condition is met.

[0065] Among them, the network model with the first sparsity can perform serial reasoning based on a batch size of 1 during reasoning, and the network model with the second sparsity can perform parallel reasoning based on the first K-1 times and the initial input in the cycle, that is, a total of k inputs, and a batch size of K. When performing feedforward reasoning based on batch sizes of different sizes, memory and bandwidth resources may be the main factors. Therefore, the time consumption of feedforward reasoning with a small batch size in this cycle will not be much different from that of feedforward reasoning with a batch size of 1, and the impact on the reasoning speed is also small.

[0066] In a possible embodiment, when the value of K is 2, when reaching the x*K-th step of reasoning, within the cycle, the first output result corresponding to the network model based on the first sparsity includes one. In this case, at the x*K-th step of reasoning, the network model based on the second sparsity performs feedforward reasoning according to the first output result, and only the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result within the cycle needs to be updated.

[0067] In this way, in the embodiment of the present disclosure, during multi-step reasoning of the network model, periodic reasoning correction is performed based on the network model with a smaller second sparsity, and the low-precision latent vector representation within the period can be updated to a higher precision to obtain a second output result with higher accuracy, thereby improving the accuracy and performance of the network model and reducing the error caused by sparse reasoning.

[0068] In addition, in the embodiments of the present disclosure, the accuracy of the target content inferred by the network model can be further improved, and the output result of the network model based on the second sparsity can be used to verify the output result of the network model based on the first sparsity. Specifically, the present disclosure also provides a possible embodiment, including:

[0069] 1) When reaching the x*Kth step of reasoning, based on multiple first output results from the (x-1)*Kth step of reasoning to the x*K-1th step of reasoning, a third output result from the (x-1)*K+1th step of reasoning to the x*K-1th step of reasoning is obtained based on a network model of the second sparsity.

[0070] 2) When the third output result is inconsistent with the first output result of the corresponding number of reasoning steps, multiple first output results from the (x-1)*K+1th reasoning step to the x*K-1th reasoning step are deleted, and the third output result from the (x-1)*K+1th reasoning step to the x*K-1th reasoning step is used to update the first output result of the corresponding number of reasoning steps for the next reasoning step.

[0071] In the disclosed embodiment, a certain number of first output results can be generated by reasoning based on a network model with a larger first sparsity, and then a network model with a smaller second sparsity can be used for parallel reasoning and verification. When the output results of the two predicted reasonings are inconsistent, starting from the inconsistent position, the inconsistent and subsequent first reasoning results are deleted, so that the third output result reasoned by the network model based on the second sparsity is used to update the first output result of the corresponding number of step reasoning. In addition, the latent vector representation of each output result can also be synchronously updated.

[0072] In this way, reasoning and verification can be performed based on the different sparsities of a network model, which can further improve the accuracy of network model reasoning and enhance the performance of the network model.

[0073] In some embodiments of the present disclosure, the content generation method in the embodiments of the present disclosure may be applied to a many-core system, the many-core system includes multiple processing cores, the network model is deployed on at least one processing core of the many-core system; the processing core is used to generate target content of the task to be generated based on the network model; the processing core is also used to:

[0074] In a multi-step reasoning process when the network model generates target content, a first output result of the first target step reasoning is obtained based on the network model with a first sparsity; when dense computing conditions are met, a second output result of the next step reasoning of the first target step reasoning is obtained based on the network model with a second sparsity and the first output result, wherein the second sparsity is less than the first sparsity.

[0075] It should be noted that the specific implementation method of the processing core executing the above steps is the same as the implementation method of each step in the above content generation method, and will not be repeated here.

[0076] The following uses a specific application scenario to illustrate the content generation method in the embodiment of the present disclosure, taking the case where the dense computing condition is determined to be satisfied when reaching the x*Kth step of reasoning as an example, see Figure 2 , which is an example diagram of multi-step reasoning within a cycle in the content generation method in an embodiment of the present disclosure.

[0077] like Figure 2 As shown, taking a cycle K as an example, K is 5, the initial input of the first step of reasoning in the cycle is "late", and the first step of reasoning to the fourth step of reasoning in the cycle K are all based on the network model of the first sparsity for reasoning, and the first output results "can't sleep on the bed" are generated respectively, and the corresponding latent vector representations of each first output result "on", "sleep", "no", and "on" can be saved. In the fifth step of reasoning, the initial input "late" and the generated first output result "can't sleep on the bed" are input into the network model of the second sparsity for parallel feedforward reasoning, and then the latent vector representation of the first output result can be updated, and the second output result "how" of the fifth step of reasoning can be predicted.

[0078] Furthermore, in the embodiment of the present disclosure, in order to improve accuracy, of course, the network model of the second sparsity can also perform reasoning based not only on the first output result within one cycle, but also on the output results of multiple previous cycles as context information. However, in the embodiment of the present disclosure, only the latent vector of the first output result of the cycle needs to be updated to reduce the calculation time.

[0079] In addition, in the embodiments of the present disclosure, the application scenarios of the content generation method are not limited. For example, it can be applied to scenarios of sparse communication and loading, scenarios with limited hardware resources, and other non-large model usage scenarios. When sparse reasoning can be performed to generate target content, the reasoning speed is improved, and a network model with smaller sparsity can be periodically used for self-correction to reduce performance losses caused by sparse reasoning, thereby improving the quality and accuracy of target content generated by the network model.

[0080] For another example, the content generation scenario is not limited to a single token prediction, but can also be applied to multi-round dialogue scenarios. Based on the network model of the second sparsity, the latent vector representation of the content of the previous round of dialogue can be updated to generate the content of the current round of dialogue.

[0081] In the embodiment of the present disclosure, in the multi-step reasoning process of the network model generating the target content, after a certain number of steps of reasoning based on the network model of the first sparsity, the network model of the second sparsity is used to perform parallel feedforward reasoning according to the first output result generated previously. The latent vector representation with lower precision corresponding to the first output result generated previously can be updated to the corresponding latent vector representation with higher precision, and the next second output result can also be inferred, which can reduce the error accumulation caused by sparse reasoning and improve the quality and accuracy of the target content generated by the network model. In the embodiment of the present disclosure, since the network models of the first sparsity and the second sparsity are essentially one network model, but the sparsity during reasoning is different, self-correction of network model reasoning is achieved. Moreover, since it is the same network model, it does not need to occupy additional storage resources, and only one network model needs to be stored.

[0082] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0083] In addition, the present disclosure also provides a content generating device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any content generating method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0084] Figure 3 A block diagram of a content generation device provided in an embodiment of the present disclosure.

[0085] Reference Figure 3 , an embodiment of the present disclosure provides a content generation device, the content generation device comprising:

[0086] The generation module 31 is used to generate the target content of the task to be generated based on the network model, wherein the network model generates the target content through multi-step reasoning. For the multi-step reasoning process, the generation module is also used to:

[0087] Based on the network model of the first sparsity, obtaining a first output result of the first target step reasoning;

[0088] When dense computing conditions are met, a second output result of the next step reasoning of the first target step reasoning is obtained based on the network model of the second sparsity and the first output result, wherein the second sparsity is less than the first sparsity.

[0089] In a possible embodiment, for the case where the dense computing condition is met, the generating module 31 is used to:

[0090] When reaching the x*Kth reasoning step, it is determined that the dense computing condition is satisfied, where K represents the number of reasoning steps in a cycle, x represents the number of cycles, and x and K are both positive integers; or,

[0091] For multi-step reasoning within any cycle, when a preset number of reasoning steps within the any cycle is reached, it is determined that the dense computing condition is satisfied.

[0092] A possible embodiment further includes a first adjustment module 32, which is used to:

[0093] Determining performance information of the network model according to the first output result;

[0094] According to the performance information, the value of K is adjusted, and / or the preset number of inference steps within the cycle is adjusted.

[0095] A possible embodiment further includes a second adjustment module 33, which is used to:

[0096] Obtaining available resource information currently corresponding to the network model;

[0097] According to the available resource information, the value of K is adjusted, and / or the preset number of inference steps within the cycle is adjusted.

[0098] In a possible embodiment, when the network model based on the second sparsity and the first output result obtains the second output result of the next step reasoning of the first target step reasoning, the generation module 31 is used to:

[0099] Based on the network model of the second sparsity, according to the first output result, feedforward reasoning is performed to update the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result;

[0100] According to the updated latent vector representation of the first output result, a second output result of the next step reasoning of the first target step reasoning is obtained.

[0101] In a possible embodiment, the value of K is greater than or equal to 3, and when reaching the x*Kth reasoning step, the first output results corresponding to the network model based on the first sparsity include at least two;

[0102] When the network model based on the second sparsity performs feedforward inference according to the first output result and updates the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result, the generation module 31 is used to: perform parallel feedforward inference on the network model based on the second sparsity according to at least two of the first output results, with K as the batch size, and update the latent vector representations corresponding to at least two of the first output results, wherein the latent vector representation corresponding to the first output result is obtained by the network model based on the first sparsity during reasoning.

[0103] A possible embodiment further includes a verification module 34, which is used to:

[0104] When reaching the x*Kth step of reasoning, according to the plurality of first output results from the (x-1)*Kth step of reasoning to the x*K-1th step of reasoning, based on the network model of the second sparsity, a third output result from the (x-1)*K+1th step of reasoning to the x*K-1th step of reasoning is obtained;

[0105] When the third output result is inconsistent with the first output result of the corresponding number of reasoning steps, multiple first output results from the (x-1)*K+1th reasoning step to the x*K-1th reasoning step are deleted, and the third output result from the (x-1)*K+1th reasoning step to the x*K-1th reasoning step is used to update the first output result of the corresponding number of reasoning steps for use in the next step of reasoning.

[0106] A possible embodiment further includes a testing module 35, which is used to:

[0107] Determine a first candidate value of K and a second candidate value of the first sparsity;

[0108] Determine, according to the first candidate value and the second candidate value, a candidate value combination corresponding to K and the first sparsity;

[0109] Based on the candidate value combination, a performance test is performed on the network model, a target value combination is determined from the candidate value combination, and according to the target value combination, a first target value of K and a second target value of the first sparsity are obtained.

[0110] In a possible embodiment, the second sparsity is 0, and when the second sparsity is 0, the network model is represented as a complete dense model, and when the first sparsity is greater than 0, the network model is represented as a sparse model;

[0111] The model data of the network model is stored in an off-chip memory, and the generation module 31 is also used to: during the execution of the multi-step reasoning process, obtain the model data of the network model from the off-chip memory to execute the multi-step reasoning process based on the model data of the network model.

[0112] Figure 4 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0113] Reference Figure 4 An embodiment of the present disclosure provides an electronic device, which includes multiple processing cores 401 and an on-chip network 402, wherein the multiple processing cores 401 are connected to the on-chip network 402, and the on-chip network 402 is used to exchange data between the multiple processing cores and external data.

[0114] One or more instructions are stored in one or more processing cores 401 , and the one or more instructions are executed by one or more processing cores 401 , so that the one or more processing cores 401 can execute the above-mentioned content generation method.

[0115] In some embodiments, the electronic device may be a brain-like chip, which can use vectorized computing and needs to load the weight information and other parameters of the neural network model through an external memory such as a double data rate (DDR) synchronous dynamic random access memory. Therefore, the batch processing used in the embodiments of the present disclosure has a higher computing efficiency.

[0116] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned content generation method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0117] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned content generation method.

[0118] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0119] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0121] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0122] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0123] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0124] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0125] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0126] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0127] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A content generation method, characterized in that: include: Based on the network model, the target content of the task to be generated is generated, wherein the network model generates the target content through multi-step reasoning, and the multi-step reasoning process includes: Based on the network model of the first sparsity, obtaining a first output result of the first target step reasoning; When dense computing conditions are met, a second output result of the next step reasoning of the first target step reasoning is obtained based on the network model of the second sparsity and the first output result, wherein the second sparsity is less than the first sparsity.

2. The content generation method according to claim 1, characterized in that: The step of obtaining a second output result of a next step of reasoning of the first target step reasoning based on the network model of the second sparsity and the first output result includes: Based on the network model of the second sparsity, according to the first output result, feedforward reasoning is performed to update the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result; According to the updated latent vector representation of the first output result, a second output result of the next step reasoning of the first target step reasoning is obtained.

3. The content generation method according to claim 2, characterized in that: The situations that satisfy the dense computing conditions include: When reaching the x*Kth reasoning step, it is determined that the dense computing condition is satisfied, where K represents the number of reasoning steps in a cycle, x represents the number of cycles, and x and K are both positive integers; or, For multi-step reasoning within any cycle, when a preset number of reasoning steps within the any cycle is reached, it is determined that the dense computing condition is satisfied.

4. The content generation method according to claim 3, characterized in that: The value of K is greater than or equal to 3, and when reaching the x*Kth step of reasoning, the first output results corresponding to the network model based on the first sparsity include at least two; The network model based on the second sparsity performs feedforward reasoning according to the first output result to update the latent vector representation obtained by the network model based on the first sparsity corresponding to the first output result, including: The network model based on the second sparsity performs parallel feedforward inference according to at least two of the first output results with K as the batch size, and updates the latent vector representations corresponding to at least two of the first output results, wherein the latent vector representations corresponding to the first output results are obtained during inference of the network model based on the first sparsity.

5. The content generation method according to claim 3, characterized in that: The method further comprises: Determining performance information of the network model according to the first output result; According to the performance information, the value of K is adjusted, and / or the preset number of inference steps within the cycle is adjusted.

6. The content generation method according to claim 3, characterized in that: The method further comprises: Obtaining available resource information currently corresponding to the network model; According to the available resource information, the value of K is adjusted, and / or the preset number of inference steps within the cycle is adjusted.

7. The content generation method according to claim 3, characterized in that: The method further comprises: When reaching the x*Kth step of reasoning, according to the plurality of first output results from the (x-1)*Kth step of reasoning to the x*K-1th step of reasoning, based on the network model of the second sparsity, a third output result from the (x-1)*K+1th step of reasoning to the x*K-1th step of reasoning is obtained; When the third output result is inconsistent with the first output result of the corresponding number of reasoning steps, multiple first output results from the (x-1)*K+1th reasoning step to the x*K-1th reasoning step are deleted, and the third output result from the (x-1)*K+1th reasoning step to the x*K-1th reasoning step is used to update the first output result of the corresponding number of reasoning steps for use in the next step of reasoning.

8. The content generation method according to claim 3, characterized in that: The method further comprises: Determine a first candidate value of K and a second candidate value of the first sparsity; Determine, according to the first candidate value and the second candidate value, a candidate value combination corresponding to K and the first sparsity; Based on the candidate value combination, a performance test is performed on the network model, a target value combination is determined from the candidate value combination, and according to the target value combination, a first target value of K and a second target value of the first sparsity are obtained.

9. The content generation method according to claim 1, characterized in that: The second sparsity is 0, and when the second sparsity is 0, the network model is represented as a complete dense model, and when the first sparsity is greater than 0, the network model is represented as a sparse model; The model data of the network model is stored in an off-chip memory, and in the process of executing the multi-step reasoning, the method further includes: The model data of the network model is obtained from the off-chip memory to execute the multi-step reasoning process based on the model data of the network model.

10. A content generating device, characterized in that: include: A generation module is used to generate target content of the task to be generated based on the network model, wherein the network model generates the target content through multi-step reasoning. For the multi-step reasoning process, the generation module is also used to: Based on the network model of the first sparsity, obtaining a first output result of the first target step reasoning; When dense computing conditions are met, a second output result of the next step reasoning of the first target step reasoning is obtained based on the network model of the second sparsity and the first output result, wherein the second sparsity is less than the first sparsity.

11. An electronic device, characterized in that: include: Multiple processing cores; as well as An on-chip network is configured to exchange data between the multiple processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the content generation method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the content generation method according to any one of claims 1 to 9.

Images