Network model reasoning method and device, network model data processing method and device, electronic equipment and medium

Abstract

The invention provides a network model reasoning method and device, a data processing method and device, electronic equipment and a medium, and the method comprises the steps: responding to the received input information of a target processing task, and carrying out the multi-step reasoning of the input information based on a network model; for any step of reasoning in the multiple steps of reasoning, under the condition that a first sub-result of the current step of reasoning is obtained, whether a first implicit vector stored in the network model meets a preset operation condition or not is determined, the first implicit vector comprises an implicit vector corresponding to the first sub-result and an implicit vector corresponding to a sub-result of historical step reasoning before the current step reasoning; under the condition that a preset operation condition is met, generating an operation instruction for the first implicit vector, and performing operation processing on the first implicit vector according to the operation instruction to obtain a second implicit vector; executing next-step reasoning based on the second implicit vector to obtain a second sub-result of the next-step reasoning; the output result for the target processing task comprises a sub-result of each step of reasoning in the multi-step reasoning.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a network model reasoning, data processing method and device, electronic equipment, and medium. Background Art

[0002] Large models usually have large-scale parameters, which brings challenges to deployment and application. In addition, when reasoning with large models, the result of each step of reasoning decoding output (for example, called token) will be fed back to the input layer of the large model, and then decoded to generate the next token until the termination condition is met. In order to reduce the amount of calculation, the key vector and value vector of each network layer of the token output at each step of reasoning can be cached in the large model to provide for the next step of reasoning. The key vector and value vector can also be called latent vector. However, as the length of the reasoning content increases, the cached latent vector will continue to increase, which will cause greater storage and computing problems, affecting the reasoning speed and performance of the large model. Summary of the invention

[0003] The present invention provides a network model reasoning, data processing method and device, electronic equipment, and medium.

[0004] In a first aspect, the present disclosure provides a network model reasoning method, the network model reasoning method comprising:

[0005] In response to receiving input information of the target processing task, performing multi-step reasoning on the input information based on the network model;

[0006] For any one of the multi-step reasoning steps, when a first sub-result of the current step reasoning is obtained, determining whether a first latent vector stored in the network model satisfies a preset operation condition, wherein the first latent vector includes a latent vector corresponding to the first sub-result and a latent vector corresponding to a sub-result of a historical step reasoning before the current step reasoning;

[0007] When a preset operation condition is met, an operation instruction for the first latent vector is generated, and the first latent vector is operated according to the operation instruction to obtain a second latent vector;

[0008] Performing the next step of reasoning based on the second latent vector to obtain a second sub-result of the next step of reasoning;

[0009] The output result for the target processing task includes a sub-result of each step of reasoning in the multi-step reasoning.

[0010] In a second aspect, the present disclosure provides a data processing method, comprising:

[0011] receiving input information of a target processing task;

[0012] Based on the network model reasoning method in the first aspect, obtaining the output result of the target processing task;

[0013] The target processing task includes any one of the following: an image processing task, a voice processing task, a text processing task, and a video processing task; and the input information includes any one of an image, a voice, a text, and a video.

[0014] In a third aspect, the present disclosure provides a network model reasoning device, the network model reasoning device comprising:

[0015] A reasoning module, configured to, in response to receiving input information of a target processing task, perform multi-step reasoning on the input information based on a network model;

[0016] a determination module, configured to determine, for any one of the multi-step reasoning steps, whether a first latent vector stored in the network model satisfies a preset operation condition when a first sub-result of a current step reasoning is obtained, wherein the first latent vector includes a latent vector corresponding to the first sub-result and a latent vector corresponding to a sub-result of a historical step reasoning before the current step reasoning;

[0017] an operation processing module, configured to generate an operation instruction for the first latent vector when a preset operation condition is met, and perform operation processing on the first latent vector according to the operation instruction to obtain a second latent vector;

[0018] The reasoning module is further used to perform a next step of reasoning based on the second latent vector to obtain a second sub-result of the next step of reasoning;

[0019] The output result for the target processing task includes a sub-result of each step of reasoning in the multi-step reasoning.

[0020] In a fourth aspect, the present disclosure provides a data processing device, including:

[0021] A receiving module, used for receiving input information of a target processing task;

[0022] An output module, used to obtain the output result of the target processing task based on the network model reasoning method in the first aspect;

[0023] The target processing task includes any one of the following: image processing task, voice processing task, text processing task, video processing task; the input information includes any one of the following: image, voice, text, video

[0024] In a fifth aspect, the present disclosure provides an electronic device comprising: a plurality of processing cores; and an on-chip network configured to exchange data between the plurality of processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the above-mentioned network model inference method or data processing method.

[0025] In a sixth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned network model inference method or data processing method when executed by a processing core.

[0026] In the seventh aspect, an embodiment of the present disclosure provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned network model inference method or data processing method.

[0027] The embodiments provided by the present disclosure perform multi-step reasoning on the input information of the target processing task based on the network model to obtain the output result of the target processing task. During the reasoning process, when it is determined that the first latent vector generated by the reasoning meets the preset operation condition, an operation instruction for the first latent vector is generated, and the first latent vector is operated and processed according to the operation instruction to obtain the second latent vector, and the second latent vector is used for the next reasoning. In this way, whether the preset operation condition is met can be judged by the autoregressive reasoning method of the network model, and the operation instruction can be generated and executed by the autoregressive reasoning method when the preset operation is met, thereby realizing the operation processing of the latent vector generated in the network model, improving efficiency, optimizing the latent vector generated in the reasoning process, meeting the resource requirements of the network model, thereby improving the reasoning speed and performance of the network model, and being realized by the autoregressive reasoning method of the network model, there is no need to rely on external control, and the cost of implementation can also be reduced.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0030] Figure 1 A flowchart of a network model reasoning method provided by an embodiment of the present disclosure;

[0031] Figure 2 A block diagram of a network model inference device provided in an embodiment of the present disclosure;

[0032] Figure 3 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0035] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0036] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0037] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0038] The parameters of network models are getting larger and larger, which brings challenges to deployment and application. Taking the large language model used for natural language processing tasks as an example, the network model is usually based on the Transformer model architecture. The Transformer model architecture contains many isomorphic layers, each of which contains an attention layer and a feed forward network (FFN) layer. The attention layer mainly processes the correlation of the token dimension (time dimension), so it can be understood as mainly involving short-term memory. The forward connection layer mainly processes the correlation between the dimensions of a single token, so it can be understood as mainly involving long-term memory.

[0039] The attention layer usually performs a linear transformation on the input vector to generate a query vector, a key vector, and a value vector, and performs a self-attention operation, and then generates an output through a linear transformation. The forward connection layer usually consists of two layers of linear transformation layers, where the first linear transformation layer acts like a matrix composed of keys. Under the query of the input vector (similar to the query), it outputs the matching degree, and then weights the second linear transformation layer (similar to the value) to obtain the output.

[0040] When the large language model is inferencing, the output of the inference result is serial and autoregressive, that is, the result of each step of decoding output (also called token) will be fed back to the input layer of the large language model, and then decoded to generate the next token until the termination condition is met. Among them, the key vector and value vector of each network layer in the network model output of each step of reasoning can be cached to provide for the next step of reasoning, avoiding recalculation and reducing the amount of calculation. The key vector and value vector can also be called the latent vector of the token. The cached key vector and value vector can be referred to as the key-value cache (kvcache). The kv cache is the main carrier of short-term memory and stores the information of tokens in a certain context.

[0041] However, as the length of the inference content increases, for example, in a conversation scenario based on a large language model, as the length of the conversation increases, the kv cache generated during reasoning continues to increase, which will cause a large storage and communication burden. In addition, in some long conversation sequences and large time scale scenarios similar to artificial intelligence personal assistant conversations, it is necessary to record user conversations and other information, further increasing the demand for converting short-term memory into long-term memory in the large language model. In addition, the increase in kv cache increases the amount of computation required to generate tokens for reasoning, which also causes a large computational burden, thereby affecting the reasoning speed and performance of the large language model.

[0042] Therefore, in the embodiment of the present disclosure, the kv cache generated by the network model during the reasoning process is mainly optimized. According to the network model reasoning method of the embodiment of the present disclosure, when multi-step reasoning is performed on the input information of the target processing task based on the network model, it is determined whether the first latent vector generated by the reasoning meets the preset operation conditions. When the preset operation conditions are met, an operation instruction for the first latent vector is generated, and the first latent vector is operated and processed according to the operation instruction to obtain a second latent vector, which is used for the next reasoning step based on the second latent vector. In this way, by judging whether the operation conditions are met through the network model, the timing of generating the operation instruction can be controlled, and the operation instruction can be automatically generated and executed through the autoregressive reasoning of the network model, thereby improving the operation processing efficiency. Furthermore, by executing the operation instruction and operating the latent vector in storage, the latent vector generated during the reasoning process can be optimized to meet the resource requirements of the network model, thereby improving the reasoning speed and performance of the network model.

[0043] The network model reasoning method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory. Alternatively, the content generation method can be executed by a server.

[0044] It should be noted that in the embodiments of the present disclosure, there is no limitation on the network model, and the network model can be used to perform any of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. The output result of the network model reasoning includes any of images, speech, text, and video.

[0045] Figure 1 A flowchart of a network model inference method provided by an embodiment of the present disclosure. Figure 1 , the method comprising:

[0046] Step S110: In response to receiving input information of the target processing task, perform multi-step reasoning on the input information based on the network model.

[0047] In the embodiments of the present disclosure, it is mainly aimed at scenarios in which content is generated based on network models. For example, the network model can be a large language model, a large visual model, a large multimodal model, etc., and there is no limitation. The network model can be used for content generation for different generation tasks. During the operation of the network model, the user can input input information of the target processing task, the network model performs inference and generates corresponding output results, and the output results can be returned to the user.

[0048] Taking the target processing task as a text processing task as an example, for example, the input information of the target processing task is: the meteorological station has just announced that a typhoon will land on the coast of a certain area today, and the maximum wind speed near the center will be 14 when it lands. After receiving the input information, the network model can infer and summarize based on the input information to output the corresponding text content. During reasoning, each step of reasoning of the network model can generate a word and output it until the termination condition is met. On the user side, each word output by each step of reasoning can be viewed through the terminal device, that is, all the text content generated for the input information is obtained.

[0049] Step S120: For any reasoning step in the multi-step reasoning, when the first sub-result of the current reasoning step is obtained, determine whether the first latent vector stored in the network model meets the preset operating conditions, and the first latent vector includes the latent vector corresponding to the first sub-result and the latent vector corresponding to the sub-result of the historical reasoning step before the current reasoning step.

[0050] In the disclosed embodiment, the network model can perform reasoning based on the input information and the previously generated sub-results as context information at each step of reasoning, and the latent vectors of the sub-results generated at each step of reasoning will be stored. This information is stored in the form of a kv cache. In this way, when reasoning is performed based on the input information and the previously generated sub-results, there is no need to repeatedly calculate the latent vectors, which can reduce the amount of calculation.

[0051] For example, the target processing task may be an image processing task, a speech processing task, a text processing task, a video processing task, etc., and the corresponding sub-results may be image processing results, speech processing results, text processing results, video processing results, etc. Taking the case where the target processing task may be a text processing task, and the input information may be text content, then the input information is inferred based on the network model, and the sub-results generated by each step of inference may be numbers, letters, words, etc. The embodiments of the present disclosure do not specifically limit the specific content of the above sub-results. Taking the case where the above sub-result is a word, for example, the latent vector of the word may be the key vector and value vector of the word.

[0052] Among them, in the embodiment of the present disclosure, it can be determined based on the network model whether the first latent vector meets the preset operation conditions. The first latent vector can represent the latent vector accumulated by multi-step reasoning. The preset operation conditions and the subsequent operation processing on the first latent vector correspond to each other. Different operation processing corresponds to different preset operation conditions. For example, the operation type of the operation processing can be a compression operation, a move operation, etc., and there is no restriction.

[0053] In a possible embodiment, when the operation type is a compression operation, satisfying the preset operation condition may include any of the following methods:

[0054] 1) When the length of the first latent vector is greater than or equal to a preset length threshold, it is determined that the preset operating condition is met.

[0055] For example, in an AI dialogue scenario, the user inputs a piece of input information, and the network model performs inference to answer, outputting a word each time the inference is performed. As the length of the answer and the dialogue increases, the kv cache continues to increase. If at the current nth step of inference, it is determined whether the length of the continuous kv cache from the 1st step to the nth step of inference is greater than or equal to the length threshold, when it is judged to be greater than or equal to the length threshold, it is determined that the preset operating conditions are met, and when it is judged to be less than the length threshold, it is determined that the preset operating conditions are not met.

[0056] 2) When the storage space occupied by the first latent vector is greater than or equal to a preset value, it is determined that the preset operating condition is met.

[0057] In the embodiment of the present disclosure, it is also possible to determine whether the preset operating conditions are met by judging the size of the storage space occupied by the kv cache. When the preset operating conditions are met, it is indicated that the first latent vector needs to be compressed. When the preset operating conditions are not met, it is indicated that the first latent vector does not need to be compressed. The size of the storage space occupied by the kv cache may be related to its length and precision. The length is positively correlated with the size of the occupied storage space, and the precision is negatively correlated with the size of the occupied storage space. The higher the precision, the larger the number of bits represented by the latent vector. For example, the longer the length of the first latent vector, the higher the precision, and the larger the storage space occupied. When the occupied storage space is greater than or equal to the preset value, the kv cache in the storage space can be compressed.

[0058] Of course, in the embodiment of the present disclosure, when determining whether the first latent vector satisfies the preset operating condition, the first latent vector is not limited to being the latent vector generated by reasoning from the first step to the current step.

[0059] Step S130: When a preset operation condition is met, an operation instruction for the first latent vector is generated.

[0060] Regarding the operation instruction for the first latent vector generated in step S130, the present disclosure provides a possible implementation method, which is based on the network model, generates the operation instruction for the first latent vector through autoregressive reasoning according to a preset instruction format and the first latent vector.

[0061] In the disclosed embodiment, the operation instruction is generated by the network model through autoregressive reasoning. When it is determined through the autoregressive reasoning of the network model that the preset operation conditions are met, the operation instruction is generated and executed. In this way, the operation instruction can also be represented by the inferred token sequence, without the need for additional reliance on external information, thereby reducing the implementation complexity and additional cost consumption.

[0062] In the embodiment of the present disclosure, the operation instruction may include an operation type, a first starting address and a first length, and a second starting address; or, the operation instruction may include an operation type, a first starting address and a first length, a second starting address and a second length.

[0063] The latent vector can be addressed by a natural number index and affects the processing of the network model through position encoding. Therefore, the natural number index of the latent vector or its corresponding position encoding is the basis for addressing it. It can be understood that the latent vector generated by the network model reasoning can be addressed to cache it in a storage unit. When making judgments and operations, the latent vector can be obtained from the corresponding storage unit according to the address index.

[0064] For example, a possible preset instruction format of an operation instruction is: <operation type><first starting address and first length><second starting address and second length>.

[0065] Among them, the operation type can indicate the type of operation to be performed by the network model, for example, the operation type includes compression operation, move operation, etc., wherein the compression operation can indicate prompting the network model to perform a summary or summary generation operation, and the sub-results of the multi-step reasoning corresponding to the first latent vector are processed by screening, summarizing, etc. to generate a second latent vector with a shorter length, thereby realizing the compression function; the move operation can indicate changing the natural number index or position encoding of the first latent vector to the target natural number index or position encoding, which is equivalent to completing the movement of the first latent vector. The move operation here can refer to the logical one, that is, the actual physical storage of the first latent vector does not necessarily need to be changed. In addition, other types of instructions can be extended as needed, such as deletion operations, etc., and there is no restriction on this.

[0066] The first starting address and the first length: represent the first starting address and the first length of the first latent vector, indicating that the network model is to operate and process the first latent vector of the specified starting address and length.

[0067] The second starting address and the second length represent the second starting address and the second length of the second latent vector, indicating the addressing and length of the second latent vector after the operation is processed by the network model (wherein the second length can be omitted), which actually determines the natural number index of the newly generated second latent vector or its position encoding.

[0068] For example, the operation instruction is: <compression><first starting address 0 first length 1222><second starting address 0 second length 442>, which can indicate that a compression operation will be performed, the first latent vector whose first starting address is 0 and whose first length is 1222 will be compressed, and the generated compressed second latent vector will be addressed to the addressing space whose second starting address is 0 and whose second length is 442.

[0069] Step S140: According to the operation instruction, the first latent vector is operated to obtain a second latent vector.

[0070] For this step S140, it can be executed by autoregressive reasoning after the network model generates the operation instruction. Specifically, the present disclosure provides a possible implementation method: based on the network model, a first latent vector corresponding to the first starting address and the first length is obtained; according to the operation type, the first latent vector is subjected to corresponding operation processing to obtain a second latent vector.

[0071] Furthermore, after obtaining the second latent vector, the second latent vector may be stored. In one possible embodiment, the address index for storing the second latent vector is determined according to the second starting address, and the second latent vector is stored based on the address index. In another possible embodiment, the address index for storing the second latent vector is determined according to the second starting address and the second length, and the second latent vector is stored based on the address index.

[0072] For example, taking the compression operation as an example, during the conversation, when the capacity of the kv cache reaches a certain level and meets the preset operation conditions, the network model will output an operation instruction of the compression operation type through autoregressive reasoning. For example, the output operation instruction is <compression><first starting address 1000 first length 500><second starting address 1000 second length 100>. After obtaining the operation instruction, the network model executes the operation instruction through autoregressive reasoning to compress the first latent vector with the first starting address of 1000 and the first length of 500, that is, to achieve compression through summarization and induction, and address the output second latent vector (that is, the summary or abstract content output by the network model under the prompt of the operation instruction) as the address area with the second starting address of 1000 and the second length of 100. In this way, through the above operation, the network model can compress a kv cache with a length of 500 to a kv cache with a length of 100, releasing a kv cache with a length of 400. The cache storage capacity and address addressing space achieve the purpose of compression, reduce the problems of large storage, communication, and computing requirements, and thus enable processing of longer time scales and dialogue sequences.

[0073] Furthermore, in the embodiment of the present disclosure, after the first latent vector is processed according to the operation instruction to obtain the second latent vector, a deletion operation may be performed on the first latent vector, which can reduce memory usage and improve memory resource utilization.

[0074] Step S150: Execute the next step of reasoning based on the second latent vector to obtain a second sub-result of the next step of reasoning; wherein the output result for the target processing task includes the sub-result of each step of reasoning in the multi-step reasoning.

[0075] For example, when the nth step of reasoning satisfies the preset operating conditions, the first latent vector is summarized and compressed to generate a second latent vector. Then, in subsequent reasoning, the second latent vector can be used as context information to generate a second sub-result for the next step of reasoning.

[0076] In the disclosed embodiment, a multi-step reasoning is performed on the input information of the target processing task based on the network model to obtain the output result of the target processing task. During the reasoning process, when the first latent vector meets the preset operation condition, an operation instruction for the first latent vector is generated, and according to the operation instruction, the first latent vector is operated and processed to obtain the second latent vector. In this way, the operation instruction can be generated and executed by the autoregressive reasoning of the network model to realize the operation processing of the latent vector, that is, the generation and execution of the operation instruction are realized based on the autoregressive reasoning of the network model, and do not need to rely on other external controls, thereby reducing the cost of implementation. In addition to operating its latent vector, it does not affect the processing process of the network model itself for the target processing task. After obtaining the second latent vector, the next reasoning is performed based on the second latent vector to obtain the second sub-result of the next reasoning. Since the operation processing of the latent vector does not need to reduce the precision representation of the latent vector, nor does it discard certain sub-results, the previous sub-results will still be involved in the next reasoning, thereby reducing the impact on the accuracy and performance of the second sub-result of the next reasoning.

[0077] The training process of the network model in the embodiment of the present disclosure is described below. In the embodiment of the present disclosure, since the generation and execution of the operation instructions are all implemented based on the autoregressive reasoning of the network model, there is no need to design and train a network model with the above functions from scratch, but only fine-tune the pre-trained network model.

[0078] Specifically, in an embodiment of the present disclosure, a possible network model training method is provided, which includes:

[0079] 1) Obtain a first training data set, which includes a first content sample, an operation instruction sample that meets an operation condition, and a second content sample after executing the operation instruction sample.

[0080] In the embodiment of the present disclosure, the data sample of the first training data set can be understood as a content-text pair, the first content sample and the operation instruction sample are input, and the second content sample is the corresponding output. The second content sample can be obtained by automatically generating through a network model under certain prompts.

[0081] For example, taking the compression operation as an example, the preset operation condition is that the length is greater than or equal to the length threshold. Then, when the latent vector length of the first content sample is greater than or equal to the length threshold, a compression processing operation instruction can be added, and the first content sample and the operation instruction are used as prompt information to prompt the network model to summarize the first content sample according to the operation instruction to obtain the second content sample.

[0082] In addition, in the embodiments of the present disclosure, a training data set in a specific scenario or field can also be used to fine-tune the network model to improve the accuracy of the target processing task in the scenario or field.

[0083] 2) Based on the first training data set, the network model is trained to obtain a trained network model.

[0084] In this way, in the embodiment of the present disclosure, the network model is fine-tuned and trained based on the first training data set, so that when the network model is used for reasoning in actual applications, it can trigger operation instructions such as output compression according to certain operation conditions and complete the execution, thereby realizing functions such as compression of latent vectors. It should be noted that through training, functions such as the generation and execution of operation instructions are stored in the network model in the form of weights.

[0085] Based on the above embodiment, the present disclosure further provides a flow chart of a data processing method, the method comprising:

[0086] S1. Receive input information of the target processing task.

[0087] The target processing task includes any one of the following: image processing task, speech processing task, text processing task, video processing task; the input information includes any one of image, speech, text, and video.

[0088] In the disclosed embodiment, there is no restriction on the target processing task and the input information.

[0089] S2. Based on the network model inference method in the above embodiment, the output result of the target processing task is obtained.

[0090] That is, in the embodiment of the present disclosure, after receiving the input information of the target processing task, multi-step reasoning is performed on the input information based on the network model to obtain the output result of the target processing task, and in the reasoning process, when the first sub-result of the current step reasoning is obtained, it is determined whether the first latent vector stored in the network model meets the preset operation conditions, and when the preset operation conditions are met, an operation instruction for the first latent vector is generated, and according to the operation instruction, the first latent vector is operated and processed to obtain the second latent vector, and the next step of reasoning is performed based on the second latent vector to obtain the second sub-result of the next step of reasoning, and then the output result for the target processing task is obtained based on the sub-results of the multi-step reasoning.

[0091] In this way, in the embodiment of the present disclosure, when processing the target processing task through the network model, the latent vector of the generated sub-result can be processed by autoregressive reasoning, which can achieve the optimization of the latent vector and improve the reasoning performance of the network model.

[0092] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0093] In addition, the present disclosure also provides a network model inference device, a data processing device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any network model inference method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0094] Figure 2 A block diagram of a network model inference device provided in an embodiment of the present disclosure.

[0095] Reference Figure 2 The embodiment of the present disclosure provides a network model reasoning device, the network model reasoning device comprising:

[0096] The reasoning module 21 is used for performing multi-step reasoning on the input information based on the network model in response to receiving the input information of the target processing task;

[0097] A determination module 22 is used to determine, for any one of the multi-step reasoning steps, whether a first latent vector stored in the network model satisfies a preset operation condition when a first sub-result of the current step reasoning is obtained, wherein the first latent vector includes a latent vector corresponding to the first sub-result and a latent vector corresponding to a sub-result of a historical step reasoning before the current step reasoning;

[0098] An operation processing module 23 is used to generate an operation instruction for the first latent vector when a preset operation condition is met, and perform an operation on the first latent vector according to the operation instruction to obtain a second latent vector;

[0099] The reasoning module 21 is further used to perform the next reasoning based on the second latent vector to obtain a second sub-result of the next reasoning;

[0100] The output result for the target processing task includes a sub-result of each step of reasoning in the multi-step reasoning.

[0101] In a possible embodiment, the operation instruction is generated by the network model through autoregressive inference;

[0102] The step of performing operation processing on the first latent vector according to the operation instruction to obtain the second latent vector is performed by the network model through autoregressive inference after generating the operation instruction.

[0103] In a possible embodiment, the operation instruction for the first latent vector is generated, and the operation processing module 23 is used to: based on the network model, generate the operation instruction for the first latent vector through autoregressive reasoning according to a preset instruction format and the first latent vector.

[0104] In a possible embodiment, the operation instruction includes an operation type, a first starting address and a first length;

[0105] When the first latent vector is operated and processed according to the operation instruction to obtain the second latent vector, the operation processing module 23 is used to:

[0106] Based on the network model, obtaining a first latent vector corresponding to the first starting address and the first length;

[0107] According to the operation type, a corresponding operation is performed on the first latent vector to obtain a second latent vector.

[0108] In a possible embodiment, the operation instruction further includes a second starting address, or the operation instruction further includes a second starting address and a second length. After obtaining the second latent vector, the operation processing module 23 is further used to:

[0109] According to the second starting address, an address index for storing the second latent vector is determined, and based on the address index, the second latent vector is stored.

[0110] Alternatively, an address index for storing the second latent vector is determined according to the second starting address and the second length, and the second latent vector is stored based on the address index.

[0111] In a possible embodiment, the operation type includes at least one of the following: a compression operation and a move operation.

[0112] In a possible embodiment, when the operation type is a compression operation and the preset operation condition is met, the determination module 22 is configured to:

[0113] When the length of the first latent vector is greater than or equal to a preset length threshold, it is determined that a preset operating condition is met; or,

[0114] When it is determined that the storage space occupied by the first latent vector is greater than or equal to a preset value, it is determined that the preset operating condition is met.

[0115] In a possible embodiment, after the first latent vector is operated and processed according to the operation instruction to obtain the second latent vector, the operation processing module 23 is further used to: perform a deletion operation on the first latent vector.

[0116] In a possible embodiment, a training module 24 is further included, and the network model is obtained by training the training module 24 in the following manner:

[0117] Acquire a first training data set, wherein the first training data set includes a first content sample, an operation instruction sample that meets an operation condition, and a second content sample after executing the operation instruction sample;

[0118] Based on the first training data set, the network model is trained to obtain a trained network model.

[0119] Based on the above embodiments, the present disclosure further provides a data processing device, the data processing device comprising:

[0120] A receiving module, used for receiving input information of a target processing task;

[0121] An output module, used to obtain the output result of the target processing task based on the above network model reasoning method;

[0122] Among them, the target processing task includes any one of the following: image processing task, voice processing task, text processing task, video processing task; the input information includes any one of the following: image, voice, text, video.

[0123] Figure 3 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0124] Reference Figure 3 An embodiment of the present disclosure provides an electronic device, which includes multiple processing cores 301 and an on-chip network 302, wherein the multiple processing cores 301 are connected to the on-chip network 302, and the on-chip network 302 is used to exchange data between the multiple processing cores and external data.

[0125] Among them, one or more instructions are stored in one or more processing cores 301, and the one or more instructions are executed by one or more processing cores 301, so that one or more processing cores 301 can execute the above-mentioned network model inference method or data processing method.

[0126] In some embodiments, the electronic device may be a brain-like chip, which can use vectorized computing and needs to load the weight information and other parameters of the neural network model through an external memory such as a double data rate (DDR) synchronous dynamic random access memory. Therefore, the batch processing used in the embodiments of the present disclosure has a higher computing efficiency.

[0127] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned network model reasoning method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium or a data processing method.

[0128] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned network model inference method or data processing method.

[0129] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0130] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0131] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0132] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0133] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0134] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0135] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0136] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0137] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0138] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A network model reasoning method, characterized in that: include: In response to receiving input information of the target processing task, performing multi-step reasoning on the input information based on the network model; For any one of the multi-step reasoning steps, when a first sub-result of the current step reasoning is obtained, determining whether a first latent vector stored in the network model satisfies a preset operation condition, wherein the first latent vector includes a latent vector corresponding to the first sub-result and a latent vector corresponding to a sub-result of a historical step reasoning before the current step reasoning; When a preset operation condition is met, an operation instruction for the first latent vector is generated, and the first latent vector is operated according to the operation instruction to obtain a second latent vector; Performing the next step of reasoning based on the second latent vector to obtain a second sub-result of the next step of reasoning; The output result for the target processing task includes a sub-result of each step of reasoning in the multi-step reasoning.

2. The method according to claim 1, characterized in that The operation instruction is generated by the network model through autoregressive reasoning; The step of performing operation processing on the first latent vector according to the operation instruction to obtain the second latent vector is performed by the network model through autoregressive inference after generating the operation instruction.

3. The method according to claim 1, characterized in that The generating an operation instruction for the first latent vector includes: Based on the network model, an operation instruction for the first latent vector is generated through autoregressive reasoning in accordance with a preset instruction format and the first latent vector.

4. The method according to any one of claims 1 to 3, characterized in that: The operation instruction includes an operation type, a first starting address and a first length; The step of operating the first latent vector according to the operation instruction to obtain a second latent vector includes: Based on the network model, obtaining a first latent vector corresponding to the first starting address and the first length; According to the operation type, a corresponding operation is performed on the first latent vector to obtain a second latent vector.

5. The method according to claim 4, characterized in that The operation instruction further includes a second starting address, or the operation instruction further includes a second starting address and a second length. After obtaining the second latent vector, the method further includes: Determining, according to the second starting address, an address index for storing the second latent vector, and storing the second latent vector based on the address index; Alternatively, an address index for storing the second latent vector is determined according to the second starting address and the second length, and the second latent vector is stored based on the address index.

6. The method according to claim 4, characterized in that The operation type includes at least one of the following: a compression operation and a move operation.

7. The method according to claim 6, characterized in that When the operation type is a compression operation, the satisfying the preset operation condition includes: When the length of the first latent vector is greater than or equal to a preset length threshold, it is determined that a preset operating condition is met; or, When the storage space occupied by the first latent vector is greater than or equal to a preset value, it is determined that the preset operating condition is met.

8. The method according to claim 1, characterized in that After the first latent vector is processed according to the operation instruction to obtain the second latent vector, the method further includes: A deletion operation is performed on the first latent vector.

9. The method according to claim 1, characterized in that: The network model is trained in the following way: Acquire a first training data set, wherein the first training data set includes a first content sample, an operation instruction sample that meets an operation condition, and a second content sample after executing the operation instruction sample; Based on the first training data set, the network model is trained to obtain a trained network model.

10. A data processing method, characterized in that: include: receiving input information of a target processing task; Based on the network model reasoning method according to any one of claims 1 to 9, obtaining the output result of the target processing task; The target processing task includes any one of the following: an image processing task, a voice processing task, a text processing task, and a video processing task; and the input information includes any one of an image, a voice, a text, and a video.

11. A network model reasoning device, characterized in that: include: A reasoning module, configured to, in response to receiving input information of a target processing task, perform multi-step reasoning on the input information based on a network model; a determination module, configured to determine, for any one of the multi-step reasoning steps, whether a first latent vector stored in the network model satisfies a preset operation condition when a first sub-result of a current step reasoning is obtained, wherein the first latent vector includes a latent vector corresponding to the first sub-result and a latent vector corresponding to a sub-result of a historical step reasoning before the current step reasoning; an operation processing module, configured to generate an operation instruction for the first latent vector when a preset operation condition is met, and perform operation processing on the first latent vector according to the operation instruction to obtain a second latent vector; The reasoning module is further used to perform a next step of reasoning based on the second latent vector to obtain a second sub-result of the next step of reasoning; The output result for the target processing task includes a sub-result of each step of reasoning in the multi-step reasoning.

12. A data processing device, characterized in that: include: A receiving module, used for receiving input information of a target processing task; An output module, used for obtaining an output result of the target processing task based on the network model inference method according to any one of claims 1 to 9; Among them, the target processing task includes any one of the following: image processing task, voice processing task, text processing task, video processing task; the input information includes any one of the following: image, voice, text, video.

13. An electronic device, characterized in that: include: Multiple processing cores; as well as A network on chip is configured to exchange data between the multiple processing cores and external data; wherein one or more instructions are stored in one or more of the processing cores, and one or more of the instructions are executed by one or more of the processing cores, so that one or more of the processing cores can execute the network model inference method described in any one of claims 1 to 9 or the data processing method described in claim 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the network model reasoning method according to any one of claims 1 to 9 or the data processing method according to claim 10.

Images