Model reasoning method, system and device and electronic equipment

Abstract

The invention provides a model reasoning method, system and device and electronic equipment, and is applied to the technical field of computers.The method comprises the steps that a target question is input into a pre-trained abstract tree generation model, an abstract tree which is generated by the abstract tree generation model and comprises multiple levels of abstracts is obtained, and the multiple levels of abstracts comprise multiple leaf node abstracts; the target problem and the multiple leaf node abstracts are spliced and then input into an end-side large model, and a key value KV matrix is calculated through the end-side large model; and respectively combining the target problem with each leaf node abstract in the plurality of leaf node abstracts to form a multi-section reasoning context, inputting the multi-section reasoning context into an end-side large model, and performing parallel reasoning on the multi-section reasoning context by combining the end-side large model with a model intermediate result to obtain a target reasoning result. The model reasoning method improves the model reasoning efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model reasoning method, system, device and electronic equipment. Background Art

[0002] As computing resources on the end-side increase, large models are expected to be deployed on the end-side, thereby achieving low-latency, high-reliability, personalized, low-cost, and privacy-protected end-side applications, improving user experience, and enriching terminal product functions and services. The existing large model based on the Transformer structure uses a serial output method, and the next token needs to wait for the previous token reasoning to end before calculation can begin; the total model reasoning time is linearly related to the length of the generated sequence. If the sequence is very long, the reasoning time will increase significantly. It can be seen that the reasoning efficiency of the existing model reasoning method is low. Summary of the invention

[0003] The embodiments of the present application provide a model reasoning method, system, device and electronic device to solve the problem of low model reasoning efficiency in existing model reasoning methods.

[0004] In order to solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a model reasoning method. The method comprises:

[0006] Inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes a plurality of leaf node summaries;

[0007] The target question and the multiple leaf node summaries are combined and input into the end-side large model, and a key-value KV matrix is ​​calculated by the end-side large model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side large model is shared during the calculation process;

[0008] The target problem is respectively combined with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and the multi-segment reasoning context is input into the end-side large model. The end-side large model is combined with the model intermediate result to perform parallel reasoning on the multi-segment reasoning context to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

[0009] Optionally, the number of the multiple reasoning contexts is n;

[0010] The step of combining the target question with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, inputting the multi-segment reasoning context into the end-side big model, and performing parallel reasoning on the multi-segment reasoning context by combining the end-side big model with the model intermediate result to obtain a target reasoning result includes:

[0011] Input the i-th reasoning context into the end-side large model, and use the model intermediate result to reason on the i-th reasoning context to obtain the i-th output sentence, where i is an integer between 1 and n;

[0012] Calculate the attention of the i-th output sentence and each leaf node summary respectively;

[0013] When the i+1th output sentence is a completion symbol or the attention value between the i+1th output sentence and the i-th leaf node summary in the i-th reasoning context is not a maximum value, determining the i-th output sentence as the i-th reasoning result;

[0014] The reasoning results from the first to the nth segments are concatenated to obtain the target reasoning result.

[0015] Optionally, the number of the multi-level summaries is n, and inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including multi-level summaries generated by the summary tree generation model comprises:

[0016] Inputting the target question into a pre-trained summary tree generation model to obtain the first level summary of the summary tree;

[0017] Repeat the first operation until j+1 equals n, wherein the first operation is: inputting the j-th level summary into the summary tree generation model to obtain the j+1-th level summary generated by the summary tree generation model, and the j+1-th level summary is a subordinate summary of the j-th level summary.

[0018] Optionally, the summary tree generation model is trained in the following manner:

[0019] Inputting a plurality of preset questions in the training question set into the end-side big model to obtain a plurality of output answers corresponding to the plurality of preset questions one by one;

[0020] The plurality of preset questions and the plurality of output answers constitute a training data set of a summary tree generation model, and the summary tree generation model is trained using the training data set.

[0021] Optionally, the plurality of preset questions include a first preset question, and a first output answer corresponds to the first preset question;

[0022] The step of forming a training data set of a summary tree generation model with the plurality of preset questions and the plurality of output answers, and training the summary tree generation model with the training data set includes:

[0023] Splitting the first output answer into a plurality of first sub-paragraphs;

[0024] Based on the attention between the plurality of first subparagraphs, calculating the attention vector between each first subparagraph and the other first subparagraphs in the plurality of first subparagraphs;

[0025] Clustering the plurality of first subparagraphs according to the attention vectors of each first subparagraph and other first subparagraphs in the plurality of first subparagraphs to obtain a plurality of second subparagraphs;

[0026] Repeat the second operation until the current sub-paragraphs cannot be clustered, and obtain the multi-level target paragraphs corresponding to the first output answer, wherein the second operation is: based on the attention between the second sub-paragraphs, calculating the attention vector between each second paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs, and clustering the plurality of second sub-paragraphs according to the attention vector between each second sub-paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs;

[0027] Performing information compression on the multi-level target paragraphs corresponding to the first output answer to obtain a multi-level summary corresponding to the first output answer;

[0028] The summary tree generation model is trained by taking the upper summary in the multi-level summary and the first preset question as input data of the summary tree generation model and taking the lower summary in the multi-level summary as output data of the summary tree generation model.

[0029] In a second aspect, an embodiment of the present application further provides a model reasoning system, the model reasoning system comprising a storage-computation integrated unit, an internal storage unit and an external storage unit;

[0030] The external storage unit is used to store the model weight of the end-side large model and the model weight of the summary tree generation model;

[0031] The storage-computation integrated unit is used to receive and store the model weight of the end-side large model and the model weight of the summary tree generation model sent by the external storage unit, calculate and store the key-value KV matrix, and perform parallel reasoning on the multiple reasoning contexts formed by combining the target problem with each leaf node summary in the multiple leaf node summaries to obtain a target reasoning result;

[0032] The internal storage unit is used to receive and store the KV matrix generated by the end-side large model during the calculation process sent by the storage and computing integrated unit.

[0033] Optionally, the summary tree generation model and the end-side large model are deployed on the model inference system, and the model inference system further includes a non-multiplication-addition computing unit;

[0034] The storage-computation integrated unit is used to process matrix multiplication and addition calculations of the client-side large model and the summary tree generation model;

[0035] The non-multiplication-addition calculation unit is used to process non-matrix multiplication-addition calculations of the client-side large model and the summary tree generation model.

[0036] In a third aspect, the embodiment of the present application further provides a model reasoning device. The model reasoning device includes:

[0037] A first acquisition module is used to input a target question into a pre-trained summary tree generation model, and obtain a summary tree including multi-level summaries generated by the summary tree generation model, wherein the last level summary of the multi-level summary includes multiple leaf node summaries;

[0038] A first calculation module, configured to combine the target question and the plurality of leaf node summaries and input them into a large end-side model, and calculate a key-value KV matrix through the large end-side model, wherein the KV matrix is ​​an intermediate result of the model, and a model weight of the large end-side model is shared during the calculation process;

[0039] The first processing module is used to combine the target problem with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and input the multi-segment reasoning context into the end-side large model, and perform parallel reasoning on the multi-segment reasoning context through the end-side large model in combination with the model intermediate result to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

[0040] In a fourth aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising a processor, wherein the processor is configured to:

[0041] Inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes a plurality of leaf node summaries;

[0042] The target question and the multiple leaf node summaries are combined and input into the end-side large model, and a key-value KV matrix is ​​calculated by the end-side large model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side large model is shared during the calculation process;

[0043] The target problem is respectively combined with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and the multi-segment reasoning context is input into the end-side large model. The end-side large model is combined with the model intermediate result to perform parallel reasoning on the multi-segment reasoning context to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

[0044] In a fifth aspect, an embodiment of the present application further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned model reasoning method when executed by the processor.

[0045] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned model reasoning method are implemented.

[0046] In a seventh aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0047] The model reasoning method of the embodiment of the present application includes inputting a target problem into a pre-trained summary tree generation model, obtaining a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes multiple leaf node summaries; splicing the target problem and the multiple leaf node summaries and inputting them into a terminal-side large model, calculating a key-value KV matrix through the terminal-side large model, wherein the KV matrix is ​​a model intermediate result, and the model weight of the terminal-side large model is shared during the calculation process; combining the target problem with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and inputting the multi-segment reasoning context into the terminal-side large model, performing parallel reasoning on the multi-segment reasoning context through the terminal-side large model in combination with the model intermediate result to obtain a target reasoning result, wherein each segment reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the terminal-side large model is shared during the parallel reasoning process.

[0048] In this way, in an embodiment of the present application, a summary tree including multi-level summaries is first generated by a pre-trained summary tree generation model, and then the end-side large model is used to infer multiple leaf node summaries in the multi-level summary. There is no dependency between the multiple leaf node summaries, so parallel reasoning can be achieved and the reasoning process can be accelerated. In the process of reasoning of the end-side large model, model weights and model intermediate results can be shared, reducing the number of data transfers within the end-side large model, thereby improving the model reasoning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 is a flow chart of the model reasoning method provided in an embodiment of the present application;

[0051] Figure 2 is a schematic diagram of a summary tree provided in an embodiment of the present application;

[0052] Figure 3 is a schematic diagram of the large model reasoning process on the end side provided in an embodiment of the present application;

[0053] Figure 4 is a schematic diagram of training data of a summary tree generation model provided in an embodiment of the present application;

[0054] Figure 5 Schematic diagram of the training process of the summary tree generation model provided in the embodiment of the present application

[0055] Figure 6 is a schematic diagram of abstract extraction provided in an embodiment of the present application;

[0056] Figure 7 It is an architecture diagram of the model reasoning system provided in the embodiment of the present application;

[0057] Figure 8 is a structural diagram of a model reasoning device provided in one embodiment of the present application;

[0058] Fig. 9 It is a structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] This application embodiment provides a model reasoning method. Figure 1 , Figure 1 is a flow chart of the model reasoning method provided in the embodiment of the present application, such as Figure 1 As shown, the following steps are included:

[0061] Step 101: input a target question into a pre-trained summary tree generation model to obtain a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes a plurality of leaf node summaries;

[0062] In this step, see Figure 2 For example, the target question Q can be “What are the types of storage and computing integration?” The target question Q is input into the pre-trained summary tree generation model to obtain the first-level summary {(introduction), (main types of storage and computing integration), (conclusion)}, and then the first-level summary is input into the summary tree generation model to obtain the second-level summary, and this is repeated until the following is obtained. Figure 2 The summary tree shown includes a multi-level summary. The multi-level summary includes multiple leaf node summaries. Figure 2 The leaf node summary of the summary tree is {(definition), (application scenario), (description), (example), (description), (example), (PIM), (advantages of different architectures), (advantages of technological development)}.

[0063] Step 102: combine the target question and the multiple leaf node summaries and input them into the end-side big model, and calculate the key-value KV matrix through the end-side big model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side big model is shared during the calculation process;

[0064] In this step, the large model on the edge refers to a large neural network model deployed on edge devices (such as smartphones, smart home products, etc.). Unlike traditional large models that are usually deployed in the cloud or data center, the large model on the edge runs on the user's local device to achieve faster and more private data processing and response.

[0065] The target question and multiple leaf node summaries are combined and input into the large end-side model, which calculates the key-value (KV) matrix of the target question and multiple leaf node summaries. During the calculation process, the model weights of the large end-side model can be shared to achieve parallel calculation, and the degree of parallelism is the input word length.

[0066] Step 103: Combine the target problem with each leaf node summary in the multiple leaf node summaries to form a multi-segment reasoning context, and input the multi-segment reasoning context into the end-side big model. Perform parallel reasoning on the multi-segment reasoning context in combination with the model intermediate result through the end-side big model to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary in the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side big model is shared in the parallel reasoning process.

[0067] In this step, the target question Q and multiple leaf node summaries {S1, S2, S3, ... Sn} are combined to form a multi-segment reasoning context C = {(Q, S1), (Q, S2), (Q, S3), ... (Q, Sn)}, which is input into the large model on the client side, and the multi-terminal reasoning context is parallelly reasoned in combination with the model intermediate results, with a parallelism of N. In the process of parallel reasoning of the large model on the client side, the model weights and model intermediate results of the large model on the client side can be shared.

[0068] In one embodiment, see Figure 3, input the target question into the pre-trained summary tree generation model, and obtain the summary tree including multi-level summaries generated by the summary tree generation model. The target question and the multiple leaf node summaries in the multi-level summary are combined and input into the end-side large model, and the key value KV matrix is ​​calculated by the end-side large model. Then the target question is combined with each leaf node summary in the multiple leaf node summaries to form a multi-segment reasoning context, and the multi-segment reasoning context is input into the end-side large model, and the end-side large model is combined with the model intermediate result to perform parallel reasoning on the multi-segment reasoning context to obtain the target reasoning result. The model weight and the intermediate result can be shared in the process of calculation and parallel reasoning of the end-side large model. In this implementation, a summary tree including multi-level summaries is first generated by a pre-trained summary tree generation model, and then the end-side large model is used to reason on the multiple leaf node summaries in the multi-level summary. There is no dependency between the multiple leaf node summaries, so that parallel reasoning can be realized, and the reasoning process can be accelerated. In the process of reasoning of the end-side large model, the model weight and the model intermediate result can be shared, which reduces the number of data transfers within the end-side large model, so the model reasoning method improves the model reasoning efficiency.

[0069] Optionally, the number of the multiple reasoning contexts is n;

[0070] The step of combining the target question with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, inputting the multi-segment reasoning context into the end-side big model, and performing parallel reasoning on the multi-segment reasoning context by combining the end-side big model with the model intermediate result to obtain a target reasoning result includes:

[0071] Input the i-th reasoning context into the end-side large model, and use the model intermediate result to reason on the i-th reasoning context to obtain the i-th output sentence, where i is an integer between 1 and n;

[0072] Calculate the attention of the i-th output sentence and each leaf node summary respectively;

[0073] When the i+1th output sentence is a completion symbol or the attention value between the i+1th output sentence and the i-th leaf node summary in the i-th reasoning context is not a maximum value, determining the i-th output sentence as the i-th reasoning result;

[0074] The reasoning results from the first to the nth segments are concatenated to obtain the target reasoning result.

[0075] In one embodiment, continue with Figure 2Take the summary tree as an example: the reasoning context of the i-th paragraph can be {What are the types of storage and computing integration, definition}. After reasoning with the end-side big model, the i-th output sentence can be "Storage and computing integration aims to integrate data storage and computing functions to reduce the delay and energy consumption caused by the frequent movement of data between memory and processor". The attention of the i-th output sentence and each leaf node summary of {(definition), (application scenario), (description), (example), (description), (example), (PIM), (different architecture advantages), (technological development advantages)} is calculated. It can be calculated that the attention value of the i-th output sentence and "definition" is the largest, and the output continues.

[0076] When the i+1th output sentence is a completion symbol, it indicates that the reasoning of the i-th reasoning context is completed, and the i-th output sentence is determined to be the reasoning result of the i-th reasoning context;

[0077] or,

[0078] The i+1th output sentence is "This architecture is particularly suitable for application scenarios that require a large amount of data processing and analysis, such as artificial intelligence and big data analysis." By calculating the attention of the i+1th output sentence and each of the leaf node summaries {(definition), (application scenario), (description), (example), (description), (example), (PIM), (advantages of different architectures), (advantages of technological development)}, it can be calculated that the attention value of the i+1th output sentence with "application scenario" is the largest, and the attention value with "definition" is not the largest, indicating that the reasoning about "definition" has ended, and the i-th output sentence is determined to be the reasoning result of the i-th paragraph.

[0079] Existing large-scale model reasoning on the end mainly adopts a decoder-only structure, which is a one-way attention structure. The text generation only focuses on the previous text and cannot take into account the following information, which easily generates repeated and contradictory content. In this implementation, by generating a summary tree including multi-level summaries in advance, the large-scale model on the end knows the information of all summaries, so that the large-scale model on the end has a global vision, and calculates the similarity between the generated content and the context during reasoning (that is, the attention value of the output sentence and all summaries can be calculated), so that the generated content is closely related to the summary, thereby improving the quality of the reasoning results generated by the large-scale model on the end.

[0080] Optionally, the number of the multi-level summaries is n, and inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including multi-level summaries generated by the summary tree generation model comprises:

[0081] Inputting the target question into a pre-trained summary tree generation model to obtain the first level summary of the summary tree;

[0082] Repeat the first operation until j+1 equals n, wherein the first operation is: inputting the j-th level summary into the summary tree generation model to obtain the j+1-th level summary generated by the summary tree generation model, and the j+1-th level summary is a subordinate summary of the j-th level summary.

[0083] In one embodiment, continue with Figure 2 The target question Q can be “What are the types of storage-computing integration?” The target question Q is input into the pre-trained summary tree generation model to obtain the first-level summary {(introduction), (main types of storage-computing integration), (conclusion)}, and then the first-level summary is input into the summary tree generation model to obtain the second-level summary {(definition), (application scenario), (near-memory computing), (in-memory computing), (PIM), (advantages of different architectures), (advantages of technological development)}, and this is repeated until the following is obtained. Figure 2 The summary tree shown includes a multi-level summary. The multi-level summary includes multiple leaf node summaries. Figure 2 The leaf node summary of the summary tree is {(definition), (application scenario), (description), (example), (description), (example), (PIM), (advantages of different architectures), (advantages of technological development)}.

[0084] In this implementation, by recursively generating a multi-level summary tree, information is gradually refined and condensed, so that originally complex data becomes clearly layered and easy to manage.

[0085] Optionally, the summary tree generation model is trained in the following manner:

[0086] Inputting a plurality of preset questions in the training question set into the end-side big model to obtain a plurality of output answers corresponding to the plurality of preset questions one by one;

[0087] The plurality of preset questions and the plurality of output answers constitute a training data set of a summary tree generation model, and the summary tree generation model is trained using the training data set.

[0088] In one implementation, the training problem set {q1, q2, q3, ..., q n}, and obtain multiple output answers {a1, a2, a3, ..., a1, a2, a3, ..., a3, a1, a2 ... n}, and then use a training data set consisting of multiple preset questions and multiple output answers {(q1, a1), (q2, a2), (q3, a3), ..., (q n , a n )} Train the summary tree generation model.

[0089] Since the inference contents of different end-side large models are different, the summary tree generated by the summary tree generation model needs to match the inference characteristics of the end-side large model, so it is difficult to use a general summary tree generation model. In this implementation, the training data set of the summary tree generation model is constructed with the participation of the end-side large model, which is conducive to the subsequent use of the summary tree generation model to generate a summary tree that meets the inference characteristics of the end-side large model.

[0090] Optionally, the plurality of preset questions include a first preset question, and a first output answer corresponds to the first preset question;

[0091] The step of forming a training data set of a summary tree generation model with the plurality of preset questions and the plurality of output answers, and training the summary tree generation model with the training data set includes:

[0092] Splitting the first output answer into a plurality of first sub-paragraphs;

[0093] Based on the attention between the plurality of first subparagraphs, calculating the attention vector between each first subparagraph and the other first subparagraphs in the plurality of first subparagraphs;

[0094] Clustering the plurality of first subparagraphs according to the attention vectors of each first subparagraph and other first subparagraphs in the plurality of first subparagraphs to obtain a plurality of second subparagraphs;

[0095] Repeat the second operation until the current sub-paragraphs cannot be clustered, and obtain the multi-level target paragraphs corresponding to the first output answer, wherein the second operation is: based on the attention between the second sub-paragraphs, calculating the attention vector between each second paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs, and clustering the plurality of second sub-paragraphs according to the attention vector between each second sub-paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs;

[0096] Performing information compression on the multi-level target paragraphs corresponding to the first output answer to obtain a multi-level summary corresponding to the first output answer;

[0097] The summary tree generation model is trained by taking the upper summary in the multi-level summary and the first preset question as input data of the summary tree generation model and taking the lower summary in the multi-level summary as output data of the summary tree generation model.

[0098] In one embodiment, Figure 4 For example, the first preset question q1 is "What are the types of storage and computing integration?", and the first output answer is a1 "Storage and computing integration technology aims to store data... the growing demand for low-power computing." Training the summary tree generation model can be divided into the following steps:

[0099] 1. Corpus splitting: Split the first output answer a1 into sub-paragraphs a1 = [sub1, sub2, ..., sub m ], where a sub-paragraph can be a natural paragraph, sentence, etc.

[0100] 2. Corpus relevance analysis: Calculate the attention between multiple first sub-paragraphs corresponding to the first output answer based on the client-side large model, and then calculate the attention vector [att1, att2, ..., att m ]. For example, in a Transformer layer, a sub-paragraph and all sub-paragraphs are obtained by K×V T The sub-paragraph attention takes the average value of all tokens in the sub-paragraph as the attention matrix between sub-paragraphs in this layer. The maximum value of the attention matrix of all layers of Transformer is taken as the final result of the attention between sub-paragraphs.

[0101] 3. Corpus clustering: see Figure 5 , cluster all first sub-paragraphs according to the attention vector, and after clustering, group the similar and connected sub-paragraphs into the second sub-paragraph a1 = [para1, para2, ..., para k ].

[0102] The second sub-paragraph is regarded as a new sub-paragraph, and steps 2 and 3 are repeated until the sub-paragraphs cannot be clustered, thereby obtaining a multi-level target paragraph corresponding to the first output answer a1.

[0103] 4. Corpus compression: compress the information of the multi-level target paragraphs to obtain a multi-level summary corresponding to the first output answer a1.

[0104] 5. Take the upper summary and the first preset question in the multi-level summary as the input data of the summary tree generation model, take the lower summary in the multi-level summary as the output data of the summary tree generation model, and train the summary tree generation model. Figure 6 Taking the summary tree generation model as an example, the L1-level “main types of storage and computing integration”, “introduction” and “conclusion”, as well as the first preset question “what are the types of storage and computing integration” are used as the input of the summary tree generation model, and the L2-level “definition”, “application scenarios”, “near-memory computing”, “in-memory computing”, “advantages of different architectures” and “technological development trends” are used as the output of the summary tree generation model.

[0105] 6. The training data in step 5 is used to fine-tune the summary tree generation model. The model structure of the summary tree generation model can be a recurrent neural network (RNN), a long short-term memory (LSTM), and a lightweight Transformer. The model size is about 100M, which is much smaller than the number of parameters of the large model on the client side.

[0106] In this implementation, the attention mechanism is used to capture the correlation between each sub-paragraph to ensure that key information points are not missed, thereby improving the accuracy of the summary. By splitting and clustering, information processing is performed at a finer granularity, and the text content can be understood and summarized more accurately. The upper-level summary and the initial question are used as input, and the lower-level summary is used as output, which can better capture the correlation between information and improve the efficiency and effect of model training.

[0107] See also Figure 7 , Figure 7 This is an architecture diagram of a model reasoning system provided by an embodiment of the present application. The model reasoning system includes a storage and computing integrated unit, an internal storage unit, and an external storage unit;

[0108] The external storage unit is used to store the model weight of the end-side large model and the model weight of the summary tree generation model;

[0109] The storage-computation integrated unit is used to receive and store the model weight of the end-side large model and the model weight of the summary tree generation model sent by the external storage unit, calculate and store the key-value KV matrix, and perform parallel reasoning on the multiple reasoning contexts formed by combining the target problem with each leaf node summary in the multiple leaf node summaries to obtain a target reasoning result;

[0110] The internal storage unit is used to receive and store the KV matrix generated by the end-side large model during the calculation process sent by the storage and computing integrated unit.

[0111] In one embodiment, the model reasoning system mainly includes an external storage unit "Dynamic Random Access Memory (DRAM)", a storage and computing integrated unit "Compute-in-Memory (CIM)" and an internal storage unit "Static Random-Access Memory-Compute In Memory Macro (SRAM-CIM Macro)".

[0112] The external storage unit is responsible for storing the model weights of the large model on the client side and the model weights of the summary tree generation model.

[0113] Storage and computing unit: responsible for attention calculation and feed-forward neural network (FFN) calculation, storing the model weights of the large model on the end and the model weights of the summary tree generation model, and reducing the number of data transfers by sharing weight data. In addition, the KV matrix in the attention calculation can also be stored, and the number of data transfers can be reduced by sharing intermediate cache data.

[0114] Internal storage unit: responsible for storing the KV matrix generated during the large model calculation process on the end side, further reducing the inference delay in the key value cache (KV cache) stage.

[0115] Traditional computing architectures have data transfer problems. Insufficient bandwidth on the end side leads to low computing power utilization, and it is difficult to further improve the speed and energy efficiency of large model inference on the end side. This implementation method proposes a model inference system based on a storage-computing integrated architecture. The model inference system includes a storage-computing integrated unit, an internal storage unit, and an external storage unit. The sharable model weights or intermediate results (KV matrix) are stored in the storage-computing integrated unit to reduce the number of data transfers, thereby improving the inference speed and energy efficiency. The intermediate results are stored in the internal storage unit, and the speed and energy efficiency of attention calculation can be further improved by taking advantage of the near-memory architecture.

[0116] Optionally, the summary tree generation model and the end-side large model are deployed on the model inference system, and the model inference system further includes a non-multiplication-addition computing unit;

[0117] The storage-computation integrated unit is used to process matrix multiplication and addition calculations of the client-side large model and the summary tree generation model;

[0118] The non-multiplication-addition calculation unit is used to process non-matrix multiplication-addition calculations of the client-side large model and the summary tree generation model.

[0119] In one implementation, the summary tree generation model and the end-side large model are deployed on a model reasoning system, and the model reasoning system also includes a non-multiplication-addition computing unit, and further, may also include a pre- and post-processing unit and a control unit. The non-multiplication-addition computing unit is used to process the non-matrix multiplication-addition calculations of the end-side large model and the summary tree generation model. The storage-computation integrated unit is used to process the matrix multiplication-addition calculations of the end-side large model and the summary tree generation model. The pre- and post-processing units are used to perform word segmentation conversion, including one-hot encoding, position encoding, top-k calculation, etc. The control unit is responsible for data scheduling and control of each unit.

[0120] In this implementation, by deploying the summary tree generation model and the end-side large model in a model reasoning system combined with a dedicated computing unit, not only is performance optimized, but hardware resources are fully utilized, system efficiency is improved, and flexibility is increased.

[0121] See also Figure 8 , Figure 8 is a structural diagram of a model reasoning device provided in one embodiment of the present application. Figure 8 As shown, the model reasoning device 800 includes:

[0122] The first acquisition module 801 is used to input the target question into a pre-trained summary tree generation model, and obtain a summary tree including multi-level summaries generated by the summary tree generation model, wherein the last level summary of the multi-level summary includes multiple leaf node summaries;

[0123] A first calculation module 802 is used to combine the target question and the multiple leaf node summaries and input them into the end-side big model, and calculate a key-value KV matrix through the end-side big model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side big model is shared during the calculation process;

[0124] The first processing module 803 is used to combine the target problem with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and input the multi-segment reasoning context into the end-side large model, and perform parallel reasoning on the multi-segment reasoning context through the end-side large model in combination with the model intermediate result to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

[0125] Optionally, the number of the multiple reasoning contexts is n;

[0126] The first processing module comprises:

[0127] A first reasoning unit is used to input the i-th reasoning context into the end-side large model, and use the intermediate result of the model to reason on the i-th reasoning context to obtain the i-th output sentence, where i is an integer between 1 and n;

[0128] A first calculation unit, used to calculate the attention of the i-th output sentence and each leaf node summary respectively;

[0129] A first determining unit is used to determine that the i-th output sentence is the i-th segment reasoning result when the i+1-th output sentence is a completion symbol or the attention value between the i+1-th output sentence and the i-th leaf node summary in the i-th segment reasoning context is not a maximum value;

[0130] The first concatenation unit is used to concatenate the reasoning results of the first to nth segments to obtain the target reasoning result.

[0131] Optionally, the number of the multi-level summaries is n, and the first acquisition module includes:

[0132] A first processing unit is used to input the target question into a pre-trained summary tree generation model to obtain a first-level summary of the summary tree;

[0133] The first execution unit is used to repeatedly execute a first operation until j+1 is equal to n, wherein the first operation is: inputting the j-th level summary into the summary tree generation model to obtain the j+1-th level summary generated by the summary tree generation model, and the j+1-th level summary is a subordinate summary of the j-th level summary.

[0134] Optionally, the summary tree generation model is trained in the following manner, and the device further includes:

[0135] A second processing module is used to input a plurality of preset questions in the training question set into the end-side large model to obtain a plurality of output answers corresponding to the plurality of preset questions one by one;

[0136] The first training module is used to form a training data set of a summary tree generation model with the multiple preset questions and the multiple output answers, and train the summary tree generation model using the training data set.

[0137] Optionally, the plurality of preset questions include a first preset question, and a first output answer corresponds to the first preset question;

[0138] The first training module includes:

[0139] A first splitting unit, used for splitting the first output answer into a plurality of first sub-paragraphs;

[0140] A first calculation unit, configured to calculate an attention vector between each first subparagraph and other first subparagraphs in the plurality of first subparagraphs based on the attention between the plurality of first subparagraphs;

[0141] a first clustering unit, configured to cluster the plurality of first subparagraphs according to the attention vectors of each first subparagraph and other first subparagraphs in the plurality of first subparagraphs, to obtain a plurality of second subparagraphs;

[0142] a second execution unit, configured to repeatedly execute the second operation until the current sub-paragraphs cannot be clustered, and a multi-level target paragraph corresponding to the first output answer is obtained, wherein the second operation is: based on the attention between the second sub-paragraphs, calculating the attention vector between each second paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs, and clustering the plurality of second sub-paragraphs according to the attention vector between each second sub-paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs;

[0143] a first compression unit, configured to compress information of the multi-level target paragraphs corresponding to the first output answer to obtain a multi-level summary corresponding to the first output answer;

[0144] The first training unit is used to train the summary tree generation model by using the upper summary in the multi-level summary and the first preset question as input data of the summary tree generation model and using the lower summary in the multi-level summary as output data of the summary tree generation model.

[0145] It should be noted that this embodiment is Figure 1 The specific implementation of the corresponding device side embodiment can be found in Figure 1 To avoid repeated description, the relevant description of the embodiment shown in will not be repeated in this embodiment, and the same beneficial effects can be achieved.

[0146] The present application also provides an electronic device. Since the principle of solving the problem by the electronic device is similar to the model reasoning method in the present application, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated. Fig. 9 As shown, the electronic device of the embodiment of the present application includes: a processor 900, which is used to read the program in the memory 920 and execute the following process:

[0147] Inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes a plurality of leaf node summaries;

[0148] The target question and the multiple leaf node summaries are combined and input into the end-side large model, and a key-value KV matrix is ​​calculated by the end-side large model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side large model is shared during the calculation process;

[0149] The target problem is respectively combined with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and the multi-segment reasoning context is input into the end-side large model. The end-side large model is combined with the model intermediate result to perform parallel reasoning on the multi-segment reasoning context to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

[0150] Optionally, the number of the multiple reasoning contexts is n;

[0151] The processor 900 is used to read the program in the memory 920 and execute the following process:

[0152] Input the i-th reasoning context into the end-side large model, and use the model intermediate result to reason on the i-th reasoning context to obtain the i-th output sentence, where i is an integer between 1 and n;

[0153] Calculate the attention of the i-th output sentence and each leaf node summary respectively;

[0154] When the i+1th output sentence is a completion symbol or the attention value between the i+1th output sentence and the i-th leaf node summary in the i-th reasoning context is not a maximum value, determining the i-th output sentence as the i-th reasoning result;

[0155] The reasoning results from the first to the nth segments are concatenated to obtain the target reasoning result.

[0156] Optionally, the number of the multi-level summaries is n, and the processor 900 is configured to read a program in the memory 920 and execute the following process:

[0157] Inputting the target question into a pre-trained summary tree generation model to obtain the first level summary of the summary tree;

[0158] Repeat the first operation until j+1 equals n, wherein the first operation is: inputting the j-th level summary into the summary tree generation model to obtain the j+1-th level summary generated by the summary tree generation model, and the j+1-th level summary is a subordinate summary of the j-th level summary.

[0159] Optionally, the summary tree generation model is trained in the following manner:

[0160] Inputting a plurality of preset questions in the training question set into the end-side big model to obtain a plurality of output answers corresponding to the plurality of preset questions one by one;

[0161] The plurality of preset questions and the plurality of output answers constitute a training data set of a summary tree generation model, and the summary tree generation model is trained using the training data set.

[0162] Optionally, the processor 900 is used to read the program in the memory 920 and perform the following process;

[0163] The step of forming a training data set of a summary tree generation model with the plurality of preset questions and the plurality of output answers, and training the summary tree generation model with the training data set includes:

[0164] Splitting the first output answer into a plurality of first sub-paragraphs;

[0165] Based on the attention between the plurality of first subparagraphs, calculating the attention vector between each first subparagraph and the other first subparagraphs in the plurality of first subparagraphs;

[0166] Clustering the plurality of first subparagraphs according to the attention vectors of each first subparagraph and other first subparagraphs in the plurality of first subparagraphs to obtain a plurality of second subparagraphs;

[0167] Repeat the second operation until the current sub-paragraphs cannot be clustered, and obtain the multi-level target paragraphs corresponding to the first output answer, wherein the second operation is: based on the attention between the second sub-paragraphs, calculating the attention vector between each second paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs, and clustering the plurality of second sub-paragraphs according to the attention vector between each second sub-paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs;

[0168] Performing information compression on the multi-level target paragraphs corresponding to the first output answer to obtain a multi-level summary corresponding to the first output answer;

[0169] The summary tree generation model is trained by taking the upper summary in the multi-level summary and the first preset question as input data of the summary tree generation model and taking the lower summary in the multi-level summary as output data of the summary tree generation model.

[0170] Among them, Fig. 9In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 900 and memory represented by memory 920. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The processor 900 is responsible for managing the bus architecture and general processing, and the memory 920 may store data used by the processor 900 when performing operations.

[0171] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned model reasoning method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0172] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0173] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0175] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model reasoning method, characterized in that: The method comprises: Inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes a plurality of leaf node summaries; The target question and the multiple leaf node summaries are combined and input into the end-side large model, and a key-value KV matrix is ​​calculated by the end-side large model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side large model is shared during the calculation process; The target problem is respectively combined with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and the multi-segment reasoning context is input into the end-side large model. The end-side large model is combined with the model intermediate result to perform parallel reasoning on the multi-segment reasoning context to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

2. The model reasoning method according to claim 1, characterized in that: The number of the multi-segment reasoning contexts is n; The step of combining the target question with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, inputting the multi-segment reasoning context into the end-side big model, and performing parallel reasoning on the multi-segment reasoning context by combining the end-side big model with the model intermediate result to obtain a target reasoning result includes: Input the i-th reasoning context into the end-side large model, and use the model intermediate result to reason on the i-th reasoning context to obtain the i-th output sentence, where i is an integer between 1 and n; Calculate the attention of the i-th output sentence and each leaf node summary respectively; When the i+1th output sentence is a completion symbol or the attention value between the i+1th output sentence and the i-th leaf node summary in the i-th reasoning context is not a maximum value, determining the i-th output sentence as the i-th reasoning result; The reasoning results from the first to the nth segments are concatenated to obtain the target reasoning result.

3. The model reasoning method according to claim 1, characterized in that: The number of the multi-level summaries is n, and the step of inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including the multi-level summaries generated by the summary tree generation model comprises: Inputting the target question into a pre-trained summary tree generation model to obtain the first level summary of the summary tree; Repeat the first operation until j+1 equals n, wherein the first operation is: inputting the j-th level summary into the summary tree generation model to obtain the j+1-th level summary generated by the summary tree generation model, and the j+1-th level summary is a subordinate summary of the j-th level summary.

4. The model reasoning method according to claim 1, characterized in that: The summary tree generation model is trained in the following way: Inputting a plurality of preset questions in the training question set into the end-side big model to obtain a plurality of output answers corresponding to the plurality of preset questions one by one; The plurality of preset questions and the plurality of output answers constitute a training data set of a summary tree generation model, and the summary tree generation model is trained using the training data set.

5. The model reasoning method according to claim 4, characterized in that: The plurality of preset questions include a first preset question, and a first output answer corresponds to the first preset question; The step of forming a training data set of a summary tree generation model with the plurality of preset questions and the plurality of output answers, and training the summary tree generation model with the training data set includes: Splitting the first output answer into a plurality of first sub-paragraphs; Based on the attention between the plurality of first subparagraphs, calculating the attention vector between each first subparagraph and the other first subparagraphs in the plurality of first subparagraphs; Clustering the plurality of first subparagraphs according to the attention vectors of each first subparagraph and other first subparagraphs in the plurality of first subparagraphs to obtain a plurality of second subparagraphs; Repeat the second operation until the current sub-paragraphs cannot be clustered, and obtain the multi-level target paragraphs corresponding to the first output answer, wherein the second operation is: based on the attention between the second sub-paragraphs, calculating the attention vector between each second paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs, and clustering the plurality of second sub-paragraphs according to the attention vector between each second sub-paragraph and the other second sub-paragraphs in the plurality of second sub-paragraphs; Performing information compression on the multi-level target paragraphs corresponding to the first output answer to obtain a multi-level summary corresponding to the first output answer; The summary tree generation model is trained by taking the upper summary in the multi-level summary and the first preset question as input data of the summary tree generation model and taking the lower summary in the multi-level summary as output data of the summary tree generation model.

6. A model reasoning system, characterized in that: The model reasoning system includes a storage-computation integrated unit, an internal storage unit and an external storage unit; The external storage unit is used to store the model weight of the end-side large model and the model weight of the summary tree generation model; The storage-computation integrated unit is used to receive and store the model weight of the end-side large model and the model weight of the summary tree generation model sent by the external storage unit, calculate and store the key-value KV matrix, and perform parallel reasoning on a multi-segment reasoning context formed by combining a target problem with each leaf node summary in a plurality of leaf node summaries to obtain a target reasoning result; The internal storage unit is used to receive and store the KV matrix generated by the end-side large model during the calculation process sent by the storage and computing integrated unit.

7. The model reasoning system according to claim 6, characterized in that: The summary tree generation model and the end-side large model are deployed on the model reasoning system, and the model reasoning system further includes a non-multiplication-addition computing unit; The storage-computation integrated unit is used to process matrix multiplication and addition calculations of the client-side large model and the summary tree generation model; The non-multiplication-addition calculation unit is used to process non-matrix multiplication-addition calculations of the client-side large model and the summary tree generation model.

8. A model reasoning device, characterized in that: The device comprises: A first acquisition module is used to input a target question into a pre-trained summary tree generation model, and obtain a summary tree including multi-level summaries generated by the summary tree generation model, wherein the last level summary of the multi-level summary includes multiple leaf node summaries; A first calculation module, configured to combine the target question and the plurality of leaf node summaries and input them into a large end-side model, and calculate a key-value KV matrix through the large end-side model, wherein the KV matrix is ​​an intermediate result of the model, and a model weight of the large end-side model is shared during the calculation process; The first processing module is used to combine the target problem with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and input the multi-segment reasoning context into the end-side large model, and perform parallel reasoning on the multi-segment reasoning context through the end-side large model in combination with the model intermediate result to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

9. An electronic device, characterized in that: comprising a processor configured to: Inputting the target question into a pre-trained summary tree generation model to obtain a summary tree including a multi-level summary generated by the summary tree generation model, wherein the multi-level summary includes a plurality of leaf node summaries; The target question and the multiple leaf node summaries are combined and input into the end-side large model, and a key-value KV matrix is ​​calculated by the end-side large model, wherein the KV matrix is ​​an intermediate result of the model, and the model weight of the end-side large model is shared during the calculation process; The target problem is respectively combined with each leaf node summary of the multiple leaf node summaries to form a multi-segment reasoning context, and the multi-segment reasoning context is input into the end-side large model. The end-side large model is combined with the model intermediate result to perform parallel reasoning on the multi-segment reasoning context to obtain a target reasoning result, wherein each segment of the reasoning context is formed by combining the target problem and each leaf node summary of the multiple leaf node summaries, and the target reasoning result is obtained by splicing the multiple segment reasoning results corresponding to the multiple segment reasoning contexts, and the model weight of the end-side large model is shared in the parallel reasoning process.

10. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the model reasoning method as claimed in any one of claims 1 to 5 when executed by the processor.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the model reasoning method according to any one of claims 1 to 5.

12. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the model reasoning method according to any one of claims 1 to 5.

Images