A task processing method and apparatus

Abstract

The application discloses a task processing method and device. The method comprises the following steps: obtaining current task information; determining reusable results that can be reused for reasoning the current task from historical results according to the current task information, wherein the historical results comprise intermediate results obtained when a plurality of task stages of historical tasks are executed; and performing reasoning of the plurality of task stages of the current task according to the reusable results and / or the current task information, so as to complete the current task.

Description

Technical Field

[0001] This application relates to the field of task processing technology, and in particular to a task processing method and apparatus. Background Technology

[0002] With the development of artificial intelligence technology, most electronic devices are now capable of calling relevant inference models to reason about input information and complete corresponding tasks. For example, they can analyze images and / or documents according to user needs to obtain analysis results. However, these technologies still suffer from problems such as long inference times and high resource consumption per inference step. Summary of the Invention

[0003] Therefore, this application discloses the following technical solution:

[0004] The first aspect of this application provides a task processing method, including:

[0005] Obtain current task information;

[0006] Based on the current task information, reusable results that can be reused for the current task are determined from the historical results. The historical results include intermediate results obtained when executing multiple task stages of the historical task.

[0007] Based on the reusable results and / or the current task information, perform reasoning for multiple task stages of the current task to complete the current task.

[0008] A second aspect of this application provides an electronic device, including a memory and a processor;

[0009] The memory is used to store computer programs;

[0010] The processor is used to execute the computer program to perform:

[0011] Obtain current task information;

[0012] Based on the current task information, reusable results that can be reused for the current task are determined from the historical results. The historical results include intermediate results obtained when executing multiple task stages of the historical task.

[0013] Based on the reusable results and / or the current task information, perform reasoning for multiple task stages of the current task to complete the current task. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 This is a flowchart of a task processing method provided in an embodiment of this application;

[0016] Figure 2 This is a flowchart illustrating a method for obtaining current task information provided in an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of current image information and current text information provided in an embodiment of this application;

[0018] Figure 4 This is a schematic diagram illustrating the execution of various task stages provided in an embodiment of this application;

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a task processing method, see [link to relevant documentation] Figure 1 The method may include the following steps.

[0022] S101, Obtain current task information.

[0023] The execution subject of the task processing method in this application can be any electronic device capable of calling the inference model to execute related inference tasks, such as smartphones, laptops, and tablets.

[0024] Current task information refers to the information that an electronic device needs to reason about at the moment. The task at hand is to perform reasoning based on this current task information to obtain the result desired by the user. For example, if an electronic device receives an image and the text "What's in this image?" from the user, this image and text constitute the current task information, and the current task requiring reasoning is to identify the objects in the image.

[0025] Current task information can be obtained through various methods such as voice recognition, keyboard input, download, reading from a specified storage address, and camera capture, without any limitation.

[0026] S102, Based on the current task information, determine the reusable results that can be reused in the historical results for the current task. The historical results include intermediate results obtained when executing multiple task stages of the historical task.

[0027] Both historical and current tasks are multi-stage tasks comprising multiple task phases (or steps). Each time a task phase is executed, the electronic device obtains an intermediate result and can then proceed to the next task phase based on that result. In this application, after completing a task, the electronic device can save the intermediate results of each task phase as the aforementioned historical results in its local memory.

[0028] A reusable result is a historical result that can be reused during the reasoning process for the current task. For example, in a historical task, an electronic device generates image feature data of photo A in order to perform reasoning based on photo A. In the current task, the electronic device needs to perform reasoning based on the same photo A. At this time, the previously generated image feature data of photo A can be reused. Therefore, this feature data is determined to be a reusable historical result, i.e., it is determined to be a reusable result.

[0029] Historical tasks include one or more tasks that the electronic device has completed up to the current moment. Optionally, historical tasks may include every task completed by the electronic device within the most recent preset time period up to the current moment, such as tasks completed within the most recent 12 hours, the most recent 60 minutes, or other time periods up to the current moment.

[0030] In this case, the electronic device saves the historical results corresponding to the historical tasks within the most recent preset time period so that reusable results can be determined from them when executing S102, and the results of historical tasks that have expired (outside the most recent preset time period) can be deleted.

[0031] Alternatively, historical tasks can only include tasks that the electronic device completed before the current task and belong to the same session as the current task. The electronic device can create or close a session based on user needs. Within a session, the electronic device can receive multiple rounds of input from the user. For example, the first round of input might be "What does this whiteboard say?", the second round might be "What is the timeline in the upper left corner?", and the third round might be "Changed the bottom right corner, let's look at the action again." Upon receiving each round of input, the electronic device can use this input and the corresponding image (or other multimedia data) as the current task information and execute the task corresponding to that round of input (i.e., the current task). Two tasks belonging to the same session mean that the two tasks correspond to different rounds of input within the same session. For example, after receiving the first round of input, the electronic device executes Task 1, which generates a summary of the whiteboard content corresponding to the first round. After receiving the second round of input, the electronic device executes Task 2, which summarizes the whiteboard timeline corresponding to the second round. Task 1 and Task 2 belong to the same session.

[0032] In this case, after the electronic device creates a new session, it can save the historical results corresponding to each historical task belonging to the session before closing the session, so that reusable results can be determined in S102, and the historical results corresponding to each historical task belonging to the session can be deleted when the session is closed.

[0033] Restricting historical tasks to the same session has the following advantages: tasks belonging to different sessions from the current task generally have significantly different task information for inference, so the probability of reusing the corresponding intermediate results is low. Intermediate results from tasks belonging to the same session are more likely to be reused. Therefore, restricting them to the same session allows the electronic device to save only intermediate results that are more likely to be reused and not save intermediate results from other sessions that are unlikely to be reused. This saves storage space and makes it easier to quickly find reusable historical results in S102.

[0034] If there are no historical tasks before the current task, for example, if the current task is the first task obtained in a session, then S102 can be skipped, and multiple task stages of the current task can be executed directly based on the current task information to complete the current task.

[0035] S103, based on reusable results and / or current task information, perform reasoning for multiple task stages of the current task to complete the current task.

[0036] In step S103, if S102 determines one or more reusable results, the electronic device executes the reasoning process of multiple task stages of the current task based on the reusable results and the current task information until the current task is completed; if S102 fails to determine reusable results, that is, all historical results cannot be reused in the current task, the electronic device executes the reasoning process of multiple task stages of the current task based only on the current task information until the current task is completed.

[0037] The beneficial effect of this embodiment is that, in the related technology, when an electronic device performs the reasoning process of a task each time, it will start from the beginning (that is, from the original task information) and execute each stage of the task one by one, which results in each reasoning requiring a long time and consuming a lot of processing resources.

[0038] After applying the processing method of this application, the electronic device can select reusable results that can be reused from the historical results obtained in the past, and directly execute the subsequent task stages based on these reusable results, without having to execute repeated task stages, thereby shortening the time for each processing task and reducing the processing resources consumed.

[0039] For example, in the example above, when the electronic device performs the current task, it directly reuses the image feature data of the previously obtained photo A, thereby saving the time and processing resources used to generate the image feature data of photo A in the current task.

[0040] Optionally, the current task information includes current text information and current image information, see [link to relevant documentation]. Figure 2 The methods for obtaining current task information may include the following steps.

[0041] S201, obtain the current text information.

[0042] S202, determine the associated region in the input image that is related to the current text information.

[0043] S203, obtain the associated region of the input image as the current image information.

[0044] In S201, the current text information can be input by the user via voice or keyboard. For example, the current text information could be "What is this whiteboard about?" or "What is the timeline in the upper left corner of the whiteboard?".

[0045] In S202, the input image can be an image uploaded by the user, or an image automatically captured, downloaded, or read by a program (e.g., an intelligent agent) in an electronic device used to execute the processing scheme of this application. For example, the intelligent agent can perform a screenshot operation on the display screen to obtain a screenshot containing all the content displayed on the screen as the input image. For example, it can obtain images such as... Figure 3 The screenshot shown.

[0046] In the associated region of S202, it can be inferred by electronic devices using data processing models with text reasoning capabilities (such as large language models).

[0047] After determining the associated region, the electronic device can use the image cropping function to crop (extract) a portion of the associated region from the input image and use this portion as the current image information.

[0048] As an example, the associated region can be determined based on the description of the image content in the current text information, in order to Figure 3 For example, if the current text information is "What does this whiteboard say?", the electronic device can determine the area where the whiteboard is located in the screenshot as the associated area based on the keyword "whiteboard", and then capture that whiteboard area as the current image information.

[0049] Alternatively, the associated region can be determined based on the description of the image's location in the current text information. For example, if the current text information is "What is the timeline in the upper left corner of the whiteboard?", the timeline area in the upper left corner can be determined as the associated region based on the keyword "upper left corner", and the timeline area of ​​the whiteboard in the screenshot can be taken as the current image information.

[0050] The advantage of obtaining the current image information using the above method is that only the part of the input image that is related to the current text information is extracted as the current image information. Other parts that are not related to the current text information may not be useful when performing the current task, so they can be excluded in advance and not included in the current image information. This can reduce the amount of data to be processed in each task stage of performing the current task, thereby improving efficiency and saving resources.

[0051] Optionally, the user-provided input image can be directly identified as the current image information. For example, if a user uploads a photo and enters the text "What kind of car is in the photo?", then the photo can be identified as the current image information, and the entered text can be used as the current text information.

[0052] Optionally, if the current task information includes current image information and current text information, based on the current task information, reusable results that can be reused for the current task are determined from historical results, including:

[0053] Based on the information comparison, determine the reusable results that can be reused for the current task from the historical results;

[0054] The information comparison includes the comparison between the current image information and the historical image information corresponding to the historical task, as well as the comparison between the current text information and the historical text information corresponding to the historical task. The historical results are obtained by processing the historical image information and / or historical text information.

[0055] Historical image information and historical text information refer to the image and text information obtained when performing historical tasks, and are obtained in the same way as above. Historical results refer to the intermediate results obtained by the electronic device after performing each task stage of the historical task based on historical image information and / or historical text information.

[0056] Information comparison results can indicate whether the current image information and historical image information match successfully or fail to match, as well as whether the current text information and historical text information match successfully or fail to match. When an electronic device can determine a reusable result, it can first determine the above information comparison results, and then determine whether the corresponding historical results can be reused in reasoning the current task based on the matching results of these two types of information, that is, determine whether each historical result can be used as a reusable result.

[0057] To improve processing efficiency, electronic devices can obtain the aforementioned information comparison results by comparing the summary of task information.

[0058] For each historical result, the electronic device can record the identifier (number, name, etc.), generation time, associated token range, and information summary of the historical result through the intermediate representation object manager. The associated token range indicates which tokens the historical result is associated with, and the information summary is a summary of the image information and / or text information used to generate the historical result.

[0059] For example, a historical result is generated based on photo A. Image feature data of photo A can be extracted and hashed (or other algorithms to obtain a digest) on this image feature data. The hash result is used as the information digest of the historical result.

[0060] Therefore, the method for determining reusable results in electronic devices can be:

[0061] Following the method for obtaining information summaries of historical results, obtain the summary corresponding to the current task information (denoted as the current information summary); compare the current information summary with the information summaries of historical results. If the current information summary of an image is the same as the information summary of an image in historical results, it means that the current image information and the historical image information have been successfully matched (i.e., the image used in the task has not changed), and if they are different, the match fails. If the current information summary of text is the same as the information summary of text in historical results, it means that the current text information and the historical text information have been successfully matched (i.e., the text used in the task has not changed), and if they are different, the match fails.

[0062] Optionally, a similarity threshold can be set (e.g., 0.85, 0.9, etc., the specific value is not limited). If the similarity between the current information summary of the image and the current information summary of the image in the historical results is greater than the similarity threshold, the image is determined to be successfully matched. If the similarity between the current information summary of the text and the information summary of the text in the historical results is greater than the similarity threshold, the text is determined to be successfully matched.

[0063] Optionally, for each historical result, its corresponding phase switching marker (PSM) information can also be recorded. The phase switching marker information can record which task stage the historical result was obtained from, and which processing module the task stage was executed in. If the processing module was switched in the middle (for example, it was originally executed by the GPU and then by the CPU), the phase switching marker information can also record the reason for the switch (such as GPU overheating, insufficient video memory, etc.).

[0064] Optionally, the electronic device can also record the method of obtaining historical results. For example, if the historical results are formed by fusing image information and text information (such as the information splicing result PCU mentioned below), the electronic device can record the fusion method used when obtaining the historical results. When determining the information comparison situation, it can check whether the fusion method corresponding to the historical results is consistent with the fusion method required in the current task. If the fusion methods are inconsistent, it can be determined that the image information and text information failed to match and the historical results will not be reused.

[0065] Using the above method, the electronic device can determine whether the current image information and historical image information are consistent, and whether the current text information and historical text information are consistent. Only when the images and / or text are consistent will the historical results obtained based on the images and / or text be used as reusable results for the current task. In this way, historical results can be reused to save time and computing power, and accurate reasoning results for the current task can be obtained.

[0066] Optionally, for cases where the task information includes both text and image information, the intermediate results shall include at least the image encoding result obtained by encoding the image information, the image projection result obtained by projecting the image encoding result onto the text, and the information splicing result that fuses the image projection result and the text information.

[0067] See Figure 4 This is a schematic diagram of the various task stages of an electronic device performing a task based on text and image information, wherein each task stage includes at least an image encoding stage, an image projection stage, and an information stitching stage.

[0068] In the image encoding stage, the electronic device uses image encoding technology in the relevant field to extract feature data of image information. The obtained feature data is the image encoding result (also referred to as the visual feature cache object, VFC). In the image projection stage, the electronic device projects the image encoding result onto the text features to obtain the image projection result (also referred to as the projection result block, PRB). In this way, the image feature data is mapped across modalities to the feature space where the text features are located. In the information splicing stage, the electronic device splices and fuses the image projection result and the text information (specifically, the extracted feature data of the text information, i.e., the text prefix) to obtain the information splicing result (also referred to as the prefix building unit, PCU).

[0069] Each task stage may also include a reasoning stage. After obtaining the information splicing result, the electronic device can input the information splicing result into a neural network model with reasoning ability (such as a large language model), and perform the reasoning stage based on the model to obtain the reasoning result of the task output by the model.

[0070] Based on the above three types of historical results, the method for determining reusable results for electronic devices in this application based on information comparison can be as follows:

[0071] In response to a successful match between current image information and historical image information, and a successful match between current text information and historical text information, each historical result is identified as a reusable result that can be reused for the current task of reasoning.

[0072] In response to a successful match between current image information and historical image information, but a failure to match between current text information and historical text information, the image encoding results and image projection results in the historical results are identified as reusable results that can be reused for the current task of reasoning.

[0073] In this embodiment, if the current image information and the historical image information of a historical task are successfully matched, and the current text information and the historical text information of the historical task are successfully matched, it means that the intermediate results that can be obtained by executing each task stage of the current task are consistent with the historical results corresponding to the historical task. Therefore, the historical results corresponding to the historical task can be directly determined as reusable results, and the current task can be directly executed based on these reusable results to obtain the corresponding reasoning results.

[0074] For example, the concatenated information from the historical task that was successfully matched can be directly input into the neural network model to perform the inference stage and obtain the inference result of the current task.

[0075] If, for a historical task, the current task information and the historical task information only fail to match text information but succeed in matching image information, it means that the image information has not changed relative to the historical task. Therefore, the historical results obtained when executing the historical task that are only related to the historical image information can be reused. Thus, the image encoding results and image projection results in the historical results are identified as reusable results.

[0076] Furthermore, since the text information matching failed, it indicates that the text information of the current task has changed compared to the historical tasks. Therefore, it is necessary to regenerate the text-related intermediate results based on the current text information. To this end, the information splicing stage of the current task can be performed on the current text information and the image projection results in the reusable results to obtain the information splicing results corresponding to the current task. Then, the information splicing results of the current task can be processed based on the large language model to obtain the inference results of the current task.

[0077] If the image information and text information fail to match between the current task and the historical tasks, the image encoding stage and image projection stage of the current task can be executed one by one based on the current image information to obtain the image projection result of the current task. Then, the information splicing stage of the current task can be executed on the current text information and the image projection result of the current task to obtain the information splicing result of the current task. Finally, the information splicing result of the current task can be processed based on the large language model to obtain the reasoning result of the current task.

[0078] If image information matching fails between the current task and historical tasks, but text information matching succeeds, historical results that are only related to text information can be identified as reusable results. For the current image information, the various task stages involving the image are executed to obtain the corresponding intermediate results. For example, the image encoding stage, image projection stage, and information stitching stage are executed sequentially on the current image information to finally obtain the information stitching result. Then, the information stitching result of the current task is processed based on the large language model to obtain the inference result of the current task.

[0079] Optionally, the current task information may contain multiple current image information and multiple current text information. Some of these may match the historical task information successfully, while others may not match. In this case, the corresponding task stage can be executed on the information that fails to match to obtain relevant intermediate results, while the corresponding historical results can be directly reused as reusable results for the information that matches successfully.

[0080] For example, the current task information includes current image information 1 and current image information 2. The former is successfully matched and the latter is not matched. Therefore, the image encoding result and image projection result corresponding to the historical image information that is successfully matched with current image information 1 can be reused (as a reusable result). For current image information 2, the image encoding stage and the image projection stage are executed in sequence to obtain the corresponding intermediate result.

[0081] Through the above embodiments, the processing method of this application selects to reuse some or all of the historical results that are only related to the unchanged information, based on the changes of the current task information relative to the historical task information, in cases such as image changes but text remains unchanged, both image and text remain unchanged, image remains unchanged but text changes, and both image and text change. For information that has changed, the corresponding task stage of the current task is still executed. In this way, both processing efficiency and the accuracy of reasoning results can be taken into account.

[0082] Optionally, based on reusable results and / or current task information, perform reasoning for multiple task phases of the current task, including:

[0083] For each task stage of the current task, if the reusable results include the intermediate results corresponding to the task stage, mark the task stage as completed to skip the task stage, and perform the reasoning for the next task stage based on the reusable results.

[0084] For each task stage of the current task, if the reusable results do not include the intermediate results corresponding to the task stage, perform inference for the task stage based on the information of the current task.

[0085] In this embodiment, after determining the reusable results, the electronic device can further determine the task stage corresponding to each reusable result, that is, determine which task stages these reusable results are intermediate results obtained by performing. For example, it can be determined that VFC corresponds to the image encoding stage, PRB corresponds to the image projection stage, and PCU corresponds to the information stitching stage in the reusable results.

[0086] Based on this, electronic devices can mark the task stages corresponding to reusable results as completed. When executing each task stage of the current task, the task stages marked as completed in the current task will no longer be executed. Only those task stages that have not been marked and do not have corresponding reusable results will be executed.

[0087] As an example, if the VFC and PRB of the historical task are determined as reusable results of the current task, then before executing the current task, the image encoding stage and the image projection stage can be marked as completed. After starting to execute the current task, these two stages can be skipped, and the information splicing stage and subsequent inference stage can be executed directly based on the current text information and the PRB in the reusable results.

[0088] Optionally, if a task stage of the current task requires multiple intermediate results, some of which can reuse historical results and others cannot be reused and still require related processing, then the task stage can not be marked as completed. For example, if the reusable results include the VFC of image 1, but the VFC of image 2 cannot be reused, then the image encoding stage is not marked as completed, and the image encoding stage is still performed on image 2 to obtain the VFC of image 2 when the current task is executed.

[0089] By using the above methods, electronic devices can skip task stages that already have reusable results and only execute the task stages in the current task that have not yet obtained reusable results, which significantly improves the execution efficiency of the current task and reduces the computing resources consumed in executing the current task.

[0090] Optionally, determining reusable results from historical results that can be reused for the current task of inference based on information comparison also includes:

[0091] In response to a successful local match between current image information and historical image information, the image encoding result and image projection result corresponding to the region in the historical results that successfully matched the historical image information are determined as reusable results;

[0092] The current image encoding result and the current image projection result are obtained by processing the failed matching regions in the current image information, and the current task is completed based on the current image encoding result, the current image projection result, and the reusable result.

[0093] In this embodiment, when the electronic device determines a reusable result, if the historical result contains historical results obtained by processing local regions of historical image information, then it can compare the overall information digest of the current image information and the historical image information, as well as the information digests of these local regions, to determine whether the local regions of the current image information and the local regions of the historical image information match successfully or fail to match.

[0094] If the comparison reveals that the current image information and historical image information fail to match as a whole, but some local areas match successfully, then the historical results can be partially reused according to the method of this embodiment.

[0095] Specifically, if the current image information and historical image information are locally matched successfully, or locally matched but not successfully, then the historical results related only to the locally matched areas in the historical image information are obtained as reusable results. At the same time, the image processing-related task stages in the current task are executed on the locally matched but not successfully matched areas in the current image information to obtain the current VFC and the current PRB corresponding to the locally matched but not successfully matched areas. Then, the current PRB, the current text information, and the PRB corresponding to the successfully matched areas in the reusable results are all executed together in the information splicing stage to obtain the information splicing result. Finally, the inference stage of the current task is executed based on the information splicing result to obtain the inference result of the current task.

[0096] Combination Figure 3 For example, suppose that when executing a historical task, VFC and PRB corresponding to the timeline area and VFC and PRB corresponding to the to-do items area were generated respectively; before executing the current task, by comparing the corresponding information digest, it is found that the overall content of the current whiteboard is different from the content of the historical task (i.e., the overall image information matching fails), the to-do items of the current whiteboard are different from the to-do items of the whiteboard in the historical task, but the timeline of the current whiteboard is the same as the timeline of the whiteboard in the historical task.

[0097] Therefore, it can be determined that the VFC and PRB of the timeline region obtained in the historical task are reusable results, and the to-do area of ​​the current whiteboard is a matching failure area. The image encoding stage and the image projection stage are performed sequentially on the to-do area of ​​the current whiteboard to obtain the VFC and PRB of the current whiteboard. Finally, based on the current text information, the VFC and PRB of the current whiteboard, and the aforementioned reusable results, the information splicing stage and the subsequent reasoning stage of the current task are performed to obtain the reasoning result of the current task.

[0098] By applying the method described above, historical results obtained from different local regions of the image can be further reused in historical tasks when local changes occur in the image. This is beneficial for improving efficiency and reducing computational load when the overall matching of current image information and historical image information fails.

[0099] Optionally, if any reusable result becomes invalid due to errors, data expiration, or accidental loss during the execution of the current task, only the task stage corresponding to that reusable result can be re-executed to obtain the intermediate structure required by the current task. For example, if the VFC and PRB of a historical task are determined to be reusable results, the image projection stage of the current task can be skipped. However, if the PRB in the reusable result is found to be invalid during the information stitching stage, the image projection stage of the current task can be re-executed based on the VFC in the reusable result to obtain the PRB of the current task. Then, the information stitching stage can continue to be executed based on the current text information and the PRB of the current task.

[0100] In other words, during the execution of the current task, the electronic device can only re-execute the task stages of those temporarily invalid reusable results, without having to re-execute every task stage. This ensures the smooth execution of the current task when reusable results fail, while also improving efficiency and saving computing power.

[0101] Optionally, for each historical result, a corresponding fallback path description (FR) can also be recorded. This information can record how the historical result is regenerated when it becomes invalid. For example, the fallback path description information may include: a specified region of the historical image information (such as...). Figure 3 The timeline region executes the image encoding and image projection stages to generate the PRB; when generating the PCU, the GPU first executes the image encoding and image projection stages to obtain the PRB, and then the CPU executes the information stitching stage to obtain the PCU; if the specified time is exceeded (e.g., 22:30), the PCU in the historical results will expire and become invalid, and the PCU needs to be rebuilt.

[0102] By recording the aforementioned rollback path description information, electronic devices can quickly restore the corresponding historical results when they become invalid, so that these historical results can be reused in a timely manner in the future.

[0103] Optionally, based on information comparison, reusable results that can be reused for the current task can be determined from historical results, including:

[0104] In response to a successful match between current image information and historical image information, and a successful match between current text information and historical text information, the available processing resources for the current task at each task stage are compared with the resources used in the corresponding task stage in the historical task.

[0105] If the available processing resources and the used resources match in any task stage, the historical results obtained by the task stage with the matching resources will be determined as reusable results that can be reused in the current task.

[0106] If the available processing resources and the resources already used in any task stage do not match, multiple task stages of the current task are executed based on the current task information to obtain the corresponding intermediate results, and the current task is completed based on the intermediate results.

[0107] Available processing resources refer to how much computing resources a processing module is expected to provide to execute a task phase after the electronic device assigns the task phase to it, including but not limited to the expected cache or video memory space and the number of computing cores.

[0108] The resources used indicate the specific amount of computing resources used by the processing module when the electronic device was executing the corresponding task stage of a historical task, including but not limited to the cache or video memory space occupied, the number of computing cores occupied, etc.

[0109] Optionally, the method for determining whether resources match may include any one or more of the following: if the available processing resources and the used resources are the same, then they are determined to match; if they are different, then they are determined to not match; if the available processing resources are less than the used resources, but the difference between the available processing resources and the used resources is not greater than a preset threshold, then they are determined to match; if the difference is greater than the preset threshold, then they are determined to not match; if the available processing resources are more than the used resources, then they are determined to match.

[0110] If there is a match between available processing resources and used resources, the reusable result can be determined based on the matching result of the image and the matching result of the text, as described in the foregoing embodiments, without further elaboration.

[0111] In the event of a mismatch, the task phase with the resource mismatch can be executed based on the current task information, as well as the task phase before the mismatched task phase, to obtain the corresponding intermediate results, and the current task can be completed based on the intermediate results.

[0112] The following example illustrates how each task stage is executed in the event of a mismatch.

[0113] Suppose that the comparison determines that the current image information and the historical image information match successfully, but the available processing resources that the CPU can provide for the information stitching stage are far less than the resources used in the information stitching stage when executing the historical task, that is, the available processing resources and the used resources do not match.

[0114] This situation indicates that the VFC and PRB obtained when executing historical tasks have high precision and large data volume. If the VFC and PRB of historical tasks are reused directly in the information splicing stage, the CPU may experience overload, slow processing speed, or affect the operation of other programs due to insufficient available processing resources.

[0115] Therefore, instead of reusing the previous high-precision VFC and PRB, the image encoding and image projection stages of the current task can be re-executed based on the current image information. The processing modules that execute the image encoding and image projection stages can be configured to generate low-precision VFC and PRB with smaller data volumes. These VFC and PRB are then input to the CPU for the information splicing stage of the current task. In this way, the amount of data processed by the processing modules (e.g., the CPU) can be kept consistent with its available processing resources, avoiding various problems that may be caused by insufficient processing resources.

[0116] The advantage of applying this embodiment is that, before executing the current task, the processing resources that each processing module can provide are assessed in advance, and the actual amount of data processed by each processing module of the electronic device is matched with the available processing resources it can provide, thus avoiding the situation where there is too much data and insufficient processing resources.

[0117] Optionally, based on reusable results and / or current task information, perform reasoning for multiple task phases of the current task, including:

[0118] Based on the resource requirements of each task stage, the task stage of the current task is assigned to multiple processing modules, enabling multiple processing modules to execute the reasoning process of the assigned task stage based on reusable results and / or current task information.

[0119] In response to the performance metrics of any processing module reaching the switching threshold, the task phase allocated to the processing module is reassigned to another processing module, causing the other processing module to re-execute the inference process of the task phase.

[0120] Multiple processing modules refer to multiple processors with different performance and uses in an electronic device. As an example, multiple processing modules include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), and an embedded neural network processor (NPU) in an electronic device.

[0121] In this embodiment, the phase-change execution scheduler of the electronic device can determine the resource requirements of each task stage before executing each task stage of the current task, such as how much cache space is needed, how much bandwidth is needed, how many computing cores are needed and what types of computing cores are needed, and allocate these task stages to each processing module for execution based on these resource requirements and the performance parameters of each processing module.

[0122] For example, the image encoding stage, which requires high throughput and occupies a large cache, can be assigned to the graphics processor, the image projection stage to the NPU, and the information stitching stage to the CPU.

[0123] Based on the above allocation, the phase change execution scheduler can monitor the performance indicators of each processing module in real time. These performance indicators can include any one or more indicators that can characterize the current load of the processing module, such as the operating frequency of the processing module's computing core, the remaining capacity of the processing module's own cache (e.g., video memory), and the temperature of the processing module.

[0124] The switching threshold can be set as needed and is not limited. Different switching thresholds can be set for different performance indicators of different processing modules. For example, the switching threshold for a GPU can be 85% of its total video memory, and the switching threshold for a CPU can be 90% of its rated operating frequency.

[0125] If the performance metrics of any processing module exceed its corresponding switching threshold, and the task phase previously assigned to this processing module has not yet been completed, the task phase of this processing module can be reassigned to other processing modules whose performance metrics have not exceeded the switching threshold, so that the task phase can be re-executed on the new processing modules. During reassignment, priority can be given to assigning it to the processing module with the lightest load.

[0126] Optionally, the phase-change execution scheduler can also monitor the performance-related status of the electronic device and processing modules, and reallocate task stages among multiple processing modules based on the status of the electronic device. For example, when the electronic device is a laptop, it can monitor whether the laptop is working in battery mode or external power mode. If it is in external power mode, the image encoding stage is assigned to the graphics processor to improve processing efficiency; if it is in battery mode, the image encoding stage is assigned to the CPU to reduce power consumption.

[0127] After each reassignment, the corresponding processing module can only re-execute the reassigned task stage. Other unaffected task stages do not need to be re-executed. Instead, the reusable results of historical tasks can be reused according to the aforementioned method, or the task results generated in the current task by executing the corresponding task stages can be reused.

[0128] As an example, if the image projection stage originally allocated to the NPU is reassigned to the CPU, then the CPU only needs to re-execute the image projection stage to obtain a new PRB. The previously obtained or reused image encoding result VFC does not need to be regenerated, nor does it need to execute the image encoding stage again.

[0129] Optionally, after a task stage is reassigned, some previously obtained intermediate results can be retained, and the corresponding task stage can continue to be executed only for the unprocessed portion. For example, the image encoding stage can be reassigned from the GPU to the CPU. Before the reassignment, the GPU had already completed encoding of a portion of the image (e.g., ...). Figure 3 Encoding the timeline region (e.g., the corresponding image encoding result) yields the corresponding image encoding result. Then, the CPU can only process the remaining unencoded portion (e.g., the timeline region). Figure 3 The meeting minutes area and to-do list area) perform the image encoding stage, and finally the image encoding results of the GPU and CPU are merged and output to the NPU to perform the next task stage.

[0130] Optional methods for saving historical results include:

[0131] The frequency of use and resource consumption of historical results are obtained. Resource consumption represents the time and / or computational resource consumption for executing the corresponding task stage to obtain historical results.

[0132] Based on usage frequency and resource consumption, multiple historical results are stored in multiple storage areas with different transmission bandwidths.

[0133] Usage frequency can be represented by the number of times historical results have been reused within the most recent preset time period (e.g., the most recent 2 hours, the most recent 40 minutes). The more times, the higher the usage frequency, and vice versa. Alternatively, it can be represented by the duration of the interval between the most recent reuse of a historical result and the current time. The longer the interval, the lower the usage frequency, and vice versa.

[0134] The time resource consumption of historical results can be represented by the duration taken by the electronic device to execute the corresponding task stage to generate the historical result. For example, for an image projection result, if the electronic device takes 10 milliseconds (ms) to execute the image projection stage based on the image encoding result to obtain the image projection result, then the time resource consumption of the image projection result is recorded as 10ms.

[0135] The computational resource consumption of historical results can be represented by the storage resources (including but not limited to video memory and system memory) occupied by the electronic device in executing the corresponding task stage to generate the historical result. For example, if the electronic device needs to occupy a total of 10 megabytes (MB) of video memory during the image projection stage to obtain an image projection result, then the computational resource consumption of the image projection result is recorded as 10MB of video memory.

[0136] The resource consumption of each historical result can be statistically analyzed and recorded in real time by the electronic device when generating the historical result during the execution of the corresponding task stage. The resource consumption of historical results may include one or more of the following: time resource consumption and computational resource consumption.

[0137] In this embodiment, the electronic device can preferentially store historical results that meet the conditions of high usage frequency and / or high resource consumption in a high-bandwidth storage area that can be read and written quickly, such as in video memory and / or system memory, and preferentially store historical results that meet the conditions of low usage frequency in a low-bandwidth storage area that is read and written slowly, such as in the solid-state drive or mechanical hard drive of the electronic device.

[0138] As an example, an electronic device can sort all historical results from high to low usage frequency and from high to low resource consumption. The historical results with the highest usage frequency (e.g., the first N) and the highest resource consumption (e.g., the first M) can be stored in video memory and / or system memory, while the remaining historical results can be stored in solid-state drives.

[0139] Optionally, historical results stored in the low-bandwidth storage area can be temporarily migrated to the high-bandwidth storage area for reuse if they are to be reused. If they are no longer needed, they can be moved back to the high-bandwidth storage area.

[0140] The advantages of storing historical results in the above manner are twofold. On the one hand, storing historical results with high usage frequency and / or high resource consumption in high-bandwidth storage areas can leverage the fast read and write performance of high-bandwidth storage areas, allowing these historical results to be reused as quickly as possible when they are reusable, effectively shortening the processing time for the current task. On the other hand, storing historical results with low usage frequency in low-bandwidth storage areas can not only reuse these historical results when necessary, but also alleviate the storage pressure on high-bandwidth storage areas such as video memory and system memory, saving high-bandwidth storage resources.

[0141] Finally, an example is provided to illustrate the implementation process of the processing method in this application.

[0142] Table 1

[0143]

[0144] Assuming the electronic device displays as follows Figure 3 The whiteboard shown displays a dialog box for an intelligent agent (meeting intelligent assistant) on the right side of the screen, which can be used to perform the aforementioned task processing methods.

[0145] When the meeting's intelligent assistant starts, it automatically creates a new session. In this session, the user inputs the text information from the first round, as shown in Table 1: "What did this whiteboard say?". At this point, there are no completed historical tasks belonging to the same session, meaning there are no reusable historical results. Therefore, the meeting's intelligent assistant captures the image of the current whiteboard area as the current image information and uses the text information from the first round as the current text information. Based on this information, it sequentially performs the current task (i.e., the task of summarizing the whiteboard content) through image encoding, image projection, information stitching, and inference stages, obtaining and outputting the inference result. Figure 3 The first response message shown.

[0146] Subsequently, the meeting's intelligent assistant obtained the second round of text information, "What is the timeline in the upper left corner?", as shown in Table 1. After comparing the information summary of the image, it confirmed that the timeline area in the current whiteboard was consistent with the task in the first round (historical task), that is, the image information remained unchanged, only the text information changed. Therefore, it reused the VFC and PRB generated for the timeline area in the first round (using the VFC and PRB of the timeline area in the first round as reusable results), generated a new PCU based only on the PRB and the text information in the second round, and executed the reasoning stage of the current task in the second round (i.e., the task of summarizing the timeline information), and obtained the reasoning result, that is, the second response information.

[0147] The meeting's intelligent assistant received the third round of text information: "The bottom right corner has been changed; please review the task again." After comparison, it was determined that the current whiteboard had undergone some changes compared to the whiteboard in the previous two rounds of tasks. Specifically, the task area was different from before, while other areas (timeline and meeting minutes) remained consistent with the first round. Therefore, the VFC and PRB obtained from processing the timeline and meeting minutes areas in the first round were used as reusable objects. Simultaneously, the image encoding and image projection stages were sequentially executed for the current task area to obtain the current VFC and PRB of the task area. Finally, based on the current VFC and PRB of the task area, the third round of text information, and the reusable results, the information splicing and reasoning stages of the current task (analyzing the task) were sequentially executed to obtain and output the reasoning result, which is the third response information.

[0148] Suppose that during the image encoding phase of the current task in the third round, the phase change execution scheduler discovers that the electronic device has switched from external power mode to battery mode and the GPU temperature is greater than the preset switching threshold. In this case, the image encoding phase previously assigned to the GPU is reassigned to the CPU and NPU, and the CPU and NPU re-execute the image encoding phase.

[0149] Optionally, historical results may also include the encoded results of skill files acquired during the execution of historical tasks (denoted as skill encoding results). A skill file is essentially a structured markup document that packages specific knowledge, experience, and operational procedures into a single file. When performing a task, the inference model can learn the corresponding skill by loading the skill file, thereby accurately executing the task according to the knowledge, experience, and operational procedures provided in the skill file. The encoding results of the skill file refer to the intermediate results obtained by encoding the vocabulary contained in the skill file into corresponding vector representations.

[0150] In this embodiment, the matching degree between the current task information and the skill coding results of multiple pre-obtained skill files can be calculated. Based on the matching degree, the skill coding result that matches the current task is determined from the multiple skill coding results as a reusable result. Before executing each task stage of the current task, the skill coding result that matches the current task is loaded into the inference model used to execute the current task so that the task can be executed based on these skill coding results.

[0151] The beneficial effect of this embodiment is that it filters multiple skill coding results based on matching degree, and only loads the skill coding results that match the current task information, thereby reducing the number of loaded skill coding results, achieving the effect of improving processing efficiency and reducing the cost of executing the current task.

[0152] To filter skill coding results, each skill coding result can be pre-divided into multiple segments according to its function, such as static segment coding, execution segment coding, and dynamic segment coding.

[0153] Static segment encoding includes the skill encoding results obtained based on the static segments of the skill file. The static segments contain skill identification information, semantic information, and trigger information that change very infrequently in the skill file. Therefore, the static segment encoding of a skill file is generally obtained using deterministic serialization encoding. The corresponding encoding results are generally fixed and have the characteristics of not relying on volatile factors such as timestamps and random numbers, hash value changes when the version changes, forced trigger segment updates, and generating consistent encoding for different instances of the same skill.

[0154] The execution segment encoding includes the skill encoding results obtained based on the execution segments of the skill file. The execution segment contains some information in the skill file that describes the execution process (workflow) and script (script), such as the description information of steps 1, 2 and 3 that need to be executed in the skill file. When encoding the execution segment, semantic preservation encoding is generally used. Some workflows in the execution segment are relatively fixed, so the execution segment encoding corresponding to this part can be saved and reused.

[0155] Dynamic segment encoding includes skill encoding results obtained based on dynamic segments in the skill file. Dynamic segments contain non-fixed parts of the execution workflow and dependency declarations in the skill file. This part of the information is highly related to the task information of each executed task. For example, it may include intermediate results related to the images and text processed by each executed task. Therefore, dynamic segment encoding is generally obtained by lazy loading encoding. This part is usually not preloaded during the stage of loading the skill file in the inference model, but is obtained dynamically during execution.

[0156] Static, execution, and dynamic sections in a skill file can be distinguished by built-in fields. For example, fields representing identification information, such as "skill_id," "name," and "version," can be identified, defining the information following these fields and preceding the next field in the skill file as identification information; fields representing semantic information (e.g., "description") can be identified, defining the information following this field and preceding the next field in the skill file as semantic information; fields representing triggering information, including but not limited to "triggers" (corresponding to the "when" tag) and "excludes" (corresponding to the "NOT for" tag), can be identified, defining the information following these fields and preceding the next field in the skill file as triggering information; and fields representing execution information, including but not limited to "workflow," "dependencies," and "scripts," can be identified, defining the information following these fields and preceding the next field in the skill file as execution information.

[0157] Therefore, identification information, semantic information, and triggering information can all be defined as static segments, execution information as execution segments, and all other information as dynamic segments.

[0158] After distinguishing the different segments (static segment, execution segment, and dynamic segment) of a skill file, a vectorized index construction step can be performed on each segment of each skill file to convert the information of these segments into corresponding vector representations, thereby obtaining the static segment encoding and semantic segment encoding of each skill file. These skill encoding results are then stored in a cached skill encoding result library with the skill file as the index.

[0159] The method for calculating the matching degree between the current task information and a skill coding result is as follows:

[0160] Obtain the current text information from the current task information, convert the current text information into the corresponding text vector representation, and obtain the query vector, denoted as v_query;

[0161] Substitute the query vector and the different skill encoding results (specifically including static segment encoding and semantic segment encoding) belonging to the same skill file into the following formula (1) to calculate the similarity between the current task information and the skill encoding results of each segment;

[0162] final_score=sim(v_query, v_skill)×w_grade×w_freq, (1).

[0163] Where v_skill represents the skill encoding result of any segment, sim() represents the calculation of the cosine similarity of the vectors, final_score represents the calculated similarity, w_grade represents the segment level of the segment corresponding to the skill encoding result, where the segment level is 1.2 for the static segment and 0.8 for the execution segment, that is, the segment level of the static segment is greater than the segment level of the execution segment, and w_freq represents the historical matching hit frequency of the skill encoding result, that is, how many historical task information the skill encoding result has successfully matched in the past.

[0164] For each skill file, after obtaining the similarity between the current task information and the skill encoding results of each segment of the skill file, these similarities can be summed to obtain the matching degree between the current task information and the skill file. For example, if the similarity between the static segment encoding of skill file A and the current task information is 0.6, and the similarity between the execution segment encoding of skill file A and the current task information is 0.8, then the matching degree between skill file A and the current task information is 0.6 + 0.8, which is 1.4.

[0165] Alternatively, the similarity between the execution segment code of the skill file and the current task information can be directly selected as the matching degree between the skill file and the current task information. Combining the previous example, the similarity between the execution segment code of skill file A and the current task information is 0.8, so the matching degree between skill file A and the current task information is 0.8.

[0166] After obtaining the matching degree between the skill files and the current task information, reusable results can be determined from the skill coding results corresponding to multiple skill files based on the matching degree. The determination method is as follows:

[0167] Sort multiple skill files in descending order of matching degree, select the top N skill files (e.g., the top 3, top 5, etc.), and determine the static segment codes corresponding to these skill files as reusable results, or determine the static segment codes and execution segment codes corresponding to these skill files as reusable results.

[0168] After determining reusable results in this way, the skill files to which these reusable results belong can be recorded in the skill dependency list (dep_list) of the current task. Before starting to execute the current task, the skill dependency list can be parsed, and the corresponding static segment codes and / or execution segment codes can be loaded from the skill coding result library according to the skill files recorded in the list (i.e., loading reusable results). Then, when the current task is executed later, each task stage can be executed according to the skills represented by these static segment codes and / or execution segment codes.

[0169] In other words, when reusable results include skill coding results, reasoning is performed on multiple task stages of the current task based on the reusable results and / or current task information, including:

[0170] Based on the current task information and the skills corresponding to the skill coding results, perform reasoning for multiple task stages of the current task.

[0171] Optionally, considering the memory usage in the system, the number of skill coding results that can be stored is limited. To solve this problem, the skill coding results stored in the skill coding result library can be updated periodically (including removal, addition, etc.) according to the following lifecycle management method.

[0172] For each skill coding result, its priority score is calculated according to the following formula (2).

[0173] eviction_score=1 / (last_access_time)×log(1+access_count)×time_saved×seg_frequency×dep_skill_num, (2).

[0174] Wherein, `eviction_score` represents the priority score; `last_access_time` represents the last time the skill encoding result was used, i.e., the time when it was most recently used to execute a task; `access_count` represents the current number of times the skill encoding result has been used, indicating how many times it was used during the execution of a task when it was last loaded for task execution; `time_saved` represents the time saved, specifically the time saved after the corresponding skill encoding result was reused. For example, if a skill encoding result originally takes 2 seconds to obtain, if the skill encoding result is reused from the skill encoding result library when executing the current task, then the corresponding time saved is 2 seconds; `seg_frequency` represents the historical usage frequency, i.e., the frequency or number of times the skill encoding result has been used to execute tasks in the past; `dep_skill_num` represents the number of dependent skills, specifically how many other skill encoding results in the current skill encoding result library depend on this skill encoding result; and `log` represents the logarithm of (1+access_count) with base 10.

[0175] The time saved for each skill coding result is equivalent to the time required to obtain that skill coding result. The time saved can be predetermined and recorded in the skill coding result database.

[0176] In summary, the method for determining the priority score of a skill coding result in this embodiment can be summarized as follows: the priority score of the skill coding result is determined based on at least one or more of the following: the last time the skill coding result was used, the current number of times it was used, the time saved, the historical frequency of use, and the number of dependent skills.

[0177] The priority score of each skill coding result can be dynamically updated based on its usage and recorded in real time in the skill coding result database. Whenever the used capacity of the skill coding result database reaches its limit, that is, when the cache pool for caching skill coding results is full, the results can be sorted in ascending order of priority score from smallest to largest, and then the skill coding results with the lowest priority score can be removed one by one from the skill coding result database until the used capacity of the skill coding results is less than the limit.

[0178] This application also provides an electronic device, see [link to relevant documentation] Figure 5 This includes a memory 501 and a processor 502;

[0179] Among them, memory 501 is used to store computer programs;

[0180] Processor 502 is used to execute computer programs to perform:

[0181] Obtain current task information;

[0182] Based on the current task information, determine the reusable results that can be reused in the historical results. The historical results include intermediate results obtained when executing multiple task stages of the historical task.

[0183] Based on reusable results and / or current task information, perform reasoning across multiple task phases of the current task to complete the current task.

[0184] The working principle of the electronic device in this embodiment can be found in the task processing method of the foregoing embodiment, and will not be repeated here.

[0185] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The systems or devices described above are divided into various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0186] In this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0187] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A task processing method, comprising: Obtain current task information; Based on the current task information, reusable results that can be reused for the current task are determined from the historical results. The historical results include intermediate results obtained when executing multiple task stages of the historical task. Based on the reusable results and / or the current task information, perform reasoning for multiple task stages of the current task to complete the current task.

2. The method according to claim 1, wherein the current task information includes current image information and current text information; The step of determining reusable results that can be reused for the current task from historical results based on the current task information includes: Based on the information comparison, determine the reusable results that can be reused for the current task from the historical results; The information comparison includes the comparison between the current image information and the historical image information corresponding to the historical task, as well as the comparison between the current text information and the historical text information corresponding to the historical task. The historical result is obtained by processing the historical image information and / or the historical text information.

3. The method according to claim 2, wherein the intermediate result includes at least an image encoding result obtained by encoding image information, an image projection result obtained by projecting the image encoding result onto text, and an information splicing result that fuses the image projection result and text information; The step of determining reusable results from historical results that can be reused for the current task based on information comparison includes: In response to a successful match between the current image information and the historical image information, and a successful match between the current text information and the historical text information, each historical result is determined as a reusable result that can be reused for the current task of reasoning. In response to a successful match between the current image information and the historical image information, and a failure to match between the current text information and the historical text information, the image encoding result and image projection result in the historical results are determined as reusable results that can be reused for the current task of reasoning.

4. The method according to claim 3, wherein determining reusable results that can be reused for the current task inference from historical results based on information comparison further includes: In response to a successful local match between the current image information and the historical image information, the image encoding result and image projection result corresponding to the region in the historical result that is successfully matched with the historical image information are determined as reusable results; The current image encoding result and the current image projection result are obtained by processing the failed matching regions in the current image information, and the current task is completed based on the current image encoding result, the current image projection result, and the reusable result.

5. The method according to claim 3, wherein determining reusable results that can be reused for the current task inference from historical results based on information comparison includes: In response to a successful match between the current image information and the historical image information, and a successful match between the current text information and the historical text information, the available processing resources of the current task at each of the task stages are compared with the used resources of the corresponding task stages in the historical tasks. If the available processing resources of any task stage match the used resources, the historical results obtained by the task stage with the matching resources will be determined as reusable results that can be reused by the current task. If the available processing resources and the used resources do not match at any task stage, multiple task stages of the current task are executed based on the current task information to obtain corresponding intermediate results, and the current task is completed based on the intermediate results.

6. The method according to claim 1, wherein the step of performing reasoning for multiple task stages of the current task based on the reusable result and / or the current task information comprises: Based on the resource requirements of each task stage, the task stage of the current task is allocated to multiple processing modules, and the multiple processing modules execute the reasoning process of the allocated task stage according to the reusable results and / or the current task information. In response to any of the processing modules reaching a switching threshold, the task phase allocated to the processing module is reassigned to another processing module, causing the other processing module to re-execute the inference process of the task phase.

7. The method according to claim 1, wherein the step of performing reasoning for multiple task stages of the current task based on the reusable result and / or the current task information comprises: For each task stage of the current task, if the reusable result includes the intermediate result corresponding to the task stage, the task stage is marked as completed to skip the task stage, and the reasoning for the next task stage is performed based on the reusable result. For each task stage of the current task, if the reusable result does not include the intermediate result corresponding to the task stage, the reasoning for the task stage is performed based on the current task information.

8. The method according to claim 1, wherein the method for saving the historical results comprises: The usage frequency and resource consumption of the historical results are obtained, wherein the resource consumption represents the time and / or computational resource consumption for executing the corresponding task phase to obtain the historical results. Based on the usage frequency and resource consumption, the historical results are stored in multiple storage areas with different transmission bandwidths.

9. The method according to claim 1, wherein the current task information includes current text information and current image information; Obtaining the current task information includes: Obtain the current text information; Identify the associated region in the input image that is related to the current text information; The associated region of the input image is obtained as the current image information.

10. An electronic device, comprising a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program to perform: Obtain current task information; Based on the current task information, reusable results that can be reused for the current task are determined from the historical results. The historical results include intermediate results obtained when executing multiple task stages of the historical task. Based on the reusable results and / or the current task information, perform reasoning for multiple task stages of the current task to complete the current task.

Images