Processing method and electronic equipment

Abstract

The invention discloses a processing method and electronic equipment, and the processing method comprises the steps: obtaining part information of a target part needed for executing a target processing task in response to the obtained target processing task; generating a corresponding feedback result based on the component information, so that a target object executes a target response operation matched with the feedback result; wherein a target association relationship exists between the target component and the target object.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and particularly to a processing method and an electronic device. Background Art

[0002] With the rapid development of information technology, in operating environments such as computer systems and electronic devices, multi-task processing has become a common working mode. In this case, multiple tasks often compete for limited target component resources such as processing models and hardware devices.

[0003] Taking the processing model as an example, the management platform that currently docks with the processing model or device resources can only respond to one processing task request at a time; when multiple task requests are in use, it usually only calls the processing model in the order of the task requests, without fully considering the diverse task requirements of users, resulting in poor user experience and easy waste of resources and reduction of overall efficiency. Summary of the Invention

[0004] An embodiment of this application provides a processing method, including: in response to obtaining a target processing task, obtaining component information of a target component required to execute the target processing task;

[0005] Generating a corresponding feedback result based on the component information, so that a target object executes a target response operation matching the feedback result;

[0006] Wherein, there is a target association relationship between the target component and the target object.

[0007] Optionally, obtaining component information of a target component required to execute the target processing task includes:

[0008] Determining the target component from at least one of the processing models deployed in the electronic device, the configured hardware components, and the configured software components based on the task information of the target processing task;

[0009] Obtaining at least one of the component category information, component usage information, and component configuration information of the target component.

[0010] Optionally, determining the target component from the processing models deployed in the electronic device based on the task information of the target processing task includes at least one of the following:

[0011] Obtaining the task content of the target processing task, and determining the processing model that can execute the task content in the processing models deployed in the electronic device as the target component;

[0012] Obtain the model identification information carried by the target processing task, and determine the processing model that matches the model identification information among the processing models deployed on the electronic device as the target component;

[0013] Obtain the task requirements of the target processing task, and determine the processing model that can meet the task requirements among the processing models deployed on the electronic device as the target component;

[0014] Obtain the task intention of the target processing task, and determine the processing model that matches the task intention among the processing models deployed on the electronic device as the target component;

[0015] Obtain the task source information of the target processing task, and determine the processing model that matches the task source information among the processing models deployed on the electronic device as the target component.

[0016] Optionally, determine the target component from at least one of the processing models, configured hardware components, and configured software components deployed on the electronic device based on the task information of the target processing task, including at least one of the following:

[0017] Obtain the user profile information of the target user who triggers the target processing task, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the user profile information;

[0018] Obtain the big data information corresponding to the target processing task, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the big data information;

[0019] Obtain the target evaluation information, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the evaluation information, where the target evaluation information is the historical evaluation information for at least one of the processing model, hardware component, and software component;

[0020] Obtain the spatial environment information where the electronic device is located, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the spatial environment information;

[0021] Obtain the task content of the target processing task, and determine the component that can execute the task content among the hardware components and / or software components configured on the electronic device as the target component;

[0022] Obtain the component identification information carried by the target processing task, and determine the component that matches the component identification information among the hardware components and / or software components configured on the electronic device as the target component;

[0023] Obtain the task requirements of the target processing task, and determine the components in the hardware components and / or software components configured by the electronic device that can meet the task requirements as the target components;

[0024] Obtain the task intention of the target processing task, and determine the components in the hardware components and / or software components configured by the electronic device that match the task intention as the target components;

[0025] Obtain the task source information of the target processing task, and determine the hardware components in the hardware components configured by the electronic device that match the task source information as the target components.

[0026] Optionally, generate a corresponding feedback result based on the component information, including at least one of the following:

[0027] When the component information indicates that the target component of the electronic device is in an idle state, generate a task execution instruction to call the target component to execute the target processing task;

[0028] When the component information indicates that the target component of the electronic device is in an occupied state, generate target reference data corresponding to the component category information and / or component usage information of the target component, so that the electronic device itself or the target application running on the electronic device performs corresponding response operations at least based on the target reference data.

[0029] Optionally, generate target reference data corresponding to the component category information and / or component usage information of the target component, including at least one of the following:

[0030] When the target component is a first processing model of the first category, calculate the waiting duration required to execute the target processing task based on the task queue information and the first model configuration information of the first processing model;

[0031] When the target component is a second processing model of the second category, calculate the waiting duration required to execute the target processing task based on the second model configuration information of the second processing model and the current task execution data;

[0032] When the target component is a third processing model of the third category, calculate the waiting duration required to execute the target processing task based on the usage duration and historical operation data of the third processing model;

[0033] When the target component is a first hardware component of the fourth category, determine the waiting duration required to execute the target processing task based on the usage plan information of the first hardware component and / or the schedule data of the electronic device;

[0034] When the target component is the first software component of the fifth category, determine the waiting duration required to execute the target processing task based on the usage plan information of the first software component.

[0035] Optionally, to enable the electronic device itself or the target application running on the electronic device to perform corresponding response operations based at least on the target reference data, including at least one of the following:

[0036] Feed back the target reference data to the target application, so that the target application performs a waiting operation or a non-waiting operation based on its configured waiting policy and the target reference data;

[0037] The electronic device itself generates a computing power sharing instruction based on the target reference data, and uses the computing power sharing instruction to request the first processing device to execute the target processing task. At least one artificial intelligence model is deployed in the first processing device.

[0038] Optionally, enable the target object to perform a target response operation that matches the feedback result, including at least one of the following:

[0039] Obtain the response operation data of the target user for the feedback result, and configure the target application running on the electronic device to perform a waiting operation or cancel the operation of calling the target component based on the response operation data;

[0040] When the waiting duration characterized by the feedback result matches the waiting policy configured by the target application, configure the target application to perform a waiting operation.

[0041] An embodiment of the present application further provides a processing method, which is applied to an electronic device. The method includes:

[0042] In response to obtaining a target processing task, obtain the model information of the target processing model, where the target processing model is an artificial intelligence model that needs to be called to execute the target processing task;

[0043] Generate target prompt information based on the model information, so that the target application that triggers the target processing task performs a target response operation based at least on the target prompt information.

[0044] An embodiment of the present application further provides an electronic device, including at least one processor and at least one processing model that can run on the processor. The processing model can be called by the target application to perform at least one of the following:

[0045] In response to obtaining a target processing task, obtain the model information of the target processing model, where the target processing model is an artificial intelligence model that needs to be called to execute the target processing task;

[0046] Generate target prompt information based on the model information, so that the target application performs a target response operation at least based on the target prompt information. Description of the Drawings

[0047] Figure 1 It is a flowchart of a processing method according to an embodiment of the present application;

[0048] Figure 2 For an embodiment of the present application Figure 1 It is a flowchart of step S100 in

[0049] Figure 3 It is a flowchart of another processing method according to an embodiment of the present application. Detailed Description of the Invention

[0050] Various solutions and features of the present application are described herein with reference to the drawings.

[0051] It should be understood that various modifications can be made to the embodiments applied herein. Therefore, the above description should not be regarded as a limitation, but only as an example of the embodiments. Those skilled in the art will think of other modifications within the scope and spirit of the present application.

[0052] The drawings included in the specification and forming a part of the specification illustrate embodiments of the present application, and together with the general description of the present application given above and the detailed description of the embodiments given below are used to explain the principles of the present application.

[0053] These and other features of the present application will become apparent from the following description of the preferred forms of the embodiments given by way of non-limiting example with reference to the drawings.

[0054] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art can surely implement many other equivalent forms of the present application.

[0055] When combined with the drawings, the above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description.

[0056] Hereinafter, specific embodiments of the present application are described with reference to the drawings; however, it should be understood that the embodiments applied are only examples of the present application, and it can be implemented in many ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details from obscuring the present application. Therefore, the specific structural and functional details applied herein are not intended to be limiting, but only as a basis for the claims and a representative basis for teaching those skilled in the art to use the present application in substantially any suitable detailed structure in a variety of ways.

[0057] This specification may use phrases such as "in one embodiment", "in another embodiment", "in yet another embodiment", or "in other embodiments", all of which may refer to one or more of the same or different embodiments according to the present application.

[0058] A processing method according to an embodiment of the present application can effectively coordinate the relationship between each target processing task and the target component by obtaining the component information of the target component required to execute the target processing task and generating a corresponding feedback result based on this information, avoiding resource idleness, and improving resource utilization efficiency and task processing efficiency. For example, when task A is using an LLM model to execute a text generation task and task B requests to execute an image generation task, the processing method provided by the present application can coordinate the relationship between task A and task B and multiple models or give personalized prompt information, avoiding resource conflicts and idleness, and improving the usage efficiency of the processing model by the electronic device or the response efficiency of the processing task.

[0059] The processing method of the present application will be described in detail below with reference to the accompanying drawings. Figure 1 is a flowchart of the processing method according to an embodiment of the present application, as Figure 1 shown, the method includes the following steps:

[0060] S100. In response to obtaining a target processing task, obtain the component information of the target component required to execute the target processing task.

[0061] Among them, the target processing task may be a task of generating specific data, a task that needs to call specific hardware, or a task that needs to use specific software or drivers, etc. Specifically, the task of generating specific data may be a text generation task, an image generation task, a video generation task, or an intelligent question answering task, etc. The task that needs to call specific hardware may be a task that needs to call a microphone, a camera, a shared processor, or a speaker, such as a video conferencing task, a video live broadcast task, etc. The task that needs to use specific software or drivers may be a task involving software calls. For example, video editing and synthesis are achieved by calling video editing software, or a rendering task needs to call the graphics card driver and graphics card resources.

[0062] The target component may be a processing model, a hardware component, or a software program, etc. Among them, the processing model may be an OCR (Optical Character Recognition) model component, a PO (Prompt Optimize) model component, an ASR model component, an Image model component, an LLM model, a Sora model (text-to-video model), or a GPT model, etc. The hardware component may be a camera, a microphone, a display screen, a speaker, a keyboard, a mouse, or a processor, etc. The software program may be a software application or a driver program, etc.

[0063] The component information of the target component mainly includes at least one of component category information, component usage information, and component configuration information. Among them, the component category information can be the category of the model to be called, the category of the hardware called, or the category of the software program called. The component usage information can be the usage status of the component. For example, the idle state or the busy state; it can also be the usage plan or usage duration of the component. The usage plan of the component can be determined by judging whether there is a task queue to be executed. The usage duration of the component can be used to evaluate the heat dissipation or heat generation of the component. The component configuration information can be hardware configuration information (such as model, manufacturer, etc.) and function configuration information such as cameras, processors, speakers, etc. The function configuration information includes the functions that the component can achieve.

[0064] Exemplarily, the target processing task is a text generation task, and the target component required to execute the text generation task is an LLM model. When the text generation task instruction is obtained, the process of obtaining the model information of the LLM model is triggered.

[0065] S200. Generate a corresponding feedback result based on the component information, so that the target object performs a target response operation matching the feedback result. Among them, there is a target association relationship between the target component and the target object.

[0066] Specifically, if it is determined according to the usage information of the target component that the target component is in an idle state, and it is determined according to the function configuration information of the target component that the target component can process the target processing task, a task execution instruction is generated.

[0067] If it is determined according to the usage information of the target component that the target component is in a busy state and can wait, a feedback prompt of the waiting time is generated; or a result that needs to wait and recommended content that can be executed during the waiting period is generated, such as some intelligent chat services can be recommended, a knowledge point can be recommended, a small game can be recommended, a painting that can be browsed or appreciated can be recommended, etc. Further, corresponding recommendations can be made based on the duration to wait.

[0068] If it is determined according to the usage information of the target component that the target component is in a busy state and cannot wait (the waiting time is too long and exceeds the waiting duration threshold), it indicates that the target component is processing a large number of tasks or the resources have been fully occupied, and in this busy state, it is not suitable for the target object to continue waiting for the target component to complete the current task. Then, different operations are performed according to the configuration strategy of the target object.

[0069] The target object can be a target processing device or a target application. Among them, the target processing device can be a smartphone, a computer, etc., and the target application can specifically be a software application running on the processing device. According to the generated feedback result, the target object performs corresponding operations. For example, in a software system, after the automatic scheduling module of the system receives the feedback result of resource adjustment, it reallocates system resources.

[0070] The target response operation can be to directly call the target component or run the target component to execute a processing task. The target response operation can also be to wait for the target component to be idle or relay the task to other devices based on different processing strategies, or adjust the task execution priority, progress, etc.

[0071] The association relationship between the target component and the target object varies according to different feedback results.

[0072] Exemplarily, assume that the target object is a smartphone and the target component is an ASR model component, which can be integrated into the smartphone. Under this association relationship, the ASR model component is used to convert the user's voice instructions into text information so that the mobile operating system and applications can understand and execute corresponding operations. For example, the user says "Open the music player" by voice. The ASR model component recognizes and understands the voice and converts it into an instruction that the operating system can recognize, and then instructs the mobile operating system to find and open the corresponding music application.

[0073] Exemplarily, assume that the target object is a picture generation application creator zone, and the target component is an Image model component (for example, a Stable Diffusion model component), which runs on an electronic device equipped with a runtime. The StableDiffusion model component is used to generate corresponding images according to the input text prompt. Since there are multiple users' image generation tasks queuing up for processing, the current Stable Diffusion model component is in a busy state. When the user inputs a text prompt through the picture generation application creator zone to generate a landscape image, the system detects that the StableDiffusion model component is in a busy state, so it generates a feedback result to prompt the user to wait. After receiving the feedback result, the picture generation application creator zone performs a waiting operation. During the waiting process, the picture generation application creator zone can display some recommended excellent landscape paintings already generated by other users to the user to enhance the user experience and reduce the boredom of waiting.

[0074] When the Stable Diffusion model component has finished processing the current task and is in an idle state, if the user inputs a text prompt through the image generation application creator zone to generate a science fiction-style image at this time, the system detects that the StableDiffusion model component is in an idle state and generates a feedback result to prompt the user that there is no need to wait. The StableDiffusion model component starts generating a science fiction-style image according to the text prompt input by the user and displays the generated image in the image generation application creator zone for the user to view and use.

[0075] In one embodiment, in the above step S100, as Figure 2 shown, obtaining the component information of the target component required to execute the target processing task specifically includes:

[0076] S110. Based on the task information of the target processing task, determine the target component from at least one of the processing models deployed in the electronic device, the hardware components configured, and the software components configured.

[0077] Among them, the task information of the target processing task includes task content, component identification information carried by the task, task requirements, task source information, etc. The task requirements can be task priority, task urgency, or specification parameters of the task result, etc. The task source information can indicate which application the task comes from, etc.

[0078] Exemplarily, assume that the task information of the target processing task is: converting a 10-minute voice file into text and requiring a relatively high recognition accuracy and the ability to recognize multiple languages. The "Multilingual ASR Model" (multilingual automatic speech recognition model) is deployed in the electronic device. It has been trained with a large amount of multilingual data, can meet the requirements of multilingual recognition, and also has good performance when processing long voices, meeting the requirements of the task for recognition ability and speed. The hardware component configured in the electronic device is a high-performance CPU, which can run the MultilingualASR Model and ensure the processing speed. The software component configured in the electronic device is an office software for converting speech to text, which is used to receive and display the recognition results. According to the above task information, the Multilingual ASR Model, the CPU, and the speech-to-text office software can be determined as the target components.

[0079] S120. Obtain at least one of the component category information, component usage information, and component configuration information of the target component.

[0080] For different types of target components, component information of the target component can be obtained according to different methods or strategies. For example, if the target component is a processing model, the model information of each model can be directly obtained based on the channels between each model and the runtime (runtime environment), or the model information can be obtained based on the model information stored in the runtime. If the target component is a hardware component, the information of each hardware can be obtained based on the monitoring software, BIOS, or the drivers of each hardware. If the target component is a software program, the information of the software program can be obtained from the properties, log files, and setting options of the software.

[0081] In one embodiment, in the above step S110, based on the task information of the target processing task, determining the target component from the processing models deployed on the electronic device includes at least one of the following:

[0082] Obtain the task content of the target processing task, and determine the processing model that can execute the task content among the processing models deployed on the electronic device as the target component. Wherein, the task content of the target processing task can be image generation, text generation, game hosting, video generation, interactive question answering, video / image processing, etc.

[0083] Exemplarily, the user inputs a description "An ancient castle surrounded by pink cherry blossoms with snowflakes floating in the sky" in an image creation application and requests to generate an image in the corresponding style. The electronic device deploys multiple processing models, for example, the ImageNet model (mainly used for image classification), the Stable Diffusion model (text-to-image model), and the GPT model. Since the target processing task is to generate an image according to the text description, the Stable Diffusion model performs well in generating complex-scene images and can well understand elements such as the castle, cherry blossoms, and snowflakes in the input text and generate corresponding images. Therefore, the Stable Diffusion model is determined as the target component.

[0084] Obtain the model identification information carried by the target processing task, and determine the processing model that matches the model identification information among the processing models deployed on the electronic device as the target component. Wherein, the model identification information can be the model name, descriptive information that can characterize the uniqueness of the model, etc. Specifically, the descriptive information that can characterize the uniqueness of the model can be the version number of the model.

[0085] Exemplarily, if the target processing task input by the user in an image recognition application carries the version number ResNet50 v2.0 of the model, the convolutional neural network model ResNet50 deployed on the electronic device can be determined as the target component according to this version number.

[0086] The task requirement of the target processing task is obtained, and a processing model that can meet the task requirement among the processing models deployed by the electronic device is determined as the target component. The task requirement of the target processing task may be a task priority, a task urgency, or a specification parameter of a task result.

[0087] For example, assume that a user submits two tasks in an image editing software at the same time. Task A1 is to make a simple color adjustment to an ordinary personal photo, and the task priority is low; Task B1 is to perform high-precision image restoration and beautification for an important commercial advertising poster, and the task priority is high. Two image editing-related processing models are deployed in the electronic device. Model M1 is a basic image adjustment model with a fast processing speed but relatively simple functions; Model M2 is an advanced image restoration and enhancement model with good processing effects but slow speed. For Task A, since it has a low priority and is only a simple color adjustment, Model M1 is sufficient to meet the requirements and can be processed quickly, so Model M1 can be determined as the target component of Task A. For Task B, since the task priority is high and the requirements for image restoration and beautification are high, although Model M2 has a slow processing speed, it can meet the high-quality task requirements, so Model M2 can be determined as the target component of Task B.

[0088] The task intent of the target processing task is obtained, and the processing model that matches the task intent among the processing models deployed by the electronic device is determined as the target component. The task intent of the target processing task can be understood as the result to be achieved by the target processing task.

[0089] For example, in a natural language processing application, the target processing task is to translate a piece of text, and the task intent is to convert the text from one language to another. The user's input can come from a specific application, such as a specialized image editing application, a translation application, etc.; it can also come from any other application, as long as the application can receive user input and trigger the target processing task. The task intent is obtained by analyzing the content of the user's input, which can be characters or voice. The system processes and analyzes the characters or voice input by the user and converts them into a form that the computer can understand.

[0090] The task source information of the target processing task is obtained, and the processing model matched with the task source information in the processing model deployed by the electronic device is determined as the target component. The task source information can be the application information that initiates the call request or the device information that generates the task.

[0091] Exemplarily, assume that the user has installed two different image applications on the mobile phone. Application E is a professional post - production photo retouching software, mainly used for fine color adjustment, defect repair, etc. of the captured photos; Application F is a simple image - sharing social software, and users mainly perform basic operations such as adding filters and cropping on it. At this time, the user initiates a processing task of deep noise reduction and color restoration for a RAW - format photo through Application E. There are multiple image - related processing models deployed in the mobile device. Model X1 is an advanced model for professional photo post - processing, capable of performing complex noise reduction, color correction, etc. on RAW - format photos; Model X2 is a lightweight image optimization model, mainly used for quickly adding common filters to ordinary photos and performing simple size adjustment. Since the task is initiated by the professional photo post - production software Application E, and the task requirement is to deeply process RAW - format photos, the function of Model X1 matches the professional positioning of Application E and the task requirement. Therefore, Model X1 can be determined as the target component for this task.

[0092] In another embodiment, in the above step S110, based on the task information of the target processing task, determining the target component from at least one of the processing models, configured hardware components, and configured software components deployed in the electronic device includes at least one of the following:

[0093] Obtain the user profile information of the target user who triggers the target processing task, and determine the target component from at least one of the processing models, hardware components, and software components based on the task information and the user profile information. Among them, the user profile information is generated based on the user habits or preferences trained or learned from historical data. For example, by analyzing historical data such as the user's past operation behaviors, usage frequencies, and usage times when using software or hardware, through training or learning by machine - learning algorithms, the user's habits and preferences are obtained. For example, a certain user is more inclined to use a specific function of a certain type of software, or has specific requirements for the performance of a certain hardware. These information constitute the user profile information.

[0094] Exemplarily, if a target processing task is to perform complex image rendering, and at the same time, according to the user profile information, it is known that this user is accustomed to using a high - performance graphics card and a specific image - processing software, then when determining the target component, select a graphics card with strong performance as the hardware component, select the image - processing software that the user is accustomed to using as the software component, and at the same time select a processing model suitable for this image - rendering task. That is to say, by combining the task information of the task itself and the user's habits and preferences, select the most suitable component for the user to complete the target processing task from the processing models, hardware components, and software components as the target component to improve the task - processing efficiency and user satisfaction.

[0095] Obtain big data information corresponding to the target processing task, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the big data information. That is, the task information can be combined, and with reference to the target component required for the same processing task in the historical big data information, the target component can be determined from at least one of the processing model, hardware component, and software component.

[0096] Exemplarily, assume that the target processing task is to quickly classify large-scale image data. By querying the big data information, it is determined that a convolutional neural network model, a high-performance GPU, and specific image data processing software were used in previous similar tasks. Then, in combination with the task information such as the specific data scale and processing speed requirements of the current target processing task, after comprehensive judgment, the final target component is determined to complete this target processing task. For example, the final target component can be the convolutional neural network model used in the past, a more powerful GPU hardware, and the optimized image data processing software.

[0097] Obtain target evaluation information, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the evaluation information. Among them, the target evaluation information is historical evaluation information for at least one of the processing model, hardware component, and software component. Specifically, the historical evaluation information can be historical scores, praising or complaining textual evaluation data, etc.

[0098] Exemplarily, assume that two text-to-image models Y1 and Y2 are deployed in an electronic device. According to the descriptions input by previous users in the image creation application, the historical score of text-to-image model Y1 is 3 points (out of 5), and the historical score of text-to-image model Y2 is 4 points (out of 5). Then, text-to-image model Y2 can be determined as the target component based on the historical scores.

[0099] Obtain the spatial environment information of the electronic device, and determine the target component from at least one of the processing model, hardware component, and software component based on the task information and the spatial environment information. Among them, the spatial environment information of the electronic device includes the current spatial location and the current privacy and security status, etc.; based on the current spatial location, a high-power model or a low-power model can be selected according to the current environmental temperature and ventilation conditions; based on the current privacy and security status, a background model or a directly interactive hardware or software can be selected.

[0100] Exemplarily, assume that the current task information is to monitor the user's task data. The user is currently running outdoors in a high-temperature environment with general ventilation. Then, according to the current high-temperature and general-ventilation environment, in order to prevent the processor from overheating due to running a high-power model and affecting performance, a low-power but relatively accurate motion data processing model is determined as the target component.

[0101] Obtain the task content of the target processing task, and determine the component in the hardware components and / or software components configured in the electronic device that can execute the task content as the target component. Among them, the target processing task can be document editing, video editing, or data statistical analysis, etc. The task content can include the specific requirements of the task, the type and format of the input data, the expected output result, etc.

[0102] Exemplarily, when the target processing task of the electronic device is to run a large 3D game, the task content includes smoothly displaying the game screen, real-time processing of various interactive operations in the game, etc. The hardware components in the electronic device that can execute this task include a high-performance CPU, a dedicated game graphics card, and a large-capacity memory; the software components include an operating system compatible with the game and the program of the game itself. These determined hardware and software components constitute the target components for the target processing task of running this large 3D game.

[0103] Obtain the component identification information carried by the target processing task, and determine the component in the hardware components and / or software components configured in the electronic device that matches the component identification information as the target component. Among them, the component identification information can be a code, name, or number that can uniquely identify a certain component. For example, if the component identification information carried by the target processing task is the model number of a specific graphics card, and there is a matching graphics card hardware in the electronic device, then this graphics card in the electronic device is determined as the target component; or if the target processing task carries the specific version identification of a certain software application, and there is a matching application in the software components installed in the electronic device, then this application can be determined as the target component.

[0104] Obtain the task requirements of the target processing task, and determine the component in the hardware components and / or software components configured in the electronic device that can meet the task requirements as the target component. Among them, the task requirements can be requirements in terms of computing power, storage capacity, graphics processing ability, data transmission speed, etc. For example, the task requirements of a high-definition video editing task include: a high-performance CPU is required for video encoding and decoding calculations, a large-capacity memory is required to store the video data and temporary files being edited, and a graphics card supporting hardware acceleration is also required to quickly render video effects, etc.; the components that meet these task requirements can be determined as the target components.

[0105] Obtain the task intention of the target processing task, and determine the components in the hardware components and / or software components configured by the electronic device that match the task intention as the target components. Among them, the task intention of the target processing task can be understood as the core purpose to be achieved by the target processing task. For example, when the user operates by opening an image editing application on the electronic device, the target processing task is "editing an image", and the task intention is "adjusting the color of a photo to be more vivid to highlight the main body". The task intention of the target processing task can be obtained based on the analysis of the user's operation behavior, the parameter settings related to the task, the prompt information of the application, etc. Determine the hardware components or software components that can match the task intention as the target components.

[0106] Obtain the task source information of the target processing task, and determine the hardware components in the hardware components configured by the electronic device that match the task source information as the target components. Among them, the task source information can be understood as the information about where this task is initiated, what triggers it, or which external factors it is related to. For example, if the task source information shows that the target processing task is triggered by an inserted USB device, then components such as the USB interface controller in the electronic device and the hardware circuit related to the connection of this USB device match the task source information, and these components can be determined as the target components.

[0107] In one embodiment, in the above step S200, generating a corresponding feedback result based on the component information includes at least one of the following:

[0108] When the component information indicates that the target component of the electronic device is in an idle state, generate a task execution instruction to call the target component to execute the target processing task.

[0109] It should be noted that the component information can be understood as the relevant data or information that can reflect the current working state of the target component in the electronic device. Exemplarily, if the target component is the CPU in a computer, the component information may include the CPU usage rate, whether there is a process being executed, etc. When this information indicates that the target component has no ongoing tasks and is in an idle state, the system generates a task execution instruction, and the role of this instruction is to make the target component execute a specific target processing task. For example, for the target component GPU in a smart phone, if the component information shows that the GPU is currently in an idle state, and there is a target processing task that requires graphic rendering at this time, the system generates a task execution instruction to call the GPU to execute this graphic rendering task.

[0110] When it is determined that the target component of the component information-representing electronic device is in an occupied state, generate target reference data corresponding to the component category information and / or component usage information of the target component, so that the electronic device itself or the target application running on the electronic device performs corresponding response operations based at least on the target reference data.

[0111] It should be noted that the target component of the electronic device being in an occupied state includes the state where the concurrent inference queue is full. In this state, newly submitted inference tasks cannot immediately enter the queue and can only wait outside the queue. The tasks already in the queue will wait to be processed in sequence. The target reference data may include the waiting duration, recommended prompts, or advice information. Among them, the waiting duration can be understood as the estimated time for a new task to wait for the target component to become idle. The recommended prompt can be to recommend using other models, devices, or components to complete the current task. The advice information can be to advise the user to change the input instruction or switch to other more suitable application programs.

[0112] The electronic device or the target application running on the electronic device makes a feedback based on the target reference data. For example, display a waiting prompt according to the waiting duration, or guide the user to use other processing methods according to the recommended prompt.

[0113] In addition, in addition to the target reference data, it is also possible to refer to the user's feedback on these data, the waiting rules pre-configured by the application or device, the processing strategies pre-configured by the application or device, etc. to perform more appropriate response operations. For example, the user's feedback on these data can be whether the user accepts the recommendation, etc. The waiting rules pre-configured by the application or device can be what operation to perform automatically after waiting for a certain period of time. The processing strategies pre-configured by the application or device can be to give priority to processing certain types of tasks, etc.

[0114] Exemplarily, when using a certain image editing application, the application needs to call the graphics processing component of the electronic device to process high-resolution images. However, at this time, the graphics processing component has been occupied by other tasks and the concurrent inference queue is full. The electronic device generates the category information, usage information, and target reference data of the graphics processing component. For example, the waiting duration (it is estimated that it will take 5 minutes to wait), the recommended prompt (recommend using another lightweight image processor on the device to process the current image), and the advice information (advise reducing the image resolution to speed up the processing). Then, the image editing application gives a waiting prompt and related advice based on these target reference data, or performs corresponding operations according to the user's feedback on these prompts (for example, the user selects to use another image processor) to complete the image processing task.

[0115] In an embodiment of the present application, further, generating target reference data corresponding to the component category information and / or component usage information of the target component includes at least one of the following:

[0116] When the target component is a first processing model of the first category, calculate the waiting duration required to execute the target processing task based on the task queue information and the first model configuration information of the first processing model.

[0117] Exemplarily, the first processing model of the first category may be an OCR component, a PO component, or an ASR model component with quantifiable duration. The task queue information of the first processing model may be the number of tasks, and the first model configuration information may be the model processing ability or inference speed, etc. Specifically, the waiting duration required to execute the target processing task can be calculated according to the current number of tasks in the task queue and the working duration of the first processing model for processing a single task.

[0118] When the target component is a second processing model of the second category, calculate the waiting duration required to execute the target processing task based on the second model configuration information of the second processing model and the current task execution data.

[0119] Exemplarily, the second processing model of the second category may be an Image component with runtime statistics of duration. The process of calculating the waiting duration required to execute the target processing task is as follows:

[0120] Based on the time used for each previously generated image by the Image component and the number of generated images, dynamically calculate the average time used by the Image component to generate each previous image. For example, the Image component previously generated an image every 1 ms or 2 ms on average.

[0121] Calculate the number of remaining images to be generated according to the total number of images to be generated and the number of generated images. For example, if 8 images need to be generated and 3 images have already been generated, then 5 more images need to be generated.

[0122] According to the average time used by the Image component to generate each previous image and the number of remaining images to be generated, calculate the time required to generate the remaining images, and use it as the waiting duration required for the execution of the target processing task. For example, if the Image component previously generated an image every 1 ms on average and 5 more images need to be generated, then it will take 5 ms to generate the remaining 5 images, that is, the target processing task needs to wait for 5 ms to execute.

[0123] When the target component is a third processing model of the third category, calculate the waiting duration required to execute the target processing task based on the usage duration and historical operation data of the third processing model.

[0124] Exemplarily, the third processing model can be an LLM model with non - quantifiable duration, and the usage duration of the third processing model can be the working duration of the text generation task in the inference queue. The historical operation data can be the average time consumption or text generation speed of the text generation task, etc. The process of calculating the waiting duration required to execute the target processing task is as follows:

[0125] Determine the number of current tasks in the task queue of the LLM model.

[0126] Based on the time used for the previously executed text generation tasks by the large - language model and the number of executed text generation tasks, dynamically calculate the average text generation duration of the large - language model.

[0127] If the number of current tasks is greater than or equal to the maximum parallel inference number of the preset large - language model, predict the waiting duration required for the execution of the first task according to the working duration and average text generation duration of the large - language model.

[0128] In the case where the target component is the first hardware component of the fourth category, based on the usage plan information of the first hardware component and / or the schedule data of the electronic device, determine the waiting duration required to execute the target processing task.

[0129] Exemplarily, the first hardware component of the fourth category can be a camera or a microphone. The target processing task is a video - conferencing task that needs to call the camera or microphone, and the schedule data of the electronic device is the meeting schedule information established in the electronic device. Then, the waiting duration required for the video - conferencing task to call the camera or microphone can be determined according to the meeting schedule information.

[0130] In the case where the target component is the first software component of the fifth category, based on the usage plan information of the first software component, determine the waiting duration required to execute the target processing task. Among them, the usage plan information can be schedule information.

[0131] Exemplarily, the first software component of the fifth category is a certain video - conferencing software. The schedule information of this video - conferencing software shows that it needs to be used for online training from 3:00 pm to 5:00 pm every Friday, and other irrelevant personnel are not allowed to use the software for meeting operations. If there is a department meeting as the target processing task that needs to use this video - conferencing software at 3:15 pm on a certain Friday, then according to the schedule information of this video - conferencing software, the waiting duration required for the department meeting to use this video - conferencing software is 1 hour and 45 minutes.

[0132] In the embodiments of the present application, further, to enable the electronic device itself or the target application running on the electronic device to perform corresponding response operations at least based on the target reference data, including at least one of the following:

[0133] Feedback the target reference data to the target application, so that the target application performs a waiting operation or a non-waiting operation based on the configured waiting policy and the target reference data. Among them, the target reference data may be a waiting duration, and the waiting policy configured by the target application may be to configure a threshold value of the waiting duration. Of course, the target application may also configure the waiting policy in combination with the task urgency or priority level. For example, for a task with a high urgency level, even if the waiting duration has not reached the threshold, it may not wait and execute directly; while for a task with a low priority level, even if the waiting duration has expired, it may continue to wait in order to give priority to processing other high-priority tasks.

[0134] The electronic device itself (for example, the Runtime running in the electronic device) generates a computing power sharing instruction based on the target reference data, and uses the computing power sharing instruction to request the first processing device to execute the target processing task. Among them, the first processing device is a device connected to the electronic device by a wired or wireless method, and at least one artificial intelligence model is deployed in the first processing device. That is to say, the electronic device uses its own runtime environment to generate a computing power sharing instruction based on the target reference data, and requests another device with an artificial intelligence model connected to the electronic device to execute the target processing task, which reflects a mechanism of computing power sharing and collaborative processing.

[0135] In one embodiment, in the above step S200, it further includes notifying the target application to call the processing model to execute the processing task after the queued task waiting to be processed expires, so that the target object performs a target response operation matching the feedback result, including at least one of the following:

[0136] Obtain the response operation data of the target user for the feedback result, configure the target application running on the electronic device, and perform a waiting operation or cancel the operation of calling the target component based on the response operation data;

[0137] In the case where the waiting duration characterized by the feedback result matches the waiting policy configured by the target application, configure the target application to perform a waiting operation.

[0138] Exemplarily, the target application may be a client application such as Lenovo Xiaotian, AI now, Creator zone, etc. When the queued task waiting to be processed expires (for example, reaches the set start processing time), the system will notify the target application to call the pre-set processing model to execute the corresponding processing task. After the processing task is executed, a feedback result is generated, and the target object performs a target response operation matching the feedback result according to this feedback result.

[0139] Specifically, the response operation data of the target user for the feedback result can be displayed and output within the session window (e.g., chat window), expansion window (e.g., an additional pop-up window), or prompt window (e.g., a prompt window on the screen) of the target application. After seeing the displayed feedback results, the target user can choose to agree or disagree. The target application running on the electronic device performs different operations based on the user's response operation data. When performing the operation of canceling the call to the target component, the target application realizes the cancel call operation by sending a cancel message to the runtime (runtime environment).

[0140] In another embodiment, as Figure 3 shown, the present application also provides a processing method, which is applied to an electronic device, and the method includes:

[0141] S1000. In response to obtaining a target processing task, obtain the model information of the target processing model, where the target processing model is an artificial intelligence model that needs to be called to execute the target processing task;

[0142] S2000. Generate target prompt information based on the model information, so that the target application that triggers the target processing task performs a target response operation at least based on the target prompt information.

[0143] Among them, the target processing task can be a task of generating specific data, a task that needs to call specific hardware, or a task that needs to use specific software or drivers, etc. Specifically, the task of generating specific data can be a text generation task, an image generation task, a video generation task, or an intelligent question and answer task, etc. The task that needs to call specific hardware can be a task that needs to call a microphone, a camera, share a processor or a speaker, such as a video conferencing task, a video live broadcast task, etc. The task that needs to use specific software or drivers can be a task involving software calls. For example, video editing and synthesis are achieved by calling video editing software, or a rendering task needs to call a graphics card driver and graphics card resources.

[0144] The target processing model can be an OCR model component, a PO model component, an ASR model component, an Image model component, an LLM model, a Sora model, or a GPT model, etc.

[0145] The target response operation can be to directly call the target component or run the target component to execute the processing task. The target response operation can also be to wait for the target component to be idle or relay the task to other devices based on different processing strategies, or adjust the task execution priority, progress, etc.

[0146] Exemplarily, the target processing task is a text generation task, and the target component required to execute the text generation task is an LLM model. When the text generation task instruction is obtained, the process of obtaining the model information of the LLM model is triggered. The target prompt information is generated based on the model information of the LLM model. For example, the target prompt information is to wait for 2 minutes. The text generation application that triggers the text generation task can select to wait for the LLM model to be idle according to the waiting duration.

[0147] The embodiment of the present application also provides an electronic device, including at least one processor and at least one processing model that can run on the processor. The processing model can be called by a target application to execute at least one of the following:

[0148] In response to obtaining a target processing task, obtain the model information of the target processing model, where the target processing model is an artificial intelligence model that needs to be called to execute the target processing task;

[0149] Generate target prompt information based on the model information, so that the target application performs a target response operation at least based on the target prompt information.

[0150] Exemplarily, the target processing task is a text generation task, and the target component required to execute the text generation task is an LLM model. When the text generation task instruction is obtained, the process of obtaining the model information of the LLM model is triggered. The target prompt information is generated based on the model information of the LLM model. For example, the target prompt information is that there is no need to wait. The text generation application that triggers the text generation task can select to directly call the LLM model or run the LLM model to execute the text generation task according to the prompt information.

[0151] The above embodiments are only exemplary embodiments of the present application and are not used to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements within the essence and protection scope of the present application, and such modifications or equivalent replacements should also be regarded as falling within the protection scope of the present application.

Claims

1. A processing method comprising: In response to obtaining a target processing task, obtaining component information of a target component required to execute the target processing task; Generate a corresponding feedback result based on the component information, so that the target object performs a target response operation matching the feedback result; There is a target association relationship between the target component and the target object.

2. The method according to claim 1, wherein obtaining component information of a target component required to perform the target processing task comprises: Determine the target component from at least one of a processing model deployed by the electronic device, a configured hardware component, and a configured software component based on the task information of the target processing task; At least one of component category information, component usage information, and component configuration information of the target component is obtained.

3. The method according to claim 2, wherein: Determining the target component from a processing model deployed by the electronic device based on the task information of the target processing task includes at least one of the following: Obtaining the task content of the target processing task, and determining a processing model that can execute the task content among the processing models deployed by the electronic device as the target component; Obtaining model identification information carried by the target processing task, and determining a processing model that matches the model identification information among processing models deployed by the electronic device as the target component; Obtaining a task requirement of the target processing task, and determining a processing model that can meet the task requirement among the processing models deployed by the electronic device as the target component; Obtaining a task intent of the target processing task, and determining a processing model that matches the task intent among processing models deployed by the electronic device as the target component; The task source information of the target processing task is obtained, and a processing model that matches the task source information among the processing models deployed by the electronic device is determined as the target component.

4. The method according to claim 2, wherein: Determining the target component from at least one of a processing model deployed by the electronic device, a configured hardware component, and a configured software component based on the task information of the target processing task includes at least one of the following: Obtaining user portrait information of a target user who triggers the target processing task, and determining the target component from at least one of the processing model, hardware component, and software component based on the task information and the user portrait information; Obtaining big data information corresponding to the target processing task, and determining the target component from at least one of the processing model, hardware component, and software component based on the task information and the big data information; Obtaining target evaluation information, determining the target component from at least one of the process model, the hardware component, and the software component based on the task information and the evaluation information, wherein the target evaluation information is historical evaluation information for at least one of the process model, the hardware component, and the software component; Obtaining spatial environment information of the electronic device, and determining the target component from at least one of the processing model, hardware components, and software components based on the task information and the spatial environment information; Obtaining the task content of the target processing task, and determining a component capable of executing the task content among hardware components and / or software components configured in the electronic device as the target component; Obtaining component identification information carried by the target processing task, and determining a component matching the component identification information among hardware components and / or software components configured in the electronic device as the target component; Obtaining a task requirement of the target processing task, and determining a component that can meet the task requirement among hardware components and / or software components configured in the electronic device as the target component; Obtaining a task intent of the target processing task, and determining a component among hardware components and / or software components configured in the electronic device that matches the task intent as the target component; The task source information of the target processing task is obtained, and the hardware component matching the task source information among the hardware components configured in the electronic device is determined as the target component.

5. The method according to claim 1, wherein: Generating a corresponding feedback result based on the component information includes at least one of the following: In a case where the component information indicates that the target component of the electronic device is in an idle state, generating a task execution instruction that calls the target component to execute the target processing task; When the component information represents that the target component of the electronic device is in an occupied state, target reference data corresponding to the component category information and / or component usage information of the target component is generated, so that the electronic device itself or the target application running on the electronic device performs a corresponding response operation based on at least the target reference data.

6. The method according to claim 5, wherein: Generating target reference data corresponding to component category information and / or component usage information of the target component includes at least one of the following: In the case where the target component is a first processing model of a first category, calculating the waiting time required to execute the target processing task based on the task queue information and the first model configuration information of the first processing model; When the target component is a second processing model of the second category, calculating the waiting time required to execute the target processing task based on the second model configuration information of the second processing model and the current task execution data; In the case where the target component is a third processing model of a third category, calculating the waiting time required to execute the target processing task based on the usage time and historical operation data of the third processing model; In the case where the target component is a first hardware component of the fourth category, determining the waiting time required to execute the target processing task based on the usage plan information of the first hardware component and / or the schedule data of the electronic device; In the case that the target component is a first software component of the fifth category, the waiting time required to execute the target processing task is determined based on the usage plan information of the first software component.

7. The method according to claim 5 or 6, wherein: So that the electronic device itself or a target application running on the electronic device performs a corresponding response operation based on at least the target reference data, including at least one of the following: Feedback the target reference data to the target application, so that the target application performs a waiting operation or a non-waiting operation based on the waiting strategy configured by the target application and the target reference data; The electronic device itself generates a computing power sharing instruction based on the target reference data, and utilizes the computing power sharing instruction to request a first processing device to perform the target processing task, wherein at least one artificial intelligence model is deployed in the first processing device.

8. The method according to claim 1, wherein: The target object is caused to perform a target response operation matching the feedback result, including at least one of the following: Obtaining response operation data of the target user to the feedback result, and configuring the target application running on the electronic device to perform a waiting operation or cancel the operation of calling the target component based on the response operation data; When the waiting time represented by the feedback result matches the waiting strategy configured for the target application, the target application is configured to perform a waiting operation.

9. A processing method, applied to an electronic device, comprising: In response to obtaining the target processing task, obtaining model information of a target processing model, wherein the target processing model is an artificial intelligence model that needs to be called to execute the target processing task; Target prompt information is generated based on the model information, so that the target application that triggers the target processing task performs a target response operation based on at least the target prompt information.

10. An electronic device comprising at least one processor and at least one processing model capable of running on the processor, wherein the processing model can be called by a target application to perform at least one of the following: In response to obtaining the target processing task, obtaining model information of a target processing model, wherein the target processing model is an artificial intelligence model that needs to be called to execute the target processing task; Target prompt information is generated based on the model information, so that the target application performs a target response operation based at least on the target prompt information.

Images