Processing method and electronic equipment

Abstract

The invention discloses a processing method and electronic equipment, and the method comprises the steps: receiving input data through an interactive interface which is constructed based on a target operating system; the target operating system is an operating system supported to run by any processor in the electronic equipment; performing intention recognition on the input data to obtain a target user intention; according to the target user intention, calling at least one processing model to obtain a processing result; wherein the processing models of different function types run based on processors of different processing types in the electronic equipment; and outputting the processing result on the interactive interface.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a processing method and an electronic device. Background Art

[0002] Currently, electronic devices such as laptops are limited by hardware and can usually only run a single AI model. Summary of the invention

[0003] In view of this, the present application provides a processing method and an electronic device as follows:

[0004] A processing method, comprising:

[0005] receiving input data through an interactive interface, wherein the interactive interface is constructed based on a target operating system; the target operating system is an operating system supported and run by any processor in the electronic device;

[0006] Performing intent recognition on the input data to obtain the target user's intent;

[0007] According to the target user intention, calling at least one processing model to obtain a processing result;

[0008] Wherein, the processing models of different functional types are run based on processors of different processing types in the electronic device;

[0009] The processing result is output on the interactive interface.

[0010] The above method preferably calls at least one processing model according to the target user intention to obtain a processing result, including:

[0011] generating at least one calling instruction according to the target user intention; one calling instruction corresponds to one processing model;

[0012] Sending the calling instruction to a corresponding processing model, wherein the processing model outputs an operation result in response to the calling instruction;

[0013] A processing result is obtained according to the operation result.

[0014] The above method, preferably, further comprises:

[0015] According to the calling instruction, obtaining initial data matching the target user's intention from a storage area of ​​the electronic device;

[0016] The initial data is sent to a processing model corresponding to the calling instruction, and the processing model processes the initial data in response to the calling instruction to output an operation result.

[0017] In the above method, preferably, the processor used by the processing model is deployed with a memory component exclusively used by the processor;

[0018] Among them, after the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor, and the model data enables the processor to execute the computing task corresponding to the processing model and output the operation result.

[0019] In the above method, preferably, the at least one processing model comprises: a first model and a second model;

[0020] Wherein, obtaining the processing result according to the operation result includes:

[0021] The operation result output by the first model is sent to the second model, so that the second model processes the operation result output by the first model and the operation result output by the second model to output the processing result.

[0022] Preferably, the above method further comprises, before calling at least one processing model:

[0023] Detecting whether the first processor corresponding to the processing model is executing computing tasks of other models;

[0024] If the first processor is executing computing tasks of other models, the computing tasks executed by the first processor are switched to be executed by the second processor in the electronic device.

[0025] In the above method, preferably, the processor used by the processing model is deployed with a memory component exclusively used by the processor;

[0026] Wherein, if the first processor is executing a computing task of another model, the method further includes:

[0027] The model data in the memory component corresponding to the first processor is migrated to the memory component corresponding to the second processor.

[0028] In the above method, preferably, the electronic device is deployed with a first processor and a second processor, the first processor runs a first operating system, and the second processor runs a second operating system; the second operating system runs based on the first operating system;

[0029] The target operating system is the first operating system, and the first operating system outputs the constructed interactive interface;

[0030] Alternatively, the target operating system is the second operating system, and the interactive interface is transmitted from the second operating system to the first operating system via a transmission interface between the first processor and the second processor, so that the first operating system outputs the interactive interface.

[0031] An electronic device, comprising:

[0032] A plurality of processors, each of which has a different processing type; any processor among the plurality of processors supports running a target operating system; and an interactive interface is constructed based on the target operating system;

[0033] The processor supporting the operation of the target operating system is used to: receive input data through the interactive interface; perform intent recognition on the input data to obtain the target user's intent; call at least one processing model according to the target user's intent to obtain a processing result; and output the processing result on the interactive interface;

[0034] The processing models of different functional types are run based on processors of different processing types in the electronic device.

[0035] The electronic device mentioned above preferably further comprises: a plurality of memory components; each of the memory components corresponds to one of the processors, and the processor exclusively occupies its corresponding memory component;

[0036] Among them, after the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor, and the model data enables the processor to execute the computing task corresponding to the processing model.

[0037] A processing device, comprising:

[0038] A data acquisition unit, configured to receive input data through an interactive interface, wherein the interactive interface is constructed based on a target operating system; the target operating system is an operating system supported and run by any processor in the electronic device;

[0039] An intention recognition unit, used to perform intention recognition on the input data to obtain the target user's intention;

[0040] A model calling unit, used to call at least one processing model according to the target user intention to obtain a processing result;

[0041] Wherein, the processing models of different functional types are run based on processors of different processing types in the electronic device;

[0042] A result output unit is used to output the processing result on the interactive interface.

[0043] A computer-readable storage medium stores a computer program / instruction, which implements any one of the processing methods described above when executed by a processor.

[0044] A computer program product comprises a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the processing method described in any one of the above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flowchart of a processing method provided in an embodiment of the present application;

[0047] Figure 2 This is an example diagram of an electronic device in an embodiment of the present application;

[0048] Figure 3 Another example diagram of an electronic device in an embodiment of the present application;

[0049] Figure 4 A partial flow chart of a processing method provided in an embodiment of the present application;

[0050] Figure 5 This is an example diagram of a poster promotion page generated by using the Wensheng graph model and the Wensheng text model in the embodiment of the present application;

[0051] Figure 6 Another example diagram of generating a poster promotion page by using the Vincent graph model and the Vincent text model in the embodiment of the present application;

[0052] Figure 7 An example diagram of memory particles being deployed on a processor in an electronic device;

[0053] Figure 8 A schematic diagram of the structure of a processing device provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0055] Fig.10 Another structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0056] Fig.11 This is the hardware architecture diagram of the notebook;

[0057] Fig.12 This is an example of an M.2 card with integrated dGPU and memory chips;

[0058] Fig.13 This is the software architecture diagram of the notebook;

[0059] Fig.14 A flowchart for generating a poster page for product A for the user in the notebook;

[0060] Fig.15 An example diagram of a user selecting the function of generating a promotional page on the interactive interface provided by Vincent's AIGC software;

[0061] Fig.16 An example diagram of the user inputting the required content in the input box of the interactive interface to generate a promotional page for product A;

[0062] Fig.17 An example diagram that directly reminds users that they need to provide more information in the interactive interface;

[0063] Fig.18 AIGC software from Weiwenshengwen will analyze the demand and retrieve the corresponding product image sample pictures from the personal knowledge base such as the photo album;

[0064] Fig.19 An example diagram of the dual-model operation in which Wenshengtu's AIGC software generates a promotional background image for product A and Wenshengwen's AIGC software generates copywriting based on user needs;

[0065] Fig. 20 An example image of the AI ​​module that generates posters for Vincent Wen’s AIGC software, which fuses images and text into posters. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] refer to Figure 1, is a flowchart of an implementation of a processing method provided in an embodiment of the present application, which can be applied to electronic devices having multiple processors of different processing types, such as a notebook computer including a central processing unit (CPU), a graphics processing unit (GPU), and a network processor (NPU). For example, Figure 2 As shown in , multiple processors in the electronic device are connected to the motherboard of the electronic device through a first interface. One or more of these processors can be integrated into one chip, such as an iGPU integrated into a CPU. One or more of these processors can be independent processors, such as a dNPU and a dGPU. The first interface can connect a hard disk (such as a solid-state drive SSD) to an interface of the motherboard, such as an M.2 interface, that is, the processor is integrated into the electronic device. For another example, Figure 3 As shown in, among the multiple processors in the electronic device, there are processors connected to the mainboard of the electronic device through a first interface, and there are processors connected to the mainboard of the electronic device through a second interface, and the second interface is an interface that supports hot plugging, such as a universal serial bus USB (Universal Serial Bus) interface, that is, among the multiple processors in the electronic device, one or more processors can be integrated in the electronic device, and these processors can be integrated in one chip, such as iGPU; or these processors can be independent processors, such as dNPU and dGPU; and one or more processors are external processors of the electronic device, such as dNPU. Each processor in the electronic device is identified by the operating system in the electronic device through a driver. The technical solution in this embodiment is mainly used to improve the data security of the call processing model.

[0068] Specifically, the method in this embodiment may include the following steps:

[0069] Step 101: receiving input data through an interactive interface.

[0070] Among them, the interactive interface is constructed based on the target operating system. The interactive interface can be an interactive interface of a target program installed based on the target operating system. For example, the target program can be a voice assistant, which can provide an interactive interface. After the voice assistant is started, the target operating system can build an interactive interface and output it. For another example, the target program can be an artificial intelligence AI (Artificial Intelligence) model that can implement Vincent van Gogh, that is, a Vincent van Gogh model. The Vincent van Gogh model can provide an interactive interface. After the Vincent van Gogh model is started, the target operating system can build an interactive interface and output it.

[0071] The target operating system may be an operating system supported by any processor in the electronic device. In a specific implementation, the electronic device is deployed with a first processor and a second processor, and of course a third processor. The first processor may be a CPU, and a first operating system, such as a Windows operating system, is running on the first processor.

[0072] Based on this, the target operating system can be the first operating system, and the first operating system constructs an interactive interface and outputs the constructed interactive interface. For example, a laptop runs a Windows operating system based on a CPU. After the voice assistant installed in the Windows operating system is started, an interactive interface is constructed in the Windows operating system and output. As a result, users can input data such as voice or text on the interactive interface.

[0073] The second processor may be an NPU, and a second operating system such as a Linux operating system may be running on the second processor (in other embodiments, the second operating system may not be loaded on the second processor). In other words, the second processor is a system on a chip (SOC).

[0074] Based on this, the target operating system may also be a second operating system, and the second operating system runs based on the first operating system. In other words, the second operating system is a subsystem of the first operating system. The interactive interface is transmitted from the second operating system to the first operating system through a transmission interface between the first processor and the second processor, such as a high-speed serial computer expansion bus PCIe (peripheral component interconnect express) interface, so that the first operating system outputs the interactive interface.

[0075] For example, a laptop runs a Windows operating system based on the CPU, while the laptop's external NPU (such as Figure 3 The Linux operating system is running on the external processor shown in the figure. A voice assistant is installed in the Linux operating system. The application icon of the voice assistant is displayed on the desktop of the Windows operating system. The user can click the application icon of the voice assistant on the desktop of the Windows operating system to start the voice assistant. The generated startup instruction is transmitted from the Windows operating system to the Linux operating system through the PCIe interface between the CPU and the NPU. The voice assistant in the Linux operating system starts in response to the startup instruction. The Linux operating system builds the interactive interface of the voice assistant and transmits the interactive interface to the Windows operating system through the PCIe interface, and the Windows operating system outputs the interactive interface. As a result, the user can input data such as voice or text on the interactive interface.

[0076] Step 102: Perform intent recognition on the input data to obtain the target user's intent.

[0077] In one implementation, in this embodiment, the target program providing the interactive interface may be used to perform intent recognition on the input data to obtain the target user's intent;

[0078] In another implementation, in this embodiment, an intent recognition model may be called to perform intent recognition on input data to obtain the target user's intent. The intent recognition model may be an AI model installed in the operating system of the electronic device, or the intent recognition model may be an AI model deployed in the cloud.

[0079] For example, in this embodiment, the Wenshengwen model can be used to identify the intent of the input data "give me a poster promotional page of product A" to obtain the target user's intention "draw a poster promotional image of product A".

[0080] Step 103: According to the target user's intention, call at least one processing model to obtain a processing result.

[0081] In a specific implementation, in step 103, the processing model to be used can be determined by analyzing the target user's intention, such as a text-to-text model, a text-to-image model, or a text-to-video model.

[0082] Among them, processing models of different functional types are run based on processors of different processing types in electronic devices. For example, the Vincent text model is run based on the NPU, and the Vincent image model or Vincent video model is run based on the GPU. In this way, the computing power of processors of different processing types can be fully utilized to perform computing tasks for the running processing models.

[0083] For example, according to the target user's intention of "drawing a poster promotional image of product A", the Vincent-Graph model and the Vincent-Text model are called, wherein the Vincent-Text model runs on the NPU of the notebook and the Vincent-Graph model runs on the GPU of the notebook. Thus, by calling the Vincent-Text model and the Vincent-Graph model, a poster promotional page of product A containing promotional text is obtained.

[0084] Step 104: Output the processing result on the interactive interface.

[0085] For example, after the user inputs "Give me a poster promotional page of product A" in the interactive interface, through this embodiment, a poster of product A containing promotional text is output in the interactive interface. In this process, the user does not need to start each processing model separately to obtain the poster promotional page of product A, which can reduce the complexity of user operations and improve user experience.

[0086] It can be seen from the above technical scheme that in a processing method provided by an embodiment of the present application, after receiving the user's input data through an interactive interface constructed by the operating system, the input data is intended to be recognized, and then one or more processing models are called according to the obtained target user intention, thereby outputting the obtained processing results to the user on the interactive interface. Since these processing models run based on processors of different processing types in electronic devices according to their respective functional types, the processors can perform computing tasks for processing models of corresponding functional types due to their respective processing types, and can give full play to the computing performance of the processor, thereby giving full play to the processing performance of the local processor in the electronic device, thereby realizing the call of one or more local processing models on the electronic device, so that multiple processing models can run on the local processor, avoiding the leakage of privacy data when calling the cloud processing model, thereby improving data security.

[0087] In one implementation, step 103 may obtain the processing result in the following manner: Figure 4 As shown in:

[0088] Step 401: Generate at least one calling instruction according to the target user's intention.

[0089] Among them, one call instruction corresponds to one processing model. For example, according to the target user intention of "drawing a poster promotion image of product A", a first call instruction and a second call instruction are generated, the first call instruction corresponds to the Wenshengwen model, and the second call instruction corresponds to the Wenshengtu model.

[0090] Step 402: Send the calling instruction to the corresponding processing model, and the processing model outputs the running result in response to the calling instruction.

[0091] Step 403: Obtain processing results according to the operation results.

[0092] In one implementation, in step 402, each call instruction can be distributed to the corresponding processing model at the same time, so that each processing model outputs the corresponding operation result respectively. Then, in step 403, the operation results output by all processing models are processed to obtain the final processing result.

[0093] For example, a first call instruction is sent to the Vincent text model, and the Vincent text model generates the promotional text in the poster promotion page in response to the first call instruction; a second call instruction is sent to the Vincent image model, and the Vincent image model generates the poster image of product A in the poster promotion page in response to the second call instruction. Then, the promotional text and the poster image of product A are processed to obtain the poster promotion page of product A containing the promotional text.

[0094] In a specific implementation, at least one processing model called includes: a first model and a second model, such as a Wenshengwen model and a Wenshengtu model. Based on this, step 403 can be implemented in the following manner:

[0095] The operation result output by the first model is sent to the second model, so that the second model processes the operation result output by the first model and the operation result output by the second model to obtain an output processing result.

[0096] For example, the text-to-text model sends the output promotional text to the text-to-image model, and the text-to-image model generates corresponding images according to the promotional text, such as Figure 5 As shown in , based on this, in this embodiment, the image corresponding to the promotional text is spliced ​​with the poster image of product A to obtain a poster promotional page of product A containing the promotional text.

[0097] In another implementation, there is a calling order between the processing models corresponding to each calling instruction. In step 402, each calling instruction can be sent to the corresponding processing model according to the calling order, so that the processing model outputs the corresponding operating results in sequence. Afterwards, in step 403, the operating result output by the last called processing model is processed to obtain the final processing result.

[0098] In a specific implementation, at least one processing model called includes: a first model and a second model, such as a Wenshengwen model and a Wenshengtu model. Based on this, step 403 can be implemented in the following manner:

[0099] The operation result output by the first model is sent to the second model, so that the second model processes the operation result output by the first model in response to its corresponding calling instruction to obtain the output operation result; then, the operation result is processed to obtain the processing result.

[0100] For example, in this embodiment, the first call instruction is first sent to the Wenshengwen model, and the Wenshengwen model generates the promotional text in the poster promotion page in response to the first call instruction, such as Figure 6 As shown in , a second calling instruction is sent to the Vincent graph model, and the promotional text output by the Vincent graph model is sent to the Vincent graph model. The Vincent graph model generates a poster image in response to the second calling instruction. The generated poster image includes the promotional text output by the Vincent graph model. Thus, the poster image including the promotional text output by the Vincent graph model is used as a poster promotional page for product A including the promotional text.

[0101] In the specific implementation, the processing model needs to use the initial data that matches the target user's intention when generating the running results. For example, when the Wenshengwen model generates promotional text, it needs information such as product features of product A, and when the Wenshengtu model generates poster images, it needs product appearance images of product A. These product features and product appearance images are all initial data that match the target user's intention "Product A".

[0102] Based on this, in this embodiment, after generating a call instruction, when sending the call instruction to the corresponding processing model (before or after), initial data matching the target user's intention can be obtained from the storage area of ​​the electronic device according to the call instruction, and then the initial data can be sent to the processing model corresponding to the call instruction. Thus, the processing model processes the initial data in response to the call instruction to output the operation result.

[0103] For example, in this embodiment, after the first call instruction and the second call instruction are generated, the product features of product A, such as product name, product model, shell color, size, improvement information relative to old products, etc., are read from the document of product A stored in the notebook according to the first call instruction, and the product features and the first call instruction are sent to the Vincent model together, so that the Vincent model can process the product features such as the improvement information relative to old products in response to the first call instruction to obtain the promotional text; and, according to the second instruction, the product appearance image of product A is read from the photo album of the notebook, and the product appearance image and the second call instruction (which may also have the promotional text) are sent to the Vincent model together, so that the Vincent model can process the product appearance image in response to the second call instruction, such as adding a background image or adjusting the color, etc., to obtain the poster image of product A.

[0104] Based on the above implementation scheme, the processors used by each processing model installed in the electronic device are respectively deployed with memory components exclusively used by the processors. The memory spaces of the memory components deployed by different processors can be the same or different. For example, Figure 7 As shown in the figure, the NPU is deployed with memory particles exclusively used by the NPU, such as 8GB, 16GB or 32GB; the GPU is deployed with memory particles exclusively used by the GPU, such as 6GB, 8GB or 12GB; the memory component deployed with the CPU is a memory stick connected to the motherboard of the notebook, such as 16GB, 32GB or 64GB.

[0105] Based on this, after the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor. The model data enables the processor to execute the computing task corresponding to the processing model and output the operation result.

[0106] For example, the model data of the Wenshengwen model is loaded from the SSD into the memory particles corresponding to the NPU, so that the Wenshengwen model performs the Wenshengwen computing task based on the model data, thereby outputting the running result, i.e., the promotional text;

[0107] For another example, the model data of the Wensheng graph model is loaded from the SSD into the memory particles corresponding to the GPU, so that the Wensheng graph model performs the Wensheng graph calculation task based on the model data, thereby outputting the operation result, that is, the poster image of product A.

[0108] Therefore, each processor accesses data from its own memory particles when executing computing tasks, which can avoid occupying the memory space of the target operating system and reduce the time consumption of data access, thereby improving the operating performance of the processing model.

[0109] In one implementation, in this embodiment, before calling at least one processing model, it can be first detected whether the processor corresponding to the processing model is occupied, that is, whether it is executing computing tasks of other models. If the processor corresponding to the processing model is occupied, the computing tasks being executed by the processor are first transferred to other processors for execution. After the processor corresponding to the processing model is in an idle state, the processing model is called again to make the processing model run based on its corresponding processor, that is, the computing tasks corresponding to the processing model are executed by the processor corresponding to the processing model.

[0110] Specifically, in this embodiment, before calling the processing model, it is detected whether the first processor corresponding to the processing model is executing the computing tasks of other models. If the first processor is executing the computing tasks of other models, the computing tasks executed by the first processor are switched to the second processor in the electronic device for execution. The second processor can be: a processor in the electronic device whose processing type matches the functional type of the other model; or, the second processor can be: any processor in the electronic device that is in an idle state.

[0111] It should be noted that the first processor refers to a processor whose processing type matches the functional type of the processing model. The second processor is a processor different from the first processor.

[0112] Furthermore, since the processor used by each processing model is deployed with an internal component exclusively used by the processor, that is, the first processor corresponding to the processing model has a memory component exclusively used by it. Based on this, if the first processor is executing a computing task of another model, while switching the computing task executed by the first processor to the second processor in the electronic device for execution, in this embodiment, the model data in the memory component corresponding to the first processor can also be migrated to the memory component corresponding to the second processor, thereby, the second processor executes the computing task of another model based on the model data migrated to the internal component corresponding to the second processor.

[0113] For example, when the Vincent model is called, it is detected that the NPU is executing the computing task of the Vincent graph model, that is, the Vincent graph model uses the memory particles corresponding to the NPU to store the model data and runs based on the NPU. At this time, the model data of the Vincent graph model in the memory particles corresponding to the NPU is migrated to the memory particles corresponding to the GPU, and the computing task of the Vincent graph model being executed on the NPU is switched to the GPU for execution, thereby making the NPU idle. At this time, the Vincent model is called again, and thus, the Vincent model uses the memory particles corresponding to the NPU to store the model data and runs based on the NPU. In this way, in this embodiment, when the called processing model is single, the NPU is used first to run the called processing model, and when a processing model that better matches the NPU is called, the processing model running on the NPU is switched to the GPU for operation, and the newly called processing model is run by the NPU. Therefore, when multiple processing models are called, for processing models of different functional types, a processor matching the processing type is used to run.

[0114] Based on the above implementation scheme, after the processing model is started, when the processing model is not in use or in an idle state, the processor corresponding to the processing model does not perform any computing tasks. At this time, the processor can enter a low-power standby mode, thereby reducing the power consumption of the processor.

[0115] refer to Figure 8 , is a schematic diagram of the structure of a processing device provided in an embodiment of the present application, and the processing device can be deployed in an electronic device having multiple processors of different processing types, such as Figure 2 or Figure 3 The electronic device shown in the figure. The technical solution in this embodiment is mainly used to improve the data security of the calling processing model.

[0116] Specifically, the device in this embodiment may include the following units:

[0117] The data acquisition unit 801 is used to receive input data through an interactive interface, where the interactive interface is constructed based on a target operating system; the target operating system is an operating system supported and run by any processor in the electronic device;

[0118] The intention recognition unit 802 is used to perform intention recognition on the input data to obtain the target user's intention;

[0119] A model calling unit 803 is used to call at least one processing model according to the target user intention to obtain a processing result;

[0120] Wherein, the processing models of different functional types are run based on processors of different processing types in the electronic device;

[0121] The result output unit 804 is used to output the processing result on the interactive interface.

[0122] It can be seen from the above technical scheme that in a processing device provided by an embodiment of the present application, after receiving the user's input data through an interactive interface constructed by the operating system, the input data is intended to be recognized, and then one or more processing models are called according to the obtained target user intention, thereby outputting the obtained processing results to the user on the interactive interface. Since these processing models run based on processors of different processing types in electronic devices according to their respective functional types, the processors can perform computing tasks for processing models of corresponding functional types due to their respective processing types, and can give full play to the computing performance of the processor, thereby giving full play to the processing performance of the local processor in the electronic device, thereby realizing the call of one or more local processing models on the electronic device, so that multiple processing models can run on the local processor, avoiding the leakage of privacy data when calling the cloud processing model, thereby improving data security.

[0123] In one implementation, the model calling unit 803 is specifically used to: generate at least one calling instruction according to the target user intention; one calling instruction corresponds to one processing model; send the calling instruction to the corresponding processing model, and the processing model outputs an operation result in response to the calling instruction; and obtain a processing result based on the operation result.

[0124] Among them, the model calling unit 803 is also used to: obtain initial data matching the target user's intention from the storage area of ​​the electronic device according to the calling instruction; send the initial data to the processing model corresponding to the calling instruction, and the processing model processes the initial data in response to the calling instruction to output the operation result.

[0125] Based on the above implementation, the processor used by the processing model is deployed with a memory component exclusively used by the processor; after the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor, and the model data enables the processor to execute the computing task corresponding to the processing model and output the running result.

[0126] In one implementation, the at least one processing model includes: a first model and a second model;

[0127] Among them, when the model calling unit 803 obtains the processing result based on the running result, it is specifically used to: send the running result output by the first model to the second model, so that the second model processes the running result output by the first model and the running result output by the second model to output the processing result.

[0128] In one implementation, before calling at least one processing model, the model calling unit 803 is also used to: detect whether the first processor corresponding to the processing model is executing computing tasks of other models; if the first processor is executing computing tasks of other models, switch the computing tasks executed by the first processor to the second processor in the electronic device for execution.

[0129] If the first processor is executing a computing task of another model, the model calling unit 803 is further used to migrate the model data in the memory component corresponding to the first processor to the memory component corresponding to the second processor.

[0130] In one implementation, the electronic device is deployed with a first processor and a second processor, the first processor runs a first operating system, the second processor runs a second operating system; the second operating system runs based on the first operating system;

[0131] Among them, the target operating system is the first operating system, and the first operating system outputs the constructed interactive interface; or, the target operating system is the second operating system, and the interactive interface is transmitted from the second operating system to the first operating system through the transmission interface between the first processor and the second processor, so that the first operating system outputs the interactive interface.

[0132] refer to Fig. 9 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device may include multiple processors 901, the processing types of the processors are different, such as GPU, NPU, CPU, etc.; any processor of the multiple processors, such as CPU or NPU, supports running a target operating system; an interactive interface is constructed based on the target operating system;

[0133] The processor supporting the operation of the target operating system is used to: receive input data through the interactive interface; perform intent recognition on the input data to obtain the target user's intent; call at least one processing model according to the target user's intent to obtain a processing result; and output the processing result on the interactive interface;

[0134] The processing models of different functional types are run based on processors of different processing types in the electronic device.

[0135] It can be seen from the above technical scheme that in an electronic device provided by an embodiment of the present application, after receiving the user's input data through an interactive interface constructed by the operating system, the input data is intended to be recognized, and then one or more processing models are called according to the obtained target user intention, thereby outputting the obtained processing results to the user on the interactive interface. Since these processing models run according to their respective functional types based on processors of different processing types in the electronic device, the processors can perform computing tasks for processing models of corresponding functional types due to their respective processing types, and can give full play to the computing performance of the processor, thereby giving full play to the processing performance of the local processor in the electronic device, thereby realizing the call of one or more local processing models on the electronic device, so that multiple processing models can run on the local processor, avoiding the leakage of privacy data when calling the cloud processing model, thereby improving data security.

[0136] In one implementation, the electronic device further includes: a plurality of memory components 902, such as Fig.10 As shown in ; each of the memory components 902 corresponds to one of the processors 901, and the processor 901 exclusively occupies its corresponding memory component 902;

[0137] Among them, after the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor, and the model data enables the processor to execute the computing task corresponding to the processing model.

[0138] Taking the electronic device as a notebook as an example, the technical solution of the present application is illustrated below:

[0139] like Fig.11 As shown in the figure, it is a hardware architecture diagram of a notebook computer. A CPU is deployed in the notebook computer. An iGPU is integrated in the CPU. An iNPU can also be integrated in the CPU. The CPU is deployed with memory such as 16GB, 32GB or 64GB. A dGPU and memory such as 6GB, 8GB or 12GB are also deployed in the notebook computer. In addition, in the notebook computer, the dNPU and memory particles such as 8GB, 16GB or 32GB are integrated into an M.2 card. The dNPU is deployed on the front of the M.2 card, and the memory particles are deployed on the back of the M.2 card. The average power consumption is less than 8 watts. The M.2 card is directly inserted into the M.2 card slot of the notebook motherboard, and the driver is installed so that the Windows operating system on the notebook computer can recognize the dNPU. Fig.12 as shown in .

[0140] In other embodiments, an external dNPU may be installed in the notebook, and the external dNPU is connected to the notebook motherboard via USB. In this case, the external dNPU can run a Linux operating system as a SOC, and the external dNPU is deployed with a memory such as 8GB, 16GB or 32GB and an embedded memory eMMC (Embedded MultiMediaCard).

[0141] In the scenario where the user opens a single large local model, based on Fig.11 As shown in the hardware architecture diagram, after the notebook operating system recognizes the dNPU, if the user opens the local target software, the target software contains the functional module of the local artificial intelligence generated content AIGC (Artificial Intelligence Generated Content), that is, the local large model. After the target software is started, the local large model corresponding to the software, such as the Wenshengwen model, will be automatically loaded onto the dNPU, thereby releasing the resources of the iGPU and dGPU.

[0142] In the scenario where the user opens multiple local large models, based on Fig.11 As shown in the hardware architecture diagram, after the notebook operating system recognizes the dNPU, if the user opens the target software such as Vincent AIGC software (which may include the Vincent model), the Vincent model will be automatically loaded onto the dNPU. When the operating system recognizes that the user has a need to use the Vincent model, it will automatically open the corresponding software such as Vincent AIGC software (which may include the Vincent model), and the Vincent model will be automatically loaded onto the dGPU. At this point, the dual models will run on the operating system at the same time. If the user does not use the model, the dNPU and dGPU where the corresponding model is located will enter low-power standby mode.

[0143] In one embodiment, taking the scenario where a user needs to draw a promotional page for product A as an example, the technical solution of the present application is illustrated below:

[0144] First, if Fig.13As shown in, it is a software architecture diagram of the notebook. Specifically, the dNPU is recognized by the operating system through the driver, and the AIGC software installed in the notebook that can realize Vincent's AIGC provides an interactive interface and an AI module with a poster generation function. The AI ​​module with a poster generation function can be understood as a sub-application in Vincent's AIGC software, such as a poster application. The user can enter the demand data on the interactive interface, such as "Give me a poster promotion page of product A", and the interactive interface provides the demand data to the AI ​​module for poster generation. When Vincent's AIGC software is started, it is loaded into the dNPU, and the model data of Vincent's AIGC software is loaded into the memory corresponding to the dNPU. When Vincent's AIGC software is closed, the dNPU resources are released. An inference engine is deployed in the dNPU, which interacts with the application through the inference interface, the built-in inference platform, the model data, and the model tool chain in the inference engine. Specifically, the AI ​​module provides the prompt word (prompt) representing the intention of poster generation to the inference engine through the inference interface according to the demand data, and requests to return the poster promotion page. Based on this, the inference engine performs the corresponding computing tasks through the built-in inference platform, model data, and model tool chain, that is, generates promotional text, and then provides the promotional text to the AIGC software of Vincent Graphics loaded into the dGPU. Vincent Graphics' AIGC software generates a poster image according to the promotional text and the target user intention corresponding to "Give me a poster page of product A", that is, a poster page of product A containing the promotional text.

[0145] In one embodiment, if Fig.14 As shown in the figure, it is a flowchart for generating a poster page of product A for the user in the notebook. Specifically, it includes:

[0146] First, the user starts the AIGC software of Wenshengwen and opens the sub-application with the poster generation function, namely the poster application, whereby the interactive interface is presented to the user; the user enters the content of the poster generation in the interactive interface, such as "give me a poster promotion page of product A";

[0147] Then, the interactive interface sends the user input to the intent recognition model in the notebook. The intent recognition model parses the input content and obtains the target user intention, such as "draw a poster promotional image of product A", and generates call instructions corresponding to the corresponding large models, such as the Vincent-text large model and the Vincent-image large model;

[0148] After that, the call instruction for generating the poster background image is sent to the AIGC software of Vincent, and the call instruction for generating the product introduction text is sent to the AIGC software of Vincent to instruct the generation of the product introduction text of product A, and the call instruction is sent to find the product image from the local album, and the product image is sent to the AIGC software of Vincent. Thus, the AIGC software of Vincent adds the poster background content to the product image to generate a product poster image with a background, and the AIGC software of Vincent sends the product poster image to the poster application in the AIGC software of Vincent;

[0149] The interactive interface displays a product image of product A, and displays product introduction text generated by Vincent's AIGC software below the product image. The product introduction text is sent to the poster application in Vincent's AIGC software, and the poster application merges the generated product introduction text with the product poster image generated by Vincent's AIGC software to generate a poster promotional page with the product introduction text.

[0150] Finally, the poster page with the product introduction text is displayed to the user on the interactive interface.

[0151] In the above scheme, Vincent's AIGC software can be loaded into the dNPU for running, Vincent's AIGC software can be loaded into the dGPU for running, and the intent recognition model can be an AI module integrated in Vincent's AIGC software or a large model independent of Vincent's AIGC software.

[0152] The following is an example of a user using Vincent's AIGC software to generate a poster page:

[0153] like Fig.15 As shown in , the user can select the function of generating a poster promotion page on the interactive interface provided by Vincent's AIGC software, that is, select the "Poster Agent" button to start the poster application, after which the interactive interface displays an input box for the user to input the required content; wherein, the interactive interface displays multiple function buttons, one of which is the "Poster Agent" button;

[0154] like Fig.16 As shown in , the user can follow the prompts and enter the required content for generating a promotional page for product A in the input box of the interactive interface, such as "give me a poster promotional page for product A";

[0155] like Fig.17 As shown in , if the input content is not particularly clear, the interactive interface can directly remind the user to provide more information, such as "Sorry, please enter more information, such as product model";

[0156] like Fig.18 As shown in , after the user provides more information, Vincent's AIGC software will analyze the demand, and then retrieve the corresponding product pictures from the personal knowledge base such as the photo album. If there are multiple product pictures 1 to n, the user can be prompted to select one or more product pictures, such as product picture 1 and product picture 2;

[0157] like Fig.19 As shown in , Vincent's AIGC software automatically sends product images and raw image requirements to Vincent's AIGC software, allowing Vincent's AIGC software to generate a promotional background image for product A and merge the product image and background image. Vincent's AIGC software generates product promotional copy based on user needs. The dual models run simultaneously to improve efficiency.

[0158] like Fig. 20 As shown in , after the raw pictures and texts are completed, the information (the fusion of the product picture and the background picture, and the product promotional copy) will be sent to the poster application in the AIGC software of Wenshengwen to generate the poster format, and the picture and text will be integrated into the poster to obtain the poster promotional page.

[0159] It can be seen that the technical solution of this application has the following advantages:

[0160] 1. Protect user privacy and improve office efficiency.

[0161] 2. Improve system computing power, optimize notebook battery life, and enhance product performance.

[0162] 3. Increase user scenario support to support more models.

[0163] 4. Provide users with a computing power foundation, based on which users can use more customized models.

[0164] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0165] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0166] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0167] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A processing method comprising: receiving input data through an interactive interface, wherein the interactive interface is constructed based on a target operating system; The target operating system is an operating system supported and run by any processor in the electronic device; Performing intent recognition on the input data to obtain the target user's intent; According to the target user intention, calling at least one processing model to obtain a processing result; Wherein, the processing models of different functional types are run based on processors of different processing types in the electronic device; The processing result is output on the interactive interface.

2. The method according to claim 1, calling at least one processing model according to the target user intention to obtain a processing result, comprising: Generate at least one calling instruction according to the target user intention; One of the calling instructions corresponds to one processing model; Sending the calling instruction to a corresponding processing model, wherein the processing model outputs an operation result in response to the calling instruction; According to the operation result, a processing result is obtained.

3. The method according to claim 2, further comprising: According to the calling instruction, obtaining initial data matching the target user's intention from a storage area of ​​the electronic device; The initial data is sent to a processing model corresponding to the calling instruction, and the processing model processes the initial data in response to the calling instruction to output an operation result.

4. The method according to claim 2, wherein the processor used by the processing model is deployed with a memory component exclusively used by the processor; in, After the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor, and the model data enables the processor to execute the computing task corresponding to the processing model and output the operation result.

5. The method according to claim 2, wherein the at least one processing model comprises: The first model and the second model; Wherein, obtaining the processing result according to the operation result includes: The operation result output by the first model is sent to the second model, so that the second model processes the operation result output by the first model and the operation result output by the second model to output the processing result.

6. The method according to claim 1 or 2, before calling at least one processing model, the method further comprises: Detecting whether the first processor corresponding to the processing model is executing computing tasks of other models; If the first processor is executing computing tasks of other models, the computing tasks executed by the first processor are switched to be executed by the second processor in the electronic device.

7. The method according to claim 6, wherein the processor used by the processing model is deployed with a memory component exclusively used by the processor; in, If the first processor is executing computing tasks of other models, the method further includes: The model data in the memory component corresponding to the first processor is migrated to the memory component corresponding to the second processor.

8. The method according to claim 1 or 2, wherein the electronic device is provided with a first processor and a second processor, the first processor runs a first operating system, the second processor runs a second operating system, and the second operating system runs based on the first operating system; in, The target operating system is the first operating system, and the first operating system outputs the constructed interactive interface; Alternatively, the target operating system is the second operating system, and the interactive interface is transmitted from the second operating system to the first operating system via a transmission interface between the first processor and the second processor, so that the first operating system outputs the interactive interface.

9. An electronic device, comprising: A plurality of processors, each of the processors having a different processing type; Any processor among the plurality of processors supports running a target operating system; An interactive interface is constructed based on the target operating system; The processor supporting the operation of the target operating system is used to: receive input data through the interactive interface; perform intent recognition on the input data to obtain the target user's intent; call at least one processing model according to the target user's intent to obtain a processing result; and output the processing result on the interactive interface; The processing models of different functional types are run based on processors of different processing types in the electronic device.

10. The electronic device according to claim 9, further comprising: A plurality of memory components; each of the memory components corresponds to one of the processors, and the processors exclusively occupy the corresponding memory components; Among them, after the processing model is called, the model data corresponding to the processing model is loaded into the memory component corresponding to the processor, and the model data enables the processor to execute the computing task corresponding to the processing model.

Images