Data processing method and electronic equipment

Abstract

The invention discloses a data processing method and electronic equipment, and the method comprises the steps: determining a task instruction based on collected user interaction data; based on the task instruction, determining a task processing result corresponding to the task instruction; determining target image data based on the task processing result; the target image data can be displayed in a target interface as image data collected by the electronic equipment.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data processing method and an electronic device. Background Technology

[0002] In scenarios involving image acquisition, such as conference sessions or video calls, electronic devices can only process and display the image data acquired by the electronic devices, resulting in limited information interaction methods and types, and making it impossible to achieve more flexible information interaction. Summary of the Invention

[0003] On the one hand, this application provides a data processing method applied to an electronic device, including:

[0004] Based on the collected user interaction data, determine the task instructions;

[0005] Based on the task instruction, determine the task processing result corresponding to the task instruction;

[0006] Based on the task processing results, target image data is determined; the target image data can be displayed on the target interface as image data collected by the electronic device.

[0007] In one possible implementation, determining the target image data based on the task processing result includes:

[0008] Determine the image data source of the electronic device;

[0009] Based on the image configuration method corresponding to the image data source, the task processing result is processed to determine the target image data, which can be used as the image collected by the image data source.

[0010] The data processing method further includes:

[0011] The target image data is sent to the target application so that the target image data is displayed in the target interface of the target application as image data collected by the electronic device.

[0012] In another possible implementation, the electronic device includes a first image acquisition unit and a second image acquisition unit, wherein the first image acquisition unit and the second image acquisition unit acquire images in different ways;

[0013] Before obtaining the target image data, the process also includes:

[0014] Obtain the first image data acquired by the first image acquisition unit;

[0015] The first image data is sent to the target application as image data collected by the electronic device, so that the first image data is displayed in the target interface of the target application;

[0016] After obtaining the target image data, the process also includes:

[0017] The target image data is configured as the second image data acquired by the second image acquisition unit;

[0018] The second image data acquired by the second image acquisition unit is sent to the target application as image data acquired by the electronic device, so that the target image data is displayed in the target interface of the target application.

[0019] In another possible implementation, the target image data is determined based on the task processing result, including at least one of the following:

[0020] If the task processing result includes video data, determine the target image data that includes the video data;

[0021] If the task processing result includes text information, generate target image data containing the text information;

[0022] If the task processing result includes an image, generate target image data including the image.

[0023] Another possible implementation includes at least one of the following:

[0024] If the task processing result includes audio data, the audio data is processed as audio to be output by the audio output unit in the electronic device, and the processed audio data is output through the audio output unit.

[0025] If the task processing result includes audio data and the electronic device has established a conference session with at least one target device, the audio data is transmitted to the target device as the audio data collected in the conference session.

[0026] If the task processing result includes text information, the text information is converted into target audio data, the target audio data is processed as the audio to be output by the audio output unit, and the processed target audio data is output through the audio output unit.

[0027] If the task processing result includes text information and the electronic device has established a conference session with at least one target device, the text information is converted into target audio data, and the target audio data is transmitted to the target device as audio data collected in the conference session.

[0028] In another possible implementation, if the task processing result includes video data, determining the target image data including the video data includes:

[0029] If the task processing result includes video data, the video data is determined to be target image data;

[0030] Alternatively, if the task processing result includes video data, obtain the actual image data collected by the electronic device, and sequentially overlay each frame of the actual image data onto a designated area of ​​each frame of the video data, and determine the overlaid video data as the target image data.

[0031] If the task processing result includes text information, generating target image data containing the text information includes:

[0032] If the task processing result includes text information, generate target image data containing the virtual character and the text information.

[0033] In another possible implementation, determining the task instructions based on the collected user interaction data includes:

[0034] If user voice is captured and the user voice includes the target statement, the task instruction is determined based on the voice segment in the user voice that is located after the target statement;

[0035] And / or, the data processing method further includes:

[0036] If the user interaction data is user voice including the target statement, generate transitional image data to represent the current state of voice command acquisition;

[0037] Before the task instruction is determined, the transitional image data is displayed on the target interface as image data acquired by the electronic device.

[0038] In another possible implementation, determining the task processing result corresponding to the task instruction based on the task instruction includes at least one of the following:

[0039] In response to the task instruction belonging to the historical task instruction of the target model, the task processing result obtained by the historical processing of the task instruction by the target model is obtained;

[0040] In response to the task instruction being an image generation instruction, the target model is used to call the image generation model to process the image generation instruction and obtain the generated image;

[0041] In response to the task instruction being a video generation instruction, the target model is used to call the video generation model to process the video generation instruction and obtain the generated video data;

[0042] In response to the task instruction being a question-answering instruction, the target model is used to perform task reasoning based on the question-answering instruction to obtain the reasoned text information.

[0043] In another aspect, this application also provides an electronic device, comprising:

[0044] Input unit, used to collect user interaction data;

[0045] The processor is configured to determine task instructions based on collected user interaction data; determine the task processing result corresponding to the task instructions based on the task instructions; determine target image data based on the task processing result; and display the target image data as image data collected by the electronic device on a target interface.

[0046] The display module is used to display the target interface.

[0047] In one possible implementation, the electronic device further includes: an audio output unit and an image acquisition unit;

[0048] The acquisition module includes: an audio acquisition unit;

[0049] The image acquisition unit is used to acquire first image data;

[0050] The audio acquisition unit is used to acquire user voice;

[0051] The processor is also used for:

[0052] If the audio acquisition unit acquires the user's voice and the user's voice includes a target statement, the task instruction is determined based on the voice segment in the user's voice that follows the target statement;

[0053] If the task processing result includes audio data, the audio data is processed as audio to be output by the audio output unit in the electronic device, and the processed audio data is sent to the audio output unit.

[0054] The audio output unit is used to output the processed audio data. Attached Figure Description

[0055] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0056] Figure 1 A flowchart illustrating the data processing method provided in this application;

[0057] Figure 2 Another flowchart illustrating the data processing method provided in this application;

[0058] Figure 3 Another flowchart illustrating the data processing method provided in this application;

[0059] Figure 4 This diagram illustrates the implementation process of the solution in one application scenario.

[0060] Figure 5 Another flowchart illustrating the data processing method provided in this application;

[0061] Figure 6 This is an example diagram of the target image data corresponding to the task processing result displayed on the target interface in this application;

[0062] Figure 7 Another flowchart illustrating the data processing method provided in this application;

[0063] Figure 8 This is yet another example of the target image data corresponding to the task processing result displayed on the target interface in this application;

[0064] Figure 9 This is an example diagram of one implementation framework of the solution in this application;

[0065] Figure 10 A schematic diagram of the component architecture of the electronic device provided in this application. Detailed Implementation

[0066] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0067] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0068] The solution proposed in this application can be applied to scenarios involving image acquisition, such as conference sessions or video calls, and can improve the richness and flexibility of information interaction.

[0069] like Figure 1 This is a flowchart illustrating a data processing method provided in this application. The method in this embodiment is applied to an electronic device, which can be any device with image acquisition capabilities, such as a mobile phone, tablet computer, or laptop computer; it can also be a conference machine used to establish a conference session, or a communication device that supports special video calls, etc., without any limitation.

[0070] The method in this embodiment may include:

[0071] S101 determines the task instructions based on the collected user interaction data.

[0072] User interaction data can be data input by the user to indicate task instructions.

[0073] The user interaction data may include, but is not limited to, one or more of the following: user-inputted voice, text, or images. Accordingly, the specific implementation for collecting this user interaction data will vary depending on the type of information it includes. For example, user-inputted voice can be obtained through an audio acquisition unit; or text or images input by the user can be obtained from the interactive interface output by the display unit.

[0074] In this application, task instructions are instructions used to specify the content of a task. For example, task instructions can be image generation instructions, video generation instructions, text generation instructions, text translation instructions, question answering instructions, and content search instructions, etc., without any specific limitations.

[0075] Specifically, the task instructions indicated by the user interaction data are determined by performing intent recognition processing, semantic analysis processing, or by using models such as large language models. This application does not impose any restrictions on the specific method used to determine the task instructions.

[0076] S102, Based on the task instruction, determine the task processing result corresponding to the task instruction.

[0077] The task processing result refers to the result generated by executing the task instruction. For example, if the task instruction is a question-answering instruction, the task processing result is the generated question reply information; similarly, if the task instruction is to generate a video, the task processing result is the generated video; and if the task instruction is a meeting minutes generation instruction, the processing result is the meeting minutes generated based on the meeting content.

[0078] In this application, the specific implementation of the electronic device determining the task processing result based on the task instruction can have multiple possibilities, without any specific limitation.

[0079] In one possible scenario, electronic devices can run intelligent programs such as intelligent agents, which have the ability to invoke target models and process task instructions through these target models.

[0080] The target model can be a machine learning model that can recognize natural language and / or other inputs (such as audio, video, images, tables, etc.) and perform comprehensive language processing tasks such as semantic analysis and question answering. In this application, the target model can also call other models to assist in processing the task instruction based on the different task types indicated by the task instruction, so as to accurately obtain the task processing results corresponding to the task instruction.

[0081] For example, determining the task processing result corresponding to a task instruction can include at least one of the following:

[0082] In response to the task instruction being an image generation instruction, the target model calls the image generation model to process the image generation instruction and obtain the generated image;

[0083] In response to the task instruction being a video generation instruction, the target model is used to call the video generation model to process the video generation instruction and obtain the generated video data;

[0084] In response to a task instruction that is a question-answering instruction, the target model is used to perform task inference based on the question-answering instruction to obtain inferred text information, which includes the answer to the question-answering instruction.

[0085] Of course, there are other possibilities for the task instructions, and correspondingly, there are other possibilities for processing the task instructions using the target model, which will not be elaborated here.

[0086] Specifically, to reduce data processing for the target model and improve the efficiency of determining task processing results, considering that users often input the same or similar task instructions, this application, after determining the task instruction, can also detect whether the task instruction belongs to a historical task instruction previously processed by the target model. Accordingly, in response to the task instruction belonging to a historical task instruction previously processed by the target model, the task processing result obtained from the historical task instruction processed by the target model is obtained. Since the target model does not need to repeatedly process the task instruction, resource consumption and the time required to process the task instruction can be reduced.

[0087] S103, Based on the task processing results, determine the target image data; the target image data can be displayed on the target interface as image data acquired by the electronic device.

[0088] In this application, the task processing result is processed into image data to obtain target image data. Therefore, the task processing result can be displayed by displaying the target image data.

[0089] The target image data can be video, or it can be at least one image or a sequence of images.

[0090] The target interface is an application interface capable of displaying image data acquired by the electronic device through the image acquisition unit. Specifically, if the electronic device currently has a real-time video interaction connection, such as a conference session or video call, with a target device outside the electronic device, the target interface may also include an application interface on the target device side for displaying the image data acquired by the electronic device.

[0091] For example, in a conference session (i.e., an online meeting) scenario, if an electronic device enters the conference session display interface, but at this time the electronic device has not yet established a conference session with other participating devices (i.e., other participants have not yet entered the conference session), then the target interface is the conference session interface displayed on the electronic device. If the electronic device has established a conference session with other participating devices, then the target interface can include both the interface locally output by the electronic device to present the image data, and the interface presented by the participating devices to display the image data acquired by the electronic device.

[0092] For example, in a video call scenario, the target interface may include the video call interface on the electronic device or a sub-interface within that video call interface used to display locally captured video. Alternatively, it may include the video call interface output by the other end of the video call, used to display the video image transmitted by the electronic device.

[0093] It is understandable that, since the target image data can display the task processing results corresponding to the task instructions, and the target image data can be displayed on the target interface as image data collected by the electronic device, in scenarios involving image acquisition such as electronic device sessions or video calls, the task processing results of the task instructions can also be seen intuitively through the target interface used to present the image data collected by the electronic device.

[0094] As can be seen from the above, after determining the task processing result corresponding to the task instruction indicated by the collected user interaction data, this application can determine the target image data based on the task processing result. Since the target image data can be displayed on the target interface as image data collected by the electronic device, in scenarios where it is necessary to display the image data collected by the electronic device through the target interface, this application can also display the task processing result corresponding to the user interaction data through the target interface, thereby improving the flexibility and diversity of information interaction.

[0095] In this application, there are several possible implementations for enabling the target image data to be displayed on the target interface as image data acquired by an electronic device. Several possible implementation methods are described below.

[0096] The following is combined Figure 2 One implementation method will be introduced. For example... Figure 2 This illustrates another flowchart of the data processing method provided in this application. The method in this embodiment may include:

[0097] S201, based on the collected user interaction data, determines the task instructions.

[0098] This step can be referred to in the relevant description of the previous embodiments.

[0099] S202, Based on the task instruction, determine the task processing result corresponding to the task instruction.

[0100] S203, Determine the image data source for the electronic device.

[0101] The image data source is used to provide the acquired image data to the electronic device.

[0102] In this embodiment, the electronic device needs to output the image data it has acquired to the target application. Therefore, the image data source is the data source used to provide image data to the target application. Based on this, the step can be: in response to the target application being in a running state, determine the image data source of the electronic device.

[0103] In this embodiment, the image data source can be an image acquisition unit of an electronic device. This image acquisition unit can be a physical image acquisition unit (such as a camera), used to acquire external images of the electronic device. Alternatively, the image data source can be a virtual camera within the electronic device. The data source for the virtual camera can be configured as needed; for example, the data source can be a physical image acquisition unit or image data generated by the electronic device. The virtual camera is a camera device simulated by software, not physical hardware. The virtual camera can act as a simulated physical camera (i.e., disguised as a real camera) to output its customized video stream to various applications within the electronic device (such as conferencing applications, video call applications, live streaming applications, and social media applications), achieving a level of flexible video acquisition and processing that physical cameras cannot provide. Compared to physical cameras that capture images through hardware, virtual cameras generate or process images through software and then simulate hardware output image data. The operating system and applications of the electronic device will recognize the virtual camera as a regular camera device, requiring no additional adaptation.

[0104] The basic working principle of a virtual camera is as follows: It simulates a virtual camera device node in the operating system's device manager through a driver / application layer interface. It intercepts application requests to access the camera and returns pre-prepared or real-time processed video data to the application according to the camera's standard data format. The entire process is completely transparent to the application. In practical applications, electronic devices are configured to use a physical image acquisition device or a virtual camera as their image data source. For example, the image data source can be configured when or after starting the target application.

[0105] S204. Based on the image configuration method corresponding to the image data source, process the task processing results to determine the target image data.

[0106] The target image data can be used as the image acquired from the image data source.

[0107] In this application, the image configuration method corresponding to the image data source is used to characterize the type of image acquisition unit that serves as the image data source, or in other words, to characterize the type of image acquisition unit that is actually connected or configured to provide the image data. For example, if the image data source in the electronic device is a physical image acquisition unit, the image configuration method can indicate that the image data is provided by the physical image acquisition unit; if the image data source of the electronic device is a virtual camera, the image configuration method indicates that the image data is provided by the virtual camera.

[0108] Because different image configuration methods correspond to different image data sources, the data format of the image data obtained by electronic devices through these sources will also differ. In order for the target image data generated based on the task processing results to be usable as an image acquired by that image data source, the data format of the target image data needs to be consistent with the data format of the image data provided by that image data source. Therefore, processing task results based on image configuration methods can involve processing the task processing results into target image data adapted to that image configuration method.

[0109] For example, the image configuration method corresponding to the image data source indicates that the image data source is a physical image acquisition unit, and the task processing result can be processed into target image data that matches the data format of the image data acquired by the physical image acquisition unit.

[0110] For example, the image configuration method corresponding to the image data source indicates that the image data source is a virtual camera. Since the data source of the virtual camera can be configured according to actual needs, there are no restrictions on the data type of the data source. Therefore, it is only necessary to process the task processing result into image data to obtain the target image data, and there are no restrictions on the data format of the target image data.

[0111] S205, the target image data is sent to the target application so that the target image data is displayed in the target interface of the target application as image data acquired by the electronic device.

[0112] The target application is one that can display image data captured by the image acquisition unit of an electronic device. For example, the target application could be a video interactive application such as an online conferencing application or an application that supports video calls.

[0113] In this application, the target interface of the target application can be used to display image data provided by the image data source of the electronic device. The target image data is obtained by processing the task processing results based on the image configuration method corresponding to the image data source. Therefore, sending the target image data to the target application is essentially equivalent to configuring the image data provided by the image data source as the target image data and sending it to the target application. For the target application, it can display the image data provided by the image data source to the target interface; therefore, the target application can display the target image data on the target interface.

[0114] In particular, if the electronic device has established a video interaction session such as a conference session or video call with at least one target device, the target application will also synchronize the image data displayed on its target interface to the target device. Therefore, the target image data can be sent to the target application, or the target application can send the target image data as image data collected by the electronic device to the target device so that the target image data can be displayed on the target device.

[0115] In this embodiment, after processing the task processing result corresponding to the task instruction based on the image configuration method of the image data source, it is only necessary to send the generated target image data to the target application. The target image data can then be displayed as the image data collected by the electronic device on the target interface of the target application without changing the target application. The task processing result determined based on the user's interaction data can be displayed on the target interface of the target application, so that the corresponding task processing result can be seen intuitively while using the target application. This not only reduces complexity but also improves the convenience, flexibility, and diversity of information interaction.

[0116] It is understood that in this embodiment, before generating the target image data, the electronic device can send image data from the image data source to the target application. Correspondingly, after sending the target image data to the target application, the electronic device can still send image data from the image data source to the target application.

[0117] To facilitate understanding, let's take a conference session scenario as an example. In this case, the target application is a conference session application. Figure 2 A specific implementation of the embodiment will be described. For example... Figure 3 This illustrates another flowchart of the data processing method provided in this application. The method in this embodiment may include:

[0118] S301, Establish a meeting session.

[0119] Establishing a conference session by an electronic device can be a connection between the electronic device and at least one target device. In this case, the conference interface of the conference session application in the electronic device can display not only the image data collected locally by the electronic device, but also the image data collected by the target device and transmitted to the electronic device.

[0120] Electronic devices can also establish a conference session while waiting for other electronic devices to join the conference session. In this case, the electronic device has not yet established a conference session connection with at least one target device. In this situation, the conference interface of the electronic device's conference session application can only display the image data acquired locally by the electronic device.

[0121] It is understood that replacing establishing a conference session with establishing a video call or other video interaction session is also applicable to this embodiment, and there is no limitation thereto.

[0122] S302, Determine the image data source for the electronic device.

[0123] S303, if the image data source is a virtual camera, configure the data source of the virtual camera to be a physical image acquisition unit in the electronic device, so as to send the real image data actually acquired by the physical image acquisition unit to the conference session application, and display the real image data on the conference interface of the conference session application.

[0124] The data source for the virtual camera is the image acquisition method of the virtual camera.

[0125] S304, If user interaction data is collected, determine the task instructions based on the user interaction data.

[0126] S305, Based on the task instruction, determine the task processing result corresponding to the task instruction.

[0127] S306, in response to the image data source being a virtual camera, determines the target image data based on the task processing results.

[0128] When the image data source is a virtual camera, it is only necessary to convert the task processing results into target image data such as video or image sequences, without restricting the data format of the target image data.

[0129] S307, switch the data source of the virtual camera to the target image data, so as to send the target image data to the conference session application, so that the target image data is displayed in the conference interface as image data collected by the electronic device.

[0130] Switching the data source of the virtual camera to the target image data essentially means configuring the target image data to be the image data captured by the virtual camera. Based on this, since the electronic device transmits the data captured by the virtual camera (which serves as the image data source) to the conference application, and the target image data is configured to be the data captured by the virtual camera, the electronic device will send the target image data to the conference application.

[0131] To facilitate a more intuitive understanding Figure 3 The solution in the embodiment, combined with Figure 4 A brief explanation is provided. Figure 4 A schematic diagram illustrating the implementation process of the solution in this application scenario is shown.

[0132] exist Figure 4 Taking a conference conversation scenario as an example, and using user interaction data as user voice collected through a microphone as an example, this will be explained.

[0133] Depend on Figure 4As can be seen, in a conference session scenario, electronic devices can be configured with a virtual camera as the image data source. Based on this, the electronic device will first configure the virtual camera's image source to be image data captured by a physical camera.

[0134] Depend on Figure 4 As can be seen, during a conference session, the real image data captured by the physical camera can be transmitted to the conference application as image data captured by the virtual camera. In this case, the conference application's interface can display the real image data captured by the electronic device. Figure 4 The example illustrates this by using a physical camera to capture images of meeting participants on the local electronic device.

[0135] When an electronic device detects a user's voice through a microphone, it can analyze the voice using a target model such as a large language model. If a task instruction is determined based on the user's voice, the electronic device can process the task instruction using the target model to obtain the task processing result. For example, if the task instruction is a video generation instruction, the target model can call a video generation model to generate video data. Based on this, the electronic device will switch the data source from the virtual camera to the target image data. Thus, the electronic device transmits the image data captured by the virtual camera to the conference application, which is essentially transmitting the target image data to the conference application, causing the conference interface of the conference application to display images of the conference participants.

[0136] As described above, in this embodiment, after the electronic device establishes a conference session, the virtual camera is used as the image data source of the electronic device. Based on this, if the user does not input user interaction data to indicate task instructions, the electronic device will configure the data source of the virtual camera as the physical image acquisition unit in the electronic device, so that the real image data actually acquired by the physical image acquisition unit can be sent to the conference session application and displayed on the conference interface of the conference session application to achieve the effect of a regular conference session.

[0137] Furthermore, during a meeting, if the electronic device receives user-input interaction data, it can process the task instructions indicated by that interaction data using methods such as a target model. After obtaining the task processing result corresponding to the task instruction and processing it into target image data, simply switching the data source of the virtual camera to that target image data allows the target image data to be displayed on the meeting interface of the meeting application. This enables users to intuitively see the task processing results of relevant task instructions during the meeting without having to jump to a specific task processing interface to view the results. Moreover, this application only needs to switch the data source of the virtual camera to switch the display content of the meeting interface without requiring any prior adaptation adjustments to the target application, making this solution highly compatible while improving the flexibility of information interaction.

[0138] Understandable, Figure 3 This example uses a virtual camera as the image data source. If the image data source is a physical image acquisition unit, the electronic device can send the real image data acquired by the physical image acquisition unit to the conference application before obtaining the target image data. Correspondingly, after obtaining the task processing results, the results can be processed based on the data format corresponding to the image data output by the physical image acquisition unit to determine the target image data. Furthermore, after obtaining the target image data, the electronic device will no longer send the real image data acquired by the image acquisition unit to the conference application; instead, it will switch to sending the target image data to the conference application. This method achieves the beneficial effects of switching image data using a virtual camera, and it eliminates the need to build a virtual camera on the electronic device, thus lowering the requirements on the electronic device side and making it more widely applicable.

[0139] certainly, Figure 3 This example uses a conference call scenario. The process is similar if the conference call scenario is replaced with a video call scenario, so I will not go into details here.

[0140] The following describes another method for displaying target image data as image data acquired by an electronic device on a target interface. For example... Figure 5This illustration shows another flowchart of the data processing method provided in this application. In this embodiment, the electronic device includes a first image acquisition unit and a second image acquisition unit. The first and second image acquisition units have different acquisition methods. For example, the first image acquisition unit is used to acquire real image data from outside the electronic device, and the data source is fixed and cannot be configured; for example, the first image acquisition unit is the aforementioned physical image acquisition unit. The second image acquisition unit, on the other hand, has a configurable image acquisition method and can acquire images based on user configuration. Therefore, the source of image data acquired by the second image acquisition unit is adjustable; for example, the second image acquisition unit can be a previously mentioned virtual camera.

[0141] The method in this embodiment may include:

[0142] S501, obtain the first image data acquired by the first image acquisition unit.

[0143] S502, the first image data is sent to the target application as image data collected by the electronic device, so that the first image data is displayed in the target interface of the target application.

[0144] As mentioned earlier, the first image data acquired by the first image acquisition unit is real image data of the exterior of the electronic device. Therefore, the image data actually acquired by the electronic device is displayed on the target interface of the target application.

[0145] It is understandable that if an electronic device has established a video interaction session such as a conference session or video session with at least one target device, after sending the first image data to the target application, while displaying the first image data on the target interface of the target application, the target application will also send the first image data to the target device so that the first image data can be displayed on the target device.

[0146] In this application, the above two steps may be repeated before the target image data is obtained, so that the first image data acquired by the first image acquisition unit can be transmitted to the target application and displayed in the target interface of the target application.

[0147] S503 determines task instructions based on the collected user interaction data.

[0148] S504, based on the task instruction, determines the task processing result corresponding to the task instruction.

[0149] S505, based on the task processing results, determines the target image data.

[0150] The above steps can be found in the descriptions of other embodiments of this application, and will not be repeated here.

[0151] S506, configure the target image data as the second image data acquired by the second image acquisition unit.

[0152] S507, the second image data acquired by the second image acquisition unit is sent to the target application as image data acquired by the electronic device, so that the target image data is displayed in the target interface of the target application.

[0153] As described above, the image acquisition method of the second image acquisition unit is configurable. Therefore, the target image data is configured to be the second image data acquired by the second image acquisition unit, and the image data provided by the second image acquisition unit to the electronic device is this target image data. Based on this, sending the second image data acquired by the second image acquisition unit to the target application is essentially sending the target image data to the target application.

[0154] In this embodiment, before acquiring the target image, the electronic device sends the first image data acquired by the first image acquisition unit to the target application, which is equivalent to establishing a data transmission channel between the first image acquisition unit and the target application. After the electronic device acquires the target image data, it switches the data transmission channel between the first image acquisition unit and the target application to the data channel between the second image data unit and the target application, so that the target application can obtain the target image data that represents the task processing result, determined by the electronic device based on user interaction data.

[0155] In one possible implementation, the electronic device runs an intelligent program that controls the transmission of image data to the target application. Based on this, before obtaining the target image data, the intelligent program can send the first image data acquired by the first image acquisition unit to the target application; after determining the target image data, the intelligent program configures the target image data as the second image data acquired by the second image acquisition unit and sends the second image data from the second image acquisition unit to the target application, thereby achieving the transmission of the target image data to the target application.

[0156] It is understood that the operation in this embodiment is in the scenario of displaying image data collected by the image acquisition unit through the target application. Therefore, in this embodiment, if the target application establishes a video interaction session such as a conference session, the electronic device obtains the first image data collected by the first image acquisition unit and performs the relevant operations of steps S402 to S407.

[0157] Similar to the previous embodiments, if the electronic device has established a video interaction session such as a conference session with at least one target device, it will also send the target image data to the target device through the target application so that the target image data can be displayed synchronously on the target device.

[0158] In this embodiment, when the electronic device sends the first image data acquired by the first image acquisition unit to the target application and displays it on the target interface of the target application, if the electronic device detects user interaction data, it can determine the task instruction based on the user interaction data. After determining the task processing result corresponding to the task instruction, the electronic device processes the task processing result into target image data and configures the target image data as the second image data acquired by the second image acquisition unit. Based on this, the electronic device only needs to switch to sending the second image data acquired by the second image acquisition unit to the target application to display the task processing result generated based on the task instruction indicated by the user interaction data on the target interface of the target application. This allows users to see the corresponding task processing results while using the target application for video calls or conference sessions, without needing to separately open other task interfaces for processing task instructions and view the task processing results. This improves the flexibility, convenience, and diversity of information display in video interactive sessions such as conference sessions.

[0159] Furthermore, if the electronic device establishes a session connection with at least one target device based on the target application, the target application can send the target image data to the target device, so that the user on the target device can also directly see the task processing result generated on the electronic device side, which can improve the convenience and flexibility of information exchange in conference interaction or video call.

[0160] In addition, through this embodiment, the electronic device only needs to control the switching of the first image acquisition unit or the second image acquisition unit to send image data to the target application, so that the actual image data acquired by the electronic device or the task processing results generated by the electronic device based on the task instructions can be flexibly displayed on the target interface of the target application, without changing the target application, thus making the complexity of presenting the task processing results through the target interface of the target application low.

[0161] In any of the above embodiments of this application, the content type of the task processing result can be various, and correspondingly, the specific implementation of determining the target image data based on the task processing result can also be various. For example, determining the target image data based on the task processing result in this application can include at least one of the following possible cases:

[0162] In the first possible scenario, if the task processing result includes video data, the target image data including that video data can be determined.

[0163] For example, if the task processing result includes video data, the video data can be directly identified as the target image data without any other special processing. Correspondingly, the target image data displayed on the target interface is the video data generated based on the task instructions indicated by the user interaction data.

[0164] For example, if the task processing result includes video data, the actual image data captured by the electronic device can also be obtained. Each frame of this actual image data is then sequentially superimposed onto a designated area of ​​each frame of the video data, and the resulting video data is identified as the target image data. Based on this, by displaying the target image data on the target interface, not only can the video data generated based on user interaction data be presented, but the actual image data captured by the electronic device can also be presented synchronously.

[0165] The actual image data acquired by the electronic device can be the image data acquired by the physical image acquisition unit in the electronic device, such as the first image data acquired by the first image acquisition unit mentioned above.

[0166] In this case, the process of sequentially overlaying each frame of the real image data onto a designated area of ​​each frame of the video data can be carried out by sequentially overlaying each frame of the real image data onto a designated area of ​​a different frame of the video data according to the order of the frames of the real image data.

[0167] The specified area can be set according to actual needs. For example, the specified interval can be the area of ​​the target range in the lower right corner of each frame of the target image data, without any specific restrictions.

[0168] In the second possible scenario, if the task processing result includes an image, target image data including that image is generated. For example, an image sequence or video is generated based on the image, where each frame in the image sequence or video is the same image. To ensure that the target interface continuously displays the image for a period of time, an image sequence or video containing multiple frames of the image can be generated, meaning that each frame in the image sequence or video is identical and is the same image used as the task processing result.

[0169] In the third possible scenario, if the task processing result includes text information, target image data containing that text information can be generated. For example, the text information can be converted into one or more images containing that text information, and based on the converted one or more images, target image data such as videos can be generated.

[0170] Furthermore, in this third possible scenario, to avoid the image content of the target image data containing text information being too monotonous, if the task processing result includes text information, this application can also generate target image data containing both virtual characters and text information. Based on this, after displaying the target image data on the target interface, the target interface can present an animation effect similar to a virtual character explaining text information, improving the richness and diversity of information interaction.

[0171] Understandably, in scenarios such as video conferencing or video calls, users may be accustomed to watching images or videos accompanied by audio. Therefore, to further enrich the information interaction, if the task processing result includes text information, this application can also convert the text information into target audio data, process the target audio data as audio to be output by the audio output unit, and output the processed target audio data through the audio output unit.

[0172] One method of processing the target audio data as the audio to be output by the audio output unit is to process the target audio data into an audio format supported by the audio output unit.

[0173] To facilitate understanding, we will continue to use a meeting session scenario as an example to illustrate and combine... Figure 6 Please provide an explanation.

[0174] Depend on Figure 6 As shown in the first image from left to right, in a conference session scenario, the conference interface of the application displays images of the participants on the electronic device side. Of course, if the electronic device has established a conference session connection with at least one target device, the interface will also display images of the conference participants on the target device side, such as... Figure 6 The meeting interface also displays images of users from other participating devices.

[0175] Based on this, such as Figure 6 As shown in the second image from left to right, if an electronic device captures the user's voice through an audio acquisition unit such as a microphone, the electronic device can determine the task processing result based on the task instructions given by the user's voice. Figure 6 To make it easier to understand, let's take the task instruction as a question-and-answer instruction as an example. Then the task processing result is the question-and-answer information in text form.

[0176] Based on this, this application generates target image data comprising multiple images based on the question-and-answer information. Each image in the target image data includes a virtual character and the question-and-answer information. To dynamically represent the virtual character speaking the question-and-answer information, the degree of mouth opening of the virtual character in different images of the target image data can be different. For example... Figure 6As shown in the rightmost image, after the target image data is generated, the meeting interface switches to display that target image data. The target image data package contains a virtual character and question-and-answer information; therefore, based on this target image data, it is possible to present the visual effect of a virtual character answering a user's question.

[0177] In addition, while generating target image data based on question and answer information, this application also converts the question and answer information into target audio data. On this basis, while the electronic device outputs target image data to the conference interface, it can process the target audio data into audio that can be output by the electronic device's speaker or other audio output unit. The processed target audio data is then output through the speaker or other unit, so that when the user on the electronic device side inputs a question via voice, the conference session application can simultaneously output text response information and audio for that question.

[0178] Furthermore, if the task processing result includes text information and the electronic device has established a conference session with at least one target device, this application can also convert the text information into target audio data and transmit the target audio data as audio data collected in the conference session to the target device, so that the target device can also hear the voice broadcast of the text information while displaying the target image data containing the text information.

[0179] It is understood that, in any of the above embodiments of this application, the task processing result determined based on the task instruction may also include audio data. In this case, the electronic device may also process the audio data as audio to be output by the audio output unit in the electronic device, and output the processed audio data through the audio output unit.

[0180] Of course, in the first to third possible cases above where the target image data is determined, the task processing result may include audio data in addition to video data, image or text information. In this case, while performing the processing in the previous cases, the electronic device may also process the audio data as the audio to be output by the audio output unit in the electronic device and output the processed audio data through the audio output unit.

[0181] Similarly, if the task processing result includes audio data and the electronic device has established a conference session with at least one target device, the audio data can also be transmitted to the target device as audio data acquired during the conference session. The specific processing method is similar to the previous method of converting text information into target audio data, and will not be repeated here.

[0182] It is understood that in the above embodiments of this application, user interaction data can be in the form of text, voice, or other forms, without any specific limitation.

[0183] However, in scenarios such as conference sessions or video calls, if information is input in text form, users may need to manually launch the intelligent program and display a text input field to input text-based user interaction information, which is relatively complex. Therefore, to reduce the complexity of user input task commands, the user interaction data in this application can be user interaction data input by the user via voice.

[0184] Furthermore, in conference sessions or video calls, performing task analysis and processing on every piece of user-input voice would not only lead to excessive data processing volume but also potentially result in misjudgments of task instructions. Therefore, to reduce data processing volume and the possibility of misjudgments, this application triggers task analysis and processing only when the user's voice includes a target statement. Specifically, if user voice is captured and contains a target statement, the task instruction is determined based on the speech segment following the target statement in the user's voice.

[0185] The target statement can be set as needed, and its purpose is to indicate the need to initiate task analysis and processing of the user's voice. For example, the target statement can be a statement used to wake up the intelligent program of an electronic device to perform task analysis and processing.

[0186] Understandably, in order to allow users to intuitively understand whether the electronic device is currently performing task analysis and processing on the user's input voice, this application can also generate transitional image data to represent the current state of voice command acquisition if the user interaction data includes the target statement. Before determining the task instruction, the transitional image data can be displayed on the target interface as image data acquired by the electronic device.

[0187] like Figure 6 As shown, after acquiring the user's voice, if the user's voice is recognized to include the target statement, the electronic device will generate transition image data and display it in the conference interface, such as... Figure 6 The image shown in the middle demonstrates that, based on this transitional image data, the user of the electronic device can determine that the device is currently acquiring and analyzing the task instructions given by the user's voice.

[0188] The specific implementation of displaying transitional image data as image data acquired by the electronic device on the target interface can be similar to the previous implementation of displaying target image data as image data acquired by the electronic device on the target interface. For example, based on the image configuration method of the electronic device's image data source, transitional image data can be generated, and then sent to the target application so that the target application's target interface can display the transitional image data. Alternatively, after obtaining the transitional image data, it can be configured as the second image data acquired by the second image acquisition unit, and then sent to the target application so that the target application's target interface can display the transitional image data.

[0189] To better understand the solution in this application, the following explanation uses user voice as the user interaction data and the target application as an application for establishing a conference session as an example, and combines one implementation method for illustration. Figure 7 This illustrates another flowchart of the data processing method provided in this application. The method in this embodiment may include:

[0190] S701, if the electronic device establishes a conference session based on the target application, it obtains the first image data acquired by the first image acquisition unit.

[0191] The first image acquisition method is fixed, and it can be regarded as a physical image acquisition unit such as a camera of an electronic device.

[0192] S702, the first image data is sent to the target application as image data acquired by the electronic device, so that the first image data is displayed in the conference interface of the target application.

[0193] When the target application is used to establish a conference session, the application interface of the target application is referred to as the conference interface.

[0194] S703, if user voice is acquired and the user voice includes the target statement, generate transitional image data to represent the current state of voice command acquisition.

[0195] S704, configure the transition image data as the second image data acquired by the second image acquisition unit, and send the second image data acquired by the second image acquisition unit as the image data acquired by the electronic device to the target application so that the transition image data is displayed in the target interface of the target application.

[0196] S705, determine the task instruction based on the speech segment in the user's speech that follows the target statement.

[0197] It should be noted that the step of generating transition image data and step S705 can be executed synchronously or in interchangeable order, without any restriction.

[0198] S706 determines the task processing result corresponding to the task instruction based on the task instruction.

[0199] For example, if the task instruction is a video generation instruction, then the electronic device can call the video generation model to generate the task processing result, which includes audio and video data. As explained above, this will not be repeated here.

[0200] S707, if the task processing result includes video data and audio data, determine that the video data is the target image data, configure the target image data as the second image data acquired by the second image acquisition unit, and send the target image data as the second image data to the target application so that the target image data is displayed in the target interface of the target application.

[0201] The image acquisition mode of the second image acquisition unit is configurable. Therefore, this application can set the image acquisition mode of the second image acquisition unit to acquire target image data or acquire transitional image data as needed. For example, the second image acquisition unit can be the virtual camera mentioned above.

[0202] S708 processes audio data as audio to be output by the audio output unit in the electronic device, and outputs the processed audio data through the audio output unit.

[0203] Specifically, if the electronic device has established a conference session with at least one target device, it can also transmit audio data as audio data captured during the conference session to the target device. For example, if audio data is transmitted as audio data captured during the conference session to the target application, the target application will then send the audio data to the target device.

[0204] It is understood that if the task processing result only includes video data, it is also applicable to this embodiment. In this case, there is no need to perform step S708.

[0205] It is understood that when an electronic device is running an intelligent program, the above operations in this embodiment can be controlled by the intelligent program to transmit image data to the target application, thereby switching the image displayed in the meeting interface of the target application from the image captured by the physical camera to the image captured by the virtual camera. Moreover, by configuring the image captured by the virtual camera as transitional image data or target image data, the meeting interface can display a prompt screen for voice capture or video data generated based on task instructions.

[0206] To understand more intuitively Figure 7 The embodiments, in combination Figure 8 Please provide an explanation.

[0207] Figure 8 In the conference session scenario shown, if user-inputted speech is detected and includes the target speech, the electronic device will switch from transmitting the image captured by the camera to the target application to transmitting transitional image data to the target application. This causes the conference interface to switch from displaying images of the participants to displaying images representing the voice acquisition and analysis, such as... Figure 8 The first two images, this part is related to Figure 6 The switching process for the first two images is similar.

[0208] If an electronic device uses video generation model data based on task instructions, the electronic device can configure the video data to be image data captured by a virtual camera. Then, the electronic device transmits the data captured by the virtual camera to the target application; in fact, it transmits the video data generated by the video generation model to the target application. The target application's meeting interface will then display the video data, such as... Figure 8 The rightmost image is shown. Figure 8 The example used is generated video data to explain the functions of a laptop.

[0209] Understandable, Figure 7 The embodiment shown is only an example of one implementation method. If the target image data is sent to the target application by adjusting the image data source in the electronic device, the other implementation process is similar, except that it is necessary to process the video data corresponding to the task instruction based on the image configuration method of the image data source to generate the target image data. It will not be described in detail here.

[0210] To facilitate understanding of how image data is acquired by switching image data sources, the following example uses an electronic device running a smart program to process user interaction data and send target image data to the target application. Figure 9 Please provide an explanation.

[0211] Figure 9 An example diagram of the implementation framework of the solution in this application scenario is shown.

[0212] To make it easier to understand, we will still use a meeting session scenario as an example.

[0213] like Figure 9 As shown, the electronic device includes: an audio acquisition unit ( Figure 9 (Taking the microphone as an example) 901, audio output unit ( Figure 9 (Taking a speaker as an example) 902 and an image acquisition unit (such as a camera) 903.

[0214] The electronic device runs target application 904 and smart program 905.

[0215] The target application can be a conference session application.

[0216] The intelligent program may include: a wake-up recognition module 906, a first audio processing module 907, a second audio processing module 908, a task processing module 909, and an image switching module 910.

[0217] The audio acquisition unit is used to collect user voice.

[0218] The first audio processing module is used to obtain the user's voice collected by the voice acquisition unit and send the user's voice to the wake-up recognition module.

[0219] The wake-up recognition module 906 is used to identify whether the user's voice includes a target statement as a wake-up word.

[0220] The task processing module 909 is used to, after the speech recognition module determines that the user's speech includes the target speech, obtain the speech segment following the target sentence from the first audio processing module, determine the task instruction based on the speech segment using a target model such as a large language model, and perform task inference based on the task instruction, or call an image generation model, video generation model, or other models to generate the task inference result. Furthermore, after obtaining the task processing result, the task processing module also processes the result into target image data and transmits the target image data to the image switching module.

[0221] The image switching module 910 is used to set the image acquisition unit as an image data source. Based on this, if the image switching unit does not obtain the target image data, the image switching module will send the image data acquired by the image acquisition unit to the target application 904 to display the image data in the application interface of the target application; if the image switching module obtains the target image data, it will transmit the target image data as image data acquired by the image acquisition unit to the target application, enabling the application interface of the target application to display the target image data.

[0222] In this application, the image switching module 910 may be a Device Media Foundation Transform (DMFT) component. The DMFT component is an intermediate layer between the image acquisition unit (such as a camera) and the target application. The DMFT can be used to post-process the raw video data acquired by the image acquisition unit and output the post-processed video data to the target application.

[0223] The first audio processing module 907 is also used to send the user's voice to the target application, so that the target application can send the user's voice to the target device that has established a conference session with the electronic device.

[0224] When the task processing module 909 determines that the task processing result includes audio data, or when it converts the text information in the task processing data into target audio data, the task processing module 909 will also send the audio data to the first audio processing module.

[0225] Correspondingly, the first audio processing module 907 is also used to send the audio data in the task processing result or the target audio data generated based on the text in the task processing result to the target application, so that the target application can send the corresponding audio data to the target device that has established a conference session with the electronic device.

[0226] In addition, the task processing module 909 is also used to send the audio data in the task processing result or the target audio data generated based on the text in the task processing result to the second audio processing module 908.

[0227] The second audio processing module 908 processes the received audio data into audio data supported by the audio acquisition unit and sends it to the audio output unit, so that the local user of the electronic device can also hear the audio data in the task processing result or the target audio data converted based on the task processing result.

[0228] Furthermore, this application also provides an electronic device in its embodiments. For example... Figure 10 As shown, a schematic diagram of the composition structure of the electronic device is presented. The electronic device includes at least: a data acquisition module 1001, a processor 1002, and a display module 1003.

[0229] The acquisition module 1001 is used to acquire user interaction data. For example, the acquisition module may include an audio acquisition unit, a touch screen, or an input device.

[0230] The processor 1002 is configured to determine a task instruction based on collected user interaction data; determine a task processing result corresponding to the task instruction based on the task instruction; and determine target image data based on the task processing result; the target image data can be displayed on a target interface as image data collected by the electronic device.

[0231] Display module 1003 is used to display the target interface.

[0232] Of course, the electronic device may also include a memory 1004 for storing programs required by the processor to perform operations.

[0233] Of course, the electronic device can also have more than Figure 10 There are no restrictions on the number of components, whether more or fewer.

[0234] In one possible implementation, the acquisition module includes an audio acquisition unit.

[0235] The electronic device also includes: an audio output unit and an image acquisition unit;

[0236] The image acquisition unit is used to acquire first image data;

[0237] The audio acquisition unit is used to acquire user voice;

[0238] The processor is also used for:

[0239] If the audio acquisition unit acquires the user's voice and the user's voice includes a target statement, the task instruction is determined based on the voice segment in the user's voice that follows the target statement;

[0240] If the task processing result includes audio data, the audio data is processed as audio to be output by the audio output unit in the electronic device, and the processed audio data is sent to the audio output unit.

[0241] The audio output unit is used to output the processed audio data.

[0242] For details regarding the specific operations performed by the processor and other modules or units, please refer to the relevant descriptions in the preceding embodiments, which will not be repeated here.

[0243] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the data processing methods provided in this application.

[0244] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the data processing methods provided in this application.

[0245] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0246] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0247] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0248] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A data processing method, applied to an electronic device, comprising: Based on the collected user interaction data, determine the task instructions; Based on the task instruction, determine the task processing result corresponding to the task instruction; Based on the task processing results, the target image data is determined; The target image data can be displayed on the target interface as image data collected by the electronic device.

2. The data processing method according to claim 1, wherein determining the target image data based on the task processing result includes: Determine the image data source of the electronic device; Based on the image configuration method corresponding to the image data source, the task processing result is processed to determine the target image data, which can be used as the image collected by the image data source. The data processing method further includes: The target image data is sent to the target application so that the target image data is displayed in the target interface of the target application as image data collected by the electronic device.

3. The data processing method according to claim 1, wherein the electronic device includes a first image acquisition unit and a second image acquisition unit, wherein the first image acquisition unit and the second image acquisition unit have different acquisition methods; Before obtaining the target image data, the process also includes: Obtain the first image data acquired by the first image acquisition unit; The first image data is sent to the target application as image data collected by the electronic device, so that the first image data is displayed in the target interface of the target application; After obtaining the target image data, the process also includes: The target image data is configured as the second image data acquired by the second image acquisition unit; The second image data acquired by the second image acquisition unit is sent to the target application as image data acquired by the electronic device, so that the target image data is displayed in the target interface of the target application.

4. The data processing method according to claim 1, based on the task processing result, determines the target image data, including at least one of the following: If the task processing result includes video data, determine the target image data that includes the video data; If the task processing result includes text information, generate target image data containing the text information; If the task processing result includes an image, generate target image data including the image.

5. The data processing method according to claim 4 further includes at least one of the following: If the task processing result includes audio data, the audio data is processed as audio to be output by the audio output unit in the electronic device, and the processed audio data is output through the audio output unit. If the task processing result includes audio data and the electronic device has established a conference session with at least one target device, the audio data is transmitted to the target device as the audio data collected in the conference session. If the task processing result includes text information, the text information is converted into target audio data, the target audio data is processed as the audio to be output by the audio output unit, and the processed target audio data is output through the audio output unit. If the task processing result includes text information and the electronic device has established a conference session with at least one target device, the text information is converted into target audio data, and the target audio data is transmitted to the target device as audio data collected in the conference session.

6. The data processing method according to claim 4 or 5, wherein if the task processing result includes video data, determining the target image data including the video data includes: If the task processing result includes video data, the video data is determined to be target image data; Alternatively, if the task processing result includes video data, obtain the actual image data collected by the electronic device, and sequentially overlay each frame of the actual image data onto a designated area of ​​each frame of the video data, and determine the overlaid video data as the target image data. If the task processing result includes text information, generating target image data containing the text information includes: If the task processing result includes text information, generate target image data containing the virtual character and the text information.

7. The data processing method according to claim 1, wherein determining the task instruction based on the collected user interaction data includes: If user voice is captured and the user voice includes the target statement, the task instruction is determined based on the voice segment in the user voice that is located after the target statement; And / or, the data processing method further includes: If the user interaction data is user voice including the target statement, generate transitional image data to represent the current state of voice command acquisition; Before the task instruction is determined, the transitional image data is displayed on the target interface as image data acquired by the electronic device.

8. The data processing method according to claim 1, wherein determining the task processing result corresponding to the task instruction based on the task instruction includes at least one of the following: In response to the task instruction belonging to the historical task instruction of the target model, the task processing result obtained by the historical processing of the task instruction by the target model is obtained; In response to the task instruction being an image generation instruction, the target model is used to call the image generation model to process the image generation instruction and obtain the generated image; In response to the task instruction being a video generation instruction, the target model is used to call the video generation model to process the video generation instruction and obtain the generated video data; In response to the task instruction being a question-answering instruction, the target model is used to perform task reasoning based on the question-answering instruction to obtain the reasoned text information.

9. An electronic device, comprising: The data collection module is used to collect user interaction data; The processor is used to determine task instructions based on the collected user interaction data; Based on the task instruction, determine the task processing result corresponding to the task instruction; Based on the task processing results, the target image data is determined; The target image data can be displayed on the target interface as image data collected by the electronic device; The display module is used to display the target interface.

10. The electronic device according to claim 9, further comprising: Audio output unit and image acquisition unit; The acquisition module includes: an audio acquisition unit; The image acquisition unit is used to acquire first image data; The audio acquisition unit is used to acquire user voice; The processor is also used for: If the audio acquisition unit acquires the user's voice and the user's voice includes a target statement, the task instruction is determined based on the voice segment in the user's voice that follows the target statement; If the task processing result includes audio data, the audio data is processed as audio to be output by the audio output unit in the electronic device, and the processed audio data is sent to the audio output unit. The audio output unit is used to output the processed audio data.

Images