Data processing method and data processing device

Abstract

The invention provides a data processing method and a data processing device.The method comprises the steps that in response to a first processing instruction for first input data, the first input data are processed into second input data, and the first input data are data input to a target application; generating third input data based on the obtained adjustment operation for the second input data; and generating a target file matched with the user intention based on the user intention represented by the third input data.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a data processing method and a data processing device. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence models have also been widely used in fields such as text recognition and image processing. However, artificial intelligence models currently have high requirements for users. Non-professional users who do not have certain professional knowledge find it difficult to use artificial intelligence models to generate accurate content, resulting in a poor user experience. Summary of the invention

[0003] The present disclosure provides a data processing method and a data processing device.

[0004] According to a first aspect of the present disclosure, there is provided a data processing method, comprising:

[0005] In response to obtaining a first processing instruction for first input data, processing the first input data into second input data, the first input data being data input to a target application;

[0006] generating third input data based on the obtained adjustment operation for the second input data;

[0007] A target file matching the user intention is generated based on the user intention represented by the third input data.

[0008] In an implementation of the present application, obtaining a first processing instruction for the first input data includes at least one of the following:

[0009] generating the first processing instruction in response to obtaining a trigger operation of a first type of control acting on a first type of application, wherein the first type of application is an application capable of responding to the first input data;

[0010] generating the first processing instruction in response to obtaining a trigger operation for first input data input to a second type of application, the second type of application being an application that can respond to or cannot respond to the first input data;

[0011] In response to obtaining first input data input to a third type of application, generating a corresponding first processing instruction based on application information of the third type of application;

[0012] A corresponding first processing instruction is generated based on the type of the first input data and / or application information of a source application.

[0013] In an implementation of the present application, the first input data is processed into the second input data, including at least one of the following:

[0014] Processing the first input data into the second input data using a processing model matching the first processing instruction;

[0015] processing the first input data into fourth input data based on the type of the first input data, and performing generation processing on first target data in the fourth input data to obtain the second input data;

[0016] The first input data input to the target application and its context data are integrated to obtain the second input data.

[0017] In an implementation of the present application, generating the third input data based on the obtained adjustment operation for the second input data includes at least one of the following:

[0018] generating third input data based on an obtained selection operation of recommended data for second target data in the second input data and / or an edit operation for the second target data;

[0019] generating third input data based on the configuration operation obtained for the configuration option in the configuration window of the second input data, wherein the configuration option is used to update the target data in the second input data;

[0020] Context data associated with the second input data is obtained, and the second input data is updated using the context data to generate third input data.

[0021] In an implementation of the present application, the third input data is generated based on the obtained selection operation of the recommended data for the second target data in the second input data and / or the editing operation for the second target data, including at least one of the following:

[0022] In response to obtaining a trigger operation for a target keyword in the second input data, outputting a recommendation word for the target keyword, replacing the target keyword with a recommendation word determined by a selection operation for the target recommendation word, and obtaining the third input data;

[0023] In response to obtaining an editing operation for a target keyword in the second input data, generating the third input data based on the edited keyword;

[0024] In response to obtaining a trigger operation for a target keyword in the second input data, an output effect of a recommendation word for the target keyword is output, and the target keyword is replaced with a recommendation word determined by a selection operation on the target output effect to obtain the third input data.

[0025] In an implementation of the present application, generating the third input data based on the obtained adjustment operation for the second input data includes at least one of the following:

[0026] In the case where the second input data is image data, in response to obtaining a trigger operation for a target object in the image data, outputting a recommended object for the target object, and replacing the target object with a recommended object determined by a selection operation for the target recommended object, to obtain the third input data;

[0027] When the second input data is audio data, in response to obtaining a trigger operation for a target segment in the audio data, outputting a recommended segment for the target segment, and replacing the target segment with a recommended segment determined by a selection operation for the target recommended segment, to obtain the third input data;

[0028] In the case where the second input data is video data, in response to obtaining a trigger operation for a target image frame in the video data, a recommended image frame for the target image frame is output, and the target image frame is replaced with the recommended image frame determined by the selection operation on the target recommended image frame to obtain the third input data.

[0029] In an implementation of the present application, generating the third input data based on the configuration operation of the configuration option in the configuration window for the second input data includes:

[0030] displaying a configuration window for the second input data, wherein the configuration window displays configuration options corresponding to the target keyword in the second input data;

[0031] In response to obtaining a triggering operation on a target configuration option, displaying a recommended option of the target configuration option;

[0032] The target keyword in the second input data is replaced by the recommended option determined by the selection operation on the target recommended option to obtain the third input data.

[0033] In an implementation of the present application, generating a target file matching the user intent based on the user intent represented by the third input data includes:

[0034] generating a second processing instruction based on the attribute information of the target application and / or the user intention represented by the third input data;

[0035] The third input data is generated and processed using a processing model or a model service corresponding to the second processing instruction to obtain a target file that matches the attribute information and the user's intention.

[0036] In an implementation of the present application, the step of generating a target file matching the user intent based on the user intent represented by the third input data further includes at least one of the following:

[0037] Obtaining user portrait information of a target user, and generating and processing the third input data with reference to the user portrait information to obtain a target file matching the user's intention;

[0038] The current environment information of the electronic device is obtained, and the third input data is generated and processed with reference to the environment information to obtain a target file that matches the user's intention.

[0039] According to a second aspect of the present disclosure, there is provided a data processing device, comprising:

[0040] a processing module, configured to, in response to obtaining a first processing instruction for first input data, process the first input data into second input data, wherein the first input data is data input to a target application;

[0041] A first generating module, configured to generate third input data based on the obtained adjustment operation for the second input data;

[0042] The second generating module is used to generate a target file matching the user intention based on the user intention represented by the third input data.

[0043] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0044] at least one processor; and

[0045] a memory communicatively coupled to the at least one processor;

[0046] An image acquisition device; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the present disclosure.

[0047] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0049] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0050] Figure 1 A schematic diagram of an implementation flow of a data processing method provided in an embodiment of the present application is shown;

[0051] Figure 2 A user interface of an image generation application provided by an embodiment of the present disclosure is shown;

[0052] Figure 3 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown;

[0053] Figure 4 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown;

[0054] Figure 5 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown;

[0055] Figure 6 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown;

[0056] Figure 7 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown;

[0057] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0059] At present, due to the high requirements of artificial intelligence models on users, it is difficult for non-professional users who do not have certain professional knowledge to use artificial intelligence models to generate accurate content, resulting in poor user experience. Therefore, in order to improve the user experience, the present application provides a data processing method and a data processing device. The electronic device provided by the present application can be a mobile phone, a computer, a tablet computer and other devices. The data processing method provided by the present disclosure can be applied in scenarios including text generation, text analysis, image generation, image processing, video generation, and video analysis.

[0060] The technical solution of the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application.

[0061] Figure 1 A schematic diagram of an implementation flow of the data processing method provided in an embodiment of the present application is shown. Figure 1 As shown, the method includes:

[0062] S101 : In response to obtaining a first processing instruction for first input data, processing the first input data into second input data.

[0063] The first input data is data input to the target application. In the present disclosure, the target application may include an image generation application (such as Lenovo's creator zone), a text generation application, a video generation application, or an intelligent agent application (such as Lenovo's AI agent Xiaotian / AI Now). Among them, the intelligent agent application refers to an intelligent assistant based on artificial intelligence technology, such as an intelligent voice assistant, a device intelligent assistant, etc. The target application may also include applications that can call or integrate image generation applications, text generation applications, video generation applications, and / or intelligent agent applications, such as social media applications or album applications, which can process the input data by calling image generation applications, text generation applications, video generation applications, and / or intelligent agent applications, or, the processing of the input data can be realized through a built-in large model (such as a raw picture model, a raw text model, a raw video model, a raw audio model, a raw UI model, etc.). For example, the target application is a social media application, which has built-in text generation models and image generation models to support article generation and / or image generation functions. Users can choose to create articles in the social media application by providing keywords, key words or scene description information. The text generation model can generate text that meets the user's intention by identifying and extracting features of keywords, key words or scene description information. The image generation model can generate images that meet the user's intention by identifying and extracting features of keywords, key words or scene description information. The text and image are combined to obtain an article that meets the user's intention.

[0064] In the present disclosure, the first input data may include character string data, multimedia data, voice data, etc. The character string data includes text and numbers, etc., and the multimedia data includes images and videos, etc.

[0065] In the present disclosure, the first input data is processed into the second input data based on the first processing instruction, and the first input data can be processed into the second input data by a corresponding processing method according to the actual application scenario and the type of the first input data provided by the user. For example, the first input data can be processed by a text generation model, a picture-to-text conversion model, a speech-to-text model, an image generation model, or an audio generation model to obtain the second input data. For example, in the application scenario of image generation, if the first input data is text data of the features of the image to be generated described by the user, the first processing instruction can be an instruction to instruct the image generation model to perform prompt word expansion on the first input data, and the image generation model performs prompt word expansion processing on the first input data to obtain the second input data.

[0066] In the present disclosure, the first processing instruction may refer to an instruction for extending the prompt word of the first input data, or an instruction for format conversion processing of the input data. For example, when the first input data is image data, the first processing instruction may be an instruction for converting the image data into text data, so as to process the first input data into a second input data in text format. For example, in the application scenario of image generation, the first input data may be text data of the features of the image to be generated described by the user. In order to make the generated image more accurate, the features of the image to be generated may be enhanced by providing prompt words to the user. Then the first processing instruction may be an instruction for expanding the prompt words to enhance the features of the image to be generated. For example, if the first input data is the text data "generate an image of a puppy" provided by the user, the features of the image to be generated may include the generated object "puppy" according to the text data "generate an image of a puppy", but the feature is relatively broad and difficult to accurately reflect the user's intention. Then the first processing instruction may be an instruction for extending the prompt words of the text data "generate an image of a puppy", and the extended prompt words include but are not limited to prompt words representing the type of puppy, prompt words representing the color of the puppy, and prompt words representing the scene where the puppy is located. For another example, if the first input data is voice data provided by the user "generate an image of a puppy", the corresponding first processing instruction may only convert the voice data into text data "generate an image of a puppy", or may include an instruction to expand the prompt word of the text data "generate an image of a puppy". For another example, if the first input data is an image of a cat provided by the user, the first processing instruction in the present disclosure may include extracting text data representing the features of the image, such as extracting the text data "playing kittens", and an instruction to expand the prompt word of the text data "playing kittens", and the prompt words that can be expanded include but are not limited to prompt words representing the type of kittens, prompt words representing the color of kittens, prompt words representing the number of kittens, and prompt words representing the scene where the kittens are located.

[0067] In the present disclosure, the type of the first input data is different from the type of the second input data. For example, in the scenario of image generation, the first input data may be voice data or image data, and the second input data may be text data obtained based on the first input data that can reflect the features of the image to be generated.

[0068] In the present disclosure, the data volume of the second input data may be the same as or different from the data volume of the first input data. For example, if the first input data is voice data or image data, and the second input data is text data obtained based on the first input data, then the data volume of the second input data is less than or equal to the data volume of the first input data; if the first input data is text data, and the second input data is voice data or image data obtained based on the first input data, then the data volume of the second input data is greater than the data volume of the first input data.

[0069] S102: Generate third input data based on the obtained adjustment operation on the second input data.

[0070] In the present disclosure, the adjustment operation on the second input data may be an adjustment operation on all the second input data, or may be an adjustment operation on the target data in the second input data. The target data of the second input data refers to key feature data that can characterize the second input data, such as keywords or key words. For example, if the second input data is text data "Please generate a puppy", the keyword in the text data is the image generation object "puppy".

[0071] In the present disclosure, the adjustment operation for the second input data can correspond to the type of the second input data. If the second input data is text data, the adjustment operation for the second input data includes but is not limited to: a change operation, a replacement operation, a deletion operation and / or an addition operation on the text data, a display parameter adjustment operation, an expansion operation and / or a replacement operation on keywords or keywords in the text data. Among them, the display parameter adjustment operation includes an adjustment operation on parameters such as the font, color, font size, handwriting and / or style of the text data. For example, in an image generation scenario, the second input data is the text data "please generate a puppy", and the adjustment operation for the text data "please generate a puppy" can be an expansion operation on the keyword "puppy" of the text data "please generate a puppy", and the type, color and quantity of the puppy can be expanded. For example, the color of the puppy can be expanded, and the expanded text data is obtained as "please generate a black puppy", and the expanded text data "please generate a black puppy" can be used as the third input data.

[0072] If the second input data is audio data, the adjustment operation for the second input data includes but is not limited to: adjusting volume, adjusting pitch and / or adjusting timbre.

[0073] If the second input data is image data, the adjustment operation for the second input data includes but is not limited to: a smearing operation, a marking operation, an element replacement operation and / or an image parameter adjustment operation, wherein the image parameter adjustment operation includes an image contrast adjustment operation, an image brightness adjustment operation and an image saturation adjustment operation.

[0074] S103: Generate a target file matching the user intention based on the user intention represented by the third input data.

[0075] In the present disclosure, the user intent represented by the third input data can be identified by the intent recognition model, and then a text file, image file, video file or multimodal multimedia data file that meets the user intent can be generated. For example, in the image generation scenario, the third input data is text data "please generate a black puppy" that can contain the user intent. The text data "please generate a black puppy" is input into the intent recognition model (intent understanding model). The intent recognition model can perform reasoning and recognize the user's intent "generate an image of a black puppy", and the image generation model can generate an image containing a black puppy according to the user's intent.

[0076] In the present disclosure, a second processing instruction input to the target application may also be monitored, the second processing instruction being an instruction for the third input data, and the third input data is processed based on the second processing instruction to generate a corresponding target file. The second processing instruction may be a voice file generation instruction, an image file generation instruction, or a video file generation instruction, etc.

[0077] With this method, the first input data is processed into the second input data input to the target application through the first processing instruction, and then the third input data is generated according to the adjustment operation on the second input data, so as to generate the target file matching the user intention by using the user intention represented by the third input data. This method can guide the user to express the user intention more accurately, or be used for the user to adjust to obtain the third input data that is more in line with the intention, so as to generate the target file that accurately matches the user intention, thereby improving the user experience.

[0078] In a possible implementation manner, obtaining a first processing instruction for the first input data includes at least one of the following methods a1 to a4:

[0079] Mode a1: In response to obtaining a trigger operation of a first type of control acting on a first type of application, generating the first processing instruction, the first type of application being an application capable of responding to the first input data.

[0080] In the present disclosure, the first category of applications may include intelligent applications, image generation applications, text generation applications, or video generation applications, etc. Among them, intelligent applications may include Lenovo Xiaotian or AI NOW, image generation applications may include Lenovo's creator zone application, text generation applications may include Lenovo's AI learning application, and video generation applications may include sora (an artificial intelligence Wensheng video large model released by OpenAI, an American artificial intelligence research company) application. The first category of applications can implement functions such as image generation, text generation, video generation, audio generation, AI image processing, image beautification, or device control in response to the first input data.

[0081] In the present disclosure, the triggering operation of the first type of control acting on the first type of application may be a click operation, a drag operation, or a double-click operation, etc. The first processing instruction may be a text generation instruction, a text polishing processing instruction, a text feature enhancement instruction, a picture-text conversion processing instruction, a voice-text conversion processing instruction, or a processing instruction combining a picture-text conversion processing instruction and a voice-text conversion processing instruction, etc.

[0082] In the present disclosure, the first type of control may include a functional control capable of expanding a prompt word of the first input data. Figure 2 A user interface of an image generation application provided by an embodiment of the present disclosure is shown. Figure 2 As shown, the user interface 200 includes a drawing area 201 and a text input area 202, wherein the text input area 202 is used for the user to input text data, and the drawing area 201 is used to display an image drawn based on the text data input by the user, and the text input area 202 includes an "enhanced speech" function option 203 and a command "send" option 204, wherein the "enhanced speech" function option 203 is a function control that can trigger the extension of the prompt word. Figure 2 As shown, the user can move the cursor 205 to the enhanced speech "enhanced speech" function option 203, and start the "enhanced speech" function option 203 by clicking the enhanced speech. Figure 2 As shown, the user can enter the text data "Help me draw a dog" in the text input area 202 and then click the enhanced speech function option 203. After the image generation application detects that the user clicks the enhanced speech function option 203, it can generate a first processing instruction indicating the expansion of the prompt word.

[0083] Mode a2: In response to obtaining a trigger operation for first input data input to the second type of application, generating the first processing instruction.

[0084] In the present disclosure, the second category of applications is an application that can respond to or cannot respond to the first input data. The second category of applications may be the same as or different from the first category of applications. For example, the second category of applications may be an image generation application or a text generation application. The second category of applications may also be a social application or a conference application, such as WeChat and Teams. If the second category of applications is a social application or a conference application, in the present disclosure, a text generation model or an image generation model may be called based on the input data to process the input data in a user chat or a conference.

[0085] In the present disclosure, the triggering operation for the first input data input to the second type of application may include a copy and paste operation, a screenshot operation, a circle operation, a marking operation or an editing operation, etc. For example, the second type of application is a chat software, and the chat software can call the image generation model. The user takes a screenshot of the text data input into the chat software. After the chat software detects the user's screenshot operation, it can obtain the captured image and extract the text data in the captured image, call the image generation model according to the text data, and the image generation model generates an instruction for expanding the prompt word of the text data as the first processing instruction.

[0086] Mode a3: In response to obtaining first input data input to a third type of application, generating a corresponding first processing instruction based on application information of the third type of application.

[0087] In the present disclosure, the third category of applications may be the same as or different from the first category of applications. The application information of the third category of applications may include an application type. In the present disclosure, a first processing instruction for the first input data may be generated based on the application type of the third category of applications. For example, if the third category of applications is an image generation application and the first input data is text data, a first processing instruction for the first input data may be generated according to an image generation rule, for example, a processing instruction for the expansion of a prompt word may be generated to expand the image feature information corresponding to the first input data, so as to facilitate the generation of a more accurate image. If the third category of applications is a text generation application and the first input data is text data, a first processing instruction for the first input data may be generated according to a text generation rule, for example, a first processing instruction for text expansion, text abbreviation or text summary extraction may be generated. If the third category of applications is an audio generation application and the first input data is text data, a first processing instruction for the first input data may be generated according to an audio generation rule, for example, a first processing instruction for increasing the volume, changing the pitch or changing the timbre may be generated.

[0088] Mode a4: Generate a corresponding first processing instruction based on the type of the first input data and / or application information of the source application.

[0089] In the present disclosure, the type of the first input data may include text, image, audio, video, etc. The source application refers to the application into which the first input data is input, for example, if the first input data is input into an image generation application, the source application is the image generation application.

[0090] In the present disclosure, a first processing instruction for the first input data may be generated according to the type of the first input data. For example, if the first input data is text data, a text enhancement instruction may be generated as the first processing instruction, and if the first input data is audio data, a volume amplification instruction may be generated as the first processing instruction.

[0091] In the present disclosure, the first processing instruction may also be generated according to the application type or function type of the source application. The application type of the source application may include an image generation type, an audio generation type, a text generation type, etc., and the function type of the source application may include a parameter adjustment function, a drawing function, and a text-to-speech function, etc. For example, if the first input data is text data and the function type of the source application is a text-to-speech function, a text-to-speech instruction may be generated as the first processing instruction. If the first input data is audio data and the function type of the source application is a parameter adjustment function, a volume amplification instruction may be generated as the first processing instruction.

[0092] In the present disclosure, a first processing instruction can also be generated based on the type of the first input data and the application type or function type of the source application. For example, if the first input data is image data and the application type of the source application is a text generation application, a first processing instruction for image-to-text conversion and text summary extraction can be generated for the first input data.

[0093] In a possible implementation manner, processing the first input data into second input data includes at least one of the following steps B1 to B3, where steps B1 to B3 are independent steps:

[0094] Step B1: Process the first input data into the second input data using a processing model that matches the first processing instruction.

[0095] In the present disclosure, the processing model that matches the first processing instruction may be an image generation model, a text generation model, or a video generation model, etc. For example, if the first processing instruction is an instruction generated by a trigger operation of a prompt word expansion function control in an image generation application, then the processing model that matches the first processing instruction is an image generation model, that is, the prompt word expansion model in the image generation application can be called to perform word expansion processing on the first input data. If the first processing instruction is an instruction generated by a trigger operation of a text expansion function control in a text generation application, then the processing model that matches the first processing instruction is a text generation model, that is, the text expansion model in the text generation application can be called to perform expansion processing on the first input data.

[0096] Step B2: Processing the first input data into fourth input data based on the type of the first input data, and generating the first target data in the fourth input data to obtain the second input data.

[0097] In the present disclosure, different types of first input data may result in different fourth input data and different processing methods. For example, if the first input data is text data, semantic analysis or intent recognition may be performed on the text data to extract keywords from the text data as the fourth input data; if the first input data is image data, image feature extraction may be performed on the image data to extract elements included in the image and convert them into corresponding text data as the fourth input data; if the first input data is audio data, audio feature extraction may be performed on the audio data to convert the audio data into corresponding text data as the fourth input data.

[0098] In the present disclosure, the first target data may include all or part of the fourth input data. For example, the first target data includes keywords in the fourth input data, and the keywords may be nouns or verbs in the fourth input data. For example, if the fourth input data is "generate a black puppy", the first target data may include the entire "generate a black puppy" of the fourth input data, and the first target data may also include the keywords "generate" and "puppy" of the fourth input data. Then, the prompt word expansion model or the raw text model in the raw image model may be used to generate the keywords "generate" and "puppy" to obtain the second input data "generate a small yellow dog running and jumping in the garden in the sun".

[0099] Step B3: Integrate the first input data input to the target application and its context data to obtain the second input data.

[0100] In the present disclosure, the first input data may be text data captured from an article or news, and the context data of the first input data is text data associated with a scene in the first input data. Therefore, the scene features of the first input data may be supplemented in combination with the context data of the first input data to generate the second input data. For example, in a short article, there is a descriptive text "On a sunny afternoon, a puppy was lying on the ground playing". If the text data "a puppy" in the short article is captured as the first input data, the scene feature "bright afternoon" and the action feature "lying on the ground" of "a puppy" may be obtained in combination with the context data of "a puppy". The scene feature "bright afternoon" and the action feature "lying on the ground" may be combined with the first input data "a puppy" to generate the second input data "a puppy lying on the ground".

[0101] In a possible implementation manner, generating the third input data based on the obtained adjustment operation on the second input data may include at least one of the following steps C1 to C3, where steps C1 to C3 are independent steps:

[0102] Step C1: generating third input data based on an obtained selection operation of recommended data for second target data in the second input data and / or an edit operation for the second target data.

[0103] In the present disclosure, the second target data may be all or part of the second input data. For example, the second target data may be keywords such as nouns, adjectives of nouns, and verbs in the second input data. The recommended data of the second target data includes other text data consistent with the type of the second target data. For example, if the second target data is text data used to describe color, the recommended data of the second target data is text data describing other colors different from the second target data. If the second target data is text data used to represent quantity, the recommended data of the second target data is text data representing a different quantity from the second target data. The selection operation of the recommended data of the second target data in the second input data may include selecting the recommended data of the second target data to replace the second target data or changing the second target data to the recommended data of the second target data, and obtaining new text data as the third input data. The editing operation of the second target data means that the user can directly edit the second target data to generate new text data as the third input data.

[0104] For example, Figure 2 In the user interface 200 shown, if the user enters the text data "help me draw a dog" in the text input area 202 and then clicks the enhanced speech function option 203, the image generation application may display the following after detecting that the user clicks the enhanced speech function option 203. Figure 3The interface shown. Figure 3 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown. Figure 3 As shown, the interface 300 displayed by the image generation application after the user clicks the enhanced speech function option 203 may include a drawing area 201, a text display area 301, an enhanced speech function option 302 and a send function option 303, wherein the drawing area 201 is used to display the generated image, the text display area 301 is used to display the text data input by the user and the text after the enhanced speech, and after the user clicks the send function option 303, the image generation application can generate an image according to the text currently displayed in the text display area 301. Figure 3 As shown, after the user inputs the text data "help me draw a dog" and clicks on the enhanced speech function, the image generation application can perform prompt word expansion processing on the text data "help me draw a dog", and then display the text data after the enhanced speech "a long-haired golden retriever, lying on the ground, the golden hair sparkles in the eyes, using a realistic style, a frontal perspective, and a 4K resolution" in the text display area 301, and use the text data after the enhanced speech as the second input data. At this time, the user can also directly click on the extended prompt words such as "golden retriever" and "lying on the ground" in the text data after the enhanced speech. After the user clicks on the extended prompt word, an interface 310 can be displayed, and the label information display area 304 is displayed in the interface 310. The user label information display area 304 is displayed. The labels in the display area 304 can be used to replace some features of the text data after the enhancement of the text display area 301, "a long-haired golden retriever, lying on the ground, the golden hair sparkles in the eyes, using a realistic style, a frontal perspective, and a 4K resolution". For example, "Husky" can be used to replace "golden retriever" so that the generated image is an image of a Husky. Alternatively, the user can directly edit the extended prompt words such as "golden retriever" and "lying on the ground", and can directly change "golden retriever" to "Husky" and / or change "lying on the ground" to "squatting on the ground", etc. After the user finishes editing, the text data displayed in the text display area 301 is the third input data, and the image generation model can generate an image based on the third input data. Figure 3As shown, the image generation application performs prompt word expansion processing on the text data "help me draw a dog" to obtain the enhanced text data "a long-haired golden retriever, lying on the ground, the golden hair sparkles in the eyes, using a realistic style, a frontal perspective, and a 4K resolution". In this text, "golden retriever", "lying on the ground", "realistic style", "frontal perspective" and "4K resolution" are key prompt words that characterize the type, action, drawing style, drawing perspective and clarity of the drawing object. These key prompt words can be displayed in the text display area 301 by highlighting, bolding or underlining, and the user can directly click or double-click these key prompt words to edit them. Specifically, after the user directly clicks or double-clicks these key prompt words, a text editing box can be displayed, and the user can enter a prompt keyword that meets his or her intention in the text editing box to replace the current prompt keyword. The text editing box can be displayed on the text display area 301; or, after the user clicks or double-clicks these key prompt words, one or more tags that can replace the clicked key prompt words can be displayed in the user tag information display area 304, such as Figure 3 The tags corresponding to the key prompt word "golden retriever" shown include "husky", "Samoyed" and "Alaska", etc. The user can replace the key prompt word by clicking on the tag that meets his or her intention.

[0105] By adopting the data processing method provided by the present invention, the user can be guided to express the user's intention more accurately or be used for user adjustment to obtain third input data that is more in line with the intention through enhanced speech function, thereby generating a target file that accurately matches the user's intention. In addition, the user can also quickly switch prompt words that meet their own intentions through multiple tags corresponding to the prompt words, thereby realizing rapid expansion of the text, thereby quickly generating a target file that accurately matches the user's intention, thereby improving the user experience.

[0106] In a possible implementation manner, generating the third input data based on the obtained selection operation of the recommended data for the second target data in the second input data and / or the editing operation for the second target data includes at least one of the following steps D1 to D3, where steps D1 to D3 are independent steps:

[0107] Step D1, in response to obtaining a trigger operation for a target keyword in the second input data, output a recommendation word for the target keyword, replace the target keyword with a recommendation word determined by a selection operation for the target recommendation word, and obtain the third input data.

[0108] In the present disclosure, the triggering operation for the target keyword in the second input data may include operations such as circling the target keyword, voice triggering the target keyword, or moving the cursor to the location of the target keyword. After the target keyword is triggered, the recommended word for the target keyword can be displayed in a pop-up window or voice broadcast. The recommended word can be other words that are consistent with the part of speech and word class of the target keyword. The recommended word can be used to replace the target keyword. The recommended word may include one or more. If there is only one recommended word, the user can replace the target keyword with the keyword after selecting the keyword. If the recommended word includes multiple recommended words, the user can select one of the recommended words to replace the target keyword. If the user has not selected any recommended word, the target keyword can be replaced by the user inputting the recommended word. If the user has not selected any recommended word and the user has not input the recommended word, it can be determined that the current target keyword will not be changed. For example, Figure 3 As shown, the user can trigger the target keyword "Golden Retriever" by moving the cursor to the location of the target keyword "Golden Retriever", so that the interface pops up multiple tags "Husky", "Samoyed" and "Alaska" in the user tag information display area 304. The multiple tags that pop up are recommended words. The user can choose a recommended word that meets his or her intention from the recommended words "Husky", "Samoyed" and "Alaska" to replace "Golden Retriever". The user may not choose to enter other recommended words that represent the type of dog, such as "Chinese Rural Dog", and replace "Golden Retriever" with "Chinese Rural Dog". Alternatively, the user may not select and enter a recommended word and retain the original target keyword "Golden Retriever".

[0109] Step D2, in response to obtaining an editing operation on the target keyword in the second input data, generating the third input data based on the edited keyword.

[0110] In the present disclosure, the editing operation for the target keyword in the second input data may include modifying the target keyword. Figure 3 As shown, if there is no recommended word that meets the user's intention among the multiple recommended words "Husky", "Samoyed" and "Alaska" that pop up in the user tag information display area 304, and the user is not satisfied with the target keyword "Golden Retriever", the user can double-click the target keyword or use voice input to make the image generation application pop up a text input box, and the user can directly enter a recommended word that meets his or her intention in the text input box to replace the target keyword "Golden Retriever".

[0111] Step D3, in response to obtaining a trigger operation for the target keyword in the second input data, output an output effect of a recommendation word for the target keyword, replace the target keyword with the recommendation word determined by the selection operation of the target output effect, and obtain the third input data.

[0112] In the present disclosure, since the text user may not be able to understand or is not very familiar with the specific objects represented by the text, for example, the recommended words are "Husky", "Samoyed" and "Alaska", and the user may not understand the appearance of "Husky", "Samoyed" and "Alaska". Therefore, after the target keyword is triggered, the output effect of the recommended word for the target keyword can be output. The output effect may include an image or voice introduction information corresponding to the recommended word for the target keyword. For example, if the recommended words are "Husky", "Samoyed" and "Alaska", images of the recommended words "Husky", "Samoyed" and "Alaska" for the target keywords can be output, so that the user can intuitively understand the meaning of the recommended word representation and facilitate user selection.

[0113] In the present disclosure, the image generation application can call an external database or an internal database, and the database stores image information and / or voice introduction information corresponding to various recommendation words. After the target keyword is triggered, the recommendation word corresponding to the target keyword can be determined, and the image information and / or voice introduction information corresponding to each recommendation word can be retrieved from the database as an output effect for output. The user can select a recommendation word that better meets his or her intention to replace the target keyword according to the output effect, thereby obtaining the replaced third input data.

[0114] Step C2: generating third input data based on the configuration operation obtained for the configuration options in the configuration window of the second input data, wherein the configuration options are used to update the target data in the second input data.

[0115] In the present disclosure, the configuration options may be options for configuring all or target data of the second input data. The target data of the second input data refers to keywords such as nouns and verbs of the second input data. Part of the text data in the second input data may be updated by providing replacement words with consistent parts of speech for the keywords to generate third input data. If the second input data is video data, the target data may also be key image frames in the video data. If the second input data is image data, the target data may also be target objects in the image data. If the second input data is audio data, the target data may also be key audio clips in the audio data.

[0116] For example, Figure 2 In the user interface 200 shown, if the user enters the text data "help me draw a dog" in the text input area 202 and then clicks the enhanced speech function option 203, the image generation application may display the following after detecting that the user clicks the enhanced speech function option 203. Figure 4 The interface shown. Figure 4 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown. Figure 4As shown, the interface 400 displayed after the image generation application detects that the user clicks on the enhanced speech function option 203 may include a drawing area 201, an original speech area 401, an enhanced speech area 402, a label information area 403, a label selection function option 404, a label area sliding button 405, a cancel function option 406 and a confirmation function option 407, wherein the drawing area is used to display the generated image, the original speech area 401 is used to display the text data input by the user, the label information area 403 is used to display the prompt words that can be selected to enhance the characteristics of the text data, the label selection function option 404 is used to select a specific prompt word, the user slides the label area sliding button 405 to display the label information hidden in the current interface, and the enhanced speech area 402 is used to display the text data generated based on the label information selected by the user and the text data input by the user. If the user clicks the cancel function option 406, the enhanced speech function will be exited, and if the user clicks the confirmation function option 407, the image will be drawn based on the text data in the currently generated enhanced speech area. Figure 4 As shown, after the user inputs the text data "Help me draw a dog" and clicks on the enhanced speech function, prompt words can be displayed in the label information area 403. The prompt words may include information such as the type of drawing object, drawing scene, drawing style, drawing perspective, and clarity of the drawing image. For example, the user selects the type of drawing object as golden retriever, the drawing scene as lying on the ground, the drawing style as realistic style, the drawing perspective as front perspective, and the clarity of the drawing image as 4K resolution in the displayed prompt words. According to the information selected by the user, the image generation application can generate enhanced text data "A long-haired golden retriever, lying on the ground, the golden hair sparkles in the eyes, using realistic style, front perspective, 4K resolution" as the third input data, and display the third input data in the enhanced speech area 402. When the user confirms that the enhanced speech is correct, he can click the confirmation function option 407 to display the following. Figure 5 The interface shown.

[0117] Figure 5 Another user interface of the image generation application provided by the embodiment of the present disclosure is shown. Figure 5 As shown, when the user confirms that the enhanced speech is correct and clicks the OK function option, the interface 500 can be displayed. The interface 500 may include the drawing area 201, the enhanced speech display area 501, the enhanced speech function option 502 and the send function option 503. The enhanced speech display area 501 displays the text data obtained after the previous enhanced speech, "a long-haired golden retriever, lying on the ground, the golden hair sparkles in the eyes, using a realistic style, frontal perspective, 4K resolution", that is, the enhanced speech display area 501 can display the text data generated by the image generation application after optimizing the text input by the user. Figure 5As shown, if the user clicks on the enhanced speech function option 502, the text enhancement can be further performed based on the text data "a long-haired golden retriever, lying on the ground, the golden hair sparkles in the light, using a realistic style, a frontal perspective, and a 4K resolution". If the user clicks on the send function option 503, the image generation model can generate an image based on the enhanced text data "a long-haired golden retriever, lying on the ground, the golden hair sparkles in the light, using a realistic style, a frontal perspective, and a 4K resolution".

[0118] In one possible implementation, generating the third input data based on the configuration operation obtained for the configuration option in the configuration window for the second input data may include steps E1 to E3:

[0119] Step E1, displaying a configuration window for the second input data, wherein the configuration window displays configuration options corresponding to target keywords in the second input data.

[0120] In the present disclosure, the configuration window may be a window for the target keyword of the second input data, and one or more configuration options for replacing the target keyword may be displayed in the configuration window. Figure 4 As shown, the configuration window may be a window 403 displaying tag information. The tags displayed in the configuration window are configuration options. The configuration options may include multiple configuration options for different target keywords of the second input data, such as Figure 4 As shown, the configuration options include options for representing the type of the drawing object "dog" in the second output data. The types of dog may include "golden retriever", "husky", "Alaskan" and "Samoyed", etc. The user can select a specific configuration option for representing the type of the drawing object by clicking button 404; the configuration options also include options for the drawing style of the drawing object "dog" in the second output data. The drawing style may include "realistic style" and "virtual style", etc. The user can select a specific configuration option for the drawing style by clicking button 404; the configuration options also include options for representing the action type of the drawing object "dog" in the second output data, options for the drawing perspective and options for image clarity, etc. The user can also slide button 405 in the pull-down label area to display more configuration options.

[0121] In the present disclosure, if there is no configuration option that meets the user's intention among the displayed configuration options, the user can double-click the original speech area 401 or the enhanced speech area 402 to pop up a prompt output box (text output box), and directly enter text that meets their own intention in the prompt output box to replace the target keyword. Among them, the configuration window and the prompt input box can be in the same window or not in the same window.

[0122] Step E2: in response to obtaining a triggering operation on a target configuration option, displaying recommended options for the target configuration option.

[0123] In the present disclosure, the recommended options of the target configuration options can be further displayed by clicking the target configuration options, sliding the target configuration options, dragging the target configuration options, and voice selecting the target configuration options. Figure 4 As shown, the user can select the configuration option representing the type of drawing object as the target configuration option by clicking button 404, and display one or more recommended options in the target configuration option, such as "Golden Retriever", "Husky", "Alaska" and "Samoyed", etc. The user can select the recommended option that meets his or her intention in the recommended options to replace the target keyword corresponding to the target configuration option.

[0124] Step E3: replacing the target keyword in the second input data with the recommended option determined by the selection operation on the target recommended option to obtain the third input data.

[0125] like Figure 4 As shown, if the user selects Husky from the recommended options such as "Golden Retriever", "Husky", "Alaska" and "Samoyed", then Husky is the target recommended option, and the target keyword "Golden Retriever" in the enhanced speech text can be replaced with Husky to obtain the third input data.

[0126] Step C3, obtaining context data associated with the second input data, and using the context data to update the second input data to generate third input data.

[0127] In the present disclosure, the keywords of the second input data may be adjusted according to the context data associated with the second input data to generate the third input data.

[0128] For example, if the second input data is a piece of text data generated by a chat assistant type, "Help me draw a dog", the image generation application can call other text data associated with the second input data in the chat software, such as historical conversations, etc. For example, historical conversations such as "I like black puppies", "Puppies lying down" and "Husky" can be obtained. Then, the second input data can be updated and processed in combination with the historical conversations, the user's intention can be analyzed, and a more specific third input data "Draw a black, lying Husky" can be generated.

[0129] In the present disclosure, the third input data may also be generated based on the obtained text update instruction for the second input data, where the text update instruction refers to an instruction for supplementing the text of the second input data.

[0130] For example, Figure 6Another user interface of the image generation application provided by the embodiment of the present disclosure is shown. Figure 6 As shown, the image generation application can provide the user with a text update interface 600, which includes a drawing area 601, a user input text area 603 and a label display area 604. The user 602 can input the second input data "Help me draw a dog". After the image generation application detects the second input data input by the user 602, it can identify the second input data, extract the keywords in the second input data, and display labels for the keywords. The labels for the keywords are other words that can replace the keywords and are consistent with the word class and part of speech of the keywords. The labels for the keywords can also be supplementary words for the keywords. For example, when the keyword is a noun that represents a specific object, the label for the keyword can be the type of object represented by the keyword. Keywords can include nouns, verbs, adjectives, etc. in text data. Figure 6 As shown, after the user 602 inputs the second input data “help me draw a dog”, the image generation application recognizes the second input data “help me draw a dog” and recognizes that the drawing object is a dog. Then, the label “golden retriever”, “husky”, “Alaskan” and “Samoyed” representing the type of the drawing object “dog” can be displayed for the drawing object. The user can select from the labels “golden retriever”, “husky”, “Alaskan” and “Samoyed”, as shown in FIG. Figure 6 As shown, user 602 has selected the label “golden retriever”. Based on the type of the drawing object selected by the user, the image generation application can also display corresponding labels for the image style of the second input data “help me draw a dog” for the user to select, such as Figure 6 As shown, the labels “realistic”, “comic”, “secondary” and “mecha style” representing the drawing style can be displayed. Based on the user 602 selecting the “comic” style, the image generation application can further display corresponding labels for the drawing perspective of the second input data “help me draw a dog”, such as Figure 6 As shown, labels representing the drawing perspectives of “front”, “side”, “top view” and “back view” can be displayed. After the user selects the “front” drawing perspective, the image generation application generates text data “Okay, I’m going to give you a picture of a long-haired golden retriever lying on the ground, with its golden hair shining in the sun, in a cartoon-style front perspective, with 4K resolution” based on the user’s selected labels “golden retriever”, “cartoon” and “front” as the third input data.

[0131] like Figure 6 As shown, the image generation application can guide the user to expand the features of the second input data in the form of a dialogue with the user. The number of expansions of the image generation application is generally not limited, such as Figure 6As shown, after the user selects the "front" drawing perspective, the image generation application can also display corresponding labels for the image clarity of the second input data "Help me draw a dog" for the user to choose, such as displaying "4K resolution" and "2K resolution" labels.

[0132] By adopting the data processing method provided by the present disclosure, prompt words can be displayed in a way of guiding user dialogue, guiding users to express their intentions more accurately or allowing users to make adjustments to obtain third input data that is more in line with their intentions, thereby generating a target file that accurately matches the user's intentions and improving the user experience.

[0133] In a possible implementation manner, generating the third input data based on the obtained adjustment operation on the second input data may include at least one of the following steps F1 to F3:

[0134] Step F1, when the second input data is image data, in response to obtaining a trigger operation for a target object in the image data, output a recommended object for the target object, replace the target object with the recommended object determined by the selection operation for the target recommended object, and obtain the third input data.

[0135] In the present disclosure, if the second data is image data, the target object in the image data may include people, objects, and backgrounds in the image. The user may trigger the target object by clicking on the position of the target object in the image, circling the area where the target object is located, painting the area where the target object is located, or sliding the area where the target object is located. Outputting a recommended object for the target object may include elements that can replace the target object. For example, if the target object is a person, the recommended object may be a person of a different gender, posture, and / or age from the target object; if the target object is a cup, the recommended object may be a cup of a different color and / or shape from the target object; if the recommended object is a landscape background, the recommended object may be a background of a different landscape or a background element of a different color, and the recommended object may include one or more.

[0136] In the present disclosure, when the second input data is image data, if the target object in the image is triggered, a recommendation list of recommended objects for the target object may pop up for the user to select, and the user may select the target recommended object by clicking or circling it, replacing the target object with the target recommended object to generate third input data.

[0137] Step F2, when the second input data is audio data, in response to obtaining a trigger operation for a target segment in the audio data, output a recommended segment for the target segment, and replace the target segment with the recommended segment determined by the selection operation for the target recommended segment to obtain the third input data.

[0138] In the present disclosure, if the second data is audio data, the target segment in the audio data may include segments of key objects such as key people and key objects involved in the audio. The audio data may be recognized to identify segments representing key objects. The user may trigger the target segment by clicking on the identifier of the target segment or sliding the audio data to select the target segment. The recommended segment of the target segment may include text data or voice data that can replace the target segment. For example, the target segment is voice information describing the weather conditions of a certain place, and the recommended segment of the target segment may be an introductory voice or text describing the local customs and practices of the place. If the recommended segment is text information, the text information may be converted into voice information first, and then the voice information may be used to replace the target segment to obtain the third input data; if the recommended segment is audio data, the target segment may be directly replaced with the recommended segment to obtain the third input data.

[0139] Step F3, in the case where the second input data is video data, in response to obtaining a trigger operation for a target image frame in the video data, output a recommended image frame for the target image frame, and replace the target image frame with the recommended image frame determined by the selection operation for the target recommended image frame to obtain the third input data.

[0140] In the present disclosure, if the second data is video data, the target image frame in the video data may include image frames of key objects such as key people and key objects in the video. The video data may be identified, and image frames representing key objects may be identified as target image frames. The user may trigger the target image frame by clicking on the identification of the target image frame. The recommended image frame of the target image frame may include image data that can replace one or more elements in the target image frame. For example, the target image frame is an image of a husky lying on the ground, and the recommended image frame of the target image frame may be an image frame corresponding to the information such as the breed, category, posture and / or quantity of the object in the target image frame after changing it. For example, the recommended image frame of the target image frame may be an image of a golden retriever lying on the ground. The recommended image frame of the target image frame may include one or more. If the user selects a recommended image frame of the target image frame, the target image frame may be replaced with the recommended image frame to obtain the third input data.

[0141] Step F1 to Step F3 are independent steps.

[0142] In a possible implementation, generating a target file matching the user intention based on the user intention represented by the third input data may include steps G1 and G2:

[0143] Step G1: Generate a second processing instruction based on the attribute information of the target application and / or the user intention represented by the third input data.

[0144] In the present disclosure, the attribute information of the target application may include the type of the target application, for example, if the type of the target application may include an image generation application, a video generation application, and an audio generation application, etc. The user intent represented by the third input data includes the intent to generate an image file, a video file, and an audio file for an object contained in the third input data.

[0145] In the present disclosure, if the target application has an image generation function, a video generation function, or an audio generation function, a second processing instruction can be generated according to the target application, and the second processing instruction refers to a processing instruction for performing image generation, video generation, or audio generation on the third input data. If the target application does not have the image generation function, video generation function, and audio generation function, for example, when the target application is a chat software, the target application generally does not have the image generation function, video generation function, and audio generation function, then the target application is required to identify the third input data to obtain the user's intention, thereby generating a second processing instruction corresponding to the user's intention, and the second processing instruction generally includes calling a processing model or model service corresponding to the user's intention, and using the called processing model to process the third input data.

[0146] Step G2: generate and process the third input data using the processing model or model service corresponding to the second processing instruction to obtain a target file that matches the attribute information and the user's intention.

[0147] In the present disclosure, the processing model corresponding to the second processing instruction may include a large model local to the electronic device, for example, an image generation model, a text generation model, and a video generation model local to the electronic device. The model service corresponding to the second processing instruction may include a service interface provided by the cloud that can realize image generation, text generation, and video generation, and the electronic device may send the third processing data to the cloud through the service interface to process the third input data using the model service in the cloud, and feed back the processing result to the electronic device.

[0148] For example, if the third input data is text data "Draw a husky lying on the ground", and the target application generally does not have image generation function, video generation function and audio generation function, then the target application can identify the user intention of the third input data "Draw a husky lying on the ground": draw an image file of a husky lying on the ground, and generate a second processing instruction of "draw an image file of a husky lying on the ground". Then the target application can call the model service interface of the cloud and send the text second processing instruction to the cloud. The cloud uses the image generation model to generate a corresponding image file for the text data "draw an image file of a husky lying on the ground" in the second processing instruction, and feeds the image file back to the electronic device. The target file is the target file that matches the user's intention.

[0149] In a possible implementation, generating a target file matching the user intention based on the user intention represented by the third input data may further include at least one of the following steps H1-H2, where steps H1 and H2 are independent steps:

[0150] Step H1, obtaining user portrait information of the target user, generating and processing the third input data with reference to the user portrait information, and obtaining a target file that matches the user's intention.

[0151] In the present disclosure, user portrait information may include user attribute information and user behavior information, etc. User attribute information may include user age, occupation and gender, etc., and user behavior information includes user behavior log data of using target application, etc. In the present disclosure, user interest preferences can be analyzed based on user portrait information, and then user interest preferences can be referred to when processing the third input data. Specifically, a clustering model, an association rule mining model or an information recommendation model can be used to identify user portraits and identify similar user groups with similar behaviors, so that the user interest preferences can be predicted based on the interest preferences of this type of user group. For example, it is identified that a user group with female gender likes to generate anime images, and the third input data can be processed based on the interest preference information of the user group to obtain an anime-style image file that matches the user's intention as the target file. In the present disclosure, the third input data can also be generated and processed with reference to the user portrait information to obtain a video file or audio file that matches the user's intention as the target file.

[0152] Step H2, obtaining the current environment information of the electronic device, generating and processing the third input data with reference to the environment information, and obtaining a target file matching the user's intention.

[0153] In the present disclosure, the environmental information of the electronic device may also affect the target file. For example, if an audio file about weather conditions needs to be generated, the environmental information of the electronic device can reflect the weather conditions at that time. The target application can use the environmental information of the electronic device and analyze the current weather conditions, and generate an audio file about the weather conditions in combination with the current weather conditions.

[0154] For another example, the environment information of the electronic device may include the application scenario information of the electronic device. For example, if the electronic device is a mobile phone, the user can interact with the electronic device through a voice assistant, and the target file is a video file, then the environment information of the electronic device may include the user's preference information for the video information expressed by voice in the environment. For example, if the user prefers a video with a 4:3 aspect ratio, the target application may collect the user's voice information and analyze the user's preference. When generating a video file, the user's preference may be referred to to generate a video file with a 4:3 aspect ratio as the target file.

[0155] Based on the same inventive concept, according to the data processing method provided by the above embodiment of the present disclosure, correspondingly, another embodiment of the present disclosure further provides a data processing device, whose structural schematic diagram is shown in FIG. Figure 7 As shown, specifically including:

[0156] A processing module 701 is used for processing first input data into second input data in response to obtaining a first processing instruction for first input data, where the first input data is data input to a target application;

[0157] A first generating module 702, configured to generate third input data based on the obtained adjustment operation for the second input data;

[0158] The second generating module 703 is used to generate a target file matching the user intention based on the user intention represented by the third input data.

[0159] With this device, the first input data is processed into second input data input to the target application through the first processing instruction, and then the third input data is generated according to the adjustment operation on the second input data, so as to generate a target file matching the user intention using the user intention represented by the third input data. This method can guide the user to express the user intention more accurately, thereby generating a target file that accurately matches the user intention, and improving the user experience.

[0160] In one possible implementation, the processing module 701 is specifically configured to obtain a first processing instruction for the first input data in at least one of the following ways:

[0161] generating the first processing instruction in response to obtaining a trigger operation of a first type of control acting on a first type of application, wherein the first type of application is an application capable of responding to the first input data;

[0162] generating the first processing instruction in response to obtaining a trigger operation for first input data input to a second type of application, the second type of application being an application that can respond to or cannot respond to the first input data;

[0163] In response to obtaining first input data input to a third type of application, generating a corresponding first processing instruction based on application information of the third type of application;

[0164] A corresponding first processing instruction is generated based on the type of the first input data and / or application information of a source application.

[0165] In one possible implementation, the processing module 701 is specifically configured to process the first input data into second input data in at least one of the following ways:

[0166] Processing the first input data into the second input data using a processing model matching the first processing instruction;

[0167] processing the first input data into fourth input data based on the type of the first input data, and performing generation processing on first target data in the fourth input data to obtain the second input data;

[0168] The first input data input to the target application and its context data are integrated to obtain the second input data.

[0169] In one possible implementation, the first generating module 702 is specifically configured to generate the third input data based on the obtained adjustment operation for the second input data in at least one of the following ways:

[0170] generating third input data based on an obtained selection operation of recommended data for second target data in the second input data and / or an edit operation for the second target data;

[0171] generating third input data based on the configuration operation obtained for the configuration option in the configuration window of the second input data, wherein the configuration option is used to update the target data in the second input data;

[0172] Context data associated with the second input data is obtained, and the second input data is updated using the context data to generate third input data.

[0173] In one embodiment, the first generating module 702 is specifically configured to generate the third input data based on the obtained selection operation of the recommended data for the second target data in the second input data and / or the edit operation for the second target data in at least one of the following ways:

[0174] In response to obtaining a trigger operation for a target keyword in the second input data, outputting a recommendation word for the target keyword, replacing the target keyword with a recommendation word determined by a selection operation for the target recommendation word, and obtaining the third input data;

[0175] In response to obtaining an editing operation for a target keyword in the second input data, generating the third input data based on the edited keyword;

[0176] In response to obtaining a trigger operation for a target keyword in the second input data, an output effect of a recommendation word for the target keyword is output, and the target keyword is replaced with a recommendation word determined by a selection operation on the target output effect to obtain the third input data.

[0177] In one possible implementation, the first generating module 702 is specifically configured to generate the third input data based on the obtained adjustment operation for the second input data in at least one of the following ways:

[0178] In the case where the second input data is image data, in response to obtaining a trigger operation for a target object in the image data, outputting a recommended object for the target object, and replacing the target object with a recommended object determined by a selection operation for the target recommended object, to obtain the third input data;

[0179] When the second input data is audio data, in response to obtaining a trigger operation for a target segment in the audio data, outputting a recommended segment for the target segment, and replacing the target segment with a recommended segment determined by a selection operation for the target recommended segment, to obtain the third input data;

[0180] In the case where the second input data is video data, in response to obtaining a trigger operation for a target image frame in the video data, a recommended image frame for the target image frame is output, and the target image frame is replaced with the recommended image frame determined by the selection operation on the target recommended image frame to obtain the third input data.

[0181] In one possible implementation, the second generation module 702 is specifically used to display a configuration window for the second input data, wherein the configuration window displays configuration options corresponding to the target keyword in the second input data; in response to obtaining a trigger operation on the target configuration option, displaying a recommended option for the target configuration option; and replacing the target keyword in the second input data with the recommended option determined by the selection operation on the target recommended option to obtain the third input data.

[0182] In one possible implementation, the second generation module 703 is specifically used to generate a second processing instruction based on the attribute information of the target application and / or the user intention represented by the third input data; and generate and process the third input data using a processing model or model service corresponding to the second processing instruction to obtain a target file that matches the attribute information and the user intention.

[0183] In one possible implementation, the second generating module 703 is specifically configured to generate a target file matching the user intention based on the user intention represented by the third input data in at least one of the following ways:

[0184] Obtaining user portrait information of a target user, and generating and processing the third input data with reference to the user portrait information to obtain a target file matching the user's intention;

[0185] The current environment information of the electronic device is obtained, and the third input data is generated and processed with reference to the environment information to obtain a target file that matches the user's intention.

[0186] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0187] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0188] like Figure 8As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0189] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0190] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0191] The electronic device 800 may further include an image acquisition device.

[0192] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), integrated systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0193] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0194] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0195] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0196] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0197] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0198] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0199] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0200] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: In response to obtaining a first processing instruction for first input data, processing the first input data into second input data, the first input data being data input to a target application; generating third input data based on the obtained adjustment operation for the second input data; A target file matching the user intention is generated based on the user intention represented by the third input data.

2. The method according to claim 1, wherein: Obtaining a first processing instruction for first input data includes at least one of the following: generating the first processing instruction in response to obtaining a trigger operation of a first type of control acting on a first type of application, wherein the first type of application is an application capable of responding to the first input data; generating the first processing instruction in response to obtaining a trigger operation for first input data input to a second type of application, the second type of application being an application that can respond to or cannot respond to the first input data; In response to obtaining first input data input to a third type of application, generating a corresponding first processing instruction based on application information of the third type of application; A corresponding first processing instruction is generated based on the type of the first input data and / or application information of a source application.

3. The method according to claim 1 or 2, wherein: Processing the first input data into second input data comprises at least one of the following: Processing the first input data into the second input data using a processing model matching the first processing instruction; processing the first input data into fourth input data based on the type of the first input data, and performing generation processing on first target data in the fourth input data to obtain the second input data; The first input data input to the target application and its context data are integrated to obtain the second input data.

4. The method according to claim 1, wherein: Generating third input data based on the obtained adjustment operation for the second input data includes at least one of the following: generating third input data based on an obtained selection operation of recommended data for second target data in the second input data and / or an edit operation for the second target data; generating third input data based on the configuration operation obtained for the configuration option in the configuration window of the second input data, wherein the configuration option is used to update the target data in the second input data; Context data associated with the second input data is obtained, and the second input data is updated using the context data to generate third input data.

5. The method according to claim 4, wherein: Generating third input data based on the obtained selection operation of recommended data for the second target data in the second input data and / or the editing operation for the second target data includes at least one of the following: In response to obtaining a trigger operation for a target keyword in the second input data, outputting a recommendation word for the target keyword, replacing the target keyword with a recommendation word determined by a selection operation for the target recommendation word, and obtaining the third input data; In response to obtaining an editing operation for a target keyword in the second input data, generating the third input data based on the edited keyword; In response to obtaining a trigger operation for a target keyword in the second input data, an output effect of a recommendation word for the target keyword is output, and the target keyword is replaced with a recommendation word determined by a selection operation on the target output effect to obtain the third input data.

6. The method according to claim 4 or 5, wherein: Generating third input data based on the obtained adjustment operation for the second input data includes at least one of the following: In the case where the second input data is image data, in response to obtaining a trigger operation for a target object in the image data, outputting a recommended object for the target object, and replacing the target object with a recommended object determined by a selection operation for the target recommended object, to obtain the third input data; When the second input data is audio data, in response to obtaining a trigger operation for a target segment in the audio data, outputting a recommended segment for the target segment, and replacing the target segment with a recommended segment determined by a selection operation for the target recommended segment, to obtain the third input data; In the case where the second input data is video data, in response to obtaining a trigger operation for a target image frame in the video data, a recommended image frame for the target image frame is output, and the target image frame is replaced with the recommended image frame determined by a selection operation on the target recommended image frame to obtain the third input data.

7. The method according to claim 4, wherein: Generating third input data based on the configuration operation of the configuration options in the configuration window for the second input data, including: displaying a configuration window for the second input data, wherein the configuration window displays configuration options corresponding to the target keyword in the second input data; In response to obtaining a triggering operation on a target configuration option, displaying a recommended option of the target configuration option; The target keyword in the second input data is replaced by the recommended option determined by the selection operation on the target recommended option to obtain the third input data.

8. The method according to claim 1, wherein: Generating a target file matching the user intention based on the user intention represented by the third input data includes: generating a second processing instruction based on the attribute information of the target application and / or the user intention represented by the third input data; The third input data is generated and processed using a processing model or a model service corresponding to the second processing instruction to obtain a target file that matches the attribute information and the user's intention.

9. The method according to claim 1 or 8, wherein: Generating a target file matching the user intention based on the user intention represented by the third input data further includes at least one of the following: Obtaining user portrait information of a target user, and generating and processing the third input data with reference to the user portrait information to obtain a target file matching the user's intention; The current environment information of the electronic device is obtained, and the third input data is generated and processed with reference to the environment information to obtain a target file that matches the user's intention.

10. A data processing device, comprising: a processing module, configured to, in response to obtaining a first processing instruction for first input data, process the first input data into second input data, wherein the first input data is data input to a target application; A first generating module, configured to generate third input data based on the obtained adjustment operation for the second input data; The second generating module is used to generate a target file matching the user intention based on the user intention represented by the third input data.

Images