Generation method and electronic equipment

Abstract

The invention provides a generation method and electronic equipment, which can be applied to the technical field of artificial intelligence. The method comprises the steps of obtaining target input data matched with a target trigger event in response to the target trigger event; obtaining target reference data associated with the target input data; performing generation processing on the target input data and the target reference data to obtain a target file; wherein the type of the target reference data is the same as or different from that of the target input data.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically, to a generation method and an electronic device. Background Art

[0002] With the development of artificial intelligence technology, its application in daily life is becoming more and more extensive. For example, content can be generated by utilizing artificial intelligence technology. However, in actual applications, there are problems such as limited generation accuracy and complex operation. Summary of the invention

[0003] A first aspect of the present disclosure provides a generation method, comprising: in response to a target trigger event, obtaining target input data that matches the target trigger event; obtaining target reference data associated with the target input data; performing generation processing on the target input data and the target reference data to obtain a target file; wherein the target reference data and the target input data are of the same or different type.

[0004] According to an embodiment of the present disclosure, obtaining target input data that matches a target trigger event includes at least one of the following: in response to obtaining a data input operation acting on a target application, determining the target input data from the input data generated by the data input operation based on the operation behavior parameters of the data input operation; in response to obtaining an adjustment operation on a first category object in the target application, taking the target object acted upon by the target adjustment operation as the target input data; in response to obtaining an adjustment operation on a display parameter of an application window of the target application, taking the change data of the display parameter as the target input data; obtaining source information of the target trigger event, and obtaining the target input data in a manner that matches the source information; in response to monitoring an operation to share target data content, taking the target data content as the target input data. as target input data; in response to a change in the usage scenario of the electronic device, using the monitored usage scenario change data as the target input data, the usage scenario including at least one of the usage environment, usage object, and usage mode of the electronic device; in response to detecting that the electronic device establishes a communication connection with the target input device and / or switches to a target position relationship, using the information of the communication connection and / or the target position relationship as the target input data; in response to detecting that the electronic device switches from a current first device form to a second device form different from the first device form, using the second device form as the target input data; wherein the target application is an application in the electronic device that can directly call the corresponding processing model, or an application that can call the corresponding processing model through the first application.

[0005] According to an embodiment of the present disclosure, determining target input data from input data generated by a data input operation based on an operation behavior parameter of the data input operation includes at least one of the following: determining an input state of a target user based on the operation behavior parameter of the data input operation, determining a screening range from the input data generated by the data input operation based on the input state, and using the input data within the screening range as the target input data; determining, as the target input data, the input data having a target operation behavior parameter among the input data generated by the data input operation.

[0006] According to an embodiment of the present disclosure, obtaining target reference data associated with the target input data includes at least one of the following: obtaining location information of the target input data, and obtaining the target reference data from the interface where the target input data is located and / or the file where the target input data is located based on the location information; obtaining attribute information of the target input data, and obtaining the target reference data from the interface where the target input data is located and / or the file where the target input data is located based on the attribute information; obtaining usage information of the target input data, and using the user profile data of the target user and / or the usage environment data of the electronic device as the target reference data based on the usage information.

[0007] According to an embodiment of the present disclosure, obtaining target reference data associated with the target input data includes at least one of the following: obtaining target display layout data and / or target context data of the target input data in a target interface based on the location information of the target input data in the current interface, and using the target layout data and / or target context data as the target reference data, where the target interface is the current application interface and / or other application interfaces of the target application presenting the target input data; in the case where the target input data is text data input to the current interface, using the context data associated with the text data and / or the display layout data of the text data in the current interface as the target reference data; in the case where the target input data is image data or editing data for an image in the current interface, using the application interface parameters of the target application presenting the target input data and / or the context data associated with the image data as the target reference data; in the case where the target input data is prompt word data capable of invoking a target processing model, using the application configuration data of the target application receiving the target input data and / or the user profile data of the target user as the target reference data.

[0008] According to an embodiment of the present disclosure, performing a generation process on the target input data and the target reference data includes: performing a first recognition process on the target input data to determine a semantic theme and / or an emotional theme represented by the target input data; performing a second recognition process on the target reference data to determine display parameters represented by the target reference data; inputting the semantic theme and / or the emotional theme represented by the target input data and the display parameters represented by the target reference data into a target processing model for a generation process to obtain the file.

[0009] According to an embodiment of the present disclosure, target input data and target reference data are generated and processed, including at least one of the following: optimizing the user intent represented by the target input data using the target reference data, inputting the obtained target prompt word data into the target processing model, so as to generate a target file using the target processing model; performing intent recognition processing on the target input data and the target reference data based on corresponding weights, and performing generation processing on the obtained target intent using the target processing model to obtain a target file; generating and processing the user intent represented by the target input data using the first processing model, and optimizing or regenerating the obtained first generation result using the second processing model based on the target reference data to obtain a target file.

[0010] According to an embodiment of the present disclosure, the generation method also includes at least one of the following: outputting the target file to a first display area of ​​an electronic device to update and display the target file in a target display area in an application interface of a target application in response to a target interactive operation, wherein the first display area is located at or not located at the application interface of the target application; outputting the target file to a current application interface of the target application to replace the foreground content and / or background content of the current application interface; obtaining feedback data from the target user on the target file, and updating model parameters of a target processing model based on the feedback data; obtaining an operating status of the electronic device, and executing or pausing the step of obtaining target input data based on the operating status.

[0011] A second aspect of the present disclosure provides a generation method, including: in response to obtaining a data input operation acting on a target application, obtaining target input data generated by the data input operation; obtaining target interface data of a target interface of a target application associated with the target input data; and generating a target file based on a user intent represented by the target input data and the target interface data.

[0012] A third aspect of the present disclosure provides an electronic device, comprising at least one processor and at least one processing model capable of running on the processor, wherein the processing model can be called by a target application to perform at least one of the following: in response to a target trigger event, obtaining target input data matching the target trigger event; obtaining target reference data associated with the target input data; generating a target file based on the user intent represented by the target input data and the target reference data; wherein the target reference data is the same or different type as the target input data, and / or the target reference data is the same or different source as the target input data.

[0013] The fourth aspect of the present disclosure provides a generation device, including: a first acquisition module, used to obtain target input data matching the target trigger event in response to a target trigger event; a second acquisition module, used to obtain target reference data associated with the target input data; and a generation processing module, used to perform generation processing on the target input data and the target reference data to obtain a target file.

[0014] A fifth aspect of the present disclosure provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0015] The sixth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 A flow chart of a generating method according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 2 A schematic diagram schematically shows a target application call processing model;

[0019] Figure 3 A flowchart for generating target input data and target reference data according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 4 The schematic diagram of the switching of the operating state of the electronic device is shown schematically;

[0021] Figure 5 A flow chart of a generating method according to another embodiment of the present disclosure is schematically shown;

[0022] Figure 6 A block diagram schematically shows an electronic device suitable for implementing the generating method according to an embodiment of the present disclosure;

[0023] Figure 7 The block diagram of a generating device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] The embodiments of the present disclosure provide a generation method and an electronic device. Before introducing the technical solution provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure are first described.

[0029] Artificial intelligence technology is widely used in the field of content generation. However, existing generation technologies are prone to problems such as inaccurate generated content, low quality, and the need for multiple modifications before use. The inventors have found that one of the reasons for this problem is that the existing generation methods usually generate content based on a single data source (such as text / pictures selected by users, or text / instructions entered by users, etc.). On the one hand, the existing generation methods only rely on a single data source, which is user input, for generation. There are information limitations (such as single content, limited perspective, etc.), which can easily lead to low accuracy of generated content and fail to meet user needs. On the other hand, this generation method based on a single data source has high requirements for users, requiring users to have good thinking logic and language expression ability, and is not used for all users. Even if users have high professional knowledge, if they want to generate accurate content, they also need to input a large amount of information by themselves, which is a large workload and a poor user experience.

[0030] In view of this, an embodiment of the present disclosure provides a generation method, including: in response to a target trigger event, obtaining target input data matching the target trigger event; obtaining target reference data associated with the target input data; performing generation processing on the target input data and the target reference data to obtain a target file; wherein the target reference data and the target input data are of the same or different type.

[0031] The following will be passed Figure 1~Figure 5 The generation method of the embodiment of the present disclosure is described in detail.

[0032] Figure 1 The flowchart of the generating method according to an embodiment of the present disclosure is schematically shown.

[0033] like Figure 1 As shown, the generating method of this embodiment includes operations S110 to S130.

[0034] In operation S110 , in response to a target trigger event, target input data matching the target trigger event is obtained.

[0035] In some embodiments, the target trigger event may refer to an operation acting on a target application, or may refer to a change in an electronic device. When a target trigger event occurs, target input data matching the target trigger event is acquired.

[0036] In some embodiments, the operations acting on the target application may include, for example, string input, voice input, gesture input operations, image replacement, insertion, editing operations, interface window editing, reconstruction operations, window parameter adjustment operations, operations of calling the target application, operations acting on the target application that can call the target processing model, etc. The target application may be, for example, a text editing application, an image processing application, office software, a social application, etc. The changes in the electronic device may include, for example, changes in the device form of the electronic device, changes in the usage mode, changes in the usage scenario or environment, etc.

[0037] Different target trigger events correspond to different target input data. The target input data may include string input data, voice input data, image input data, operation data, device form change data, mode change data (such as a mode that can be used to provide AI function services), scene or environment change data, etc.

[0038] For example, when a user performs a string operation in a text editing application, the string "Today is sunny" input by the user is obtained as the target input data. Or, in response to a change in the form of an electronic device, the device form change data is obtained as the target reference data, where the device form change data may include, for example, the angle change between different bodies of the electronic device, the current state information of a component or structure, the position change, the user interface change, etc.

[0039] In operation S120 , target reference data associated with the target input data is obtained.

[0040] In some embodiments, target reference data associated with target input data may be acquired from multiple dimensions such as content dimension, user dimension, and scene dimension.

[0041] The target reference data of the content dimension may, for example, include the context data of the input string or voice data, the layout information or theme information of the input page, or the theme information of the entire file or application; the target reference data of the user dimension may, for example, include the user portrait of the user of the target application; the target reference data of the scene dimension may, for example, be the scene or environment data adapted to the current usage scene or usage environment.

[0042] In some embodiments, acquiring target reference data associated with the target input data from the content dimension may include, for example, acquiring data that is spatially (or logically) associated with the target input data.

[0043] Taking the target application as a presentation making software as an example, if the target input data is string data or voice data input on the third page of the current presentation, the text information, image information, and theme style information in other pages adjacent to the third page can be obtained as the target reference data. In addition, the target reference data can also be the preceding and following paragraphs or chapter paragraphs of the target input data on the third page, or the layout information or theme style information of the third page, or the theme style information of the current presentation, or the theme style information of the target application (such as the unified design style and elements followed by all presentations in the presentation making software).

[0044] In some embodiments, obtaining target reference data associated with target input data from the user dimension may include, for example, obtaining user portrait information of the user. User portrait information may include the user's habits or preferences in the current application (i.e., the target application), and may also include the user's habits or preference settings in associated applications. For example, after receiving the target input data, the user (such as the user information of the target application) may be determined based on the target input data and the user portrait information of the user may be used as the target reference data.

[0045] The target application may be set up with multiple ports, different ports focus on different functions, and users play different roles in different ports. Users may include, for example, users on the creation side and users on the demand side, and the user portrait information of users with different roles may be different. Users on the creation side refer to users who use the target application to create, produce or generate content. For example, designers upload their own works on the creation side. The corresponding user portrait information may be the functions that are frequently accessed when processing the works, editing operations, and the style of the uploaded works. Users on the demand side refer to users who need to obtain, consume or use the content generated by the creation side, such as users who browse and download other people's works. The corresponding user portrait information may be related information of the works that are frequently liked and commented on when browsing the works (such as the style and content of the works, etc.).

[0046] The associated application may be an application of similar type and / or complementary function to the current application. For example, if the current application is image editing software, the associated application may be other applications for image editing. If the current application is a social media platform, the associated application may be an application for content creation and image editing. For example, when a user inputs target input information in application A, in addition to the user's editing habits in application A, the user's editing habits and preferences in associated application B can also be used as target reference data.

[0047] Taking the target application as an image processing application and the target input data as image input data as an example, the associated applications of the target application may include applications that generate images (such as painting applications, photography applications, etc.), and other applications for image processing or image sharing (such as social media platforms, image sharing communities, etc.).

[0048] In some embodiments, obtaining target reference data associated with target input data from a scenario dimension may include, for example, obtaining scenario or environment data adapted to the current usage scenario or usage environment, etc. In different usage scenarios and usage environments, there may be different user requirements, device function limitations, and different user behavior patterns. Therefore, when obtaining target reference data associated with target input data, scenario or environment data adapted to the current usage scenario or usage environment may be obtained.

[0049] It should be noted that the target reference data may be of the same or different type as the target input data. For example, when the target input data is a string, the target reference data may be another string related to the target input data (for example, it may be text data of the previous and next chapters or pages of the string), or it may be image data, page layout information, paragraph layout information, etc. that are different from the target reference data type, or it may be user portrait information of the user who performs string input, etc. For example, if the target input data is text data, image data and audio data associated with the text data (such as belonging to the same interface, the same file, created by the same user, etc.) may be obtained as the target reference data. And / or

[0050] The target reference data may have the same or different sources as the target input data. Continuing with the example of the target input data being a string, the target reference data may belong to the same document or the same system as the target input data. For example, the target reference data may be text content, page layout information, etc. in the document to which the target input data belongs. The target reference data may also belong to a different document or system than the target input data. For example, the target reference data may be text content, theme style information, etc. from another document or another application.

[0051] In operation S130, target input data and target reference data are generated to obtain a target file.

[0052] In some embodiments, a multimodal target file that meets the requirements is generated by intelligently analyzing and processing the target input data and the target reference data. Among them, the target input data is used to provide the main information of the generated content, and the target reference data is used to constrain and guide the generated content, and to assist in understanding the meaning and intention of the input data, thereby improving the accuracy of the target file generation. The target file can be obtained by calling the corresponding artificial intelligence (AI) processing model, or calling an application deployed with an AI processing model to perform tasks such as picture generation, text generation, video generation, image processing, and text editing processing, wherein the target file can be, for example, an image, text, video, animated image, template data (such as a presentation template), user interface, audio file, etc. The target file obtained based on the generation process can be used to update the target input data's action object. For example, the image obtained after the generation process can be inserted into the interface to which the target input data belongs to obtain a new interface containing the newly inserted image.

[0053] In some embodiments, the target file can be directly output at the current editing position (for example, inserting the target file to the current position of the cursor), or displayed in an adjacent area of ​​the current editing area (for example, displaying the target file in the adjacent area after determining the position and size of the adjacent area). The target file can also be displayed in a window of a specific application that can call the model (taking the target file as an image as an example, the target file can be displayed in the main window, extended window or floating window of the application used to perform the image processing task), or the generated target file can be displayed in a separate window area. For example, the target input data is "generate a picture of a snake", and a colorful cartoon-style picture of the user can be obtained by obtaining the user portrait of the user, that is, the target reference data is "colorful, cartoon style", then the target input data and the target reference data are generated and processed, and the resulting target file is a colorful cartoon snake picture.

[0054] The generation method provided by the embodiment of the present disclosure automatically obtains target input data by capturing and analyzing target trigger events, thereby improving the accuracy and speed of obtaining target input data. The target reference data associated with the target input data is automatically obtained based on the received target input data, enriching the information source, quickly understanding the background information related to the target input data and user needs, improving the accuracy, diversity and flexibility of target file generation, and making the generated target file more in line with the actual situation and user needs.

[0055] In one possible implementation, Figure 1Based on the generation method shown in FIG. 1 , operation S110 “obtaining target input data matching the target trigger event” may further include at least one of the following operations S1101 to S1108. It should be noted that for other implementation details of the generation method of this embodiment, please refer to Figure 1 The embodiment part of the generation method shown will not be repeated here.

[0056] Operation S1101 , in response to obtaining a data input operation acting on a target application, determining target input data from input data generated by the data input operation based on an operation behavior parameter of the data input operation.

[0057] In some embodiments, data input operations may include, for example, string input operations, voice input operations, gesture input operations, etc. Different data input operations may be performed by acting on different input devices or interfaces.

[0058] For example, the string input operation can be performed by handwriting operations acting on the writing area on the display screen, input operations acting on the virtual keyboard, trigger operations acting on the target buttons of the electronic device, and other operations to input the string. Among them, the trigger operation acting on the target button of the electronic device may include, for example, keyboard tapping or quick reply buttons, and the target button includes a single physical button or a combination of physical buttons set on the body of the electronic device. For example, a quick reply message can be input through a physical button on a single side of a smartphone, or a string copied from another source can be input through Ctrl+c and Ctrl+v. The voice input operation can be performed by configuring the microphone control or the switch state of the camera on the application interface of the electronic device. The gesture input operation can be performed by gesture input through the interactive data acting on the sensing area of ​​the electronic device, such as gesture control input using a touch screen, camera, or radar.

[0059] For example, in response to the user drawing a graphic in the canvas window through gestures such as sliding, clicking, and long pressing on the touch screen, the gesture trajectory is captured and displayed. When the user completes drawing, the gesture trajectory drawn by the user is used as target input data.

[0060] In response to obtaining a data input operation acting on the target application, the system enters a waiting state, waiting for the user to complete editing. When the user stops editing (pauses for more than a set time or the cursor leaves the text box), the system considers that the user has completed the input. For example, in response to monitoring a handwriting operation acting on the writing area on the display screen, the system enters a waiting state. When the handwriting operation pauses for more than 1 minute, it is considered that the user has completed the input, and the input data generated by the user's handwriting operation is determined as the target input data.

[0061] Figure 2The schematic diagram of the target application call processing model is schematically shown.

[0062] like Figure 2 As shown, the target application mentioned in the embodiment may be an application in the electronic device that can directly call the corresponding processing model, or may be an application that can call the corresponding processing model through the first application.

[0063] Among them, the first application can be, for example, Lenovo's Creator zone, Xiaotian, AI now, etc. The first application can directly call the large model deployed on the main body, and the target application can be any third-party application (such as social applications, word processing software, presentation software, etc.). For example, the target application can be an application that has an internally integrated processing model calling interface, and can directly call the corresponding processing model through the internally integrated calling interface. Or, the target application cannot directly call the corresponding processing model, but the target application and the first application can communicate with each other, then the target application can call the large model deployed on the main body or the cloud through the calling interface between the first application.

[0064] Operation S1102 : in response to obtaining an adjustment operation on a first category object in a target application, taking a target object on which the target adjustment operation acts as target input data.

[0065] In some embodiments, the first category image may be, for example, a picture, and the adjustment operation of the first category object may be, for example, a replacement or editing operation on the picture. The adjustment operation may be triggered in a variety of ways, such as clicking, dragging, long pressing, etc., and the target input data may be determined according to the object on which the adjustment operation is performed. For example, if the adjustment operation is to edit a picture, the target image is the picture being edited, and if the adjustment operation is to replace a picture, the target object is the new picture to be replaced.

[0066] In addition, the target object may also include control icons or element icons related to the image, and the control icons and element icons may be used to trigger corresponding image editing functions or image interface changes. For example, the control icons may include filter buttons (different filter buttons correspond to different filter information, and different filter effects can be selected and applied to the image by clicking the filter buttons), adjustment buttons (used to adjust the hue, color temperature, saturation, size, etc. of the image), etc. Element icons may include stickers (used to add decorations or marks to the image, etc.), text boxes (used to add text or labels to the image), etc. For example, if the target adjustment operation is "click filter button A", the clicked filter button A is used as the target object, and the target input data includes the filter information corresponding to the filter button A.

[0067] Operation S1103 : in response to obtaining an adjustment operation of a display parameter of an application window of a target application, using change data of the display parameter as target input data.

[0068] In some embodiments, the adjustment operation may be an edit or replace operation on the interface elements or control elements in the current application window. The interface elements may include static or dynamic components (such as text, background, icons, etc.) that constitute the visual content of the application window. For example, the icon color in the application window may be modified, the background display form may be modified, etc., through editing operations. The icons in the application window may be replaced with icons of other themes, the current background may be replaced with other pictures, etc., through replacement operations. The control elements may include dynamic components with interactive functions, and corresponding technical effects may be achieved through editing operations. For example, a single-line text box may be replaced with a multi-line text box, the animation effect in the video may be adjusted, the display mode of the picture may be adjusted, etc., through editing the control elements.

[0069] In some embodiments, the adjustment operation may also be an operation to adjust the display layout of the presentation, for example, adjusting the position of each element in the presentation, adjusting the stacking relationship of multiple elements overlapping at the same position, adjusting the arrangement of each element in the presentation, etc.

[0070] In some embodiments, the adjustment operation may also be to adjust the display parameters of the application window, such as size, color, transparency, etc. For example, the color of the application window is changed from color scheme 1 to color scheme 2.

[0071] In some embodiments, different adjustment operations correspond to different display parameter change data. The change data may include data that changes after the adjustment operation is performed, such as window color, area size, stacking relationship, layout relationship, transparency, style, etc.

[0072] For example, if you move a text box in a presentation from the upper left corner to the lower right corner, the target input data you get is the change data of the text box position (i.e. the original coordinates and new coordinates of the text box). Or, if you change the color scheme of the browser window from the default light theme to the dark theme, the target input data you get is the change data of the window color scheme, as well as the change data of the interface element's color, contrast and other attributes.

[0073] Operation S1104: obtaining source information of a target trigger event, and obtaining target input data in a manner matching the source information.

[0074] In some embodiments, when the source information of the target trigger event is different, the input method for obtaining the target input data is different. For example, when the user performs a paste operation, the source of the target trigger event is the "clipboard", and the text content is read from the clipboard as the target input data. When the user imports a picture from another application, the picture is received and parsed through the inter-application communication interface as the target input data. When the user inputs audio data into a smart phone, the voice data is captured through the microphone module as the target input instruction.

[0075] Operation S1105: In response to monitoring an operation of sharing the target data content, use the target data content as the target input data.

[0076] In some embodiments, monitoring an operation of sharing a certain image, a certain piece of text, or a certain audio to a specific object triggers an operation of enhancing, beautifying, or processing according to sharing requirements on the file to be shared, where the target data content may be an image, text, audio, video, link data, etc. For example, in a social media application, when the user selects a picture and clicks the "Share" button on the application interface, the picture is determined as the target input data.

[0077] Operation S1106: In response to a change in the usage scenario of the electronic device, use the monitored usage scenario change data as the target input data, where the usage scenario includes at least one of the usage environment, usage object, and usage mode of the electronic device.

[0078] In some embodiments, the usage scenario may include, for example, the usage environment of the electronic device (such as location, light, etc.), the usage object (such as the user, etc.), and the usage mode (such as the low-power mode, high-performance mode, etc.). A change in the usage scenario may trigger some matching changes. For example, when the usage scenario changes from daytime to night, the theme style may switch from a light color style to a dark color style. When the usage object changes, the user information, user preferences, etc. after the switch are obtained, and a processing operation adapted to the current user is triggered. When the usage mode changes, corresponding processing operations are performed in the current mode. For example, in the high-performance mode, an AI processing model may be considered to be called for automatic enhancement processing.

[0079] Operation S1107: In response to detecting the establishment of a communication connection between the electronic device and the target input device and / or switching to the target position relationship, use the information of the communication connection and / or the target position relationship as the target input data.

[0080] In some embodiments, the target input device may include, for example, a keyboard, a mouse, a pen, a writing board, a microphone, etc. The communication connection information may include, for example, the connection type (such as a wired connection, a wireless connection, etc.) between the target input device and the electronic device, the device identification, the connection status, etc.

[0081] For example, when the target input device is a keyboard, when the keyboard establishes target communication with the electronic device, the communication connection information between the keyboard and the electronic device is obtained as the target input data. When the target input device is a pen device set in a storage position of the electronic device, when it is detected that there is a first position relationship between the pen device and the first device, the target position relationship is used as the target input data, wherein the first position relationship can be, for example, writing after taking the pen device out of the storage position, and the storage position can be provided with a sensor, and when the pen device is taken out of the storage position, it can be determined that the pen device has been taken out according to the sensor change data.

[0082] Operation S1108: in response to detecting that the electronic device is switched from the current first device form to a second device form different from the first device form, using the second device form as target input data.

[0083] In some embodiments, switching the device form may include, for example, switching the first device to a form that is more convenient for the user to perform drawing input or trajectory input, so as to generate a corresponding picture based on the trajectory or stick figure input by the user. For example, the electronic device may be a foldable screen mobile phone, the first device form of which may be a folded state, and the second device form may be an unfolded state. When it is detected that the foldable screen mobile phone switches from a portable folded state to an unfolded state, the unfolded state is used as the target input data, so as to perform a picture generation operation based on the trajectory or stick figure input by the user.

[0084] The disclosed embodiment can accurately match and obtain target input data related to the target trigger event in response to different target trigger events, which helps to reduce the interference of irrelevant data and helps to improve the accuracy and relevance of the target input data. In addition, the target input data in the present disclosure can be input through a variety of target trigger events, including but not limited to direct data input operations, adjustment operations on the display parameters of the target application window, changes in the use scenarios of electronic devices, changes in the form of electronic devices, etc., providing users with a wealth of input options, allowing users to choose the most appropriate input method based on actual conditions and preferences, thereby improving user experience.

[0085] In a possible implementation, operation S1101 “determining target input data from input data generated by the data input operation based on the operation behavior parameters of the data input operation” may further include at least one of the following operations S1101a to S1101b.

[0086] Operation S1101a, determining the input state of the target user based on the operation behavior parameters of the data input operation, determining a filtering range from the input data generated by the data input operation based on the input state, and taking the input data within the filtering range as the target input data.

[0087] In some embodiments, the operation behavior parameters may include the user's input speed, pause duration, number of deletions and cancellations, cursor movement frequency, etc. The input state of the target user is determined by parsing the operation behavior parameters of the data input operation, where the input state may include, for example, terminating input, suspending input, thinking, etc.

[0088] For example, when there is no input operation for a long time (the pause time exceeds the first preset threshold) and the input content is complete (no unclosed symbols), the input state of the target user is determined to be terminated input. When the user switches to other applications or windows, the input state of the target user is determined to be suspended input. When the user's operation behavior parameters show high-frequency short pauses (such as the operation interval time is less than the second preset threshold) and are accompanied by deletion / modification operations, or the cursor moves frequently but there is no new content, the input state of the target user is determined to be thinking. Among them, the first preset threshold and the second preset threshold can be set, for example, according to the user's historical input behavior.

[0089] In some embodiments, after determining the input state of the target user, a screening range is determined from the input data generated by the target input operation based on the input state, and the target input data is extracted according to the screening range. The screening range corresponding to the terminated input state may include, for example, all final contents generated by this data input operation. The screening range corresponding to the "thinking" state may include, for example, input data within a valid duration. For example, when it is recognized that the user is in the "thinking" state, a pause duration may be set, and the target input data may be screened based on the pause duration, and the data input within the pause duration may be determined as the target input data. Assuming that the pause duration is 30s, the target input data is the data input within a pause of less than 30s.

[0090] Operation S1101b: determining data having target operation behavior parameters in the input data generated by the data input operation as target input data.

[0091] In some embodiments, the operation behavior parameter may be a behavior characteristic generated by the user when performing a data input operation. Different behavior characteristics can indirectly reflect the user's emotional state when inputting the current data and the degree of importance attached to the current data, etc. The behavior characteristics may include, for example, the force of keyboard tapping, the thickness of handwriting, the volume intensity, the amplitude of gestures, the number of repetitions, etc. The target operation behavior parameter may include, for example, a relatively large force of keyboard tapping, relatively thick handwriting, a relatively large volume, a relatively large amplitude of gestures, or a relatively large number of repetitions. The data that meets at least one target operation behavior parameter is determined as the target input data.

[0092] In some embodiments, for example, the operation data of the user can be collected through various sensors to obtain the operation behavior parameters of the user, and the target operation behavior parameters are screened from the collected user operation behavior parameters through a plurality of preset thresholds. Among them, the preset threshold can be set according to the user's historical input habits. For example, in the case where the data input operation is a string input, the target operation behavior parameter can be that the force of keyboard tapping is greater than 60 grams, and the input data generated by this input operation is determined as the target input data. In the case where the data input operation is a voice input operation, the target operation behavior parameter can be that the sound is greater than 50 decibels, and the input data generated by this input operation is determined as the target input data.

[0093] In a possible implementation manner, on the basis of the Figure 1 shown generation method, operation S120, "obtain the target reference data associated with the target input data", may further include at least one of the following operations S1201 to S1203. It should be noted that for other implementation details of the generation method in this embodiment, please refer to the Figure 1 embodiment part of the shown generation method, which will not be elaborated here.

[0094] Operation S1201, obtain the location information of the target input data, and obtain the target reference data from the interface where the target input data is located and / or the file where it is located based on the location information.

[0095] In some embodiments, the location information may include, for example, the input location or the presentation location of the target input data. For example, it may include the location of a newly added element in the application window interface or the location where an existing element is presented after rearrangement, and may also include the original location or the changed location in the file, etc. The target reference data may be data belonging to the same interface as the target input data, or data belonging to the same file as the target input data. The target reference data may include, for example, data obtained from an interface, input box, or display area capable of presenting or inputting the target input data, or other data extracted from the file where the target input data is located, such as data of other interfaces in the file.

[0096] Operation S1202: Obtain attribute information of target input data, and acquire target reference data from an interface and / or a file where the target input data is located based on the attribute information.

[0097] In some embodiments, the attribute information of the target input data may include, for example, the data type and data source of the target input data. The target reference data may be data different from the attribute information of the target input data. For example, if the target input data is text, you may try to extract the image data in the interface and / or file where the target input data is located as the target reference data. The target reference data may also be data from a different source than the target input data. For example, the target input data and the target reference data both come from file A, or the target input data comes from file A and the target reference data comes from file B, or the target input data belongs to application A and the target reference data belongs to application B associated with application A. By acquiring target reference data of the same and / or different data type and the same and / or different source as the target input data, the richness of the target reference data can be effectively improved.

[0098] Operation S1203: Obtain usage information of the target input data, and use user portrait data of the target user and / or usage environment data of the electronic device as target reference data based on the usage information.

[0099] In some embodiments, the usage information may include, for example, usage information, usage information, and user information of the target input data. The usage information is used to characterize the purpose for which the target input information is used, such as whether the input image information is to be shared on social media or for product display. The usage information is used to characterize how the target input information is used, such as which device or application is used. The user information is used to characterize which user the target input data comes from, such as user account information, historical behavior data, and the like. The user portrait data of the user in the current scene and / or the usage environment data of the electronic device can be determined based on the usage information, wherein the user portrait data may include, for example, the user's age, gender, behavior preference, etc., and the usage environment data of the electronic device may include device resolution, display mode, light intensity of the current environment, geographic location, etc. For example, the usage information of the target input data is obtained, and the target input data is input by user A and used to share on social media. Then, by analyzing the user's historical behavior in social media applications, it is found that the user portrait data includes: liking to use a specific filter style and adding geographic location tags to pictures, and the obtained user portrait data is used as the target reference information. Through the usage information of the target input data, the application scenarios with the same purpose / method / user as the current target input data are accurately matched, and the user portrait data and / or usage environment data in the same scenario are used as the target reference data, effectively improving the correlation between the target reference data and the target input data. In the same or similar application scenarios, the user's behavior patterns and preferences often have certain commonalities, which helps to maintain the consistency of the target input data and the target reference data.

[0100] The generation method provided by the embodiment of the present disclosure jointly obtains target reference data associated with the target input data from three dimensions: location information, usage information, and attribute information of the target input data, thereby improving the accuracy and richness of the obtained target reference information and providing more comprehensive support for subsequent generation processing.

[0101] In one possible implementation, Figure 1 Based on the generation method shown in FIG. 1 , operation S120 “obtaining target reference data associated with target input data” may further include at least one of the following operations S12001 to S12004. It should be noted that for other implementation details of the generation method of this embodiment, please refer to Figure 1 The embodiment part of the generation method shown will not be repeated here.

[0102] Operation S12001, based on the position information of the target input data in the current interface, obtain the target display layout data and / or target context data in the target interface, and use the target layout data and / or target context data as target reference data. The target interface is the current application interface and / or other application interface of the target application that presents the target input data.

[0103] In some embodiments, the current interface may be, for example, the current front page for displaying files, or may be an application interface of any application.

[0104] The location information of the target input data may include the coordinates of the target input data, the chapter to which it belongs, the page to which it belongs, and other information. Based on the location information of the target input data, the positional relationship between the target input data and other interface element objects can be determined, and the target display layout data and / or target context data in the target interface can be obtained.

[0105] Optionally, the target display layout data and / or target context data may include only the display layout data and / or target context data of the current interface where the target input information is located, or may include the layout data of the entire document and the contextual connections in the entire document, or may include the layout data or contextual relationships under a specific chapter or topic.

[0106] For example, if the target input data is located at the second paragraph of page 3 of chapter 2, then the target reference data may include only the display layout data of page 3 of chapter 2 and context data that has a contextual relationship with paragraph 2; or it may include the layout reference data of all pages of chapter 2 and context data that has a contextual relationship with page 3 (page 2 of chapter 2 and page 4 of chapter 2); or it may include the layout data and context data of the entire file.

[0107] Operation S12002: When the target input data is text data input into the current interface, context data associated with the text data and / or display layout data of the text data in the current interface are used as target reference data.

[0108] In some embodiments, when the type of the target input data is text data, it may be considered to obtain context data associated with the text data and / or display layout data of the text data in the current interface as reference data. Optionally, the context data may include data that is logically and / or spatially associated with the target input data, and the display layout data may include the positional relationship between the input text and other element objects in the current interface, and may also include the positional relationship between element objects in other interfaces with the same structure as the current interface of the input text.

[0109] Continuing with the example that the target input data is located in the second paragraph of the third page of the second chapter, the data belonging to the same chapter as the target input data (data of the second chapter), the data belonging to the same page as the target input data (data of the third page), the titles of the chapters in the file, and other data can determine the emotional attributes or semantic attributes of the target input data by obtaining the context data associated with the target input data, so as to be used for subsequent generation processing. For example, if the target reference file contains words such as "future city", "smart device", and "bright future", the emotional attributes of the target input data can be determined to be positive, innovative, and expected. The target reference data can also include display layout data formed by the relative position relationship between the target input data and other element objects on the third page, and / or display layout data of other interfaces with the same interface structure as the target input data. For example, if the current interface is a chapter page, the display layout data of other chapter pages in the file can be obtained as the target reference data for subsequent image generation or template generation based on the target reference data.

[0110] Operation S12003, when the target input data is image data or editing data for an image in the current interface, application interface parameters of a target application presenting the target input data and / or context data associated with the image data are used as target reference data.

[0111] In some embodiments, when the target input data is image data or editing data for an image in the current interface, the application interface parameter data of the target application used to present the target input data can be obtained as the target reference data. Optionally, the target reference data may include, for example, the relative position relationship between interface elements, size, display layout, color parameters, interface style attributes, etc. For example, when a user uploads a picture, the current window size, resolution, commonly used editing tools (such as "high contrast filter"), etc. can be used as target reference information to change the subsequent image generation or template generation. For example, the size of the generated image can be determined based on the application window size, and the image generation style can be determined based on commonly used editing tools.

[0112] In some embodiments, the target reference data may also be context data associated with the image data. The context data may include, for example, external information of the input image and other images that are contextually related to the current image.

[0113] For example, when a user uploads a picture, the shooting time, location, shooting weather, usage scenario data (such as whether the picture is a social media cover, e-commerce product picture, etc.), pictures in the same series, etc. can be obtained as target context data. For example, an image processing template can be generated based on the picture style of pictures in the same series, which can be used for subsequent unified processing of pictures of the same type. It is also possible to determine applicable image editing operations (such as adjusting the image exposure, saturation, filters, etc.) based on the weather and location at the time of shooting, and adjust the picture style or generate a new image based on the image editing operation.

[0114] Operation S12004, when the target input data is prompt word data capable of invoking a target processing model, application configuration data of a target application receiving the target input data and / or user portrait data of a target user are used as target reference data.

[0115] In some embodiments, the application configuration data may be the application interface style, application attribute style, complexity of application interface configuration elements, display layout parameters, etc., and image generation or template generation may be performed through the application interface parameters. User portrait data may include, for example, the user's favorite picture style, color preference, commonly used picture themes, etc. For example, if the target input data is "generate a minimalist login interface", the corresponding attribute information (such as white and light gray color, sans serif font, configuration element is a simple line light), display layout parameters, user commonly used templates, etc. are obtained from the application configuration data of the target application as target reference information.

[0116] Figure 3 The flowchart of generating target input data and target reference data according to an embodiment of the present disclosure is schematically shown.

[0117] like Figure 3 As shown, in Figure 1 Based on the generation method shown in FIG. 1 , the generation method of this embodiment generates target input data and target reference data, and can further include operations S1301 to S1303. It should be noted that other implementation details of the generation method of this embodiment can be found in Figure 1 The embodiment part of the generation method shown will not be repeated here.

[0118] In operation S1301, a first recognition process is performed on the target input data to determine the semantic theme and / or emotional theme represented by the target input data.

[0119] In some embodiments, for example, a natural language processing (NLP) module may be used to perform keyword extraction, semantic analysis, and sentiment analysis on the target input data to determine the semantic theme and / or sentiment theme represented by the target input data.

[0120] For example, when the target input data is text data, algorithms such as TF-IDF, TextRank or BERT can be used to extract keywords from the target input data, and the input text can be analyzed using a pre-trained semantic model to determine the semantic theme of the input text, and the sentiment tendency (positive, negative, neutral) of the text can be judged through a sentiment dictionary or sentiment classification model (such as VADER, TextBlob) to obtain the sentiment theme represented by the target input data. When the target input data is image data, pre-trained CNN models (such as VGG, ResNet, Inception, etc.) or custom models can be used to extract features from the image to capture the key information in the image, and the extracted features can be classified and identified to obtain the semantics represented by the image, and the emotional information represented by the image can be obtained by analyzing the content and visual style of the image.

[0121] In operation S1302, a second recognition process is performed on the target reference data to determine a display parameter represented by the target reference data.

[0122] In some embodiments, the display parameters can be used to adjust the visual effect of the target file, and the visual style of the target file to be generated is determined by analyzing the target reference information. Taking the target input data as text data as an example, the target reference data may include, for example, context data associated with the text data and display layout data of the text data in the current interface. By performing a second recognition process on the target reference data, features related to the display parameters are extracted from the target reference data, and the extracted features are analyzed to determine the display parameters. Among them, the display parameters may include, for example, color schemes (such as primary colors, secondary colors, background colors, etc.), graphic elements (such as icons, borders, background patterns, etc.), style information (such as realism, cartoons, etc.), and layout rules (element alignment, spatial distribution, hierarchy, etc.).

[0123] In operation S1303, the semantic theme and / or emotional theme represented by the target input data and the display parameters represented by the target reference data are input into the target processing model for generation processing to obtain a target file.

[0124] In some embodiments, the semantic themes and / or emotional themes represented by the target input data can be input into the target processing model simultaneously with the display parameters, or the semantic themes and / or emotional themes represented by the target input data can be input into the target processing model first, and then the display parameters can be input into the target processing model.

[0125] For example, after obtaining the semantic theme and / or emotional theme represented by the target input data and the display parameters, the two are simultaneously passed to the target processing model as input feature parameters, and the target processing model generates and outputs the target file. Alternatively, the semantic theme and / or emotional theme of the target input data is first input into the target processing model, and the target processing model performs preliminary generation processing based on the input semantic / emotional theme to generate an intermediate state file, and then the display parameters are input into the target processing model, and the target processing model performs further generation processing based on the display parameters and the intermediate state file to generate and output the target file.

[0126] Taking the target input data as text data and the target file as an image as an example, the text data is firstly subjected to a first recognition process to determine the semantic theme and / or emotional theme represented by the text data. The target reference data associated with the text data may include, for example, the layout data of the current interface and / or file, the title, the subtitle, the image in the current page and / or file, etc., and the display parameters represented by the target reference data are determined by performing a second process on the target reference data. For example, if the title is "Intelligent Development of Future Cities" and the subtitle is "Urban Innovation Brought by Science and Technology", the style information in the display parameters should be futuristic and technological, and the image tone in the current page and / or file is blue, black and white, then the color scheme in the display parameters is consistent with the image color code. The semantic theme and / or emotional theme represented by the target input data and the display parameters represented by the target reference data are input into the target processing model for generation processing, and a picture with a futuristic, technological, and image tone of blue, black and white can be obtained. The visual style of the picture is consistent with the style of other content in the file, and can reflect the core information that the target input text wants to express.

[0127] The generation method provided by the embodiment of the present disclosure performs a first recognition process on the target input data, thereby accurately capturing the core semantics and emotional tendency of the target input data, providing a basis for the accuracy of content generation, and determines the display parameters by performing a second recognition process on the target reference data, so that the generated target file not only matches the target input data in terms of content, but also meets personalized needs in many aspects such as visual style and layout, thereby effectively improving the accuracy of the target file, meeting the personalized needs of users, and improving the user experience.

[0128] In one possible implementation, Figure 1 Based on the generation method shown in FIG. 1 , operation S130 “generating and processing the target input data and the target reference data” may further include at least one of the following operations S1301 to S1303. It should be noted that for other implementation details of the generation method of this embodiment, please refer to Figure 1 The embodiment part of the generation method shown will not be repeated here.

[0129] Operation S1301 : optimizing the user intention represented by the target input data by using the target reference data, and inputting the obtained target prompt word data into the target processing model, so as to generate a target file by using the target processing model.

[0130] In some embodiments, target reference data can be used to assist in identifying user intent, and an intent understanding model can be used to identify user intent of adjusted input data to obtain target prompt words for guiding a target processing model to generate a target file.

[0131] For example, the target input data is "generate a picture of a future city". By analyzing the target reference data, it can be known that the user prefers cyberpunk style and cold color style, and the resolution requirement of the application configuration is 4k. Based on the target reference data, the style, resolution and other details are supplemented for the user intention, and the user intention is expanded and improved. The user intention after optimization can be, for example, "generate a cyberpunk-style cold-toned future city picture with a resolution of 4k". The user intention of the adjusted input data is identified through the intention understanding model to obtain the target prompt word, and the target prompt word is input into the target processing model to generate a text file, image file, video file, animation file, or multimodal multimedia data file that matches the user intention, thereby improving the accuracy of target file generation and content richness.

[0132] Operation S1302, performing intent recognition processing on the target input data and the target reference data based on the corresponding weights, and generating processing on the obtained target intent using the target processing model to obtain the target file.

[0133] In some embodiments, the intent recognition process can be combined with weights, and the target input data and the target reference data can be subjected to intent recognition based on different weights. For example, by assigning weights, the target processing model takes into account the importance of different elements in the target file when generating the obtained target intent, so that the target processing model focuses on the main intent during the generation process. For example, the weight of the main intent represented by the target input data is set to 0.7, and the weight of the secondary intent represented by the target reference data is set to 0.3. Then, when the main intent and the secondary intent conflict, the main intent is given priority for generation. For example, the main intent indicates that the picture style is a high-saturation tone, and the display parameters (i.e., the secondary intent) obtained based on the target reference data indicate that the picture in the file is in black and white. Then, when the target processing model generates the obtained target intent, it follows the main intent to generate a picture with a high-saturation tone.

[0134] Operation S1303: Generate the user intention represented by the target input data using the first processing model, and optimize or regenerate the first generated result based on the target reference data using the second processing model to obtain the target file.

[0135] In some embodiments, a dual model may be used for phased processing to obtain a target file, wherein the first phase performs preliminary generation based on target input data, and the second phase performs optimization processing on the preliminary generated file based on the target reference file.

[0136] For example, a first processing model can be used to generate and process the user intent represented by the target input data to obtain a first image, wherein the first image can reflect the semantic theme and / or emotional theme represented by the target input data, and then the second processing model can optimize or regenerate the first image based on the target reference data, such as optimizing the detail information, color tone, style, etc. in the first image, so that the optimized first image echoes other content in the file and maintains style consistency.

[0137] In one possible implementation, Figure 1 Based on the generation method shown in FIG. 1 , at least one of the following operations S1401 to S1403 may be further included. It should be noted that other implementation details of the generation method of this embodiment can be found in Figure 1 The embodiment part of the generation method shown will not be repeated here.

[0138] Operation S1401, outputting a target file to a first display area of ​​an electronic device, so as to update and display the target file in a target display area in an application interface of a target application in response to a target interactive operation, wherein the first display area is located in or not located in the application interface of the target application.

[0139] In some embodiments, a preview of the target file is provided through a first display area of ​​the electronic device, wherein the first display area may be a sidebar area of ​​the target application, or a floating window different from the application window, or a session window area of ​​other applications (such as a session window in a bar or Creator zone).

[0140] When the user performs a specific target interaction operation, the target file is updated from the first display area to the target display area in the application interface of the target application. Optionally, the target interaction operation may include, for example, a click operation, a drag operation, a voice command, a gesture operation, and the like. For example, when the user clicks a designated button in the first display area, the target file is updated and displayed in the target display area in the target application interface. Alternatively, the user long presses the target file in the first display area and drags the target file to the target display area designated on the application interface of the target application. After the user releases his hand, the target file is updated and displayed in the target display area. Optionally, the interaction operation may also include a first interaction operation, and in response to the first interaction operation, the target file is regenerated. The first interaction operation may, for example, include inputting first input data, adjusting the current target file based on the first data, and regenerating the target file.

[0141] Operation S1402: output the target file to the current application interface of the target application to replace the foreground content and / or background content of the current application interface.

[0142] In some embodiments, the target file can be used to replace specific content in the current application interface of the target application. The target file can be an element in the current application interface, or it can be template data applied to the current application interface of the target application (for example, it can include the layout information, format information, color scheme, etc. of the current application interface). When the target file is an element in the current application interface, some elements in the current application interface are replaced with the target file. When the target file is the template data of the current application interface, the template of the current page is replaced based on the target file, including but not limited to changing the arrangement of each element in the current page, position information, font style, color scheme, etc.

[0143] Operation S1403: obtaining feedback data of the target user on the target file, and updating model parameters of the target processing model based on the feedback data.

[0144] In some embodiments, after receiving user feedback data on the target file, the model parameters of the target processing model are updated based on the feedback data to further optimize the generation capability of the model and meet specific needs. Among them, the model parameters may include operator configuration parameters, RAG library, etc. By collecting and analyzing user feedback data, problems and deficiencies in the model are discovered in a timely manner to improve the accuracy of the model. In addition, the user's feedback data can reflect the user's personalized needs and preferences to a certain extent. By updating the model parameters of the target processing model through feedback data, the performance of the model in personalized services can be improved.

[0145] Operation S1404: obtaining an operating state of the electronic device, and executing or suspending the step of obtaining target input data based on the operating state.

[0146] In some embodiments, the operating status of the electronic device is monitored in real time, and a dynamic decision is made based on the operating status of the electronic device whether to continue the step of obtaining the target input data.

[0147] Optionally, the operating state of the electronic device may include, for example, a dormant state, a waiting state, and a throttling state. When the electronic device is in a dormant state or a waiting state, the step of obtaining the target input data may be performed. When the electronic device is in a throttling state, the step of obtaining the target input data is suspended. For example, the operating state of the electronic device may be combined with a target triggering event, and the operating state of the electronic device may be adjusted based on the target triggering event.

[0148] Figure 4 The schematic diagram shows the principle diagram of switching the operating state of an electronic device.

[0149] like Figure 4 As shown, after the electronic device is started, it is in a dormant state, waiting for a target trigger event. When a target trigger event occurs, the electronic device enters a waiting state, continuously monitors user operations, and executes the steps of obtaining target input data. When the target trigger event occurs too frequently, the electronic device enters a throttling state, suspends the steps of obtaining target input data, until the throttling state returns to a dormant state. A time window can be set. If the total number of target trigger events occurring within the time window is greater than a preset threshold (the maximum number of target trigger events allowed to occur within the time window), the electronic device enters a throttling state, and suspends the steps of obtaining target input data. When the electronic device enters the throttling state, it is necessary to maintain the throttling state within the throttling time to protect the electronic device from excessive load and ensure the stability and reliability of the electronic device. After the throttling time ends, the electronic device returns to a dormant state or enters other states according to user operations. In addition, the operating state of the electronic device may also include a file generation state. When the target trigger event stops, the system enters a file generation state, analyzes the target input data, and generates files.

[0150] Flexibly adjusting the data acquisition strategy according to the operating status of the electronic device can help save resources, avoid forcibly executing data acquisition when the electronic device is not in an appropriate state, reduce operation delays caused by resource competition or excessive system load, and suspend data acquisition when the electronic device is in a throttling state. This can help prevent the system from being overloaded due to excessive resource consumption and reduce the risk of crashes or freezes.

[0151] Figure 5 The flowchart of the generation method according to another embodiment of the present disclosure is schematically shown. It should be noted that other implementation details of the generation method of this embodiment can be found in Figure 1 The embodiment part of the generation method shown will not be repeated here.

[0152] like Figure 5 As shown, the generating method of this embodiment includes operations S510 to S530.

[0153] In operation S510 , in response to obtaining a data input operation acting on a target application, target input data generated by the data input operation is obtained.

[0154] In some embodiments, the data input operation may include, for example, a string input operation, a voice input operation, a gesture input operation, etc., and the target input data may include, for example, text, voice, image, stick figure handwriting, etc. For example, a data input listener may be set in the target application to listen to the user's data input operation in the target application. When the data input operation is listened to, the data content corresponding to the input operation is parsed and extracted as the target input data. Optionally, in response to obtaining the data input operation acting on the target application, the system updates from the dormant state to the waiting state, waiting for the user to complete the editing. For example, the user opens the presentation software and enters the text "Smart devices for future cities" on page 4 of the presentation. At this time, the listener listens to the user performing the data input operation, and the system enters the waiting state, waiting for the user to complete the data input. When the user stops inputting (such as pausing for more than a set time or the cursor leaves the text box), the user's input text in the presentation is captured and the text is used as the target input data. In operation S520, the target interface data of the target interface of the target application associated with the target input data is obtained.

[0155] In some embodiments, the target interface data may include, for example, display layout information, context information, interface theme style, etc. of the target interface. The target interface can be determined by the location information of the target input file in the current interface and the attribute information of the current interface, and the target interface data can be extracted from the target interface. Among them, the target interface can be the interface where the target input data is located, or it can be another interface that has an association relationship with the interface where the target data is located. The display layout information may include, for example, the position of elements in the interface, the shape of elements, the attributes of elements, and the relative position relationship between elements, and the context information is data associated with the target input data, for example, it may include titles, subtitles, adjacent paragraphs, etc. The interface theme style may include, for example, the color scheme of the interface, the font style, the icons and pictures used in the interface, the arrangement of interface elements, etc.

[0156] Continuing with the presentation in operation S510 as an example, the target interface data (such as the text data, image data, display layout information, etc. in the page) is extracted from page 4 of the presentation, the main title, subtitle, and chapter title of the chapter to which the current page belongs in the presentation are extracted as target interface data, and the interface theme style and display layout information of other pages in the presentation are extracted as target interface data.

[0157] In operation S530 , a target file is generated based on the user intention represented by the target input data and the target interface data.

[0158] In some embodiments, the target input data can be processed by natural language processing, machine learning and other technologies for keyword extraction, semantic and sentiment analysis, etc. to determine the user intent (such as semantic theme and / or emotional theme) represented by the target input data. And based on the target interface data, display parameters are determined. The display parameters can be used to adjust the visual effect of the target file. The display parameters can include, for example, color schemes (such as primary colors, auxiliary colors, background colors, etc.), style information (such as realism, cartoons, etc.), etc. By analyzing the target interface data, the visual style of the target file to be generated is determined, and the target file is generated based on the user intent and display parameters. Optionally, when the target input data and the target interface data are analyzed to determine the user intent and display parameters, the system state is updated to the generation processing state, and the target file is generated based on the user intent and display parameters. Optionally, in response to the completion of the generation of the target file, the system returns to the dormant state.

[0159] Continuing with the presentation in operation S520 as an example, by extracting keywords, we get the keywords: future city, smart device. And perform semantic analysis and sentiment analysis on the target input data to understand the overall meaning and get the emotions expressed by the target input data. And determine the visual style of the target file based on the overall layout, context data, title and subtitle of the slides in the presentation. If the title is "Intelligent Development of Future Cities", the subtitle is "Urban Innovation Brought by Technology", and the color scheme used in the target interface data tends to be blue and cold tones, then the output target file can be a blue and cold tones picture with a sense of technology and futuristic style, which effectively ensures that the target file echoes other content in the displayed document.

[0160] In some embodiments, the generated target file can be displayed in the sidebar of the presentation, so that the user can preview the generated target file, and the user can choose to insert a picture, regenerate or adjust the picture style. In the case where the user chooses to insert a picture, the target file is inserted into the corresponding area of ​​the presentation. Optionally, the insertion position of the target file can be adjusted according to the overall layout of the page to be inserted to avoid the target file from conflicting with the existing layout of the page. Continuing with the presentation in operation S520 as an example, based on the overall layout of page 4, the insertion position of the target file is adjusted, and the target file is inserted into the specified position of page 4.

[0161] In some embodiments, in response to obtaining a data input operation acting on a target application, it is determined whether the user operation is too frequent. If the user operation frequency is too high, a throttling state is entered, and obtaining the data input operation acting on the target application is stopped. A prompt message is generated to prompt the user to operate later to avoid system overload. After the throttling time ends, the system returns to a sleep state.

[0162] The generation method provided by the embodiment of the present disclosure automatically obtains the target input data by capturing and analyzing the data input operation, thereby improving the accuracy and speed of obtaining the target input data. The target interface data associated with the target input data is automatically obtained based on the received target input data, and the background information related to the target input data and user needs are quickly understood, thereby improving the accuracy, diversity and flexibility of target file generation, so that the generated target file is more in line with the actual situation and user needs.

[0163] Figure 6 A block diagram of an electronic device suitable for implementing the generating method according to an embodiment of the present disclosure is schematically shown. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0164] like Figure 6 As shown, the electronic device 600 includes: a processor 410 and at least one processing model 620 capable of running on the processor 610, and the processing model 620 can be called by a target application to perform at least one of the following: in response to a target trigger event, obtaining target input data matching the target trigger event; obtaining target reference data associated with the target input data; generating a target file based on the user intent represented by the target input data and the target reference data; wherein the target reference data is the same or different type as the target input data, and / or the target reference data is the same or different source as the target input data.

[0165] Figure 7 The block diagram of a generating device according to an embodiment of the present disclosure is schematically shown.

[0166] like Figure 7 As shown, the generating device 700 includes a first obtaining module 710 , a second obtaining module 720 and a generating processing module 730 .

[0167] The first obtaining module 710 is used to obtain target input data matching the target trigger event in response to the target trigger event. In one embodiment, the first obtaining module 710 can be used to perform the operation S110 described above, which will not be described in detail here.

[0168] The second obtaining module 720 is used to obtain target reference data associated with the target input data. In one embodiment, the second obtaining module 720 can be used to perform the operation S120 described above, which will not be described in detail here.

[0169] The generation processing module 730 is used to perform generation processing on the target input data and the target reference data to obtain the target file. In one embodiment, the generation processing module 730 can be used to perform the operation S130 described above, which will not be described in detail here.

[0170] The target reference data and the target input data may be of the same or different type.

[0171] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0172] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0173] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0174] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the generation method provided by the embodiment of the present disclosure.

[0175] When the computer program is executed by the processor 610, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0176] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0178] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A generation method, comprising: In response to a target trigger event, obtaining target input data matching the target trigger event; obtaining target reference data associated with the target input data; Performing generation processing on the target input data and the target reference data to obtain a target file; The target reference data and the target input data may be of the same or different type.

2. The method according to claim 1, wherein obtaining target input data matching the target trigger event comprises at least one of the following: In response to obtaining a data input operation acting on a target application, determining the target input data from input data generated by the data input operation based on an operation behavior parameter of the data input operation; In response to obtaining an adjustment operation on a first category object in a target application, taking a target object acted upon by the target adjustment operation as the target input data; In response to obtaining an adjustment operation on a display parameter of an application window of a target application, using change data of the display parameter as the target input data; Obtaining source information of the target trigger event, and obtaining the target input data in a manner matching the source information; In response to monitoring an operation of sharing target data content, using the target data content as the target input data; In response to a change in the usage scenario of the electronic device, using the monitored usage scenario change data as the target input data, the usage scenario including at least one of the usage environment, usage object, and usage mode of the electronic device; In response to detecting that the electronic device establishes a communication connection with the target input device and / or switches to a target position relationship, using the information of the communication connection and / or the target position relationship as the target input data; In response to detecting that the electronic device is switched from a current first device form to a second device form different from the first device form, using the second device form as the target input data; The target application is an application in the electronic device that can directly call the corresponding processing model, or an application that can call the corresponding processing model through the first application.

3. The method according to claim 2, wherein: Determining the target input data from the input data generated by the data input operation based on the operation behavior parameters of the data input operation includes at least one of the following: Determine the input state of the target user based on the operation behavior parameters of the data input operation, determine a screening range from the input data generated by the data input operation based on the input state, and use the input data within the screening range as the target input data; The data having the target operation behavior parameters in the input data generated by the data input operation is determined as the target input data.

4. The method according to claim 1, wherein obtaining target reference data associated with the target input data comprises at least one of the following: Obtaining location information of the target input data, and acquiring the target reference data from an interface and / or a file where the target input data is located based on the location information; Obtaining attribute information of the target input data, and acquiring the target reference data from an interface and / or a file where the target input data is located based on the attribute information; The usage information of the target input data is obtained, and based on the usage information, the user portrait data of the target user and / or the usage environment data of the electronic device are used as the target reference data.

5. The method according to claim 1 or 4, wherein obtaining target reference data associated with the target input data comprises at least one of the following: Based on the position information of the target input data in the current interface, the target display layout data and / or target context data of the target input data in the target interface are acquired, and the target layout data and / or target context data are used as the target reference data, wherein the target interface is the current application interface and / or other application interface of the target application presenting the target input data; In the case where the target input data is text data input into the current interface, using context data associated with the text data and / or display layout data of the text data in the current interface as the target reference data; In a case where the target input data is image data or editing data for an image in the current interface, using application interface parameters of a target application presenting the target input data and / or context data associated with the image data as the target reference data; In the case where the target input data is prompt word data capable of invoking a target processing model, application configuration data of a target application receiving the target input data and / or user portrait data of a target user are used as the target reference data.

6. The method according to claim 1, wherein the generating process of the target input data and the target reference data comprises: Performing a first recognition process on the target input data to determine a semantic theme and / or an emotional theme represented by the target input data; Performing a second recognition process on the target reference data to determine a display parameter represented by the target reference data; The semantic theme and / or emotional theme represented by the target input data and the display parameters represented by the target reference data are input into a target processing model for generation processing to obtain the target file.

7. The method according to claim 1, wherein the generating process of the target input data and the target reference data comprises at least one of the following: Optimizing the user intention represented by the target input data using the target reference data, and inputting the obtained target prompt word data into a target processing model, so as to generate the target file using the target processing model; Performing intent recognition processing on the target input data and the target reference data based on corresponding weights, and performing generation processing on the obtained target intent using a target processing model to obtain the target file; The user intention represented by the target input data is generated using a first processing model, and the first generated result is optimized or regenerated based on the target reference data using a second processing model to obtain the target file.

8. The method according to claim 1 or 7, further comprising at least one of the following: The target file is output to a first display area of ​​the electronic device, so that the target file is updated and displayed in a target display area in an application interface of a target application in response to a target interactive operation, wherein: The first display area is located at or not located at the application interface of the target application; Outputting the target file to a current application interface of the target application to replace the foreground content and / or background content of the current application interface; Obtaining feedback data of a target user on the target file, and updating model parameters of a target processing model based on the feedback data; The operating state of the electronic device is obtained, and the step of obtaining target input data is executed or suspended based on the operating state.

9. A generation method comprising: In response to obtaining a data input operation acting on a target application, obtaining target input data generated by the data input operation; Obtain target interface data of a target interface of a target application associated with the target input data; A target file is generated based on the user intention represented by the target input data and the target interface data.

10. An electronic device comprising at least one processor and at least one processing model capable of running on the processor, wherein the processing model can be called by a target application to perform at least one of the following: In response to a target trigger event, obtaining target input data matching the target trigger event; obtaining target reference data associated with the target input data; generating a target file based on the user intention represented by the target input data and the target reference data; The target reference data and the target input data may be of the same or different type, and / or the target reference data and the target input data may be of the same or different source.

Images