Content generation method and electronic device

Abstract

The embodiments of the present disclosure provide a content generation method and an electronic device which involve the field of artificial intelligence. The method includes: displaying or partially displaying a plurality of pictures input by a user in a preset manner on a content generation function interface; determining a target application and a target generation style; generating associated content with the target generation style for the plurality of pictures in response to a content generation instruction; and displaying the plurality of pictures and the associated content on a specified interface of the target application in response to a sending instruction. Alternatively, the above method executed by electronic apparatus may be executed by using artificial intelligence models.

Description

DescriptionTitle of Invention :CONTENT GENERATION METHOD AND ELECTRONIC DEVICETechnical Field

[0001] The present disclosure relates to a field of artificial intelligence, and more specifically to a content generation method, an electronic device, a computer-readable storage medium and a computer program product.Background Art

[0002] With the development of artificial intelligence technology, especially the development of multimodal Artificial Intelligence Generated Content (AIGC) technology, it brings a lot of convenience to people's life. Meanwhile, with the development of intelligent terminals and the continuous enrichment of applications, there are more and more scenarios in which people share or publish graphic content through various applications (such as social media APP, review APP, takeaway APP, shopping APP, etc.). Therefore, using AIGC technology to automatically generate copywriting based on pictures can improve work efficiency for people.

[0003] However, the existing technology still has some limitations, such as difficulty in controlling the quality of the copywriting, cumbersome operation for users, etc., which makes it difficult to fully satisfy users' needs.Summary of Invention

[0004] According to a first aspect of embodiments of the present disclosure, there is provided a content generation method, the method includes: displaying or partially displaying a plurality of pictures input by a user in a preset manner on a content generation function interface; determining a target application and a target generation style; generating associated content with the target generation style for the plurality of pictures in response to a content generation instruction; and displaying the plurality of pictures and the associated content on a specified interface of the target application in response to a sending instruction.

[0005] Alternatively, the preset manner of inputting the plurality of pictures by the user includes one of inputting the plurality of pictures selected by the user into the content generation function interface by sharing; or inputting the plurality of pictures selected by the user into the content generation function interface by dragging; or inputting the plurality of pictures selected by the user into the content generation function interface by a voice instruction.

[0006] Alternatively, determining the target generation style includes: determining the target generation style based on a user operation; or determining the target generation style based on analyzing the plurality of pictures.

[0007] Alternatively, generating the associated content with the target generation style for the plurality of pictures in response to the content generation instruction includes: pre-processing the plurality of pictures in response to the content generation instruction to obtain a picture with a preset size; generating, based on the picture with the preset size and the target generation style, the associated content with the target generation style for the plurality of pictures.

[0008] Alternatively, pre-processing the plurality of pictures to obtain the picture with the preset size includes: stitching the plurality of pictures, adjusting the stitched picture to the preset size, and obtaining the picture with the preset size; or adjusting a size of each of the plurality of pictures based on the preset size, and stitching each of the adjusted pictures to obtain the picture with the preset size.

[0009] Alternatively, after obtaining the picture with the preset size, the method further includes: adjusting a compression rate of the picture with the preset size to a preset compressionrate; generating the associated content based on the picture with the preset size and the preset compression rate.

[0010] Alternatively, displaying the plurality of pictures and the associated content on the specified interface of the target application in response to the sending instruction includes: sending identification information respectively corresponding to the plurality of pictures and the associated content to the target application at the same time, such that the plurality of pictures and the associated content are displayed on the specified interface of the target application.

[0011] Alternatively, displaying the plurality of pictures and the associated content on the specified interface of the target application in response to the sending instruction includes: sending identification information respectively corresponding to the plurality of pictures to the target application such that the plurality of pictures are displayed on the specified interface of the target application; triggering a content editing function on the specified interface, and copying the associated content to the specified interface such that the plurality of pictures and the associated content are displayed on the specified interface of the target application.

[0012] Alternatively, displaying the plurality of pictures and the associated content on the specified interface of the target application in response to the sending instruction includes: creating a specific picture classification for the plurality of pictures; creating a virtual screen and launching the target application in the virtual screen; triggering a picture selection function of the target application in the virtual screen, and displaying the plurality of pictures in the specified interface of the target application in the virtual screen based on the specific picture classification; triggering a content editing function on the specified interface in the virtual screen and copying the associated content to the specified interface such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the virtual screen; switching display content of the virtual screen to a home screen for displaying such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the home screen.

[0013] Alternatively, triggering the picture selection function of the target application in the virtual screen, and displaying the plurality of pictures in the specified interface of the target application in the virtual screen based on the specific picture classification includes: in response to triggering the picture selection function of the target application in the virtual screen, entering a picture selection interface of the target application; selecting the plurality of pictures from the picture selection interface based on the specific picture classification; triggering entry into the specified interface of the target application and displaying the plurality of pictures in the specified interface.

[0014] According to a second aspect of embodiments of the present disclosure, there is provided a content generation device, the device includes: an inputting module configured to display or partially display a plurality of pictures input by a user in a preset manner on a content generation function interface; and determine a target application and a target generation style; a content generating module configured to generate associated content with the target generation style for the plurality of pictures in response to a content generation instruction; and a sending module configured to display the plurality of pictures and the associated content on a specified interface of the target application in response to a sending instruction.

[0015] Alternatively, the inputting module is configured to: input the plurality of pictures selected by the user into the content generation function interface by sharing; or input the plurality of pictures selected by the user into the content generation function interface by dragging.

[0016] Alternatively, the inputting module is configured to: determine the target generation style based on a user operation; or determine the target generation style based on analyzing the plurality of pictures.

[0017] Alternatively, the content generating module is configured to: pre-process the plurality of pictures in response to the content generation instruction to obtain a picture with a preset size; generate, based on the picture with the preset size and the target generation style, the associated content with the target generation style for the plurality of pictures.

[0018] Alternatively, the content generating module is configured to: stitch the plurality of pictures, adjust the stitched picture to the preset size, and obtain the picture with the preset size; or adjust a size of each of the plurality of pictures based on the preset size, and stitch each of the adjusted pictures to obtain the picture with the preset size.

[0019] Alternatively, the content generating module is configured to: adjust a compression rate of the picture with the preset size to a preset compression rate; generate the associated content based on the picture with the preset size and the preset compression rate.

[0020] Alternatively, the sending module is configured to: send identification information respectively corresponding to the plurality of pictures and the associated content to the target application at the same time, such that the plurality of pictures and the associated content are displayed on the specified interface of the target application.

[0021] Alternatively, the sending module is configured to: send identification information respectively corresponding to the plurality of pictures to the target application such that the plurality of pictures are displayed on the specified interface of the target application; trigger a content editing function on the specified interface, and copy the associated content to the specified interface such that the plurality of pictures and the associated content are displayed on the specified interface of the target application.

[0022] Alternatively, the sending module is configured to: create a specific picture classification for the plurality of pictures; create a virtual screen and launch the target application in the virtual screen; trigger a picture selection function of the target application in the virtual screen, and display the plurality of pictures in the specified interface of the target application in the virtual screen based on the specific picture classification; trigger a content editing function on the specified interface in the virtual screen and copy the associated content to the specified interface such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the virtual screen; switch display content of the virtual screen to a home screen for displaying such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the home screen.

[0023] Alternatively, the sending module is configured to: in response to triggering the picture selection function of the target application in the virtual screen, enter a picture selection interface of the target application; select the plurality of pictures from the picture selection interface based on the specific picture classification; trigger entry into the specified interface of the target application and displaying the plurality of pictures in the specified interface.

[0024] According to a third aspect of the present disclosure, there is provided an electronic apparatus that may include: at least one processor; and at least one memory storing computerexecutable instructions, wherein the computer-executable instructions, when run by the at least one processor, cause the at least one processor to perform the method of an exemplary embodiment of the present disclosure.

[0025] According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium storing computer programs or instructions, wherein the computerprograms or instructions, when executed by at least one processor, cause the at least one processor to perform the method of an exemplary embodiment of the present disclosure.

[0026] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program, wherein the computer program, when executed by a processor, implements the method of exemplary embodiments of the present disclosure.

[0027] The embodiments of the present disclosure enable a user to input multiple pictures simultaneously and intelligently generate high-quality associated content based on the multiple pictures, which is easy to operate and improves the user experience.

[0028] It should be understood that the above general description and the detailed descriptions that follow are merely exemplary and explanatory and do not limit the present disclosure.Brief Description of Drawings

[0029] The accompanying drawings herein are incorporated into and form part of the specification, illustrate embodiments consistent with the disclosure, which are used in conjunction with the specification to explain the principles of the disclosure and do not constitute an undue limitation of the disclosure.

[0030] FIG. 1 is a flowchart of a content generation method according to an exemplary embodiment of the present disclosure.

[0031] FIGS. 2 to 5 are schematic diagrams of inputting multiple images into a content generation function interface according to an exemplary embodiment of the present disclosure.

[0032] FIGS. 6 to 11 are schematic diagrams of pre-processing multiple images according to an example embodiment of the present disclosure.

[0033] FIG. 12 is a schematic diagram of generating associated content according to an example embodiment of the present disclosure.

[0034] FIGS. 13 and 14 are schematic diagrams of a first sending operation of an example embodiment of the present disclosure.

[0035] FIGS. 15 and 16 are schematic diagrams of a second sending operation of an example embodiment of the present disclosure.

[0036] FIGS. 17 and 18 are schematic diagrams of a third sending operation of an example embodiment of the present disclosure.

[0037] FIG. 19 is a schematic diagram of publishing multiple pictures and associated content according to an exemplary embodiment of the present disclosure.

[0038] FIG. 20 is a block diagram illustrating a content generation device according to an exemplary embodiment of the present disclosure.

[0039] FIG. 21 is a block diagram illustrating an electronic apparatus according to an exemplary embodiment of the present disclosure.

[0040] FIG. 22 illustrates a schematic diagram of a structure of an electronic apparatus applicable to an exemplary embodiment of the present disclosure.Description of Embodiments

[0041] The following description with reference to the accompanying drawings is provided to aid in a thorough understanding of various embodiments of the present disclosure as defined by claims and equivalents thereof. This description includes various specific details to aid in understanding but should only be considered exemplary. Accordingly, those ordinary skills in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the presentdisclosure. In addition, descriptions of well-known features and structures may be omitted for the sake of clarity and brevity.

[0042] The terms and phrases used in the claims and the following description are not limited to dictionary meaning thereof, but are used only by the inventor to enable a clear and consistent understanding of the present disclosure. Accordingly, it should be apparent to those skilled in the art that, the following description of the various embodiments of the present disclosure is provided for an illustrative purpose only and is not intended to a purpose of limiting the present disclosure as defined by the appended claims and equivalents thereof.

[0043] It should be understood that, “a”, “an” and “the” in a singular form may also include a plural reference, unless the context clearly indicates otherwise. Thus, for example, a reference to a “part surface” includes a reference to one or more such surfaces. When it refers to one element as being “connected” or “coupled” to another element, the one element may be directly connected or coupled to the other element, or it may refer to a connection relationship between the one element and the other element established through an intermediate element. In addition, “connected” or “coupled” as used herein may include wirelessly connected or wirelessly coupled.

[0044] The term “include” or “may include” refers to the presence of a function, operation, or component of the corresponding disclosure that may be used in the various embodiments of the present disclosure, and does not limit the presence of one or more additional functions, operations, or features. In addition, the terms “include” or “have” may be interpreted to denote certain features, figures, steps, operations, constituent elements, components, or combinations thereof, but should not be interpreted to exclude the possibility of the presence of one or more other features, figures, steps, operations, constituent elements, components, or combinations thereof.

[0045] The term “or” as used in the various embodiments of the present disclosure includes any of the listed terms and all combinations thereof. For example, “A or B” may include A, may include B, or may include both A and B. When describing a plurality of (two or more) items, the plurality of items may refer to one, more, or all of the plurality of items if a relationship among the plurality of items is not explicitly defined. For example, for the description “a parameter A comprises Al, A2, A3”, it may be implemented as parameter A comprising Al, A2 or A3, or as parameter A comprising at least two of the three items of the parameter Al, A2, A3.

[0046] All terms (including technical or scientific terms) used in the present disclosure have the same meaning as understood by those skilled in the art to which the present disclosure belongs, unless defined differently. Common terms as defined in dictionaries are interpreted to have a meaning consistent with the context in the relevant technology art and should not be interpreted in an idealized or overly formalistic manner, unless expressly so defined in the present disclosure.

[0047] At least part of the functions in a device or electronic apparatus provided in the embodiments of the present disclosure may be implemented through an Al model, such as, at least one of a plurality of modules of the device or electronic apparatus may be implemented through the Al model. A function associated with Al may be performed through the nonvolatile memory, the volatile memory, and the processor.

[0048] The processor may include one or more processors. At this time, the one or more processors may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, or may be a graphics-only processing unit such as agraphics processing unit (GPU), a visual processing unit (VPU), and / or an Al-dedicated processor such as a neural processing unit (NPU).

[0049] The one or more processors control processing of input data in accordance with a predefined operating rule or artificial intelligence (Al) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning.

[0050] Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or an Al model of a desired characteristic is made. The learning may be performed in a device or electronic apparatus itself in which Al according to embodiments is performed, and / or may be implemented through a separate server / system.

[0051] The Al model may consist of a plurality of neural network layers. Each layer has a plurality of weight values, and performs a neural network calculation by calculating between the input data of this layer (such as, a calculation result of the previous layer and / or the input data of the Al model) and the plurality of weight values of the current layer. Examples of neural networks include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted Boltzmann Machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a generative adversarial networks (GAN), and a deep Q-network.

[0052] The learning algorithm is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0053] The methods provided in the present disclosure may involve one or more of technical fields such as speech, language, image, video, or data intelligence.

[0054] Alternatively, when involving the field of speech or language, in the content generation method according to the present disclosure, a speech signal, which is an analog signal, may be received via speech input devices (e.g., a microphone), and the speech part is converted into computer readable text using an automatic speech recognition (ASR) model. The user's intent of utterance may be obtained by interpreting the converted text using a natural language understanding (NLU) model. The ASR model or NLU model may be an artificial intelligence model. The artificial intelligence model may be processed by an artificial intelligence-dedicated processor designed in a hardware structure specified for artificial intelligence model processing. Language understanding is a technique for recognizing and applying / processing human language / text and includes, e.g., natural language processing, machine translation, dialog system, question answering, or speech recognition / synthesis.

[0055] Alternatively, when involving the field of image or video, in the content generation method according to the present disclosure, output data may be obtained by using image data as input data for an artificial intelligence model. The method of the present disclosure may involve the field of visual understanding in the artificial intelligence technology, and the visual understanding is a technique for recognizing and processing things as does human vision and includes, e.g., object recognition, object tracking, image retrieval, human recognition, scene recognition, 3D reconstruction / localization, or image enhancement.

[0056] Alternatively, when involving the field of data intelligence processing, in the content generation method according to the present disclosure, in the reasoning or predicting stage, an artificial intelligence model can be used to perform predictions by using real-time input data.Processors of the electronic apparatus may perform a pre-processing operation on the data to convert into a form appropriate for use as an input for the artificial intelligence model. Reasoning and prediction is a technique of logically reasoning and predicting by determining information and includes, e.g., knowledge-based reasoning, optimization prediction, preference-based planning, or recommendation.

[0057] In the present application, the artificial intelligence model may be obtained by training. Here, “obtained by training” means that a predefined operation rule or artificial intelligence model configured to perform a desired feature (or purpose) is obtained by training a basic artificial intelligence model with multiple pieces of training data by a training algorithm. The artificial intelligence model may include a plurality of neural network layers. Each of the plurality of neural network layers includes a plurality of weight values and performs neural network computation by computation between a result of computation by a previous layer and the plurality of weight values.

[0058] At present, an intelligent solution for generating copywriting based on multiple pictures has a problem that the quality of the generated copywriting is not high, especially when the content of the pictures is complicated. The reason is that the current intelligent solution is to analyze an entire publishing interface after uploading the pictures, that is, the picture(s) acquired by a model is / are a screenshot of the current interface or pictures in the current entire interface layout view, which will result in each uploaded picture being small in size and having low clarity, and the quality of the pictures will have a direct impact on the quality of the copywriting generated by the model. In addition, the operation of the current intelligent solution is cumbersome and not user-friendly.

[0059] Based on this, the present disclosure is designed to enable a user to simultaneously input multiple pictures and intelligently generate high-quality associated content based on the multiple pictures. Embodiments of the present disclosure will be described in detail below with reference to FIGS. 1 to 22.

[0060] FIG. 1 is a flowchart of a content generation method according to an exemplary embodiment of the present disclosure. The method shown in FIG. 1 may be performed by an application / logic / circuit / module / unit of the present disclosure for generating associated content based on pictures. The application / logic / circuitry / module / unit of the present disclosure for generating associated content based on pictures may be installed in any electronic device / terminal. In the following, the application / logic / circuitry / module / unit for generating associated content based on pictures may be referred to as a content generation application or by other names.

[0061] Referring to FIG. 1, at step S101, a plurality of pictures input by a user in a preset manner are displayed or partially displayed on a content generation function interface. Here, the content generation function interface may be an interface of a content generation application of the present disclosure. The user may activate or enable the content generation application in some way and input a plurality of pictures selected from any other application interface into the content generation function interface. In the present disclosure, the content may be used interchangeably with copywriting.

[0062] As an example, the preset manner for the user to input the plurality of pictures into the content generation function interface may be: to input the plurality of pictures selected by the user into the content generation function interface by means of sharing; or to input the plurality of pictures selected by the user into the content generation function interface by means of dragging.

[0063] In the present disclosure, the user may select pictures from either application interface. For example, the user selects a plurality of pictures in an application interface and activates a sharing function to input the plurality of pictures into the content generation function interface; or the user selects a plurality of pictures in an application interface and inputs the plurality of pictures into the content generation function interface by dragging and dropping.

[0064] For example, the user may select multiple pictures that the user wants to share in any application interface of the terminal, such as a photo album, my files, a browser page, etc., and then the selected pictures may be sent to the content generation application of the present disclosure through a specific interface / portal and displayed in the content generation function interface.

[0065] In a scenario where a user selects pictures from a photo album, the user may select a plurality of pictures in the photo album that need to be sent to a target application, click on a share button to display multiple icons for applications, select the icon of the content generation application such that this enables access to the content generation function interface, where the plurality of pictures may be displayed or partially displayed in the content generation function interface, as shown in FIG. 2.

[0066] In a scenario where the user selects pictures from my files, the user may select multiple pictures in my files that need to be sent to a target application, click a share button to display multiple icons for applications, select the icon of the content generation application such that this enables access to the content generation function interface, where the multiple pictures may be displayed or partially displayed, as shown in FIG. 3.

[0067] For example, in a scenario of using a screen search function, the user may select multiple pictures in the screen that need to be sent to a target application, click a share button such that multiple icons for applications may be displayed, select the content generation application such that this enables access to the content generation function interface, as shown in FIG. 4. The user may launch screen search applications through corresponding shortcut launch methods to perform a series of operations related to screen content, such as searching, sharing and so on.

[0068] For another example, in a scenario of using an intelligent drag-and-drop function, the user selects multiple pictures to be sent to a target application and performs a drag-and-drop operation, at this time multiple icons for applications will be displayed on a sidebar, and then continues to drag the pictures to the icon of the content generation application to enter the content generation function interface, as shown in FIG. 5. The user may drag the content or part of the content on the screen to the edge of the screen to quickly complete the relevant operations, such as sharing, jumping, etc. through an intelligent dragging function.

[0069] As another example, in a scenario of using a voice assistant function, the user selects multiple pictures that need to be sent to a target application, sends the selected multiple pictures to the content generation application through a voice command such that this enables access to the content generation function interface. For example, after the user selects multiple pictures that the user wants to share in any application interface such as a photo album, my files, a browser page, etc. in the terminal, the user may send a voice command such as “input selected pictures to content generation application” or other specific voice commands to the terminal, and the terminal may input the selected pictures to the content generation application in response to the voice command.

[0070] The above methods of inputting multiple pictures to the content generation application are only exemplary, and the present disclosure is not limited thereto.

[0071] At step SI 02, a target application and a target generation style are determined.

[0072] As an example, the user may select the target application from a plurality of applications via the content generation function interface. For example, a plurality of icons or titles of to-be-selected applications may be presented to the user in the content generation function interface in the form of a drop-down menu for the user to select.

[0073] In determining the target generation style, the target generation style may be determined based on a user operation. Alternatively, the target generation style may be determined based on analyzing the input plurality of pictures.

[0074] As an example, the target generation style may be determined based on the user's selection among a plurality of preset generation styles in the content generation function interface. For example, names of the plurality of preset generation styles may be presented to the user in the content generation function interface in the form of a drop-down menu for the user to select. As another example, the user may customize the target generation style by means of inputting (text input or voice input). As yet another example, the content generation application may determine the target generation style based on analyzing the input plurality of pictures, e.g., the content generation application may intelligently determine a target generation style based on the content of the input plurality of pictures. When at least two of a user-selected target generation style, a user-input target generation style, and an intelligently determined target generation style are present, the user-selected target generation style or the user-input target generation style may be used as the final target generation style.

[0075] At step SI 03, in response to a content generation instruction, associated content with the target generation style for the plurality of pictures is generated.

[0076] The content generation application of the present disclosure may include a neural network model for generating content based on pictures. The neural network model may generate associated content with a plurality of pictures based on the input pictures, a target generation style, and / or a target application.

[0077] After determining the target application and the target generation style, the content generation application may generate the associated content based on the plurality of pictures input by the user, the target application and the target generation style.

[0078] In order to reduce the time for generating the content, the input multiple pictures may be pre-processed first. According to embodiments of the present disclosure, the pre-processing may include at least one of adjusting a size / resolution of a picture, stitching pictures, and adjusting a compression rate of a picture. Adjusting the compression rate of the picture means adjusting a file size of the picture without changing the size / resolution of the picture.

[0079] According to embodiments of the present disclosure, the input plurality of pictures may be pre-processed to obtain a picture of a preset size, and then associated content may be generated based on the picture of the preset size, a target generation style, and / or a target application.

[0080] The input plurality of pictures may be pre-processed based on a resolution / size required by the neural network model to process pictures, such that the pictures are input at the resolution / size that is most suitable for the model's requirements.

[0081] The resolution that is most suitable for the model's requirements may be defined as a resolution at which a picture is input to the neural network model to achieve a combined optimal effect of recognizing content in the picture, a quality of generated copywriting and time consumption. When training the neural network model, a corresponding training set is used, and resolutions of pictures in the training set are usually within a certain range. Therefore, if a resolution of an oversized picture is modified to the optimal resolution range, it will not have a significant impact on recognizing content in the picture and the quality of generatedcopywriting, and at the same time, it will reduce the time required for generating the copywriting. Different neural network models require different resolutions, the most suitable resolution required by the model may be adaptively determined according to the specific neural network model used.

[0082] There are various ways of pre-processing of a resolution / size of a picture, and different pre-processing methods may be used in different scenarios, and examples of pre-processing are explained below.

[0083] A size of each picture in the input plurality of pictures may be adjusted based on a preset size, and each of the adjusted pictures may be stitched to obtain a picture of the preset size; or the input plurality of pictures may be stitched, and the stitched picture may be adjusted to the preset size to obtain a picture of the preset size.

[0084] For example, all of the input pictures are subjected to a resolution modification operation (i.e., resizing the pictures), and then all of the pictures (or several of the pictures) are stitched. The stitching method may be horizontal stitching, vertical stitching, nine-grid stitching, or different pictures may be assigned different layout weights and then stitched based on the layout weights.

[0085] It is assumed that the optimal input resolution of the neural network model used is a*b, when the user selects one picture, if a length or a width of the picture is greater than a and b respectively, the resolution of the picture will be adjusted to a*b. If the length of the resolution of the picture is less than a, the length of the picture may be retained in its original size; similarly, if the width of the resolution of the picture is less than b, the width of the picture may be retained in its original size. When the user selects two pictures, if a length of an original resolution of a certain picture is less than a / 2, the length of the picture may be retained in its original size; similarly, if a width of an original resolution of a certain picture is less than b / 2, the width of the picture may be retained in its original size. If a length and a width of a certain picture are greater than a / 2 and b / 2 respectively, the resolution of the picture is adjusted to (a / 2) x (b / 2), and so forth.

[0086] FIG. 6 illustrates two ways of modifying a resolution according to example embodiments of the present disclosure. In a case where the optimal input resolution of the neural network model is axb, when the user inputs different numbers of pictures, the input pictures are modified in resolution and stitched in different ways. On the left side (a) of FIG.6, one way of modifying a resolution is illustrated, and on the right side (b) of FIG. 6, another way of modifying a resolution is illustrated. In FIG. 6, each block represents the maximum resolution of each picture after adjusting the resolution.

[0087] When the user selects nine pictures (a~ i), the resolution of each picture is not the same. The nine-grid stitching method may be used. It is assumed that the optimal input resolution of the neural network model used is axb, the resolution of each picture is adjusted to a maximum of (a / 3)x(b / 3), and then these pictures are stitched in the order of left to right and top to bottom, as shown in FIG. 7. If a length of an original resolution of a certain picture is less than (a / 3), the length of the picture retains its original size; similarly, if a width of an original resolution of a certain picture is less than (b / 3), the width of the picture retains its original size. In FIG. 7, after adjusting the size of each picture, a compression rate of each picture may be selectively adjusted, and then these pictures are stitched together into a single picture.

[0088] Another example is that the user selects nine pictures (a~i), each of which has a different resolution. The nine-grid stitching method may be used, each picture is stitched together at its original size. For example, pictures a, b and c are arranged in a first row from left to right, pictures d, e and f are arranged in a second row, and pictures g, h and i are arranged in a thirdrow, and stitched in sequence. Then the stitched picture is adjusted to the most suitable resolution / size a*b of the model, as shown in FIG. 8. In addition, after adjusting the size of the stitched picture, a compression rate of the stitched picture may be selectively adjusted.

[0089] According to embodiments, a size of each picture may be adjusted according to a preset size based on at least one of a number of input plurality of pictures, width and height values of each picture in the plurality of pictures, and a weight of each picture, and then each adjusted picture may be stitched to obtain a picture of the preset size.

[0090] For example, layout weights may be assigned to multiple input pictures, and a resolution of each picture may be modified according to the layout weights, and then each picture is stitched together, as shown in FIG. 9. In FIG. 9, picture a is assigned the maximum layout weight, while pictures b~i are assigned the same smaller layout weight. The picture with the large weight may account for a larger proportion of the stitched picture. Therefore, the resolution of the picture a may be adjusted to a relatively large resolution. After stitching each picture, a compression rate of the stitched picture may be selectively adjusted.

[0091] As another example, several pictures may be selected from the input plurality of pictures for resolution modification and stitching processing, as shown in FIG. 10. In FIG. 10, four pictures are selected for resolution modification and stitching from the input nine pictures. After stitching each picture, a compression rate of the stitched picture may be selectively adjusted.

[0092] As yet another example, pictures with a height or width greater than a certain threshold may be subjected to resolution modification by modifying the width value (or height value), and stitching is performed, and then the resolution of the stitched picture is modified to a target resolution. As shown in FIG. 11, it is assumed that the optimal input resolution of the neural network model used is a*b, when the user inputs nine pictures (a~i), a threshold for targeting the height may be a / 3, and a threshold for targeting the width may be b / 3. The heights and widths of the pictures a~i are adjusted to a / 3 or b / 3.

[0093] After obtaining the stitched picture, a compression rate of the stitched picture may be selectively be adjusted. For example, the compression rate of a picture of a preset size is adjusted to a preset compression rate.

[0094] In addition, the compression rate of each picture may also be adjusted to a different value separately before stitching the pictures.

[0095] By adjusting the resolutions of the pictures and adjusting the compression rates of the pictures, the quality requirements of the generated copywriting are satisfied while the time consumption requirements are also satisfied.

[0096] As an example, after the user selects a target application and a target generation style for generating copywriting, pre-processed pictures (or multiple original pictures) are input to the neural network model, the neural network model outputs corresponding copywriting by combining the input pictures and prompt words corresponding to the selected target application and the target generation style. As shown in FIG. 12, after selecting a target application (APP1) and a target generation style (Style 1), corresponding copywriting may be output after clicking a “generate” button. After generating the copywriting, if the user is not satisfied with the content of the currently generated copywriting, the user may re-select a target generation style, and then click on the “generate” button again to generate associated content again until the user is satisfied. The target generation style and application illustrated in FIG. 12 are only exemplary, and the present disclosure is not limited thereto.

[0097] At step SI 04, in response to a sending instruction, the plurality of pictures and the associated content are displayed in a specified interface of the target application.

[0098] A sending operation may be employed for a corresponding type of the target application to realize displaying the plurality of pictures selected by the user and the associated content on the specified interface of the target application.

[0099] For a target application that supports receiving multiple pictures and text, identification information (e.g., a uniform resource identifier (URI)) corresponding to each of the input multiple pictures and the generated associated content may be sent to the target application at the same time, so that the multiple pictures and the associated content may be displayed on a specified interface of the target application, as shown in FIG. 13. In the present disclosure, instead of sending the pictures to the target application, identification information (e.g., URIs) of the pictures may be sent to the target application, so that the target application may find the corresponding original pictures based on the identification information of the pictures, so that the pictures published by the user are clear.

[0100] For example, when the user enters a sending instruction through the content generation function interface, the content generation application can start the target application through an application or logic (such as intent) used for communication between components, and at the same time send URIs of multiple pictures selected by the user together with copywriting to the target application in the intent, and then the target application receives the pictures and the copywriting and loads them directly and displays them on a publishing interface of the target application. The user can click a publish button to publish the content to the social platform, as shown in FIG 14.

[0101] For a target application that supports receiving multiple pictures but cannot receive text at the same time, identification information corresponding to each of multiple pictures may be sent to the target application, so that the multiple pictures are displayed on a specified interface of the target application. A content editing function is triggered on the specified interface and the associated content is copied to the specified interface, so that the multiple pictures and the associated content are displayed on the specified target interface of the target application, as shown in FIG. 15.

[0102] For example, the content generation application can start the target application through the intent, and at the same time send URIs of pictures selected by the user to the target application in the intent, the target application receives the pictures and loads them directly and displays them on a publishing interface of the target application, and the content generation application uses a simulated clicking function to click on an editing text position on the publishing interface of the target application, and pastes the copywriting into an editing text box. The user clicks a publish button to publish the content to the social platform with one click, as shown in FIG. 16.

[0103] For a target application that only supports receiving a single picture and text, a specific picture classification may be created for multiple input pictures, a virtual screen is created and the target application is launched in the virtual screen. A picture selection function of the target application is triggered in the virtual screen, and based on the specific picture classification, the multiple pictures are displayed in the specified interface of the target application in the virtual screen, a content editing function on the specified interface is triggered in the virtual screen, and the associated content is copied to the specified interface, so that the multiple pictures and the associated content are displayed in the specified interface of the target application in the virtual screen, and then the display content of the virtual screen is switched to a home screen for displaying, so that the multiple pictures and the associated content are displayed in the specified interface of the target application on the home screen. As shown in FIG. 17, the content generation application may call an intelligent categorization application(such as a photo function that may provide classification for pictures of a third-party application to help users easily manage and search for photos in a photo album of a cell phone) to create a specific picture classification in the intelligent categorization application for the multiple input pictures. Simulated clicking is performed on the target application in the virtual screen to enter a picture selection interface, and at this point various picture categories (including the specific picture classification mentioned above) output by the intelligent categorization application are displayed. Simulating the selection of the specific picture classification in the virtual screen and copying the associated content to the specified interface makes it possible to display the selected multiple pictures and the associated contents on the specified interface of the virtual screen, and then switching the display content of the virtual screen to the home screen makes it possible to display the multiple pictures and the associated content on the specified interface of the target application on the home screen.

[0104] As another example, a specific picture classification may be created for a plurality of pictures, a virtual screen may be created and a target application may be launched in the virtual screen, and access to a specified interface of the target application, such as a publishing interface, may be triggered in the virtual screen. A picture selection function is triggered via the specified interface of the virtual screen, and the plurality of pictures are displayed in the specified interface in the virtual screen based on the specific picture classification. A content editing function on the specified interface is triggered in the virtual screen, and the associated content is copied to the specified interface, so as to display the plurality of pictures and the associated content in the specified interface in the virtual screen. The display content of the virtual screen is switched to the home screen for displaying, such that the plurality of pictures and the associated content are displayed in the specified interface of the target application on the home screen.

[0105] For example, the content generation application may use an intelligent categorization application to create a specific picture classification / category for pictures selected by the user, and store URIs of the pictures selected by the user under the specific picture category. The specific picture category may be a temporary category, and the intelligent categorization application may delete the specific picture category after transmission of the pictures is completed. The content generation application creates a virtual screen, simulates a click on the target application, and launches it on the virtual screen. Continuing with a simulated click on an add picture button, it enables to enter an add picture screen (i.e., the picture selection screen) of the target application. At this point, a search button is selected in the add picture screen so that picture categories provided by the intelligent categorization application are displayed. The specific picture category mentioned above is displayed in the picture categories, simulated clicking is performed on that specific picture category, and then simulated clicking / sliding is performed to select all the pictures in that category. While selecting the pictures, the clicking may be performed sequentially or simultaneously on all the pictures, or a sliding action may be simulated to select all the pictures. The content generation application simulates a click on a copywriting input box of the target application, and copies and pastes the generated copywriting into the copywriting input box. When the above steps are completed, the content generation application displays a publishing interface of the target application to the home screen. This allows the user to see the publishing interface of the target application, and the user can click a publish button to publish the content to the social platform, as shown in FIG.18.

[0106] FIG. 19 is a schematic diagram of publishing multiple pictures and associated content according to an exemplary embodiment of the present disclosure.

[0107] Referring to FIG. 19, the user selects a plurality of pictures to be sent to a target application in an application such as a photo album or my files, clicks on a share button, displays a plurality of icons of applications, selects the icon of the content generation application therefrom such that this enables access to the content generation function interface, and the plurality of pictures may be displayed or partially displayed in the content generation function interface. In the content generation function interface, a style of copywriting you want to generate (i.e. target generation style) and a target application are selected, a generate button is clicked, and after generating the copywriting, a send button is clicked to send the pictures and the copywriting directly to an editing interface of the target application, and then a publish button is clicked in the editing interface to complete the publication of the pictures and the copywriting. The user can operate simply and intuitively, saving energy and time.

[0108] FIG. 20 is a block diagram illustrating a content generation device according to an exemplary embodiment of the present disclosure.

[0109] Referring to FIG. 20, the content generation device 200 may include an inputting module 201, a content generating module 202, and a sending module 203. The names and numbers of the above modules are merely exemplary and the present disclosure is not limited thereto.

[0110] The inputting module 201 may display or partially display a plurality of pictures input by a user in a preset manner on a content generation function interface.[oni] The inputting module 201 may determine a target application and a target generation style.

[0112] The content generating module 202 may generate associated content with the target generation style for the plurality of pictures in response to a content generation instruction.

[0113] The sending module 203 may display the plurality of pictures and the associated content on a specified interface of the target application in response to a sending instruction.

[0114] According to the embodiments, the inputting module 201 may input the plurality of pictures selected by the user into the content generation function interface by sharing; or input the plurality of pictures selected by the user into the content generation function interface by dragging; or input the plurality of pictures selected by the user into the content generation function interface by a voice instruction. For example, the user selects a plurality of pictures in a specific application and activates a sharing function to input the plurality of pictures into the content generation function interface; or the user selects a plurality of pictures in a specific application and inputs the plurality of pictures into the content generation function interface by dragging. For example, the plurality of pictures may be input by referring to the manners illustrated in FIGS. 2 to 5.

[0115] According to the embodiments, the inputting module 201 may determine the target generation style based on a user operation; or determine the target generation style based on analyzing the plurality of pictures. For example, the user can select the target generation style among a plurality of preset generation styles, or the user can enter the target generation style in text form or voice form.

[0116] According to the embodiments, the content generating module 202 may pre-process the plurality of pictures in response to the content generation instruction to obtain a picture with a preset size; generate, based on the picture with the preset size and the target generation style, the associated content with the target generation style for the plurality of pictures. For example, the associated content may be generated with reference to the manner of FIG. 12.

[0117] According to the embodiments, the content generating module 202 may stitch the plurality of pictures, adjust the stitched picture to the preset size, and obtain the picture withthe preset size; or adjust a size of each of the plurality of pictures based on the preset size, and stitch each of the adjusted pictures to obtain the picture with the preset size. For example, the plurality of pictures may be pre-processed with reference to the manners illustrated in FIGS. 6 to 11.

[0118] According to the embodiments, the content generating module 202 may adjust a compression rate of the picture with the preset size to a preset compression rate; generate the associated content based on the picture with the preset size and the preset compression rate. For example, the plurality of pictures may be pre-processed with reference to the manners illustrated in FIGS. 6 to 11.

[0119] According to the embodiments, the sending module 203 may send identification information respectively corresponding to the plurality of pictures and the associated content to the target application at the same time, such that the plurality of pictures and the associated content are displayed on the specified interface of the target application. For example, a corresponding sending operation may be performed with reference to the manners illustrated in FIGS. 13 and 14.

[0120] According to the embodiments, the sending module 203 may send identification information respectively corresponding to the plurality of pictures to the target application such that the plurality of pictures are displayed on the specified interface of the target application; trigger a content editing function on the specified interface, and copy the associated content to the specified interface such that the plurality of pictures and the associated content are displayed on the specified interface of the target application. For example, a corresponding sending operation may be performed with reference to the manners illustrated in FIGS. 15 and 16.

[0121] According to the embodiments, the sending module 203 may create a specific picture classification for the plurality of pictures; create a virtual screen and launch the target application in the virtual screen; trigger a picture selection function of the target application in the virtual screen, and display the plurality of pictures in the specified interface of the target application in the virtual screen based on the specific picture classification; trigger a content editing function on the specified interface in the virtual screen and copy the associated content to the specified interface such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the virtual screen; switch display content of the virtual screen to a home screen for displaying such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the home screen.

[0122] According to the embodiments, the sending module 203 may enter a picture selection interface of the target application, in response to triggering the picture selection function of the target application in the virtual screen; select the plurality of pictures from the picture selection interface based on the specific picture classification; trigger entry into the specified interface of the target application and displaying the plurality of pictures in the specified interface. For example, a corresponding sending operation may be performed with reference to the manners illustrated in FIGS. 17 and 18.

[0123] In the above, the content generation method according to exemplary embodiments of the present disclosure has been described.

[0124] In the following, an electronic apparatus according to embodiments of the present disclosure is briefly described. FIG. 21 is a block diagram illustrating an electronic apparatus according to an exemplary embodiment of the present disclosure. Referring to FIG. 21, the electronic apparatus 1100 may include a memory 1101 and a processor 1102, wherein theprocessor 1102 is coupled to the memory 1101 and configured to perform any method described above.

[0125] FIG. 22 illustrates a schematic diagram of a structure of an electronic apparatus applicable to an exemplary embodiment of the present disclosure. As shown in FIG. 22, the electronic apparatus 4000 shown in FIG. 22 includes: a processor 4001 and a memory 4003. Wherein the processor 4001 and the memory 4003 are coupled, e.g., through a bus 4002. Alternatively, the electronic apparatus 4000 may further include a transceiver 4004 which may be used for data interaction between the electronic apparatus and other electronic apparatuses, such as transmitting of data and / or receiving of data. It should be noted that, each of the processor 4001, the memory 4003, and the transceiver 4004 is not limited to one in a practice application, and the structure of the electronic apparatus 4000 does not constitute a limitation of the embodiments of the present disclosure. Alternatively, the electronic apparatus may be the first network node, the second network node, or the third network node.

[0126] The processor 4001 may be a Central Processing Unit (CPU), general purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware part, or any combination thereof. It may implement or perform various exemplary logic boxes, modules, and circuits described in conjunction with the disclosed contents of the present disclosure. The processor 4001 may also be a combination that implements computing functions, such as a combination containing one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0127] The bus 4002 may include a pathway to transfer information between the above components. The bus 4002 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, and the like. The bus 4002 may be classed as an address bus, a data bus, a control bus, and the like. For ease of representation, only one bold line is shown in FIG. 22, but it does not mean that there is only one bus or one type of bus.

[0128] The memory 4003 may be a Read Only Memory (ROM) or other types of static storage apparatuses that can store static information and instructions, a Random Access Memory (RAM) or other types of dynamic storage apparatuses that can store information and instructions, may be an Electrically Erasable Programmable Read Only Memory (EEPROM), Compact Disc Read Only Memory (CD-ROM) or other optical disc storages, an optical disc storage (including a compressed disc, laser disc, optical disc, digital universal disc, Blu-ray disc, etc.), a disk storage medium, other magnetic storage apparatuses, or any other medium that can be used to carry or store computer programs and can be read by a computer, it is not limited herein.

[0129] The memory 4003 is used to store computer programs or executable instructions for performing the embodiments of the present disclosure, and is controlled for execution by the processor 4001. The processor 4001 is used to execute the computer programs or executable instructions stored in the memory 4003 to implement the steps shown in the preceding method of the embodiments.

[0130] An embodiment of the present disclosure provides a computer readable storage medium storing computer programs or instructions, the computer programs or instructions, when being executed by at least one processor may perform or implement the steps in the preceding method of the embodiments and corresponding contents.

[0131] An embodiment of the present disclosure provides a computer program product including computer programs, the computer programs, when being executed by a processor,may implement the steps shown in the preceding method of the embodiments and corresponding contents.

[0132] The terms “first”, “second”, “third”, “fourth”, “1”, “2” and the like (if exists) in the specification and claims of the present disclosure and the above drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequence. It should be understood that, data used as such may be interchanged in appropriate situations, so that the embodiments of the present disclosure described here may be implemented in an order other than the illustration or text description.

[0133] It should be understood that, although each operation step is indicated by an arrow in the flowcharts of the embodiments of the present disclosure, an implementation order of these steps is not limited to an order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of the present disclosure, the implementation steps in the flowcharts may be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include a plurality of sub steps or stages, based on an actual implementation scenario. Some or all of these sub steps or stages may be executed at the same time, and each sub step or stage in these sub steps or stages may also be executed at different times. In scenarios with different execution times, an execution order of these sub steps or stages may be flexibly configured according to a requirement, which is not limited by the embodiment of the present disclosure.

[0134] The above text and accompanying drawings are provided as examples only to assist readers in understanding the present disclosure. They are not intended and should not be interpreted as limiting the scope of the present disclosure in any way. Although certain embodiments and examples have been provided, based on the content disclosed herein, it is apparent to those skilled in the art that, changes can be made to the illustrated embodiments and examples without departing from the scope of the present disclosure, and other similar implementation methods based on the technical concepts of the present disclosure also belongs to a protection scope of the embodiments of the present disclosure.

Claims

Claims

1. A content generation method comprising:displaying or partially displaying a plurality of pictures input by a user in a preset manner on a content generation function interface;determining a target application and a target generation style;generating associated content with the target generation style for the plurality of pictures in response to a content generation instruction; anddisplaying the plurality of pictures and the associated content on a specified interface of the target application in response to a sending instruction.

2. The method according to claim 1, wherein the preset manner of inputting the plurality of pictures by the user comprises one of:inputting the plurality of pictures selected by the user into the content generation function interface by sharing; orinputting the plurality of pictures selected by the user into the content generation function interface by dragging; orinputting the plurality of pictures selected by the user into the content generation function interface by a voice instruction.

3. The method according to claim 1, wherein determining the target generation style comprises:determining the target generation style based on a user operation; ordetermining the target generation style based on analyzing the plurality of pictures.

4. The method according to claim 1, wherein generating the associated content with the target generation style for the plurality of pictures in response to the content generation instruction comprises:pre-processing the plurality of pictures in response to the content generation instruction to obtain a picture with a preset size;generating, based on the picture with the preset size and the target generation style, the associated content with the target generation style for the plurality of pictures.

5. The method according to claim 4, wherein pre-processing the plurality of pictures to obtain the picture with the preset size comprises:stitching the plurality of pictures, adjusting the stitched picture to the preset size, and obtaining the picture with the preset size; oradjusting a size of each of the plurality of pictures based on the preset size, and stitching each of the adjusted pictures to obtain the picture with the preset size.

6. The method according to claim 4 or 5, wherein after obtaining the picture with the preset size, the method further comprises:adjusting a compression rate of the picture with the preset size to a preset compression rate;generating the associated content based on the picture with the preset size and the preset compression rate.

7. The method according to claim 1, wherein displaying the plurality of pictures and the associated content on the specified interface of the target application in response to the sending instruction comprises:sending identification information respectively corresponding to the plurality of pictures and the associated content to the target application at the same time, such that the plurality of pictures and the associated content are displayed on the specified interface of the target application.

8. The method according to claim 1, wherein displaying the plurality of pictures and the associated content on the specified interface of the target application in response to the sending instruction comprises:sending identification information respectively corresponding to the plurality of pictures to the target application such that the plurality of pictures are displayed on the specified interface of the target application;triggering a content editing function on the specified interface, and copying the associated content to the specified interface such that the plurality of pictures and the associated content are displayed on the specified interface of the target application.

9. The method according to claim 1, wherein displaying the plurality of pictures and the associated content on the specified interface of the target application in response to the sending instruction comprises:creating a specific picture classification for the plurality of pictures;creating a virtual screen and launching the target application in the virtual screen; triggering a picture selection function of the target application in the virtual screen, and displaying the plurality of pictures in the specified interface of the target application in the virtual screen based on the specific picture classification;triggering a content editing function on the specified interface in the virtual screen and copying the associated content to the specified interface such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the virtual screen;switching display content of the virtual screen to a home screen for displaying such that the plurality of pictures and the associated content are displayed in the specified interface of the target application in the home screen.

10. The method according to claim 9, wherein triggering the picture selection function of the target application in the virtual screen, and displaying the plurality of pictures in thespecified interface of the target application in the virtual screen based on the specific picture classification comprises:in response to triggering the picture selection function of the target application in the virtual screen, entering a picture selection interface of the target application;selecting the plurality of pictures from the picture selection interface based on the specific picture classification;triggering entry into the specified interface of the target application and displaying the plurality of pictures in the specified interface.

11. A content generation device comprising:an inputting module configured to display or partially display a plurality of pictures input by a user in a preset manner on a content generation function interface; and determine a target application and a target generation style;a content generating module configured to generate associated content with the target generation style for the plurality of pictures in response to a content generation instruction; and a sending module configured to display the plurality of pictures and the associated content on a specified interface of the target application in response to a sending instruction.

12. An electronic device comprising:at least one processor; andat least one memory storing computer-executable instructions,wherein the computer-executable instructions, when run by the at least one processor, cause the at least one processor to perform the method in any one of claims 1 to 10.

13. A computer-readable storage medium storing instructions, wherein the instructions, when run by at least one processor, cause the at least one processor to perform the method in any one of claims 1 to 10.

Images