Data processing method, image generation method and electronic equipment

Abstract

The invention discloses a data processing method, an image generation method and electronic equipment, and the method comprises the steps: obtaining attribute information of a plurality of to-be-processed objects and / or association information among the plurality of to-be-processed objects in response to the obtained plurality of to-be-processed objects, the to-be-processed objects including or not including a response result generated by the electronic equipment in response to the intention of a target user; and performing corresponding processing on the plurality of to-be-processed objects based on the attribute information and / or the associated information to obtain a target file, the target file comprising at least one of object contents represented by the plurality of to-be-processed objects. Therefore, under the condition that the to-be-processed object is the image, fusion processing is performed on the plurality of images based on the attribute information of each image element and / or the association information between the image elements, so that each image element is naturally fused together, and a natural and beautiful effect is achieved.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a data processing method, an image generating method, and an electronic device. Background Art

[0002] In the field of modern image processing and generation, users often want to fuse multiple objects or elements (such as people, objects, scenery, etc.) into one image. Existing image fusion systems often only focus on the generation of image content, which can easily lead to unnatural generation results and make it difficult to achieve the expected results. Summary of the invention

[0003] The present application provides a data processing method, an image generating method, and an electronic device.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, a data processing method is provided, comprising:

[0006] In response to obtaining a plurality of objects to be processed, obtaining attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed, wherein the objects to be processed include or do not include response results generated by the electronic device in response to the target user's intention;

[0007] Based on the attribute information and / or the association information, the multiple objects to be processed are processed accordingly to obtain a target file, wherein the target file includes at least one of the object contents represented by the multiple objects to be processed.

[0008] In a second aspect, an image generation method is provided, comprising:

[0009] In response to the target user's intention to generate a plurality of image elements, obtaining attribute information of the plurality of image elements and / or association information between the plurality of image elements;

[0010] configuring display layout information between the plurality of image elements based on the attribute information and / or the association information to generate a target image based on the display layout information;

[0011] The target users intend to come from the same user or different users.

[0012] According to a third aspect, an electronic device is provided, comprising: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the methods of the first and second aspects when running the computer program.

[0013] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the methods of the first and second aspects when executed by a processor.

[0014] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein the computer program implements the steps of the methods of the first and second aspects when executed by a processor.

[0015] The present application provides a data processing method, including: in response to obtaining multiple objects to be processed, obtaining attribute information of the multiple objects to be processed and / or association information between the multiple objects to be processed, the objects to be processed include or do not include response results generated by the electronic device in response to the intention of the target user; based on the attribute information and / or the association information, corresponding processing is performed on the multiple objects to be processed to obtain a target file, the target file includes at least one of the object contents represented by the multiple objects to be processed. In this way, when the object to be processed is an image, based on the attribute information of each image element and / or the association information between the image elements, a fusion process is performed on the multiple images, so that each image element is naturally fused together, thereby achieving a natural and beautiful effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the data processing method in the embodiment of the present application;

[0017] Figure 2 This is a schematic diagram of the structure of the data processing device in the embodiment of the present application;

[0018] Figure 3 The process diagram of the image generation method in the embodiment of the present application is as follows Figure 1 ;

[0019] Figure 4 This is a schematic diagram of the structure of the image generating device in the embodiment of the present application;

[0020] Figure 5 This is a schematic diagram of the architecture of the image generation system in the embodiment of the present application;

[0021] Figure 6 The process diagram of the image generation method in the embodiment of the present application is as follows Figure 2 ;

[0022] Figure 7 It is a schematic diagram of the structure of the electronic device in the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this embodiment and are not intended to limit this application.

[0025] In the following description, references to “some embodiments,” “this embodiment,” “this embodiment,” and examples, etc., describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0026] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the present embodiment described here can be implemented in an order other than that illustrated or described here.

[0027] In this embodiment, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, object A and / or object B may represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0028] The present application embodiment provides a data processing method, Figure 1 Schematic diagram of the data processing method in the embodiment of the present application, which is applied to electronic devices, which may be smart phones, tablet computers, desktop computers and other devices. Figure 1 As shown, the data processing method may include:

[0029] S101: In response to obtaining a plurality of objects to be processed, obtaining attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed, wherein the objects to be processed include or do not include response results generated by the electronic device in response to the target user's intention.

[0030] In an embodiment of the present application, the object to be processed may be at least one type of object among text, image, voice and video, or it may be an object generated and processed by at least one of text, image, voice and video using an artificial intelligence generation model (such as a text-to-image model, a image-to-image model, a text-to-text model, a text-to-video model, etc.).

[0031] In one achievable manner, the object to be processed includes at least one type of object among text, image, voice and video input by the user. The user may be one user or multiple users, that is, one user may input multiple objects to be processed in one electronic device, or multiple objects to be processed in multiple electronic devices; multiple users may input multiple objects to be processed in one electronic device, or multiple users may input one object to be processed in their respective electronic devices.

[0032] For example, when the object to be processed is an image, the attribute information of the object in the image includes at least one of the following: type information, size, shape, color, position, importance, priority, and the association information between objects includes at least one of the following: primary and secondary relationship, symmetry, hierarchy, and display rules. When the object to be processed is text, its attribute information includes at least one of the following: type information, language information, and emotional information, and the association information includes at least one of the following: logical order and display rules.

[0033] In another achievable manner, the object to be processed includes a response result generated by the electronic device in response to the target user's intention. The implementation steps may be: using the artificial intelligence model built into the electronic device to respond to the user's intention and generate a response result, i.e., the object to be processed.

[0034] The object to be processed is generated based on the user's intention represented by the user's input data. For example, a prompt word is entered in Lenovo Xiaotian or Lenovo Shengtu application, and the prompt word is generated and processed using the artificial intelligence model preset locally on Lenovo computers (i.e. electronic devices) or preset in the cloud to generate the image or video or multimedia file content that the target user wants, i.e. the object to be processed. Multimedia files are files containing multiple information forms such as text, images, audio, video, animation, etc.

[0035] As a further example, if the prompt word is "a puppy running on the grass in the sun", the prompt word is processed using the artificial intelligence model preset locally on the Lenovo computer or preset in the cloud, and the generated objects to be processed include sunshine, grass, running, and puppy. If the prompt word is "a puppy running and barking on the grass in the sun", the prompt word is processed using the artificial intelligence model preset locally on the Lenovo computer or preset in the cloud, and the generated objects to be processed include sunshine, grass, running, and puppy, as well as the text content of the puppy barking or "woof woof woof".

[0036] For example, input an image into the Lenovo Xiaotian or Lenovo Shengtu application, process the image using the artificial intelligence model pre-installed locally on the Lenovo computer or pre-installed in the cloud, and identify at least one image element included in the image, namely, the object to be processed.

[0037] As a further example, if the image includes mountains, water, cattle, sheep and green plants, the image is processed using an artificial intelligence model preset locally on the electronic device or in the cloud, and the objects to be processed obtained include mountains, water, cattle, sheep and green plants; then, illustratively, the attribute information of multiple objects to be processed may be the size, shape, color, position, importance, etc. of the mountains, water, cattle, sheep and green plants, and the association information between the objects to be processed may be the primary and secondary relationships, symmetrical relationships, hierarchical relationships, etc. between the mountains, water, cattle, sheep and green plants, which can reflect the display layout between the mountains, water, cattle, sheep and green plants.

[0038] In another achievable manner, the object to be processed includes at least one type of object among text, image, voice and video input by the user, and also includes a response result generated by the electronic device in response to the target user's intention. In an embodiment of the present application, different processing operations are performed on different objects to be processed to obtain attribute information of the object to be processed and / or association information between the objects to be processed. Exemplarily, if the object to be processed is an image, image analysis operations such as target detection and feature extraction are performed on the image to obtain attribute information of image elements in the image and / or association information between image elements. Alternatively, if the object to be processed is text, the text is segmented to obtain attribute information of keywords in the text and / or association information between keywords.

[0039] S102: performing corresponding processing on a plurality of objects to be processed based on the attribute information and / or the associated information to obtain a target file, wherein the target file includes at least one of the object contents represented by the plurality of objects to be processed.

[0040] In the embodiment of the present application, the corresponding processing includes at least one of the following: fusion processing, display parameter adjustment, format conversion processing and generation processing.

[0041] For example, if the object to be processed is an image, based on the attribute information of the object in the image and the association information between the objects, multiple images are fused to obtain a fused image, i.e., a target file; or, based on the attribute information of the object in the image and the association information between the objects, the display parameters of multiple images are adjusted to obtain an adjusted image, i.e., a target file. If the multiple objects to be processed include text and images, the text is format-converted to an image, and fused with the remaining images to obtain a fused image, i.e., a target file; or, the image is format-converted to a text document or a text illustration, and generated with the remaining text to obtain the generated text, i.e., a target file.

[0042] In the embodiment of the present application, the target file includes at least one of the object contents represented by the multiple objects to be processed, that is, the number of object contents is reduced. It should be noted that the reduction in the number of object contents may be due to the fact that some object contents are screened out due to conflicts or positional relationships or other reasons, or may be erased by the user's adjustment input.

[0043] For example, if two object contents are located in the same position, the unimportant object contents can be filtered out. Alternatively, if a certain object content is unimportant in the target file and does not affect the overall effect, the object content can be deleted. Alternatively, the user instructs to filter out a certain object content.

[0044] In an embodiment of the present application, when the object to be processed is an image, multiple images are fused based on the attribute information of each image element and / or the association information between the image elements, so that the various image elements are naturally integrated together, thereby achieving a natural and beautiful effect.

[0045] In some embodiments, obtaining a plurality of objects to be processed includes at least one of the following:

[0046] In response to obtaining first input data input to a target application, splitting the user intent represented by the first input data, and using object contents represented by multiple sub-user intents obtained by splitting as the multiple objects to be processed;

[0047] In response to obtaining second input data input to the target application, generating a response result for the second input data, and using a plurality of object contents in the response result as the plurality of objects to be processed;

[0048] In response to obtaining a plurality of third input data input to the target application, obtaining a plurality of user intentions represented by the plurality of third input data, and using the object contents represented by the plurality of user intentions as the plurality of objects to be processed;

[0049] In response to obtaining fourth input data input to the target application, outputting a response result for the fourth input data, and in the case of obtaining fifth input data for the response result, taking at least one object content in the fifth input data and the response result as the multiple objects to be processed;

[0050] The obtained multiple object contents selected by at least one user and / or the received multiple object contents sent from the target device are used as the multiple objects to be processed, wherein the multiple object contents are of the same type or different types.

[0051] In the embodiments of the present application, the target application may be a raw image application (such as Lenovo's creator zone), a raw text application, a raw video application, and an intelligent application (such as Lenovo Xiaotian or AI NOW), or it may be an application that can call the above applications (i.e., raw image applications, raw text applications, raw video applications, and intelligent applications), such as conference applications, social media applications, and photo album applications. These applications (i.e., conference applications, social media applications, and photo album applications) can realize the effect of model processing services by calling the above applications, or these applications may have built-in large models to realize large model processing capabilities.

[0052] In the embodiment of the present application, the first input data includes at least one of the following: string data such as text, voice data, image data and other multimedia data.

[0053] For example, when the first input data is text data, the user intent represented by the text data is split to obtain multiple keywords, namely, sub-user intents, and the object contents represented by the multiple keywords are used as multiple objects to be processed. Alternatively, when the first input data is image data, the user intent represented by the image data is split to obtain multiple image elements, namely, sub-user intents, and the multiple image elements are used as multiple objects to be processed.

[0054] As a further example, the text data input to the raw image application is "a puppy running on the grass in the sun", and the keywords after splitting include generating sunshine, generating grass, generating a running puppy, and the object content represented by generating sunshine, generating grass, and generating a running puppy (sunlight, grass, running puppy) are used as the objects to be processed. Alternatively, the image data input to the raw image application includes mountains, water, cattle, sheep, and green plants, and the image elements after splitting include generating mountains, generating water, generating cattle, generating sheep, and generating green plants, and the object content represented by generating mountains, generating water, generating cattle, generating sheep, and generating green plants (mountains, water, cattle, sheep, and green plants) are used as the objects to be processed.

[0055] In response to obtaining second input data input to the target application, generating a response result for the second input data, and using a plurality of object contents in the response result as the plurality of objects to be processed;

[0056] In the embodiment of the present application, the second input data includes at least one of the following: string data such as text, voice data, multimedia data such as image data, etc. The response result of the second input data includes at least one of the following: generated image, audio, text, video and user interface (UI) prototype.

[0057] Based on this, when the second input data input to the text generation application is text data, in response to the text data, new text data for the text data, that is, a response result, is generated, multiple keywords in the new text data are identified, and the object content represented by the keywords is used as the object to be processed. The number of segments of the text data is at least one segment. Alternatively, when the second input data input to the image generation application is image data, in response to the image data, new image data for the image data, that is, a response result, is generated, and multiple image elements (i.e., object content) in the new image data are identified as objects to be processed. The number of frames of the image data is at least one frame. Alternatively, when the second input data input to the audio generation application is audio data, in response to the audio data, new audio data for the audio data, that is, a response result, is generated, and multiple audio segments (i.e., object content) in the new audio data are identified as objects to be processed. Alternatively, when the second input data is video data, in response to the video data, new video data for the video data, that is, a response result, is generated, and multiple frames of images (i.e., object content) in the new video data are identified as objects to be processed.

[0058] For example, the second input data input to the text or image application is text, such as "a puppy running on the grass in the sun", which can be polished, and the response result generated for the second input data is "a puppy running on the grass in the sun, its tail is raised high, showing endless joy and vitality", and its keywords include sunshine, grass, running, puppy and tail raised high, that is, the object to be processed. Alternatively, the second input data is an image, such as a scene representing a herder herding cattle and sheep on the grassland, and the response result generated for the second input data is an image after adjusting the image parameters, and the image elements in the adjusted image, namely the herder, cattle, sheep and grassland, are used as the object to be processed.

[0059] In response to obtaining a plurality of third input data input to the target application, obtaining a plurality of user intentions represented by the plurality of third input data, and using the object contents represented by the plurality of user intentions as the plurality of objects to be processed;

[0060] In an embodiment of the present application, the third input data includes at least one of the following: string data such as text, voice data, multimedia data such as image data. In the case where multiple third input data are different data input by the same user multiple times, the multiple user intentions represented by the multiple third input data are multiple different intentions of the same user. In the case where multiple third input data are different data input by multiple users, the multiple user intentions represented by the multiple third input data are different intentions of multiple users. The object content represented by the user intention includes at least one of the following: image content, text content, voice content, and video content.

[0061] In an embodiment of the present application, the user intent represented by the third input data is obtained. The implementation step may be to perform intent analysis on the third input data to obtain the corresponding user intent. For example, when the third input data is an image, the image is analyzed for intent to obtain the image scene content, that is, the user intent. Alternatively, when the third input data is text, the text is analyzed for intent to obtain the text content such as an emotional tag. Alternatively, the third input data is speech, and the speech is analyzed for intent to obtain the text content of the speech.

[0062] As a further example, when the third input data is an image, the user intention represented by the image is to generate an image of a "puppy running on the grass in the sun", then the object content to be generated in the image, namely the sun, grass, running and puppy, are the objects to be processed.

[0063] In response to obtaining fourth input data input to the target application, outputting a response result for the fourth input data, and in the case of obtaining fifth input data for the response result, taking at least one object content in the fifth input data and the response result as the multiple objects to be processed;

[0064] In an embodiment of the present application, the fourth input data includes at least one of the following: string data such as text, multimedia data such as voice data, and image data. Outputting a response result for the fourth input data can be understood as displaying at least one of a generated image, text, or outputting a generated audio clip. The fifth input data may be replacement content for the generated response result or additional new content. If it is an additional addition, the additional new content and at least one object content in the response result are used as objects to be processed. If it is a replacement, the replacement content and the object content in the response result other than the replaced content are used as objects to be processed.

[0065] For example, when the fourth input data is an image of a puppy running, a generated image (response result) of a puppy running on the grass generated based on the image is displayed. When a new background image for the generated image is obtained, such as a desert, the grass background is replaced by the desert background, and the desert background image and the image elements in the generated image other than the grass background are used as objects to be processed. Alternatively, when additional image elements for the generated image are obtained, such as sunlight, the additional sunlight image elements and the generated image are used as objects to be processed.

[0066] The obtained multiple object contents selected by at least one user and / or the received multiple object contents sent from the target device are used as the multiple objects to be processed, wherein the multiple object contents are of the same type or different types.

[0067] In an embodiment of the present application, the multiple object contents selected by at least one user may include at least one of the following: multiple images, multiple text segments, multiple audio segments, and multiple visual objects (i.e., display objects in the image) selected by the same user, and may also include at least one of the following: different types of images, texts, audios, and videos selected by multiple users. That is to say, the same user inputs multiple object contents of different or the same type to the same electronic device as objects to be processed; or, different users input different or the same type of object contents to different electronic devices as objects to be processed. Alternatively, multiple object contents of different or the same type sent by the target device are taken as objects to be processed. Alternatively, multiple object contents of different or the same type selected by at least one user and multiple object contents of different or the same type sent by the target device are taken as objects to be processed.

[0068] For example, if the same user selects multiple frames of images, the multiple frames of images are the objects to be processed, and the multiple frames of images may be fused subsequently. Alternatively, if the same user selects images and texts, the images and texts are the objects to be processed, and the images may be converted into texts, and then generated with the texts selected by the user, or the texts may be converted into images or text illustrations, and then fused with the images selected by the user. Alternatively, if different users select images, texts, and videos respectively, the images, texts, and videos are the objects to be processed, and the texts may be converted into images or text illustrations, and then fused with the images and videos selected by the users to obtain new videos. Alternatively, if different users select images, the images of different users are the objects to be processed, and the images of different users may be fused subsequently. Alternatively, if the same user selects images and texts, and also receives videos and audios from the target device, the images, texts selected by the user, and the videos and audios of the target device are the objects to be processed, and the texts may be converted into images or text illustrations, and then generated with the images selected by the users, and the videos and audios of the target device to obtain new videos with sound effects.

[0069] In some embodiments, obtaining the attribute information of the plurality of objects to be processed and / or the association information between the plurality of objects to be processed includes:

[0070] Obtaining source information of the plurality of objects to be processed;

[0071] In the case where the source information represents that the multiple objects to be processed are response results generated by the electronic device in response to the target user's intention, performing an operation of obtaining attribute information of the multiple objects to be processed and / or performing an operation of obtaining association information between the multiple objects to be processed;

[0072] It should be noted that, when the source information represents that the object to be processed comes from the response result generated by the electronic device in response to the target user's intention, since the object to be processed is based on the local generation model of the electronic device or the cloud generation model, obtaining the attribute information of multiple objects to be processed and / or the association information between multiple objects to be processed is, on the one hand, to verify whether the response result generated by the electronic device complies with the standard or is logical or meets the user's intention, and on the other hand, to assist the corresponding processing operations on multiple objects to be processed, such as further optimization processing, such as beautification, polishing, and updating.

[0073] For example, if the object to be processed comes from an image, and the image content is "a puppy running on the grass in the sun", then the attribute information of the sun, the grass, and the puppy, as well as the association information between the sun, the grass, and the puppy, are obtained.

[0074] or, in the case where the source information indicates that the multiple objects to be processed are from multiple object contents input by the same user, performing an operation of obtaining attribute information of the multiple objects to be processed;

[0075] It should be noted that, when multiple objects to be processed come from multiple object contents input by the same user, considering that the user may input different types of object contents, fusion may be performed based on attribute information of each object content.

[0076] For example, if multiple objects to be processed come from an image and a text input by the user, and the image content is "a puppy running on the grass in the sun", then the attribute information of the sun, grass, and puppy, as well as the association information between the sun, grass, and puppy are obtained; if the text content is "a herd of cattle and sheep running on the grassland", then the grassland, cattle, and sheep are obtained. The key information, i.e., attribute information, is obtained. Furthermore, based on these attribute information and association information, a fused image of "a herd of cattle and sheep and a puppy running on the grassland" is generated.

[0077] or, in the case where the source information represents that the multiple objects to be processed are from multiple object contents input by different users, performing an operation of obtaining attribute information of the multiple objects to be processed and association information between the multiple objects to be processed;

[0078] It should be noted that when the object contents come from different users, it is necessary to know the attributes of the object contents input by each user and the possible association relationships, so as to merge, automatically edit or automatically generate multiple objects to be processed according to the attributes and association information.

[0079] For example, if one user inputs living room image 1, and the image content includes a sofa and a coffee table, and another user inputs living room image 2, and the image content includes green plants, a dining table, a sofa, and ornaments, then the attribute information of the sofa and the coffee table, as well as the positional relationship between the sofa and the coffee table, are obtained respectively, and the attribute information of the green plants, the dining table, the sofa, and the ornaments, as well as the positional relationship between the green plants, the dining table, the sofa, and the ornaments are obtained respectively, and based on this information, a living room image including green plants, the dining table, the sofa, the ornaments and the coffee table is further generated.

[0080] or, in the case where the source information represents that the multiple objects to be processed are from multiple object contents sent by the target device, performing an operation of obtaining attribute information of the multiple objects to be processed and association information between the multiple objects to be processed;

[0081] It should be noted that, considering the situations of different devices, it is necessary to obtain the attribute information of multiple objects to be processed and the association information between the multiple objects to be processed, so as to facilitate the subsequent corresponding processing of the multiple objects to be processed, so that the obtained target file is a file that conforms to the target version or format.

[0082] Exemplarily, if the first image taken by the first device is in PNG format and the second image taken by the second device is in JPEG format, the image format is the attribute information, the first image is an image of a living room, and the second image is an image of a puppy running on the grass, the association information between the two images can be that the elements in the living room image are the main elements, and the puppy is the secondary element. The fused image is a fused image that contains the living room elements and the puppy elements, and the format is PNG format (i.e., the target format).

[0083] Or, in a case where the source information represents that the multiple objects to be processed are from object contents represented by the target user's intention, an operation of obtaining association information between the multiple objects to be processed is performed.

[0084] It should be noted that in the case of multiple user intentions, since image generation is required, the main consideration is the positional relationship, proportional relationship, primary and secondary relationship, symmetry, hierarchical structure and other related information between objects.

[0085] For example, if the image input by one user includes a sofa element, the image input by another user includes a coffee table element, and the image input by another user includes a green plant element, considering that the coffee table is usually directly in front of the sofa, the green plant element can be on the coffee table or on the side of the coffee table. The fused image is a fused image of a coffee table directly in front of the sofa with green plants placed on the coffee table, or green plants placed on the side of the coffee table.

[0086] On the basis of the above embodiments, different information acquisition operations can also be performed based on the type of the object to be processed. For example, when the object to be processed is an image, in addition to obtaining the attribute information of each object in the image, it is also necessary to obtain the association information between the objects. Alternatively, when the object to be processed is a video, only the association information between the image frames can be obtained. Alternatively, when the object to be processed is a text, only the association information can be obtained without obtaining the attribute information.

[0087] In some embodiments, obtaining attribute information of multiple objects to be processed includes at least one of the following:

[0088] In a case where the plurality of objects to be processed are from object contents represented by the target user's intention, using first description information of the object contents as attribute information of the object to be processed;

[0089] In the embodiment of the present application, the object content represented by the target user's intention includes at least one of the following: objects, backgrounds, people, animals, styles, tones, and scenes. The first description information of the object content refers to information describing the object content, and the first description information may include at least one of the following: type, size, shape, color, location, importance, and priority.

[0090] For the case where the first description information of the object content is used as the attribute information of the object to be processed, for example, if the object to be processed comes from an image (i.e., the object content that the user intends to represent), and the image content is "a lady is lying on the right side of a black sofa holding a white cat", then the first description information about the sofa is a black sofa, and the attribute information of the sofa includes the color, i.e., black; the first description information about the lady is lying on the right side of the sofa, and the attribute information of the lady includes the posture, i.e., lying down, and the position, i.e., the right side of the sofa; the first description information about the cat is a white cat lying in the lady's arms, and the attribute information of the cat includes the color, i.e., white, the position, i.e., in the lady's arms, and the posture, i.e., lying down.

[0091] In a case where the plurality of objects to be processed are from object contents input by at least one user or sent by a target device, using first description information of the object contents and / or a first recognition result obtained by recognizing the object contents as attribute information of the objects to be processed;

[0092] In the embodiment of the present application, when multiple objects to be processed are from object contents input by at least one user or sent by a target device, the object contents may be a single element object, an entire image, or descriptive input data, i.e., text. In the case where the object content is an entire image, the image is recognized and processed to identify each image element in the image, as well as display parameters of the image elements such as display size, color, position, and style, i.e., the first recognition result, which is the attribute information corresponding to the image. In the case where the object content is text, keywords in the text are recognized as the first recognition result, i.e., the attribute information of the text.

[0093] Regarding the case where the first recognition result obtained by identifying the content of the object is used as the attribute information of the object to be processed, for example, if the object to be processed comes from an image, and the image content is "a lady is lying on the right side of a black sofa holding a white cat", it is recognized and processed, and the first recognition result obtained indicates that the sofa is black, that is, the attribute information of the sofa includes the color, i.e., black, and indicates that the lady is lying on the right side of the sofa, that is, the attribute information of the lady includes the posture, i.e., lying down, and the position, i.e., the right side of the sofa, and indicates that the white cat is lying in the lady's arms, that is, the attribute information of the cat includes the color, i.e., white, the position, i.e., in the lady's arms, and the posture, i.e., lying down.

[0094] In the case that the multiple objects to be processed are from response results generated by the electronic device in response to the target user's intention, the presentation parameters of each object to be processed in the response result are used as the attribute information of the object to be processed.

[0095] In an embodiment of the present application, when multiple objects to be processed are response results generated by an electronic device in response to the target user's intention, if the response result is a generated image, the presentation parameters are the display parameters of each image element in the image; if the response result is a generated audio, the presentation parameters are the audio parameters of each audio segment in the image; if the response result is a generated UI, the presentation parameters are the display parameters of each control icon in the UI; if the response result is a generated text, the presentation parameters are the position of each text segment in the text file, the contextual relationship, etc.

[0096] Exemplarily, if the target user intends a text, and the text content is "a puppy running on the grass in the sun", the electronic device responds to the text to generate an image corresponding to the text content (i.e., the response result), that is, the object to be processed is an image, and the presentation parameters of the image include red sunshine, green grass, and a black puppy running, that is, the attribute information of the image.

[0097] In some embodiments, obtaining the association information between the multiple objects to be processed includes at least one of the following:

[0098] In the case where the multiple objects to be processed are response results generated by the electronic device in response to the target user's intention, the first association parameter of each object to be processed in the response result and / or the second association parameter between different response results are used as the association information between the objects to be processed;

[0099] In an embodiment of the present application, when multiple objects to be processed come from response results generated by an electronic device in response to the target user's intention, the response result may be a generated image, audio, text or UI interface.

[0100] If the response result is an image, the first associated parameter may be the positional relationship, background-foreground relationship, primary-secondary relationship, symmetry relationship, hierarchical relationship, etc. of each image element in the image. If the response result is audio, the first associated parameter may be the channel relationship, timing relationship, etc. If the response result is text, the first associated parameter may be the logical contextual relationship, positional relationship, timing relationship, etc. If the response result is a UI interface, the first associated parameter may be the hierarchical relationship, positional relationship, etc.

[0101] For example, if multiple objects to be processed come from an image of a living room content (i.e., the response result), and the multiple objects to be processed include a sofa, a coffee table, a TV cabinet, and two green plants, then the first associated parameter may be that the TV cabinet is located directly in front of the sofa, the coffee table is located between the sofa and the TV cabinet, and the two green plants are symmetrically placed on both sides of the TV cabinet.

[0102] If the response result is multiple images, the second association parameter may be the temporal relationship, spatial conversion relationship, overall and layout relationship, scaling relationship, etc. between different images. If the response result is multiple generated audio segments, the second association parameter may be the temporal relationship, position relationship, channel relationship, etc. between different audio segments. If the response result is multiple generated text segments, the second association parameter may be the logical context relationship, temporal relationship, etc. between different text segments.

[0103] For example, a puppy is continuously photographed running on the grass, and the multiple images obtained in this way have corresponding shooting times, so the association information between the images is determined according to the shooting time. Alternatively, for the images of the four seasons of spring, summer, autumn and winter of the same scene, by identifying the season corresponding to each image, the association information between the four images of the same scene is determined according to the order of spring, summer, autumn and winter.

[0104] In a case where the multiple objects to be processed are from object contents represented by the target user intention, using the second description information of the object contents and / or the weight parameters between different user intentions as association information between the objects to be processed;

[0105] In the embodiment of the present application, the second description information is a description that limits the spatial position relationship, background foreground relationship, primary and secondary relationship, symmetry relationship, hierarchical relationship, etc. between different objects.

[0106] For example, if the object to be processed comes from an image (i.e., the object content represented by the target user's intention), which is an image of the placement of a sofa, a coffee table, a TV cabinet, and two green plants, then the second description information can be that the TV cabinet is located directly in front of the sofa, the coffee table is located between the sofa and the TV cabinet, and the two green plants are symmetrically placed on both sides of the TV cabinet.

[0107] It should be noted that, in the case where multiple objects to be processed come from multiple user intentions of multiple users, the association information between the objects to be processed may include weight parameters between different user intentions, that is, different weight parameters are configured according to different users.

[0108] In one achievable method, different weights may be configured according to different identities of different users, and the weight may be understood as a high possibility of being recognized or a greater proportion of consideration.

[0109] For example, if multiple objects to be processed are input from teachers and students respectively, the teacher's weight parameter can be configured to be greater than the student's weight parameter.

[0110] In another achievable method, in different scenarios, the weight parameters of user intent corresponding to users of different identities are different. For example, different weight parameters are configured for teachers of different ranks, and different weight parameters are configured for students of different grades.

[0111] For example, if multiple objects to be processed are input from teacher 1, teacher 2, student 1, and student 2 respectively, and the rank of teacher 1 is greater than that of teacher 2, then the weight parameter of teacher 1 is configured to be greater than the weight parameter of teacher 2, and the grade of student 1 is higher than the grade of student 2, then the weight parameter of student 1 is configured to be greater than the weight parameter of student 2, and the weight parameter of the teacher is configured to be greater than the weight parameter of the student.

[0112] In a case where the plurality of objects to be processed are from object contents input by a user or sent by a target device, using second description information of the object contents and / or a second recognition result obtained by recognizing the object contents as association information between the objects to be processed;

[0113] It should be noted that, in the case of one or more images input by a user or sent by a target device, each image needs to be processed to identify the relevance of multiple objects in the image, such as primary and secondary relationships, symmetry and hierarchy.

[0114] For example, if multiple objects to be processed come from an image of a living room, the image is identified and processed, and the second identification result indicates that the TV cabinet is located in front of the sofa, the coffee table is located between the sofa and the TV cabinet, and two green plants are symmetrically placed on both sides of the TV cabinet, that is, the association information between the sofa, the coffee table, the TV cabinet and the green plants. Alternatively, if multiple objects to be processed come from images of the four seasons of spring, summer, autumn and winter in the same scene, by identifying the season corresponding to each image, and then determining the association information between the four images in the same scene in the order of spring, summer, autumn and winter.

[0115] In the case that the multiple objects to be processed are object contents input by multiple users or sent by multiple target devices, the second description information of the object contents and weight parameters of different users / target devices are used as association information between the objects to be processed.

[0116] In an embodiment of the present application, for the case where multiple objects to be processed come from object contents input by multiple users or sent by multiple target devices, the association information between the objects to be processed includes weight parameters of different users / target devices, that is, different weight parameters are configured according to different users / target devices.

[0117] Exemplarily, different weights are configured according to different identities of different users, and the weight can be understood as a high possibility of being recognized or a large proportion of consideration. If multiple objects to be processed are input from teachers and students respectively, the weight parameter of the teacher can be configured to be greater than the weight parameter of the student. Alternatively, if multiple objects to be processed are input from teacher 1, teacher 2, student 1, and student 2 respectively, and the rank of teacher 1 is greater than the rank of teacher 2, then the weight parameter of teacher 1 is configured to be greater than the weight parameter of teacher 2, and the grade of student 1 is higher than the grade of student 2, then the weight parameter of student 1 is configured to be greater than the weight parameter of student 2, and the weight parameter of the teacher is configured to be greater than the weight parameter of the student.

[0118] For example, different weights are configured according to different target devices, and the weight can be understood as a high probability of being recognized or a large proportion of consideration. If multiple objects to be processed come from smartphones and tablets respectively, the weight parameter of the smartphone is configured to be greater than the weight parameter of the tablet.

[0119] In some embodiments, obtaining the attribute information of the plurality of objects to be processed and / or the association information between the plurality of objects to be processed includes at least one of the following:

[0120] In the case where the objects to be processed are from at least one image, determining attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed based on initial display parameters of each object to be processed in the image and / or third association parameters between different images;

[0121] In the embodiment of the present application, the initial display parameters include attribute information of image elements, such as at least one of size, shape, color, and position, and association information between image elements, such as at least one of primary and secondary relationships, symmetry, and hierarchical structure. The third association parameter between different images includes at least one of a temporal relationship, a spatial conversion relationship, a relationship between the whole and the part, and a scaling relationship between different images. Then, the attribute information of the multiple objects to be processed includes the attribute information of the image elements, and the association information between the multiple objects to be processed includes the association information between the image elements and the association information between different images.

[0122] For example, if the object to be processed comes from an image, and the image content is "a puppy running on the grass in the sun", then the initial display parameters of the image include the attribute information of the sun, the attribute information of the grass, and the attribute information of the running puppy, as well as the associated information that the sun is in the upper left of the grass, the puppy is on the grass, and the puppy is in the lower right of the sun. If the object to be processed comes from multiple images, i.e., images of the four seasons of spring, summer, autumn and winter of the same scene, then the third associated parameter is the associated information between the four images in the same scene in the order of spring, summer, autumn and winter.

[0123] In the case where the objects to be processed are from at least one image, the at least one image is recognized and processed using a target recognition model to obtain attribute information of the multiple objects to be processed and / or association information between the multiple objects to be processed.

[0124] In an embodiment of the present application, each image is identified using a target recognition model to identify the attribute information of each object to be processed in each image, such as at least one of size, color, and shape, and / or based on a pre-established association relationship model, the association information between the objects to be processed, such as at least one of primary and secondary relationships, symmetry, and hierarchy, is identified. Here, the association information between multiple objects to be processed can automatically adjust the layout so that multiple objects to be processed blend naturally together. This method avoids the tedious manual adjustment process of the user, while ensuring a high degree of visual and semantic consistency of the generated image, achieving a natural and beautiful effect.

[0125] For example, if the object to be processed comes from an image, and the image content is "a puppy running on the grass in the sun", the target recognition model is used to identify and process the image to identify the attribute information of the sun, the attribute information of the grass, and the attribute information of the running puppy, as well as related information such as the sun is in the upper left of the grass, the puppy is on the grass, and the puppy is in the lower right of the sun.

[0126] In some embodiments, performing corresponding processing on the plurality of objects to be processed based on the attribute information and / or the association information includes:

[0127] Based on the attribute information and / or the association information, the plurality of objects to be processed are fused using a target fusion strategy to obtain the target file;

[0128] In the embodiment of the present application, the target fusion strategy includes one of the following: a fusion template, a fusion process, and a fusion model. Based on the attribute information and / or association information between multiple objects to be processed, the multiple objects to be processed are fused using the fusion template, fusion process, or fusion model to obtain a target file.

[0129] Exemplarily, when the object to be processed is an image, based on the attribute information and association information between the image elements, a fusion model is used to fuse multiple images to obtain a fused image of the multiple images. Alternatively, when the object to be processed is text, based on the attribute information and association information between the texts, a fusion model is used to fuse multiple texts to obtain a fused text of the multiple texts.

[0130] As a further example, if one image is a grass background and the other image is a running puppy, then based on the attribute information of the grass, i.e., its color is green, and the attribute information of the puppy, i.e., its color is black and its posture is running, the two images are fused using a fusion model to obtain a fused image of the puppy running on the grass.

[0131] In the case where the object to be processed comes from at least one image, adjusting image parameters of the multiple objects to be processed based on the attribute information and / or the association information to obtain a target image;

[0132] In the embodiment of the present application, based on the attribute information and / or association information between the images, the image parameters (such as color, saturation, brightness and contrast) of the multiple objects to be processed are adjusted using the reference template or the standard image parameters or the historical image parameters to obtain the target image. In addition, manual adjustment can also be considered, that is, based on the attribute information and / or association information, the user manually adjusts the image parameters of the multiple objects to be processed on the display interface to obtain the target image.

[0133] For example, in an image of a living room, the sofa is black, and the coffee table is on the right side of the sofa. If you want the entire living room image to be warm in color, and the coffee table is usually directly in front of the sofa, then you can adjust the color of the sofa to light brown or beige, and adjust the position of the coffee table to directly in front of the sofa, so that you get the target image.

[0134] Performing format conversion processing on at least one of the multiple objects to be processed based on the attribute information and / or the association information to obtain the target file;

[0135] It should be noted that the reason for performing format conversion processing on at least one of the multiple objects to be processed is to take into account the different types of the multiple objects to be processed or the different generation requirements of the target files. The format conversion processing may be image to text, text to image, video to image, etc.

[0136] Exemplarily, if the target file generation requirement is text, and the multiple objects to be processed include images, it is necessary to convert the images into text documents or text illustrations, and combine with the remaining texts to generate a target file of text type. If the target file generation requirement is an image, and the multiple objects to be processed include text, it is necessary to convert the text into an image, and combine with the remaining images to generate a target file of image type. If the multiple objects to be processed include text and images, the images are converted into text documents or text illustrations, and combined with the remaining texts to generate a target file of text type; or the text is converted into an image, and combined with the remaining images to generate a target file of image type.

[0137] Based on the attribute information and / or the association information, the target processing model is used to generate and process the multiple objects to be processed to obtain the target file.

[0138] In the embodiment of the present application, the target processing model can be understood as a generative large model, which uses attribute information and / or association information as input to the generative large model to generate and process multiple objects to be processed and output a target file.

[0139] Exemplarily, the object to be processed is an image, and the attribute information of the image elements in multiple images and the association information between the image elements are used as the input of the target processing model to perform image fusion or generation processing to obtain a fused image. The object to be processed is text, and the attribute information of the text in multiple paragraphs and the association information between the texts are used as the input of the target processing model to perform text fusion or generation processing to obtain a fused text.

[0140] In order to implement the method of the above-mentioned embodiment of the present application, based on the same inventive concept, a data processing device is also provided in the embodiment of the present application. Figure 2 Schematic diagram of the structure of the data processing device in the embodiment of the present application. Figure 2 As shown, the data processing device 20 includes:

[0141] A first response unit 201, configured to obtain, in response to obtaining a plurality of objects to be processed, attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed, wherein the objects to be processed include or do not include a response result generated by the electronic device in response to an intention of a target user;

[0142] The first processing unit 202 is configured to perform corresponding processing on the multiple objects to be processed based on the attribute information and / or the association information to obtain a target file, wherein the target file includes at least one of the object contents represented by the multiple objects to be processed.

[0143] In an embodiment of the present application, when the object to be processed is an image, multiple images are fused based on the attribute information of each image element and / or the association information between the image elements, so that the various image elements are naturally integrated together, thereby achieving a natural and beautiful effect.

[0144] In some embodiments, the first response unit 201 is further configured to respond to at least one of the following:

[0145] In response to obtaining first input data input to a target application, splitting the user intent represented by the first input data, and using object contents represented by multiple sub-user intents obtained by splitting as the multiple objects to be processed;

[0146] In response to obtaining second input data input to the target application, generating a response result for the second input data, and using a plurality of object contents in the response result as the plurality of objects to be processed;

[0147] In response to obtaining a plurality of third input data input to the target application, obtaining a plurality of user intentions represented by the plurality of third input data, and using the object contents represented by the plurality of user intentions as the plurality of objects to be processed;

[0148] In response to obtaining fourth input data input to the target application, outputting a response result for the fourth input data, and in the case of obtaining fifth input data for the response result, taking at least one object content in the fifth input data and the response result as the multiple objects to be processed;

[0149] The obtained multiple object contents selected by at least one user and / or the received multiple object contents sent from the target device are used as the multiple objects to be processed, wherein the multiple object contents are of the same type or different types.

[0150] In some embodiments, the first response unit 201 is further used to obtain source information of the multiple objects to be processed; when the source information represents that the multiple objects to be processed are response results generated by the electronic device in response to the target user's intention, perform an operation of obtaining attribute information of the multiple objects to be processed and / or perform an operation of obtaining association information between the multiple objects to be processed;

[0151] In the case where the source information indicates that the multiple objects to be processed are from multiple object contents input by the same user, performing an operation of obtaining attribute information of the multiple objects to be processed;

[0152] In the case where the source information indicates that the multiple objects to be processed are from multiple object contents input by different users, performing an operation of obtaining attribute information of the multiple objects to be processed and association information between the multiple objects to be processed;

[0153] In the case where the source information represents that the multiple objects to be processed are from multiple object contents sent by the target device, performing an operation of obtaining attribute information of the multiple objects to be processed and association information between the multiple objects to be processed;

[0154] In a case where the source information represents that the multiple objects to be processed are from object contents represented by the target user's intention, an operation of obtaining association information between the multiple objects to be processed is performed.

[0155] In some embodiments, the first response unit 201 is further configured to respond to at least one of the following:

[0156] In a case where the plurality of objects to be processed are from object contents represented by the target user's intention, using first description information of the object contents as attribute information of the object to be processed;

[0157] In a case where the plurality of objects to be processed are from object contents input by at least one user or sent by a target device, using first description information of the object contents and / or a first recognition result obtained by recognizing the object contents as attribute information of the objects to be processed;

[0158] In the case that the multiple objects to be processed are from response results generated by the electronic device in response to the target user's intention, the presentation parameters of each object to be processed in the response result are used as the attribute information of the object to be processed.

[0159] In some embodiments, the first response unit 201 is further configured to respond to at least one of the following:

[0160] In the case where the multiple objects to be processed are response results generated by the electronic device in response to the target user's intention, the first association parameter of each object to be processed in the response result and / or the second association parameter between different response results are used as the association information between the objects to be processed;

[0161] In a case where the multiple objects to be processed are from object contents represented by the target user intention, using the second description information of the object contents and / or the weight parameters between different user intentions as association information between the objects to be processed;

[0162] In a case where the plurality of objects to be processed are from object contents input by a user or sent by a target device, using second description information of the object contents and / or a second recognition result obtained by recognizing the object contents as association information between the objects to be processed;

[0163] In the case that the multiple objects to be processed are object contents input by multiple users or sent by multiple target devices, the second description information of the object contents and weight parameters of different users / target devices are used as association information between the objects to be processed.

[0164] In some embodiments, the first response unit 201 is further configured to respond to at least one of the following:

[0165] In the case where the objects to be processed are from at least one image, determining attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed based on initial display parameters of each object to be processed in the image and / or third association parameters between different images;

[0166] In the case where the objects to be processed are from at least one image, the at least one image is recognized and processed using a target recognition model to obtain attribute information of the multiple objects to be processed and / or association information between the multiple objects to be processed.

[0167] In some embodiments, the first processing unit 202 is further configured to perform fusion processing on the plurality of objects to be processed based on the attribute information and / or the association information using a target fusion strategy to obtain the target file;

[0168] In the case where the object to be processed comes from at least one image, adjusting image parameters of the multiple objects to be processed based on the attribute information and / or the association information to obtain a target image;

[0169] Performing format conversion processing on at least one of the multiple objects to be processed based on the attribute information and / or the association information to obtain the target file;

[0170] Based on the attribute information and / or the association information, the target processing model is used to generate and process the multiple objects to be processed to obtain the target file.

[0171] Based on the above embodiments, the present application also provides an image generation method. Figure 3 Schematic diagram of the process of the image generation method in the embodiment of the present application. Figure 3 As shown, the image generation method includes the following steps:

[0172] S301: In response to the target user's intention to generate multiple image elements, obtain attribute information of the multiple image elements and / or association information between the multiple image elements.

[0173] In the embodiment of the present application, the input of the raw image application is multiple images input by the target user, and multiple image elements in the multiple images are output. The attribute information of the image element includes at least one of the following: type, size, shape, color, position, importance and priority. The association information between the image elements includes at least one of the following: primary and secondary relationship, symmetry, hierarchy and display rules.

[0174] S302: configuring display layout information between multiple image elements based on attribute information and / or association information to generate a target image based on the display layout information; wherein the target user intentions are from the same user or different users.

[0175] In the embodiment of the present application, based on the attribute information of multiple image elements and / or the association information between multiple image elements, the display layout between the image elements is adjusted to generate a target image that meets the user's intention. The display effect of the target image generated in this way is better than the display effect of the image generated directly using the target user's intention.

[0176] For example, in an image of a living room, the sofa is black (i.e., attribute information), and the coffee table is on the right side of the sofa (i.e., associated information). If you want the entire living room image to be warm in color, and the coffee table is usually directly in front of the sofa, then you can adjust the color of the sofa to light brown or beige, and adjust the position of the coffee table to directly in front of the sofa, so that you get the target image.

[0177] In an embodiment of the present application, display layout information between multiple image elements is configured based on the attribute information of each image element and / or the association information between image elements, and then multiple images are fused based on the display layout information to generate a target image, so that the target image presents a natural and beautiful effect.

[0178] In some embodiments, configuring the display layout information between the plurality of image elements based on the attribute information and / or the association information includes at least one of the following:

[0179] Obtaining target visual perception information, and configuring display layout information between the plurality of image elements based on the attribute information and / or the association information and in combination with the target visual perception information;

[0180] In an embodiment of the present application, the target visual perception information may include potential visual information such as texture features of image elements, and information about the current spatial environment in which the electronic device is located, that is, environmental data that can be perceived, such as ambient brightness, color temperature, light angle, viewer, etc., so that the generated display layout conforms to the current environment or the visual angle of the current user.

[0181] For example, for an image of a living room, the sofa is black (i.e., attribute information), and the coffee table is on the right side of the sofa (i.e., associated information). The entire image has a warm tone (i.e., target visual perception information). Then, the color of the sofa can be adjusted to light brown or beige, and the position of the coffee table can be adjusted to be directly in front of the sofa, so as to obtain the target image.

[0182] For example, for an image of a living room, the sofa is black (i.e., attribute information), and the coffee table is on the right side of the sofa (i.e., associated information). The entire image presents a warm tone, and the spatial environment information in which the electronic device is currently located is a night environment (i.e., target visual perception information). Then, the color of the sofa can be adjusted to light brown or beige, the position of the coffee table can be adjusted to be directly in front of the sofa, and the brightness of the entire image can be dimmed to obtain the target image.

[0183] The user portrait information of the target user is obtained, and based on the attribute information and / or the association information, the display layout information between the multiple image elements is configured in combination with the user portrait information.

[0184] In the embodiment of the present application, the user image can reflect the user's preferences, historical records, etc.

[0185] Exemplarily, the user's preference may be a preference for a certain color tone, such as warm tones or cool tones, and then, based on the attribute information of each image element and / or the association information between image elements, the image parameters of certain image elements are adjusted to make the display layout between the adjusted or configured multiple image elements more natural and beautiful.

[0186] Exemplarily, the user's historical records may include preset color, position requirements, etc. for fixed image elements, and then combined with the attribute information of each image element and / or the association information between image elements, the image parameters and / or position of the fixed image elements are adjusted to configure the display layout information between multiple image elements.

[0187] The present application also provides an image generating device. Figure 4 Schematic diagram of the composition structure of the image generating device in the embodiment of the present application. Figure 4 As shown, the image generating device 40 includes:

[0188] The second responding unit 401 is configured to generate a plurality of image elements in response to the target user's intention to obtain attribute information of the plurality of image elements and / or association information between the plurality of image elements.

[0189] The second processing unit 402 is used to configure display layout information between multiple image elements based on attribute information and / or association information to generate a target image based on the display layout information; wherein the target user intention is from the same user or different users.

[0190] In some embodiments, the second response unit 401 is further configured to respond to one of the following:

[0191] Obtaining target visual perception information, and configuring display layout information between the plurality of image elements based on the attribute information and / or the association information and in combination with the target visual perception information;

[0192] The user portrait information of the target user is obtained, and based on the attribute information and / or the association information, the display layout information between the multiple image elements is configured in combination with the user portrait information.

[0193] In an embodiment of the present application, display layout information between multiple image elements is configured based on the attribute information of each image element and / or the association information between image elements, and then multiple images are fused based on the display layout information to generate a target image, so that the target image presents a natural and beautiful effect.

[0194] Based on the above embodiments, the present application provides an architecture diagram of an image generation system. Figure 5 is a schematic diagram of the image generation system in the embodiment of the present application, such as Figure 5 As shown, the architecture of the image generation system includes an input management layer 50, an intent analysis layer 51, a feature extraction layer 52, a fusion generation layer 53, a user interaction layer 54 and a collaboration management layer 55.

[0195] The input management layer 50 is mainly used for users to upload background images and images to be fused, and to obtain users' historical operation records, usage preferences and style preferences.

[0196] The intent analysis layer 51 includes an image intent analysis model and a text intent analysis model, wherein the image intent analysis model is used to analyze the scene, visual elements, texture features, etc. of the background image and the image to be fused, and the text intent analysis model is used to parse the text description into a semantic vector to supplement the image intent information.

[0197] The feature extraction layer 52 includes an image feature extraction module and an image texture analysis model, wherein the image feature extraction module is used to extract feature vectors such as texture, color, and illumination of the image based on a convolutional neural network (CNN), and the image texture analysis model is used to analyze the texture matching degree between the background image and the image to be fused, providing a reference for image fusion.

[0198] The fusion generation layer 53 includes a deep fusion generation model and a multi-scheme generation module. The deep fusion generation model is used to combine the intention vector and the image feature vector to automatically adjust the image's lighting, perspective, color, and occlusion relationship to ensure that the fused image is natural and coordinated. The multi-scheme generation module is used to generate a variety of fusion schemes for the user to choose from based on the user's historical operations and image features.

[0199] The user interaction layer 54 includes a drag-and-drop operation interface and an intelligent recommendation system. The drag-and-drop operation interface allows users to view the fusion effect in real time by dragging the image to be fused to a specified position on the background image. The intelligent recommendation system is used to provide users with fusion solutions, and different effects can be directly viewed after clicking.

[0200] The collaborative management layer 55 includes multi-person editing, editing record viewing and management, implementation feedback and adjustment. That is, it supports multiple users to participate at the same time, view other people's operations and generation progress in real time, and supports real-time feedback and adjustment.

[0201] Based on this, the embodiment of the present application also provides a flowchart of an image generation method, Figure 6 The process diagram of the image generation method in the embodiment of the present application is as follows Figure 2 ,like Figure 6 As shown, the following steps are included:

[0202] S60: User uploads input.

[0203] Task: The user uploads the background image and the image to be fused through the system.

[0204] Technical implementation:

[0205] Provides a file upload interface that supports multi-format images (such as JPEG, PNG) and text input.

[0206] After the image is input, the system receives and stores the file through the API interface (such as through the REST API, GraphQL interface).

[0207] Subtasks:

[0208] File upload: The front end uses a file selection box, and the back end receives and saves the file stream.

[0209] Text description reception: Convert voice commands into text through form input or speech-to-text modules (such as Google Speech API).

[0210] S61: Data preprocessing.

[0211] Task: Preprocess the input data to ensure the efficiency and accuracy of subsequent processing.

[0212] Technical implementation:

[0213] Image resizing: Use image processing libraries (such as OpenCV or Pillow) to uniformly resize and resize uploaded images.

[0214] Image normalization: Normalize the image (such as normalizing the RGB value to the range of 0-1) to ensure input consistency.

[0215] Text cleaning: Clean the text entered by the user, remove stop words and punctuation marks, and perform word segmentation (such as using word segmentation tools such as NLTK and Jieba).

[0216] Subtasks:

[0217] Image format conversion and image resizing.

[0218] Text segmentation and cleaning.

[0219] S62: Image intent analysis.

[0220] Task: Analyze the image content of the background image and the image to be fused, and identify the scene, object, style and other intentions in the image.

[0221] Technical implementation:

[0222] Use pre-trained image classification or object detection models (such as ResNet, YOLO) for object recognition and scene classification.

[0223] Extract high-level semantic information of images, such as main objects, scene categories (indoor, outdoor, natural scenery, city, countryside, etc.), and emotional labels (tranquil, active, etc.).

[0224] Subtasks:

[0225] Image semantic classification: Classify images based on the ResNet model.

[0226] Object detection: Use YOLO or Mask R-CNN to detect and segment objects in the image.

[0227] S63: Textual intention analysis.

[0228] Task: Perform semantic analysis on the text description entered by the user and extract the user's intention to assist image fusion.

[0229] Technical implementation:

[0230] Use natural language processing tools (such as BERT, GPT) to perform deep semantic analysis on text to extract keywords, emotions, and potential image description intentions.

[0231] Text vectorization: Convert the processed text semantic information into vector form for use by subsequent models.

[0232] Subtasks:

[0233] Text Sentiment Analysis: Use a pre-trained sentiment analysis model to label text sentiment (such as positive, negative, etc.).

[0234] Keyword extraction: Extract key descriptive words related to the visuals from the text.

[0235] S64: Image feature extraction.

[0236] Task: Extract feature vectors of the background image and the image to be fused for subsequent image fusion process.

[0237] Technical implementation:

[0238] Use Convolutional Neural Network (CNN) to extract the underlying visual features of the image, such as size, color, texture, shape, etc.

[0239] Extract multi-scale features of images, including local features (such as SURF / SIFT) and global features (such as HOG, LBP, etc.).

[0240] Subtasks:

[0241] Color feature extraction: Perform color histogram analysis on the image to obtain color distribution information.

[0242] Texture feature extraction: Use Gabor filter or LBP method to extract image texture information.

[0243] Shape feature extraction: extract the main edge and contour information in the image.

[0244] S65: Texture matching analysis.

[0245] Task: Analyze the texture consistency of the background image and the image to be fused, and determine the possibility of image fusion.

[0246] Technical implementation:

[0247] Use texture matching algorithms (such as SIFT, ORB) to match the texture features of the images to ensure that the texture features of the two images are similar to improve the naturalness of the fusion effect. If the value is greater than a certain threshold, weighted average,

[0248] Subtasks:

[0249] Extract SIFT features and calculate the similarity of feature points.

[0250] Use the RANSAC algorithm to remove unmatched feature points.

[0251] S66: Selection and configuration of fusion models.

[0252] Task: Based on the intent analysis and feature extraction results, select the appropriate fusion model and adjust the parameters for image generation.

[0253] Technical implementation:

[0254] Use deep learning-based image generation models (such as GAN, StyleGAN) for image fusion. Improve the quality of generated images by adjusting the model's hyperparameters (such as learning rate, training rounds, loss function, etc.).

[0255] The generation model is fine-tuned according to the user's intention features (such as sentiment tags and keywords) to ensure that the output image conforms to the user's expected style.

[0256] Subtasks:

[0257] Image illumination adjustment: Adjust the illumination of the image to be fused based on the scene’s illumination model (such as the Phong model) to keep it consistent with the illumination of the background image.

[0258] Perspective transformation: Use the perspective transformation matrix to perform geometric transformation on the image to be fused so that it maintains the same perspective as the background image.

[0259] S67: Multiple solution generation.

[0260] Task: Generate multiple fusion solutions for users to choose from and provide personalized customization options.

[0261] Technical implementation:

[0262] The system uses different combinations of model parameters (such as different color adjustments and lighting modes) to generate multiple solutions for users to choose from.

[0263] Provide an automatic scoring system (such as based on SSIM or PSNR indicators) to evaluate the quality of each solution.

[0264] Subtasks:

[0265] Lighting adjustment solution generation: Generate multiple fusion solutions through different lighting models.

[0266] Color harmony scheme generation: Generate different color harmony schemes using color mapping technology.

[0267] S68: Drag-and-drop interface.

[0268] Task: Provide a visual drag-and-drop operation interface so that users can view the fusion effect in real time.

[0269] Technical implementation:

[0270] Use HTML5 and front-end frameworks (such as React or Vue.js) to build a draggable interface. Users can drag the image to be fused to the specified position of the background image and view the fusion effect in real time.

[0271] Subtasks:

[0272] Image Preview: Live preview of images in the browser using Canvas or WebGL.

[0273] Drag and drop function: The drag and drop function of images is realized through the Drag and Drop API of HTML5.

[0274] Intelligent Recommendation System

[0275] Task: Recommend the best fusion solution for users based on their historical operations and current intentions.

[0276] Technical implementation:

[0277] Use collaborative filtering algorithms or content-based recommendation algorithms to recommend fusion solutions that suit the user's style based on the user's historical operations and preferences.

[0278] Subtasks:

[0279] Recommendation model training: Train the recommendation model based on the user's historical data.

[0280] Real-time update: The recommendation list is updated in real time based on the user's current operations.

[0281] S69: Image post-processing and optimization.

[0282] Task: After the fusion is completed, the generated image is post-processed to ensure that the image quality meets the requirements.

[0283] Technical implementation:

[0284] Denoising: Use image denoising techniques (such as non-local mean filtering or bilateral filtering) to remove noise during the image fusion process.

[0285] Edge smoothing: Use edge detection algorithms (such as Canny edge detection) to identify the boundaries in the image and perform smoothing to avoid obvious fusion marks after image fusion.

[0286] Color correction: Adjust the overall color of the image to make it more natural through color correction techniques (such as tone mapping and histogram equalization).

[0287] Subtasks:

[0288] The non-local means filtering algorithm is used to remove noise from the generated image.

[0289] Use edge smoothing algorithms to adjust the transition area between objects and background in an image.

[0290] Make global or local color adjustments to your entire image to ensure consistent color.

[0291] S70: Export and download image formats.

[0292] Task: Export the final processed image to the format required by the user and provide a download link.

[0293] Technical implementation:

[0294] Multi-format support: supports exporting multiple image formats such as JPEG, PNG, TIFF, etc. Users can choose the appropriate format.

[0295] Resolution selection: Provides image export options with different resolutions (such as HD, 4K), and users can select output quality according to their usage.

[0296] Add watermark or copyright information: Add watermark or copyright information to the exported image according to user needs.

[0297] Subtasks:

[0298] The user selects the export format and resolution, and the image format conversion and resolution adjustment are performed in the background.

[0299] Once the image is exported, a download link is generated and provided to the user.

[0300] S71: Multiple output channels support.

[0301] Mission: In addition to providing a download function, it also supports sharing the generated images directly to social media or cloud storage platforms.

[0302] Technical implementation:

[0303] Social sharing: Integrate social platforms (such as Facebook, Instagram, Weibo, etc.) through API, allowing users to share generated images with one click.

[0304] Cloud storage upload: Provides the function of directly uploading to cloud storage services (such as Google Drive, Dropbox, Alibaba Cloud), making it convenient for users to store and access them later.

[0305] Subtasks:

[0306] Use OAuth authorization mechanism to access the APIs of major social media to achieve direct sharing of pictures.

[0307] Cloud storage integration: Upload images through the API interface of the cloud storage service and generate a sharing link or direct download address.

[0308] S72: Real-time feedback mechanism.

[0309] Task: Provide a real-time feedback mechanism so that users can instantly view the generated effects and make adjustments.

[0310] Technical implementation:

[0311] Real-time generation and updating is achieved. When users drag, drop and adjust images, the system background automatically updates and renders new fusion effects in real time.

[0312] Subtasks:

[0313] Real-time preview: After the image is generated in the background, the result is immediately transmitted to the front-end through WebSocket for display.

[0314] User evaluation system: provides an interface for users to rate each generated solution or provide feedback. The system adjusts subsequent recommendations based on the feedback.

[0315] S73: Multiplayer collaboration support.

[0316] Mission: Allow multiple users to collaborate in real time and simultaneously blend and adjust images.

[0317] Technical implementation:

[0318] Use WebSocket or a real-time database (such as Firebase or Socket.IO) to achieve multi-user collaboration, so that users can view other users' operations in real time.

[0319] Subtasks:

[0320] Real-time synchronization: Real-time synchronization of user operations is achieved through WebSocket.

[0321] Collaboration interface: Displays other users’ operations and status on the user interface.

[0322] S74: User operation log record.

[0323] Task: Record every operation of the user, including uploading images, selecting fusion solutions, image generation, etc., to facilitate subsequent optimization and data analysis.

[0324] Technical implementation:

[0325] Use a database (such as MySQL, MongoDB) to record each user's operation log, including detailed information such as time and operation content.

[0326] Clean up operation logs regularly to maintain system performance and data security.

[0327] Subtasks:

[0328] Implement a user behavior log system: record the user's operation steps and selected solutions.

[0329] Data storage optimization: Archive and clean up data regularly to prevent excessive data from affecting system performance.

[0330] S75: Data analysis and user profiling.

[0331] Task: Generate user portraits by analyzing user behavior data and optimize recommendation systems and model parameters.

[0332] Technical implementation:

[0333] Use data analysis tools (such as Pandas and NumPy) to count and analyze user behavior data to generate a profile of each user, including preferences, habits, etc.

[0334] Update the recommendation model based on user portraits to make the system more intelligent and personalized.

[0335] Subtasks:

[0336] Data cleaning and analysis: Use data analysis tools to clean and analyze user operation logs and feedback data.

[0337] Update recommendation model: Based on the analysis results, update the parameters of the recommendation system to provide an image fusion solution that better meets user needs.

[0338] S76: Model and system performance optimization.

[0339] Task: Optimize the image generation model and overall system performance to improve response speed and image quality.

[0340] Technical implementation:

[0341] Model optimization: Reduce the size of the image generation model and improve computational efficiency through methods such as model pruning and quantization.

[0342] GPU acceleration: In the image generation and processing stages, full use is made of the parallel computing capabilities of the GPU to improve the system's response speed.

[0343] Caching and asynchronous processing: cache the image results of high-frequency requests to reduce the time of repeated generation; and use asynchronous processing to improve the response speed of the system.

[0344] Subtasks:

[0345] Model quantization: Use TensorFlow Lite or ONNX to quantize the model to reduce the amount of calculation.

[0346] GPU computing: Use CUDA or TensorFlow's GPU acceleration module to handle the image generation process.

[0347] Asynchronous task processing: Use asynchronous frameworks such as Celery to handle long tasks and improve user experience.

[0348] S77: Scalability and fault tolerance mechanism.

[0349] Mission: Improve the scalability and fault tolerance of the system and ensure stable operation of the system under high concurrency.

[0350] Technical implementation:

[0351] Microservice architecture: The system is split into multiple microservices (such as image processing service, recommendation service, etc.), and each service is deployed independently to facilitate expansion and maintenance.

[0352] Load balancing: Distribute requests across multiple servers through a load balancer (such as NGINX, HAProxy) to reduce the pressure on a single server.

[0353] Fault-tolerance processing: Set up retry and timeout mechanisms for key operations (such as image uploading and generation) to ensure that the system can automatically recover when a failure occurs.

[0354] Subtasks:

[0355] Deploy microservice architecture: Divide the system into multiple independent modules, such as user management, image processing, recommendation services, etc., and deploy them separately.

[0356] Implement load balancing: Use a load balancer to distribute user requests to different servers.

[0357] Fault-tolerance mechanism: Set up request retry and timeout mechanisms to ensure high availability of the system.

[0358] Based on the above content, the following conclusions are drawn:

[0359] 1. Intelligent image layout based on object attribute recognition and association modeling

[0360] Through image analysis technology, the size, color, shape and other attributes of each object in the image are automatically identified, and a model of the association relationship between objects is established. Through this modeling, the system can understand the primary and secondary relationships, symmetry and hierarchy of image elements, and automatically adjust the layout so that multiple objects blend together naturally. This method avoids the user's tedious manual adjustment process, while ensuring that the generated image is highly consistent in vision and semantics, achieving a natural and beautiful effect.

[0361] (1) Object attribute recognition and association modeling: Through deep learning and image analysis technology, the system can automatically identify the attributes of each image object (such as size, color, shape, etc.) and establish an association relationship model between objects. This model is used to understand the primary and secondary relationships, symmetry, and hierarchy between objects, ensuring the intelligence and rationality of object layout when fusing images.

[0362] (2) Dynamic layout adjustment: Based on the object attributes and their correlation model, the system can automatically optimize and adjust the layout of image objects, avoiding the tedious process of manual layout adjustment. The generated image is natural, beautiful and logically consistent.

[0363] 2. Intelligent intent analysis and multimodal information fusion

[0364] By building an intelligent intent analysis model, the intent information in the input image and text is analyzed to understand the context, visual elements and semantic relationships. The system can not only deeply combine multiple images and text information through multimodal fusion, but also optimize the image fusion method based on these analysis results. This makes the generated images not only meet the user's visual needs, but also more coherent and reasonable in semantics, greatly improving the quality of the generated effect and user experience.

[0365] (1) Intelligent intent analysis model: This model can analyze the intent information of the background image and the image to be fused, including context information, visual elements, and semantic relationships, so as to understand the user’s real needs and guide the image fusion process accordingly. This greatly improves the semantic rationality and visual consistency of the generated image.

[0366] (2) Multimodal information fusion: Multimodal fusion of multiple images and text information. Through intent analysis and image texture feature extraction, deep understanding and fusion of multiple data sources, the generated images are richer and deeper, and the user experience is more personalized.

[0367] 3. Image texture feature vector extraction and fusion optimization based on deep learning

[0368] By using deep learning technology to extract image texture features such as color, lighting, and material, the system can automatically understand the potential visual information in the image. During the image fusion process, the system automatically adjusts according to these features to make parameters such as lighting, perspective, and texture highly matched, ensuring seamless fusion of various image elements. Through this optimization model, the generated image is not only natural and realistic, but also more visually consistent, avoiding the common fusion inconsistency problem in traditional image processing methods.

[0369] (1) Image texture feature extraction: Deep learning technology is used to extract the potential texture features of images, such as color, lighting, and material. These features provide data support for the optimization of image fusion, making the visual effects of different images such as lighting, perspective, and color highly consistent, and generating more natural and realistic images.

[0370] (2) Deep fusion generation model: Based on the texture features and intent analysis of the image, the system automatically adjusts the position, size, perspective, lighting and other parameters of the fused image to ensure that the visual and semantic fusion effects of the image elements are realistic and natural.

[0371] 4. Intelligent recommendation fusion solution and real-time drag-and-drop operation interface

[0372] The system can intelligently recommend multiple image fusion solutions based on the user's historical preferences, the characteristics of the current input image, and the multimodal fusion results. Users can quickly preview different solutions by clicking, greatly simplifying the tedious steps of image adjustment. At the same time, the intuitive real-time drag-and-drop operation interface allows users to place image elements on the background image in real time by simply dragging, and instantly observe and fine-tune the fusion effect. This intelligent recommendation and real-time feedback mechanism greatly improves the user's operating efficiency and experience.

[0373] (1) Intelligent recommendation of fusion solutions: The system can intelligently recommend a variety of image fusion solutions based on user history, preferences, and current image features. Users can preview different fusion effects with a single click without complicated operations, which improves operational efficiency and user experience.

[0374] (2) Real-time drag-and-drop multi-image fusion interface: It provides an intuitive drag-and-drop interface, where users can blend images into the background by simply dragging and dropping, and can view and fine-tune the fusion effect in real time. This interface simplifies the operation process, reduces the complexity of image processing, and is suitable for general users.

[0375] 5. Support multi-person collaboration and optimize user personalized experience

[0376] The system supports multi-person collaboration mode, allowing multiple users to view each other's operations and generation progress in real time in the same project, greatly improving the team's creative efficiency. In addition, the system continuously optimizes the image fusion solution it provides based on each user's operating habits, historical records and feedback to ensure a personalized service experience. At the same time, the system has dynamic self-optimization capabilities, can iteratively improve based on user feedback, continuously improve functions and user experience, and provide users with more personalized solutions and a better operating environment.

[0377] (1) Support for multi-person collaboration: The system supports real-time collaboration among multiple people. Each user can see the operations and progress of others, which greatly improves the efficiency of team collaboration and is suitable for team creation scenarios.

[0378] (2) User experience optimization and personalized services: By recording user operating habits and preferences, the system can continuously optimize the recommended integration solutions and provide users with personalized services to ensure that users always enjoy an optimized experience. In addition, the system has dynamic iteration capabilities and can self-adjust and optimize functions based on user feedback.

[0379] The advantages of using this application are:

[0380] Intelligent layout: The system can automatically adjust the layout according to the properties and relationships of different objects in the image, reducing the tedious work of manual adjustment by users.

[0381] Multimodal information fusion: The present invention can process multiple images and text information at the same time. Through the intention analysis model and the texture feature vector extraction model, different information is deeply integrated. The generated image has clear logic and rich details.

[0382] Intelligent analysis of intent information: Through the intelligent intent analysis module, the present invention can obtain the intent information of different image inputs, and reasonably fuse the images according to the intent, thereby improving the semantic rationality of the generated images.

[0383] Natural fusion effect: The present invention solves the problem of unnatural fusion effect of traditional methods by automatically adjusting the lighting, perspective and texture relationship of the image, making the generated image more realistic and coordinated.

[0384] Easy operation: Users can complete image fusion operations by dragging and dropping and clicking, without having to manually set complex parameters. The system intelligently recommends fusion solutions, greatly reducing the operation threshold and improving user experience.

[0385] Support multi-person collaboration: The present invention supports multi-person collaborative image generation. Each user can view the operations and generation progress of others in real time, thereby improving team collaboration efficiency.

[0386] User experience optimization and personalized service: The system can make personalized solution recommendations based on user operation records and feedback, which improves user experience. At the same time, through the analysis of user behavior, the system has the ability to dynamically iterate to ensure continuous optimization of system functions.

[0387] The present application also provides another electronic device, Figure 7 Schematic diagram of the structure of the electronic device in the embodiment of the present application. Figure 7 As shown, the electronic device 70 includes: a processor 701 and a memory 702 configured to store a computer program that can be run on the processor;

[0388] The processor 701 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0389] Of course, in practical applications, Figure 7 As shown, the components in the electronic device 70 are coupled together via a bus system 703. It is understood that the bus system 703 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 703 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 7 Various buses are labeled as bus system 7003.

[0390] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0391] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0392] In an exemplary embodiment, the present application also provides a computer-readable storage medium for storing a computer program.

[0393] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0394] Illustratively, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by a processor of an electronic device to complete the steps of any of the aforementioned methods.

[0395] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0396] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0397] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to program instructions, and the above program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0398] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0399] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0400] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0401] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A data processing method, comprising: In response to obtaining a plurality of objects to be processed, obtaining attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed, wherein the objects to be processed include or do not include response results generated by the electronic device in response to the target user's intention; Based on the attribute information and / or the association information, the multiple objects to be processed are processed accordingly to obtain a target file, wherein the target file includes at least one of the object contents represented by the multiple objects to be processed.

2. The method according to claim 1, wherein: Obtain multiple objects to be processed, including at least one of the following: In response to obtaining first input data input to a target application, splitting the user intent represented by the first input data, and using object contents represented by a plurality of sub-user intents obtained by splitting as the plurality of objects to be processed; In response to obtaining second input data input to the target application, generating a response result for the second input data, and using a plurality of object contents in the response result as the plurality of objects to be processed; In response to obtaining a plurality of third input data input to the target application, obtaining a plurality of user intentions represented by the plurality of third input data, and using the object contents represented by the plurality of user intentions as the plurality of objects to be processed; In response to obtaining fourth input data input to the target application, outputting a response result for the fourth input data, and in the case of obtaining fifth input data for the response result, taking at least one object content in the fifth input data and the response result as the multiple objects to be processed; The obtained multiple object contents selected by at least one user and / or the received multiple object contents sent from the target device are used as the multiple objects to be processed, wherein the multiple object contents are of the same type or different types.

3. The method according to claim 1, wherein: Obtaining the attribute information of the plurality of objects to be processed and / or the association information between the plurality of objects to be processed includes: Obtaining source information of the plurality of objects to be processed; In the case where the source information represents that the multiple objects to be processed are response results generated by the electronic device in response to the target user's intention, performing an operation of obtaining attribute information of the multiple objects to be processed and / or performing an operation of obtaining association information between the multiple objects to be processed; or In the case where the source information indicates that the multiple objects to be processed are from multiple object contents input by the same user, performing an operation of obtaining attribute information of the multiple objects to be processed; or, In the case where the source information indicates that the multiple objects to be processed are from multiple object contents input by different users, performing an operation of obtaining attribute information of the multiple objects to be processed and association information between the multiple objects to be processed; or, In the case where the source information indicates that the multiple objects to be processed are from multiple object contents sent by the target device, performing an operation of obtaining attribute information of the multiple objects to be processed and association information between the multiple objects to be processed; or, In a case where the source information represents that the multiple objects to be processed are from object contents represented by the target user's intention, an operation of obtaining association information between the multiple objects to be processed is performed.

4. The method according to any one of claims 1 to 3, wherein: Obtaining attribute information of the plurality of objects to be processed includes at least one of the following: In a case where the plurality of objects to be processed are from object contents represented by the target user's intention, using first description information of the object contents as attribute information of the object to be processed; In a case where the plurality of objects to be processed are from object contents input by at least one user or sent by a target device, using first description information of the object contents and / or a first recognition result obtained by recognizing the object contents as attribute information of the objects to be processed; In the case that the multiple objects to be processed are from response results generated by the electronic device in response to the target user's intention, the presentation parameters of each object to be processed in the response result are used as the attribute information of the object to be processed.

5. The method according to any one of claims 1 to 3, wherein: Obtaining association information between the multiple objects to be processed includes at least one of the following: In the case where the multiple objects to be processed are response results generated by the electronic device in response to the target user's intention, taking the first association parameter of each object to be processed in the response result and / or the second association parameter between different response results as the association information between the objects to be processed; In the case where the multiple objects to be processed are from object contents represented by the target user intention, using the second description information of the object contents and / or the weight parameters between different user intentions as association information between the objects to be processed; In a case where the plurality of objects to be processed are from object contents input by a user or sent by a target device, using second description information of the object contents and / or a second recognition result obtained by recognizing the object contents as association information between the objects to be processed; In the case that the multiple objects to be processed are object contents input by multiple users or sent by multiple target devices, the second description information of the object contents and weight parameters of different users / target devices are used as association information between the objects to be processed.

6. The method according to claim 1, wherein: Obtaining the attribute information of the plurality of objects to be processed and / or the association information between the plurality of objects to be processed includes at least one of the following: In the case where the objects to be processed are from at least one image, determining attribute information of the plurality of objects to be processed and / or association information between the plurality of objects to be processed based on initial display parameters of each object to be processed in the image and / or third association parameters between different images; In the case where the objects to be processed are from at least one image, the at least one image is recognized and processed using a target recognition model to obtain attribute information of the multiple objects to be processed and / or association information between the multiple objects to be processed.

7. The method according to claim 1, wherein: Performing corresponding processing on the multiple objects to be processed based on the attribute information and / or the association information includes: Based on the attribute information and / or the association information, the plurality of objects to be processed are fused using a target fusion strategy to obtain the target file; In the case where the object to be processed comes from at least one image, adjusting image parameters of the multiple objects to be processed based on the attribute information and / or the association information to obtain a target image; Performing format conversion processing on at least one of the multiple objects to be processed based on the attribute information and / or the association information to obtain the target file; Based on the attribute information and / or the association information, the target processing model is used to generate and process the multiple objects to be processed to obtain the target file.

8. A method for generating an image, comprising: In response to the target user's intention to generate a plurality of image elements, obtaining attribute information of the plurality of image elements and / or association information between the plurality of image elements; configuring display layout information between the plurality of image elements based on the attribute information and / or the association information to generate a target image based on the display layout information; The target users intend to come from the same user or different users.

9. The method according to claim 8, wherein: Configuring display layout information between the plurality of image elements based on the attribute information and / or the association information includes at least one of the following: Obtaining target visual perception information, and configuring display layout information between the plurality of image elements based on the attribute information and / or the association information and in combination with the target visual perception information; The user portrait information of the target user is obtained, and based on the attribute information and / or the association information, the display layout information between the multiple image elements is configured in combination with the user portrait information.

10. An electronic device comprising: a processor and a memory configured to store a computer program capable of being executed on the processor, Wherein, the processor is configured to execute the steps of the method according to any one of claims 1 to 9 when running the computer program.

Images