Image generation method, image display device, storage medium and program product

Abstract

The invention provides an image generation method, an image display device, a storage medium and a program product, which can be applied to the technical field of artificial intelligence and image processing. The image generation method comprises the steps of obtaining environment information for a current scene in response to an image generation instruction; according to the environment information, at least one influence factor is determined, and each influence factor is used for indicating an image generation effect based on the environment information; and according to the image generation instruction and the at least one influence factor, generating a target image for the current scene.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and image processing technology, and more specifically, to an image generation method, an image display device, a storage medium and a program product. Background Art

[0002] With the development of computer technology, posters can be used to display information through text and / or images. When designing a poster, factors such as text and its font, image, and the layout of text and image need to be considered. In one example, an artificial intelligence model can be used to design a poster, that is, an artificial intelligence model is used to generate a poster based on the input text and / or image. However, the method of using an artificial intelligence model to design a poster cannot adapt to the adaptive generation of posters in different scenarios and cannot guarantee the efficiency of poster generation. Summary of the invention

[0003] In view of this, the present disclosure provides an image generating method, an image display device, a storage medium and a program product.

[0004] According to one aspect of the present disclosure, there is provided an image generation method, comprising: in response to an image generation instruction, acquiring environmental information for a current scene; determining at least one influencing factor based on the environmental information, wherein each of the influencing factors is used to indicate an image generation effect based on the environmental information; and generating a target image for the current scene based on the image generation instruction and the at least one influencing factor.

[0005] According to another aspect of the present disclosure, there is provided an image display device, comprising: a processor, for acquiring environmental information for a current scene in response to an image generation instruction; determining at least one influencing factor based on the environmental information, wherein each of the influencing factors is used to indicate an image generation effect based on the environmental information; and generating a target image for the current scene based on the image generation instruction and the at least one influencing factor; and a display device, communicatively connected to the processor, for displaying the target image.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor implements the method described in the present disclosure.

[0007] According to another aspect of the present disclosure, a computer program product is provided. The computer program product includes computer executable instructions. When the computer executable instructions are executed, they are used to implement the method described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0009] Figure 1 The system architecture of the image display device according to the embodiment of the present disclosure is schematically shown;

[0010] Figure 2 The flowchart of the image generation method according to the embodiment of the present disclosure is schematically shown;

[0011] Figure 3A An example schematic diagram schematically shows a poster generation model processing process according to an embodiment of the present disclosure;

[0012] Figure 3B An example schematic diagram of a poster edge expansion model processing process according to an embodiment of the present disclosure is schematically shown;

[0013] Figure 3C An example schematic diagram schematically shows a dynamic poster model processing process according to an embodiment of the present disclosure;

[0014] Figure 4A An example schematic diagram of influencing factors in an indoor scene according to an embodiment of the present disclosure is schematically shown;

[0015] Figure 4B An example schematic diagram schematically shows a process of generating a target image in an indoor scene according to an embodiment of the present disclosure;

[0016] Figure 5A An example schematic diagram of influencing factors in an outdoor scene according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 5B An example schematic diagram schematically shows a process of generating a target image in an outdoor scene according to an embodiment of the present disclosure;

[0018] Figure 6 An example schematic diagram schematically shows a target image generation process according to an embodiment of the present disclosure;

[0019] Figure 7 An example schematic diagram schematically shows a target image generation process according to another embodiment of the present disclosure;

[0020] Figure 8 An example schematic diagram schematically shows a target image generation process according to another embodiment of the present disclosure;

[0021] Fig. 9 An example schematic diagram schematically shows a target image generation process according to another embodiment of the present disclosure;

[0022] Fig.10 An example schematic diagram schematically illustrates screen factors according to an embodiment of the present disclosure;

[0023] Fig.11 An example schematic diagram schematically shows a target image generation process according to another embodiment of the present disclosure;

[0024] Fig.12 An example schematic diagram schematically shows a target image generation process according to another embodiment of the present disclosure;

[0025] Fig.13 An example schematic diagram schematically shows a process of generating a target image by an image generation method according to an embodiment of the present disclosure; and

[0026] Fig.14 An example schematic diagram of a process of generating a target image by expanding an image according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] A poster can refer to a way of presenting information through text and images. When designing a poster, you need to consider factors such as the text and its font, the image, and the layout of the text and image. This is difficult for users who have no basic knowledge of poster design.

[0033] In one example, an artificial intelligence model can be used to design a poster, that is, the artificial intelligence model generates a poster based on input text and / or images.

[0034] However, since the AI ​​model only supports generating posters from a piece of text, or changing the content based on an image, the implementation scenario is relatively single and cannot adapt to the adaptive generation of posters in different scenarios. In addition, since the AI ​​model does not have the ability to generate and reuse local content, each poster generation needs to be completely regenerated, even for the same scene, which cannot guarantee the efficiency of poster generation.

[0035] To this end, the present disclosure provides an image generation method, an image display device, a storage medium, and a program product, which can be applied to the fields of artificial intelligence technology and image processing technology. The image generation method includes: in response to an image generation instruction, obtaining environmental information for a current scene; determining at least one influencing factor based on the environmental information, wherein each influencing factor is used to indicate an image generation effect based on the environmental information; and generating a target image for the current scene based on the image generation instruction and the at least one influencing factor.

[0036] Figure 1 The system architecture of the image display device according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0037] like Figure 1As shown, the image display device 100 according to this embodiment may include a processor 110 and a display device 120. The display device 120 may include a first terminal device 101, a second terminal device 102, and a third terminal device 103, for example.

[0038] The processor 110 is configured to obtain environmental information for a current scene in response to an image generation instruction, determine at least one influencing factor based on the environmental information, and each influencing factor is used to indicate an image generation effect based on the environmental information, and generate a target image for the current scene based on the image generation instruction and the at least one influencing factor.

[0039] The image generation instruction may refer to a command or parameter for instructing the processor 110 to generate a specific image. In one example, the image generation instruction may include specific requirements for aspects such as image content, style, size, and resolution to ensure that the target image generated based on the image generation instruction can meet expectations. For example, the image generation instruction may include at least one of an image and text. The influencing factors may be used to indicate the image generation effect of the target image finally generated during the image generation process. The image generation effect may include aspects such as quality, content, and style.

[0040] The display device 120 is connected to the processor 110 through the network 130 for displaying the target image.

[0041] According to the embodiments of the present disclosure, the processor can respond to the image generation instruction, actively obtain the environmental information of the current scene, and through in-depth analysis of the environmental information, it can determine at least one influencing factor for indicating the image generation effect. Since each influencing factor accurately characterizes the role of the environmental information in the image generation process, it can provide accurate guidance for subsequent image generation. On this basis, since the display device connected to the processor can display the generated target image in real time, it provides intuitive visual feedback to the user, enhances the interactivity between the user and the image display device, and enables the user to instantly evaluate the image generation effect and adjust or optimize it as needed.

[0042] It should be noted that the image generation method provided in the embodiment of the present disclosure can generally be executed by the processor 110. Accordingly, the image generation device provided in the embodiment of the present disclosure can generally be set in the processor 110.

[0043] It should be understood that Figure 1 The number of processors and display devices in the embodiment is only for illustration. Any number of processors and display devices may be provided according to implementation requirements.

[0044] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0045] The above describes the system architecture of the image display device. Figure 2 For example, the image generation process of the present disclosure is further described.

[0046] Figure 2 The flowchart of the image generating method according to the embodiment of the present disclosure is schematically shown.

[0047] like Figure 2 As shown, the image generating method 200 includes operations S210 to S230.

[0048] In operation S210 , in response to an image generation instruction, environmental information for a current scene is acquired.

[0049] In operation S220, at least one influencing factor is determined according to the environmental information.

[0050] In operation S230, a target image for the current scene is generated according to the image generation instruction and at least one influencing factor.

[0051] The image generation instruction may be used to trigger an image generation operation. The image generation instruction may include at least one of an image and text. After receiving the image generation instruction, environmental information for the current scene may be acquired. The environmental information may be used to characterize the environment in which the display device is located. The environmental information may include at least one of the following: environmental image, lighting information, sound information, date information, temperature information, object information.

[0052] The method of obtaining environmental information can be configured according to actual business needs and is not limited here. For example, an image acquisition device can be used to capture images of the environment in which the display device is located to obtain an environmental image. On this basis, the environmental image can be identified to obtain scene information, object information and weather information. Alternatively, a sensor can be used to sense the current scene to obtain lighting information, temperature information and sound information. Alternatively, date information can be determined based on the current date of the system.

[0053] After obtaining the environmental information, at least one influencing factor may be determined based on the environmental information. Each influencing factor is used to indicate the image generation effect based on the environmental information. The image generation effect may include aspects such as quality, content, and style. The influencing factors may be configured according to actual business needs and are not limited here. For example, the influencing factors may include at least one of the following: scene factors, lighting factors, sound factors, time factors, temperature factors, weather factors, object factors, screen factors, and local factors.

[0054] In one example, the light factor may characterize the light intensity. The sound factor may characterize the sound intensity. The time factor may characterize whether the current date is a preset date. The temperature factor may characterize the temperature. The weather factor may characterize whether the weather information is preset weather. The object factor may characterize the object features of the object information. The screen factor may characterize the screen size of the display device.

[0055] After obtaining at least one influencing factor, a target image for the current scene may be generated according to the image generation instruction and the at least one influencing factor. The target image may be a poster.

[0056] According to an embodiment of the present disclosure, by acquiring the environmental information of the current scene, the image generation process can dynamically adapt to different environmental conditions. By analyzing the environmental information, at least one influencing factor for indicating the image generation effect can be obtained. On this basis, by generating a target image for the current scene according to the image generation instruction and at least one influencing factor, customized generation of the target image can be achieved.

[0057] The above describes the image generation process of the present invention. Figure 3A , Figure 3B and Figure 3C Some methods of generating a target image using different candidate image generation models are described.

[0058] In one example, the image generation instruction can be used to indicate at least one image generation model. For example, the candidate image generation model may include one of the following: a poster generation model, a poster margin expansion model, and a dynamic poster model. The poster generation model can be used to generate a static poster based on at least one of the text and the image in the image generation instruction. The poster margin expansion model can be used to expand the poster based on at least one of the text and the image in the image generation instruction to generate a static poster after the margin expansion. The dynamic poster model can be used to generate a dynamic poster based on the input image to be generated.

[0059] After receiving the image generation instruction, the image generation model can be determined from a plurality of candidate image generation models according to the image generation instruction. On this basis, the image generation instruction and at least one influencing factor can be input into the image generation model to obtain a target image.

[0060] Figure 3A An example schematic diagram of a poster generation model processing process according to an embodiment of the present disclosure is schematically shown.

[0061] like Figure 3A As shown, the image generation instruction includes text 301, and the image generation model indicated by the image generation instruction is a poster generation model 310 for realizing static image generation. The text 301 and at least one influencing factor are input into the poster generation model 310 to obtain a target image 302. In this case, the poster generation model 310 can intelligently recognize the text 301 and input the target image 302 corresponding thereto. The generation process of the poster generation model 310 can be shown as the following formula (1).

[0062] A=A(x)(1)

[0063] Among them, A represents the poster generation model and x represents the text.

[0064] Figure 3B An example schematic diagram of a poster margin expansion model processing process according to an embodiment of the present disclosure is schematically shown.

[0065] like Figure 3B As shown, the image generation instruction includes an image 303, and the image generation model indicated by the image generation instruction is a poster edge expansion model 320 for realizing image edge expansion generation. The image 303 and the size 304 are input to the poster edge expansion model 320 to obtain a target image 305. In this case, the poster edge expansion model 320 can parse the information in the image 303, and output text and images that match its content and fill the periphery to obtain the target image 302. The generation process of the poster edge expansion model 320 can be shown in the following formula (2).

[0066] B=B(y,z)(2)

[0067] Among them, B represents the poster expansion model, y represents the image, and z represents the size.

[0068] Figure 3C An example schematic diagram of a dynamic poster model processing process according to an embodiment of the present disclosure is schematically shown.

[0069] like Figure 3CAs shown, the image generation instruction includes an image 306, and the image generation model indicated by the image generation instruction is a dynamic poster model 330 for realizing dynamic generation. The image 306 is input into the dynamic poster model 330 to obtain a target image 307. In this case, the dynamic poster model 330 can parse the information in the image 306 and input a dynamic target image 307 that matches it. The generation process of the dynamic poster model 330 can be shown as the following formula (3).

[0070] C=C(x)(3)

[0071] Among them, C represents the dynamic poster model and x represents the image.

[0072] According to an embodiment of the present disclosure, by intelligently screening out the image generation model that best suits the current task from multiple candidate image generation models based on at least one image generation model indicated by an image generation instruction, and inputting the image generation instruction and at least one influencing factor into the image generation model to obtain the target image, the efficiency and accuracy of image generation can be guaranteed.

[0073] In one example, the environmental information may include an environmental image taken for the current scene. After obtaining the environmental image, the environmental image may be recognized to obtain scene information. Scene information may characterize various elements and their relationships in a specific environment or a specific scene. For example, scene information may include the shape, size, color, and position of objects in the specific scene. The recognition method may be configured according to actual business needs and is not limited here. For example, the recognition method may include at least one of the following: a method based on image recognition technology, a method based on point cloud data processing, a method based on multimodal fusion, and a method based on statistical pattern recognition.

[0074] Methods based on image recognition technology can refer to identification by extracting key features from images. Methods based on point cloud data processing can refer to identifying objects and spatial structures in a scene through methods such as point cloud segmentation, clustering, and feature extraction. Methods based on multimodal fusion can refer to scene recognition by combining information from multiple modalities such as images, sounds, and texts. Methods based on statistical pattern recognition can refer to the use of statistical methods to classify and identify scenes.

[0075] After obtaining the scene information, the target scene type can be determined from multiple candidate scene types according to the scene information. Each candidate scene type corresponds to a candidate image style. The candidate scene types and candidate image styles can be configured according to actual business needs and are not limited here. For example, the candidate scene types may include: indoor scenes and outdoor scenes. Indoor scenes may include supermarket scenes, exhibition scenes, home scenes, and office scenes, etc. Outdoor scenes may include square scenes and highway scenes, etc.

[0076] For example, in a supermarket scenario, the purpose of the target image is to attract customers' attention to promote product sales, so the candidate image style corresponding to the supermarket scenario may be lively and eye-catching; in an exhibition scenario, the purpose of the target image is to display the theme and content of the exhibition or display to guide the audience to visit, so the candidate image style corresponding to the exhibition scene may be creative and artistic; in a home scene, the purpose of the target image is to be used for home decoration and atmosphere creation, so the candidate image style corresponding to the home scene may be warm and comfortable; in an office scene, the purpose of the target image is to be used for corporate culture promotion and work reminders, so the candidate image style corresponding to the office scene may be formal and professional.

[0077] For another example, in a square scene, the purpose of the target image is to promote activities to attract people's attention, so the style of the candidate image corresponding to the square scene may be warm and cheerful; in a highway scene, the purpose of the target image is to convey information related to the highway, so the style of the candidate image corresponding to the highway scene may be concise and steady.

[0078] After determining the target scene type from multiple candidate scene types, the candidate image style of the target scene type can be determined as a scene factor. For example, when the target scene type is a supermarket scene, the candidate image style corresponding to the target scene type is lively and eye-catching, so lively and eye-catching can be used as scene factors. Alternatively, when the target scene type is a highway scene, the candidate image style corresponding to the target scene type is concise and steady, so concise and steady can be used as scene factors.

[0079] According to the embodiments of the present disclosure, by recognizing the captured environment image, the type of the current scene can be accurately identified from multiple candidate scene types, thereby ensuring that the subsequent image generation process can closely fit the actual environment. On this basis, by automatically matching the candidate image style suitable for the target scene type and determining it as one of the factors affecting the image generation effect, the accuracy of image generation is improved.

[0080] The above describes the method of using different candidate image generation models to generate a target image in the present disclosure. Figure 4A and Figure 4B The influencing factors in indoor scenes and the process of generating target images for indoor scenes are explained.

[0081] When the target scene type includes an indoor scene, the environmental information may include at least one of the following: lighting information, sound information, and the current date. Indoor scenes may include supermarkets, exhibitions, homes, and offices. Different indoor scenes have different requirements for poster generation effects. For example, for supermarket scenes, since the environment is more complex, posters can be generated adaptively according to the environment. For another example, for exhibition scenes, it is necessary to generate more formal posters that take date factors into consideration.

[0082] Figure 4A An example schematic diagram of influencing factors in an indoor scene according to an embodiment of the present disclosure is schematically shown; Figure 4B An example schematic diagram of a process of generating a target image in an indoor scene according to an embodiment of the present disclosure is schematically shown.

[0083] Combination Figure 4A and Figure 4B As shown, when the target scene type includes an indoor scene, the influencing factors may include a scene factor 405 , a lighting factor 406 , a sound factor 407 , and a time factor 408 .

[0084] In 400A, scene factor 405 may be obtained by the following operations: using an image acquisition device to capture the current scene to obtain an environment image. Recognizing the environment image to obtain scene information. Based on the scene information, scene factor 405 is determined. Scene factor 405 is used to characterize the style of the candidate image corresponding to the target scene type.

[0085] The illumination factor 406 may be obtained by the following operation: the illumination intensity of the current scene is detected by the light intensity acquisition device 403 to obtain illumination information. According to the illumination information, the illumination factor 406 is determined. The illumination factor 406 is used to characterize the illumination intensity.

[0086] The sound factor 407 may be obtained by the following operation: using the sound collection device 402 to detect the sound intensity of the current scene to obtain sound information. According to the sound information, the sound factor 407 is determined. The sound factor 407 is used to characterize the magnitude of the sound intensity.

[0087] The time factor 408 may be obtained by the following operations: using the display device 404 to determine the current date. Based on the current date, the time factor 408 is determined. The time factor 408 is used to characterize whether the current date is a preset date. The preset date can be configured according to actual business needs and is not limited here. For example, the preset date may be a holiday or season. For example, if the preset date is the Spring Festival, a more festive target image may be generated. For another example, if the preset date is summer, a cooler style may be generated.

[0088] In 400B, based on the generation type, the image generation model can be determined in the poster generation model 310, the poster expansion model 320 and the dynamic poster model 330, and the scene factor 405, the lighting factor 406, the sound factor 407 and the time factor 408 are input into the image generation model to obtain the target image 409.

[0089] Therefore, when the target scene type includes an indoor scene, the generation process of each candidate image generation model is as shown in the following equations (4) to (6).

[0090] A=A(x,e,t,v,d) (4)

[0091] B=B(y,z,e,t,v,d) (5)

[0092] C=C(y,e,t,v,d) (6)

[0093] Among them, A represents the poster generation model, B represents the poster expansion model, C represents the dynamic poster model, x represents the text, y represents the image, z represents the size, e represents the scene factor, t represents the lighting factor, v represents the sound factor, and d represents the time factor.

[0094] According to the embodiments of the present disclosure, by considering the lighting information, sound information and current date in the indoor scene and indicating the image generation effect based on these factors respectively, the indoor environment in the real world can be simulated more comprehensively. This multi-dimensional environmental perception can make the subsequently generated target image more suitable for indoor scenes.

[0095] The above describes the influencing factors of the indoor scene and the method of generating the target image for the indoor scene in the present disclosure. Figure 5A and Figure 5B The influencing factors in outdoor scenes and the process of generating target images for outdoor scenes are explained.

[0096] In the case where the target scene type includes an outdoor scene, the environmental information may include at least one of the following: temperature information and weather information. Outdoor scenes may include squares, roads, and high-rise buildings, etc. Different outdoor scenes have different requirements for poster generation effects.

[0097] Figure 5A An example schematic diagram of influencing factors in an outdoor scene according to an embodiment of the present disclosure is schematically shown; Figure 5B An example schematic diagram of a process of generating a target image in an outdoor scene according to an embodiment of the present disclosure is schematically shown.

[0098] Combination Figure 5A and Figure 5BAs shown, when the target scene type includes an outdoor scene, the influencing factors may include scene factor 505 , temperature factor 506 , and weather factor 507 .

[0099] In 500A, the scene factor 505 may be obtained by the following operations: using an image acquisition device to capture the current scene to obtain an environment image. Recognizing the environment image to obtain scene information. Based on the scene information, determining the scene factor 505. The scene factor 505 is used to characterize the style of the candidate image corresponding to the target scene type.

[0100] The temperature factor 506 may be obtained by the following operation: using the temperature sensing device 504 to detect the temperature of the current scene to obtain temperature information. According to the temperature information, the temperature factor 506 is determined. The temperature factor 506 is used to characterize the high or low temperature.

[0101] The weather factor 507 may be obtained by the following operations: parsing the environment image to obtain weather information. According to the weather information, determining whether the weather factor 507 is represented. The weather factor 507 is used to represent whether the weather information is the preset weather.

[0102] In 500B, based on the generation type, the image generation model can be determined in the poster generation model 310, the poster expansion model 320 and the dynamic poster model 330, and the scene factor 505, the temperature factor 506 and the weather factor 507 are input into the image generation model to obtain the target image 508.

[0103] Therefore, when the target scene type includes an outdoor scene, the generation process of each candidate image generation model is as shown in the following equations (7) to (9).

[0104] A=A(x,w,p) (7)

[0105] B=B(y,z,w,p) (8)

[0106] C=C(y,w,p) (9)

[0107] Among them, A represents the poster generation model, B represents the poster expansion model, C represents the dynamic poster model, x represents the text, y represents the image, z represents the size, w represents the weather factor, and " represents the temperature factor.

[0108] According to the embodiments of the present disclosure, by considering the temperature information and weather information in the outdoor scene and indicating the image generation effect based on these factors respectively, the outdoor environment in the real world can be simulated more comprehensively. This multi-dimensional environmental perception can make the subsequently generated target image more suitable for outdoor scenes.

[0109] The above describes the different influencing factors and the method of generating the target image in the indoor scene and the outdoor scene of the present disclosure. In addition, the influencing factors may also include object factors, which will be described below.

[0110] In one example, since posters generated for different scenes may target different groups of people, poster generation can also combine the characteristics of the currently observed group of people to generate posters of different styles. For example, after obtaining the environmental image, the environmental image can also be identified to obtain object information. Object information can characterize the state of the object. For example, object information can include name, age, gender, height, and weight. The identification method can be configured according to actual business needs and is not limited here.

[0111] On this basis, object features can be analyzed according to object information, and object factors that characterize the object features of the object information can be determined. The object features can include at least one of the following: elderly, children, middle-aged people, males or females. For example, in a kindergarten, the most distributed people may be children, so a cartoon-style target image can be generated. For another example, in a nursing home, the most distributed people may be the elderly, so a target image with larger content and softer style can be generated.

[0112] Therefore, based on the object factors, the generation process of each candidate image generation model is as shown in the following equations (10) to (12).

[0113] A=A(x,f) (10)

[0114] B=B(y,z,f) (11)

[0115] C=C(y,f) (12)

[0116] Among them, A represents the poster generation model, B represents the poster expansion model, C represents the dynamic poster model, x represents the text, y represents the image, z represents the size, and f represents the object factor.

[0117] According to the embodiments of the present disclosure, by identifying objects in environmental images and using their object features as object factors, it is possible to ensure that object information is accurately expressed in the subsequent image generation process, thereby helping to improve the accuracy of image content.

[0118] The above describes the different influencing factors of the present disclosure. Figure 6 The method of generating the target image is described.

[0119] In one example, the image generation instruction may include at least one of text and an image. Before the image generation instruction and the at least one influencing factor are input into the image generation model, a match may be performed based on at least one of the text and the image, the at least one influencing factor and a pre-configured mapping relationship, and based on the obtained matching result, it is determined whether it is necessary to generate the target image through the image generation model again.

[0120] Since the text and image and at least one influencing factor input this time are the same as the text and image and at least one influencing factor input in the past, a pre-configured mapping relationship can be set to achieve poster sharing. That is, after each target image is generated, the text and image input this time are taken as candidate objects, at least one influencing factor is taken as a candidate influencing factor, and the generated target image is taken as a candidate image for associated storage.

[0121] After obtaining the text and image input this time and at least one influencing factor, the text and image can be matched with multiple candidate objects in the mapping relationship, and at least one influencing factor can be matched with multiple candidate influencing factors in the mapping relationship. If there are candidate objects that match the text and image, and candidate influencing factors that match the influencing factors, the candidate image corresponding to the candidate object and the candidate influencing factors can be directly output as the target image. If there are no candidate objects that match the text and image, and candidate influencing factors that match the influencing factors, the text and image input this time and at least one influencing factor can be input into the image generation model to generate the target image.

[0122] Figure 6 An example schematic diagram of a target image generation process according to an embodiment of the present disclosure is schematically shown.

[0123] like Figure 6 As shown, in 600 , the mapping relationship 610 may include a correspondence between a candidate object 601 , a candidate influencing factor 602 , and a candidate image 603 .

[0124] After obtaining the image generation instruction 604 and the influencing factors 605, matching can be performed based on the image generation instruction 604 and the multiple candidate objects 601 in the mapping relationship to obtain an intermediate matching result. The intermediate matching result can indicate whether there is a candidate object 601 in the multiple candidate objects 601 that matches at least one of the text and the image in the image generation instruction 604.

[0125] For example, when the intermediate matching result indicates that there is no candidate object 601 matching the image generation instruction 604 among the multiple candidate objects 601, the image generation instruction 604 and the influencing factor 605 can be input into the image generation model 608 to obtain the target image 609. Alternatively, when the intermediate matching result indicates that there is a candidate object 601 matching the image generation instruction 604 among the multiple candidate objects 601, the influencing factor 605 can continue to be matched with the candidate influencing factor 602 corresponding to the candidate object 601 matching the image generation instruction 604 to obtain the matching result 606. After obtaining the matching result 606, operation S610 can be performed.

[0126] In operation S610 , it is determined whether there is a candidate object matching the image generation instruction and a candidate influencing factor matching the influencing factor in the matching result representation mapping relationship.

[0127] If yes, the candidate image 603 corresponding to the matched candidate object 601 and the candidate influencing factor 602 can be directly determined as the target image 607. If no, the image generation instruction 604 and the influencing factor 605 can be input into the image generation model 608 to obtain the target image 609.

[0128] According to an embodiment of the present disclosure, a mapping relationship is preconfigured, and matching is performed in the mapping relationship according to an image generation instruction and at least one influencing factor to obtain a matching result. When the matching result represents that there is a candidate object that matches the image generation instruction and a candidate influencing factor that matches the influencing factor in the mapping relationship, the candidate image corresponding to the candidate object and the candidate influencing factor is directly determined as the target image. This can avoid repeated generation of images with the same object and influencing factor, thereby improving image generation efficiency and realizing image reuse.

[0129] The above describes the method for generating a target image provided by an embodiment of the present disclosure. Figure 7 How to generate a target image with audio is described.

[0130] Figure 7 An example schematic diagram of a target image generation process according to another embodiment of the present disclosure is schematically shown.

[0131] like Figure 7As shown, in 700, after obtaining at least one influencing factor 701, at least one influencing factor 701 can also be input into a preset model 702 to obtain a target audio 703. The preset model 702 can be used to recommend a target audio 703 with a relatively matching style according to the at least one influencing factor 701. On this basis, at least one influencing factor 701, the target audio 703 and at least one of the text and the image in the image generation instruction 704 can be input into an image generation model 705 to obtain a target image 706. The target image 706 is an audio poster. In addition, the volume of the target audio 703 is adaptively adjusted based on whether the surrounding environment is noisy.

[0132] According to an embodiment of the present disclosure, a target image is generated by combining target audio, image generation instructions and at least one influencing factor, which not only takes visual elements into consideration but also incorporates auditory elements. This multi-sensory combination can create a richer and three-dimensional experience for users, enhance the user's sense of immersion, and improve the image generation effect.

[0133] The above describes how to generate a target image with audio in the present disclosure. Figure 8 A method for generating a target image through an image edge expansion process provided by another embodiment of the present disclosure is described.

[0134] Figure 8 An example schematic diagram of a target image generation process according to another embodiment of the present disclosure is schematically shown.

[0135] like Figure 8 As shown, in 800, in the process of generating the target image 806 through the image edge expansion process, the edge expansion size of the edge expansion area 801 and the first position information 803 of the area to be placed 802 can be obtained. Through the first position information 803, at least one of the text and the image in the image generation instruction can be placed arbitrarily in the edge expansion area 801. For example, the first position information 803 may include the coordinates of the center point. On this basis, the edge expansion size of the edge expansion area 801, the first position information 803, at least one of the text and the image in the image generation instruction, and at least one influencing factor 804 can be input into the image generation model 805 to obtain the target image 806.

[0136] According to the embodiments of the present disclosure, by obtaining the edge expansion size and the first position information of the area to be placed, accurate layout guidance can be provided for image edge expansion generation. On this basis, by inputting the edge expansion size, the first position information, the image generation instruction and at least one influencing factor into the image generation model to obtain the target image, accurate image layout control is achieved, and the flexibility of image edge expansion is enhanced.

[0137] The above describes the method of generating a target image through the image edge expansion process of the present invention. Fig. 9 A method for realizing local image generation is described.

[0138] Fig. 9 An example schematic diagram of a target image generation process according to another embodiment of the present disclosure is schematically shown.

[0139] like Fig. 9 As shown, in 900, the image generation instruction may include a reference image 901 and second position information 903. Reference image 901 refers to a generated image, and there is a region to be updated in the reference image 901 where the image generation effect does not meet expectations, and the second position information 903 may be used to indicate the position of an image 902 in the region to be updated in the reference image 901, and the size of the image 902 in the region to be updated is less than or equal to the size of the reference image.

[0140] The indication method of the area to be updated can be configured according to actual business needs and is not limited here. For example, on a screen device with a touch function, the indication method can be to draw an area with a finger, and then calculate the coordinates of the finger falling and lifting to obtain the second position information 903. Alternatively, on a screen device without a touch function, the indication method can be to segment the reference image 901 to obtain multiple candidate image blocks, and select at least one target image block from the multiple candidate image blocks based on the remote control to obtain the second position information 903.

[0141] After obtaining the second position information 903, the reference image 901 and the second position information 903 can be input into the image generation model 904, so that the model can partially regenerate the image 902 of the area to be updated in the reference image 901 based on the reference image 901 and the current material, and obtain the updated target image 905. Since there is no need to regenerate the entire image, the generation time is shortened and the model calculation amount is reduced.

[0142] Therefore, the local image generation process of each candidate image generation model is as shown in the following equations (13) to (15).

[0143] A=A(x,r) (13)

[0144] B=B(y,z,r) (14)

[0145] C=C(y,r) (15)

[0146] Among them, A represents the poster generation model, B represents the poster expansion model, C represents the dynamic poster model, x represents the text, y represents the image, z represents the size, and r represents the second position information.

[0147] Before generating the target image, the image generation model can determine whether the second position information (i.e., r) exists. If the second position information exists, there is no need to generate the complete target image, but only partial generation based on the second position information is sufficient. If the second position information does not exist, the complete target image can be generated again.

[0148] According to an embodiment of the present disclosure, the position of the area to be updated in the reference image can be accurately specified through the second position information in the image generation instruction, thereby ensuring that only the specified area to be updated will be regenerated, while other areas remain unchanged, thereby realizing the update of the local content of the image, thereby saving image generation time and energy consumption of the device.

[0149] The above describes the method for realizing local image generation in the present disclosure. In addition, the influencing factors may also include screen factors. Fig.10 Explain the screen factor.

[0150] Fig.10 An example schematic diagram of screen factors according to an embodiment of the present disclosure is schematically shown.

[0151] Since different types of screens have different display effects or requirements for the generated posters, the screen characteristics can also be used as an influencing factor for poster generation. For example, a 2k screen can only display resolutions of 2k and below. For another example, the display effect of a special-shaped screen is different from that of a normal screen.

[0152] like Fig.10 As shown in 1000, taking the ordinary screen 1001 as an example, the width (ie, w 1 ) and high (i.e. h 1 ), determine a first screen resolution, and use the first screen resolution as the screen factor of the ordinary screen 1001.

[0153] Taking the special-shaped screen 1002 as an example, the maximum horizontal distance of the special-shaped screen 1002 (i.e., w 2 ) and the maximum longitudinal distance (i.e. h 2 ), determine the second screen resolution, and use the second screen resolution as the screen factor of the special-shaped screen 1002.

[0154] Therefore, based on the screen factors, the generation process of each candidate image generation model is shown in the following equations (16) to (18).

[0155] A=A(x,s) (16)

[0156] B=B(y,z,s)(17)

[0157] C=C(y,s) (18)

[0158] Among them, A represents the poster generation model, B represents the poster expansion model, C represents the dynamic poster model, x represents the text, y represents the image, z represents the size, and s represents the screen factor.

[0159] The above describes the screen factors of the present disclosure. Fig.11 A target image generation process provided by another embodiment of the present disclosure is described.

[0160] Fig.11 An example schematic diagram of a target image generation process according to another embodiment of the present disclosure is schematically shown.

[0161] like Fig.11 As shown, in 1100, the reference image 1101 and the image generation instruction 1102 are input into the image generation model 1103 to obtain the target style 1104 of the reference image 1101, and then the target image 1106 is generated based on the target style 1104 and the object 1105 in the image generation instruction 1102. The object 1105 may include at least one of text and an image. The target image 1106 is a poster having the same target style 1104 as the reference image 1101 and generated using the object 1105 input this time, thereby achieving the generation of the target image according to the style of the existing image.

[0162] The above describes the different influencing factors provided by the present disclosure. Fig.12 The overall target image generation process is described.

[0163] Fig.12 An example schematic diagram of a target image generation process according to another embodiment of the present disclosure is schematically shown.

[0164] like Fig.12 As shown, in 1200, in response to the image generation instruction, environmental information 1210 for the current scene may be acquired. At least one influencing factor is determined based on the environmental information 1210. The influencing factor may include at least one of the following: scene factor 1201, illumination factor 1202, sound factor 1203, time factor 1204, temperature factor 1205, weather factor 1206, object factor 1207, screen factor 1208, and local factor 1209.

[0165] After obtaining at least one influencing factor, a plurality of candidate objects 601 and a plurality of candidate influencing factors 602 in the pre-configured mapping relationship 610 may be matched based on the image generation instruction and the at least one influencing factor.

[0166] When there is a candidate object 601 matching the image generation instruction and a candidate influencing factor 602 matching the influencing factor in the preconfigured mapping relationship 610 , a candidate image 603 corresponding to the candidate object 601 and the candidate influencing factor 602 may be determined as the target image 1211 .

[0167] When there is no candidate object 601 matching the image generation instruction and no candidate influencing factor 602 matching the influencing factor in the pre-configured mapping relationship 610, the image generation instruction and at least one influencing factor can be input into the image generation model 1211 to obtain a target image 1212 for the current scene.

[0168] The following will take the image generation process and image edge expansion process as examples. Fig.13 and Fig.14 The process of generating the target image based on different methods is explained respectively.

[0169] Fig.13 An example schematic diagram of a process of generating a target image by an image generation method according to an embodiment of the present disclosure is schematically shown.

[0170] like Fig.13 As shown, in 1300, taking the supermarket scene as an example, an environmental image 1302 can be obtained by shooting using an image acquisition device 1301. The environmental image 1302 is analyzed to obtain at least one influencing factor 1303 that characterizes the surrounding environment. The at least one influencing factor 1303 can guide the generation of a poster based on the surrounding environment. On this basis, by inputting at least one influencing factor into the image generation model 1304, a target poster 1305 is obtained, which can realize adaptive poster adjustment generation in different environments, thereby intelligently generating a poster that is particularly harmonious with the current environmental atmosphere. In addition, a target audio can also be added to the target poster 1305 to make consumers more inclined to buy this product.

[0171] Fig.14 An example schematic diagram of a process of generating a target image by expanding an image according to an embodiment of the present disclosure is schematically shown.

[0172] like Fig.14 As shown, in 1400 , after obtaining the image generation instruction 1401 , the edge expansion size of the edge expansion area 801 and the first position information 1403 of the area to be placed can be further obtained.

[0173] On this basis, at least one of the text and the image in the image generation instruction 1401 , the edge expansion size of the edge expansion area 1402 and the first position information 1403 can be input into the image generation model 1404 to obtain the target image 1405 .

[0174] The above are only exemplary embodiments, but are not limited thereto, and may also include other image generation methods known in the art, as long as customized generation of a target image can be achieved.

[0175] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the image generation method, model training method and image processing method according to the embodiments of the present disclosure are implemented.

[0176] In the present disclosure, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0177] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the image generation method, model training method and image processing method provided by the embodiment of the present disclosure.

[0178] When the computer program is executed by the processor, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0179] According to the embodiments of the present disclosure, program codes for executing the computer programs provided by the embodiments of the present disclosure may be written in any combination of one or more programming languages.

[0180] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0181] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for generating an image, comprising: In response to the image generation instruction, acquiring environmental information for the current scene; Determine at least one influencing factor according to the environmental information, wherein each of the influencing factors is used to indicate an image generation effect based on the environmental information; and generate a target image for the current scene according to the image generation instruction and the at least one influencing factor.

2. The method according to claim 1, wherein: The environmental information includes an environmental image captured for the current scene; Determining at least one influencing factor according to the environmental information includes: Determining a target scene type from a plurality of candidate scene types according to scene information obtained by identifying the environment image, wherein each of the candidate scene types has a candidate image style; and The candidate image styles of the target scene type are determined as scene factors.

3. The method according to claim 2, wherein: The target scene type includes an indoor scene, and the environmental information includes at least one of the following: lighting information, sound information, and current date; Determining at least one influencing factor according to the environmental information includes at least one of the following: Determining a lighting factor according to the lighting information; Determine the sound factor according to the sound information; and / or Based on the current date, a time factor is determined.

4. The method according to claim 2, wherein: The target scene type includes an outdoor scene, and the environmental information includes at least one of the following: temperature information and weather information; Determining at least one influencing factor according to the environmental information includes at least one of the following: Determining a temperature factor based on the temperature information; and / or Weather factors are determined according to the weather information.

5. The method according to claim 2, wherein: Determining at least one influencing factor according to the environmental information includes: The object factor is determined according to the object information obtained by recognizing the environment image.

6. The method according to any one of claims 1 to 5, further comprising, before generating a target image for the current scene according to the image generation instruction and the at least one influencing factor: According to the image generation instruction and the at least one influencing factor, a match is performed in a pre-configured mapping relationship to obtain a matching result, wherein: The mapping relationship includes a correspondence between candidate objects, candidate influencing factors and candidate images; as well as In response to the matching result indicating that there is a candidate object matching the image generation instruction and a candidate influencing factor matching the influencing factor in the mapping relationship, a candidate image corresponding to the candidate object and the candidate influencing factor is determined as the target image.

7. The method according to any one of claims 1 to 5, further comprising: Determining a target audio frequency according to the at least one influencing factor; Generating a target image for the current scene according to the image generation instruction and the at least one influencing factor comprises: The target image is generated according to the target audio, the image generation instruction and the at least one influencing factor.

8. The method according to any one of claims 1 to 5, wherein: The image generation instruction is used to instruct at least one image generation model; Generating a target image for the current scene according to the image generation instruction and the at least one influencing factor comprises: Determining the image generation model from a plurality of candidate image generation models according to the image generation instruction; and The image generation instruction and the at least one influencing factor are input into the image generation model to obtain the target image.

9. The method according to claim 8, wherein: In the case where the generation type is an image edge expansion type, the method further includes: Acquire the edge expansion size and first position information of the area to be placed, wherein the first position information includes the coordinates of the center point; The step of inputting the image generation instruction and the at least one influencing factor into the image generation model to obtain the target image comprises: The edge expansion size, the first position information, the image generation instruction and the at least one influencing factor are input into the image generation model to obtain the target image.

10. The method according to claim 1, wherein: The image generation instruction includes a reference image and second position information, wherein the second position information is used to indicate the position of the image of the area to be updated in the reference image, and the size of the image of the area to be updated is smaller than or equal to the size of the reference image; The method further comprises: The image of the to-be-updated area in the reference image is regenerated according to the second position information to obtain an updated target image.

11. An image display device, comprising: a processor, for acquiring environmental information for a current scene in response to an image generation instruction; Determining at least one influencing factor according to the environmental information, wherein each of the influencing factors is used to indicate an image generation effect based on the environmental information; and generating a target image for the current scene according to the image generation instruction and the at least one influencing factor; A display device is connected to the processor for displaying the target image.

12. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images