Article background display method and system for intelligent show window and intelligent show window

Abstract

The invention relates to the technical field of artificial intelligence, discloses an article background display method and system for an intelligent show window, a computer storage medium, a computer program product and the intelligent show window, and is used for solving the problem that the background effect displayed by an existing intelligent show window system cannot be adjusted according to displayed articles and viewing audiences. And the object background display effect is not good. The method comprises the following steps: acquiring an article image of a displayed target article, and acquiring a figure image of an audience who watches a show window at present; identifying the collected image information of the to-be-displayed article to obtain first text description information of the appearance of the displayed article; identifying the collected image information of the person viewing the show window to obtain second text description information of the character characteristics; and inputting the first text description information and the second text description information into an image generation model to generate a show window background image, and displaying the show window background image.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system, computer storage medium, computer program product and smart window for displaying object background in a smart window. Background Art

[0002] In recent years, the technology of AI-generated content (AIGC) in the field of image generation and processing has continued to mature, and its application in multimedia and advertising fields has become more and more extensive. The use of AI-generated content can make the generation and customization of image content more efficient and intelligent. Specifically, traditional window displays usually rely on manual design and layout, which is not only time-consuming but also difficult to change in real time. With the development of AI technology, smart windows use computer vision and deep learning algorithms to realize the intelligent generation and adjustment of display content, improving the display effect and viewer experience.

[0003] However, in the prior art, when AI is used to generate content for background display of items in smart windows, the solution is mostly to take photos of items and then generate promotional pictures of the items. The process of generating pictures requires professionals to input prompt words for the use of the items and the corresponding scene requirements, generate them in the cloud, obtain the pictures and then transmit them to the display terminal through the network system for display. This requires personnel with experience in prompt words to operate, and everyone has different perceptions of the environment, so it is difficult to have a relatively uniform generation effect; and the background effect generated by this generation method is determined by the merchant user. When displaying, it is not only not accurate in matching the characteristics of the displayed items, but also cannot be adjusted according to the audience watching the display, and does not take into account the individual preference differences of the final audience. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem in the prior art that the background effect displayed by the existing intelligent window system cannot be adjusted according to the displayed objects and the viewing audience, resulting in poor object background display effect.

[0005] A first aspect of the present invention provides a method for displaying the background of an object in a smart window, comprising: obtaining an object image of a target object to be displayed, and obtaining a person image of a viewer currently viewing the window; identifying the collected image information of the object to be displayed, and obtaining first text description information of the appearance of the displayed object; identifying the collected image information of a person viewing the window, and obtaining second text description information of the person's characteristics; generating a window background image based on the first text description information and the second text description information, and displaying the window background image.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, the identifying the collected image information of the object to be displayed to obtain the first text description information of the appearance of the displayed object includes: preprocessing the image information of the object to be displayed, and identifying the object body contained in the preprocessed image information of the object to be displayed; identifying the object body based on a multimodal language model, generating a natural language description corresponding to the appearance of the object body, and obtaining the first text description information.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the identifying the collected image information of the person viewing the window display to obtain the second text description information of the person's characteristics includes: preprocessing the image information of the person viewing the window display, and identifying the character subject contained in the preprocessed character image information; identifying the character subject based on a multimodal language model, generating a natural language description corresponding to the character characteristics of the character subject, and obtaining the second text description information.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, the window display includes a first screen and a second screen that is at an angle to the first screen; generating a window display background image based on the first text description information and the second text description information, and displaying the window display background image includes: calling an aesthetic design model, reasoning about the background design in combination with the first text description information and the second text description information, and obtaining background prompt words; inputting the background prompt words into an image generation model, and based on the positional relationship between the first screen and the second screen, generating and displaying background images suitable for the first screen and the second screen, respectively.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, acquiring the object image of the target item on display includes: receiving first video information inside the smart window, and performing image sampling on the first video information at first preset time intervals; identifying whether there are any items placed in the current smart window based on the internal image of the window obtained by image sampling; if there are any items placed, capturing the object image of the target item on display.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, obtaining the character image of the viewer currently viewing the window display includes: receiving second video information in front of the smart window display, and performing image sampling on the second video information at second preset time intervals; identifying whether there is someone currently viewing the window display based on the image obtained by the image sampling; and if there is someone viewing the window display, capturing the character image of the viewer currently viewing the window display.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the object background display method for a smart window also includes: determining whether the objects displayed in the smart window have been replaced; if replaced, recording the replacement time of the objects in the smart window; calculating the object replacement frequency based on the replacement time; and generating a first preset time based on the object replacement frequency.

[0012] Optionally, in a seventh implementation of the first aspect of the present invention, the object background display method for a smart window further includes: determining whether the audience viewing the smart window has changed; if changed, recording the audience change time; calculating the audience change frequency based on the audience change time; and generating a second preset time based on the audience change frequency.

[0013] Optionally, in an eighth implementation manner of the first aspect of the present invention, generating a window background image based on the first text description information and the second text description information, and displaying the window background image also includes: determining a viewing position of a current viewer based on a person image of the viewer currently viewing the window; and displaying the background images suitable for the first screen and the second screen after stretching and adjusting them according to the viewing position and the angle information between the first screen and the second screen.

[0014] The second aspect of the present invention provides an object background display system for a smart window, comprising: an image acquisition module, used to obtain an object image of a target object to be displayed, and to obtain a person image of a viewer currently viewing the window; a first recognition module, used to recognize the collected image information of the object to be displayed, and obtain first text description information of the appearance of the displayed object; a second recognition module, used to recognize the collected image information of a person viewing the window, and obtain second text description information of the person's characteristics; a background generation module, used to generate a window background image based on the first text description information and the second text description information, and to display the window background image.

[0015] A third aspect of the present invention provides a smart display window, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory to enable the smart display window to execute the steps of the above-mentioned method for background display of objects in the smart display window.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the above-mentioned method for background display of items in a smart window.

[0017] A fifth aspect of the present invention provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the method for background display of objects in a smart window as described above are implemented.

[0018] In the technical solution provided by the present invention, an object image of the target object to be displayed is obtained, and a person image of the audience currently viewing the window display is obtained; the collected image information of the object to be displayed is identified to obtain first text description information of the appearance of the displayed object; the collected image information of the person viewing the window display is identified to obtain second text description information of the person's characteristics; the first text description information and the second text description information are input into the image generation model to generate a window display background image, and the window display background image is displayed. This method enables the smart window to intelligently adjust the display background according to the displayed objects and the audience viewing the objects, so that the generated display background is more matched with the displayed objects and can meet the preferences of different viewing groups. In addition, a system, a computer-readable storage medium, a computer program product and a smart window provided by the present invention also solve corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 This is a schematic diagram of the steps of a first embodiment of a method for displaying item backgrounds in a smart window according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the steps of a second embodiment of the method for displaying item backgrounds in a smart window according to an embodiment of the present invention;

[0022] Figure 3 This is a first working schematic diagram of the smart window in an embodiment of the present invention;

[0023] Figure 4 This is a second working schematic diagram of the smart window in an embodiment of the present invention;

[0024] Figure 5 is a third working schematic diagram of the smart window in an embodiment of the present invention;

[0025] Figure 6 Schematic diagram of the workflow of the smart window in an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of an embodiment of an item background display system for a smart window in an embodiment of the present invention;

[0027] Figure 8 Schematic diagram of another embodiment of the object background display system for a smart window in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms, and should not be construed as limiting the present invention to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and is more convenient for fully conveying the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0029] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0030] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.

[0031] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0034] See also Figure 1 The first embodiment of the method for displaying item background in a smart window in the embodiment of the present invention includes:

[0035] S101, obtaining an object image of a target object on display and a person image of a viewer currently viewing the window display;

[0036] It is understandable that the execution subject of the present invention may be an item background display device or a smart window terminal, which is not limited here. The embodiment of the present invention is described by taking the smart window as the execution subject.

[0037] The smart window in this embodiment can be used to display background images related to the target item currently displayed in the window to viewers viewing the window. In specific implementation, the smart window can respond to an item display request and obtain an item image of the target item currently displayed and a person image of the viewer currently viewing the window.

[0038] When obtaining the object image of the target object currently on display and the person image of the audience currently viewing the window display, the image acquisition module in the smart window display can respectively capture the images of the target object placed in the window display and the audience viewing the window display.

[0039] In a specific implementation, the smart window can automatically determine whether a background image of the window can be generated. For example, it can detect whether there are viewers in the current window and whether there are target items on display. If both are present, it is determined that a background image needs to be generated, and the image of the target item on display and the image of the viewer currently viewing the window are obtained. When detecting whether there are viewers in the current window and whether there are target items on display, infrared sensing detection or other detection methods can be used; in a preferred implementation, the image information of the image in the window and the image in front of the window can be obtained through an image acquisition module, and recognition can be performed based on the image information, and the content contained in the recognized image can be used to determine whether there are viewers in the current window and whether there are items on display.

[0040] S102, identifying the collected image information of the object to be displayed, and obtaining first text description information of the appearance of the object to be displayed;

[0041] After obtaining the image information of the target object being displayed, the features of the object contained in the image information are identified, and the features are converted into a natural language description to obtain the first text description information, wherein the first text description information is mainly for the appearance shape, type information, color and other contents of the target object displayed in the current image information.

[0042] S103, identifying the collected image information of the person viewing the window display, and obtaining second text description information of the person's characteristics;

[0043] After obtaining the image information of the person viewing the window display, the features of the person contained in the image information are identified, and the features are converted into a natural language description to obtain second text description information, wherein the second text description information mainly targets the gender, age, and clothing style of the person viewing the window display contained in the current image information.

[0044] S104: Generate a window background image based on the first text description information and the second text description information, and display the window background image.

[0045] In combination with the first text description information and the second text description information in the aforementioned step, the self-trained aesthetic design model is called to infer the window background design based on the described objects and the characteristics of the people viewing the window, and the prompt words required for generating the window background image are output. The window background image is generated according to the prompt words, and the window background image is displayed.

[0046] The method in the embodiment of the present invention enables the smart window to intelligently adjust the display background according to the displayed items and the audience viewing the items, so that the generated display background is more matched with the displayed items and can meet the preferences of different viewing groups.

[0047] Please see Figure 2-6 The second embodiment of the method for displaying the background of items in a smart window in the embodiment of the present invention includes:

[0048] S201, obtaining an object image of a target object on display, and obtaining an image of a person currently viewing the window display;

[0049] The smart display window described in this embodiment includes a first screen and a second screen, wherein the second screen forms a preset angle with the first screen. Figure 3 The first screen is located at the rear background position of the target object displayed in the smart window, the second screen is connected to the first screen and is arranged at a right angle below, and the displayed target object can be placed above the second screen.

[0050] The smart window in this embodiment further includes a processor and an image acquisition module. The processor can be used to control the image acquisition module and the screen of the window, and process and generate images, etc. The various models described later can be configured in the processor. The number of the image acquisition modules can be one or more. In a preferred embodiment, the number of the image acquisition modules can be two, including a first image acquisition module and a second image acquisition module. The first image acquisition module is used to acquire the object image of the target object on display, and the second image acquisition module is used to acquire the image of the person of the audience currently viewing the window.

[0051] Please continue reading Figure 4The first image acquisition module can acquire the first video information inside the smart window in real time, and perform image sampling based on the acquired first video information at a first preset time interval to obtain an image inside the window; based on the image inside the window obtained by image sampling, it is identified whether there are objects placed in the current smart window; if there are objects placed, an object image of the target object on display is acquired or the image inside the window is cropped to obtain an object image of the target object on display.

[0052] Please continue reading Figure 5 , the second image acquisition module can acquire the second video information in front of the smart window in real time, and perform image sampling based on the acquired second video information at a second preset time interval to obtain the image in front of the window; based on the image in front of the window obtained by image sampling, it is identified whether there is someone in front of the current smart window viewing the window and the items inside; if there is someone viewing the window, the image of the person viewing the window is acquired or the image in front of the window is cut to obtain the image of the person viewing the window. Among them, the image of the person viewing the window currently includes not only the image information of the person, but also the environment information of the person viewing the window.

[0053] S202, preprocessing the image information of the object to be displayed, and identifying the object body contained in the preprocessed image information of the object to be displayed;

[0054] After obtaining the image information of the object to be displayed, it is first preprocessed. The image preprocessing steps include filtering and denoising, illumination correction, and image edge enhancement. The preprocessed image information of the object to be displayed is identified to obtain the object body contained in the image information of the object to be displayed.

[0055] S203, identifying the object body based on the multimodal language model, generating a natural language description corresponding to the appearance of the object body, and obtaining first text description information;

[0056] The object body is identified based on a pre-trained multimodal language model to obtain a natural language description corresponding to the appearance of the object body. The multimodal language model is trained to generate natural language description information of the object features based on the image of the object body identified in the image, including information such as the appearance shape, the type of object, and the color of the object. The first text description information obtained in this step is the appearance shape, the type of object, and the color of the object, etc., which are generated in natural language based on the object body obtained in the previous step.

[0057] S204, preprocessing the image information of the person viewing the window display, and identifying the person subject contained in the preprocessed image information of the person;

[0058] S205, identifying the character subject based on the multimodal language model, generating a natural language description corresponding to the character characteristics of the character subject, and obtaining second text description information;

[0059] Step S204 and step S205 in this embodiment are similar to the contents of step S202 and step S203 in the aforementioned embodiment, and also pre-process the image information of the person viewing the window display to identify the person subject contained in the pre-processed image information of the person. The person subject is identified according to a pre-trained multimodal language model. In addition to the function in S203, the multimodal language model is also configured to generate a natural language description of the person's characteristics based on the image of the person subject, to generate a natural language description of the environmental characteristics based on the environment where the person subject is located, and other functions. In step S205, the second text description information of the person's characteristics obtained by identifying the person subject based on the multimodal language model includes information such as the age, gender, and dressing style of the person subject.

[0060] In addition, the second text description information described in this embodiment also includes a natural language description of the environment in which the person is located. In the case where the second text description information also includes a natural language description of the environment in which the person is located, while acquiring the image information of the person viewing the window display, the environment information in which the person viewing the window display is currently located is also extracted, and a natural language description of the environment is generated based on the environment information.

[0061] S206, calling the aesthetic design model, combining the first text description information and the second text description information to perform background design reasoning, and obtaining background prompt words;

[0062] After obtaining the first text description information and the second text description information, the first text description information and the second text description information are input into a pre-trained aesthetic design model. The aesthetic design model is constructed based on a deep learning algorithm, and can specifically be a unimodal or multimodal large language model. The aesthetic design model is trained to perform design reasoning based on the first text description information and the second text description information of the text description, and can reason about a more popular window background design based on the items displayed in the window and the characteristics of the people viewing the window, and output background prompt words for the window background design.

[0063] S207: input the background prompt words into the image generation model, and based on the positional relationship between the first screen and the second screen, generate and display background images suitable for the first screen and the second screen respectively.

[0064] The image generation model may be a diffusion image generation model, which may generate a window background according to background prompt words, and display the window background on the first screen and the second screen.

[0065] In this embodiment, due to the positional relationship between the first screen and the second screen, the backgrounds displayed in the first screen and the second screen need to be adjusted accordingly so that they respectively display related but not identical images. Specifically, a pre-trained image generation model can be called to generate the image specifically, and the image generation model can be constructed based on a diffusion model and a low-rank adaptation (LoRA) model.

[0066] The image generation model is pre-trained and can be adjusted based on the size ratio and angle information between the first screen and the second screen and the window background image generated by the image generation model to generate two matching background images of the first screen and the second screen.

[0067] In a preferred embodiment, there may be multiple second screens, which may be located above and on the left and right sides of the target object, and each second screen and the first screen form a storage space similar to a cuboid or a cube. The low-rank adaptive model described in this embodiment can perform brightness adjustment, stretching adjustment, and other processing on the generated background image according to the positional relationship between each second screen and the first screen to obtain a background image with better display effect, and display the adjusted background image.

[0068] In a preferred embodiment, when the image information of the person viewing the window is obtained in step S204 and step S205, the position of the person in the image information is also identified to obtain the relative position of the person viewing the window and the window, including the relative distance and the sight height, etc. Specifically, the relative position can be obtained by identifying the position and area occupied by the task in the image information. When the image is subsequently adjusted in step S207, the image adjustment parameters of the first screen and each second screen image can be calculated according to the relative distance and the sight height, and the background image can be adjusted and displayed based on the image adjustment parameters.

[0069] In a specific embodiment, please continue to refer to Figure 6In a specific workflow of a smart window, after the smart window starts working, after the video signal is connected through the image acquisition module, the video is sampled every 1 second, and it is judged whether there is someone staying in front of the smart window based on the sampled image; if there is someone staying, the semantic recognition of the person and the environment is performed, and if no one is staying, the video signal continues to be sampled and processed. At the same time, the smart window can be displayed, and in this embodiment, the sampling interval can be set according to the frequency of item replacement, and the image is collected according to the set interval for sampling the image of the item, and the semantic recognition of the item is performed. Based on the semantic recognition results of the person and the environment and the semantic recognition results of the item, the program is called to generate a background image and transmitted to the screen for display. For specific steps, please refer to the contents of the aforementioned steps S201-S207 in this embodiment.

[0070] The method provided in the embodiment of the present invention enables the smart window to intelligently adjust the display background according to the displayed items and the audience viewing the items, and provide different backgrounds for different audiences in different scenes to display the goods, so that the generated background can meet the preferences of different viewing groups, and overall make the entire window more targeted to show the appearance style that the current audience is more likely to be interested in, so as to attract more audiences. And the generated display background can be adjusted according to the displayed items, so that the display effect and the displayed items are more matched. At the same time, the background can be adjusted according to the positional relationship between the screens used to display the background and the position of the audience, so that the display effect is more beautiful.

[0071] The above describes the method for displaying the background of items in a smart window according to an embodiment of the present invention. The following describes the system for displaying the background of items in a smart window according to an embodiment of the present invention. Figure 7 , an embodiment of the object background display system for a smart window in an embodiment of the present invention includes:

[0072] The image acquisition module 701 is used to acquire the image of the target object on display and the image of the person currently viewing the window display;

[0073] A first recognition module 702 is used to recognize the collected image information of the object to be displayed, and obtain first text description information of the appearance of the displayed object;

[0074] The second recognition module 703 is used to recognize the collected image information of the person viewing the window display, and obtain the second text description information of the person's characteristics;

[0075] The background generation module 704 is used to generate a window background image based on the first text description information and the second text description information, and display the window background image.

[0076] The method provided in the embodiment of the present invention enables the smart window to intelligently adjust the display background according to the displayed items and the audience viewing the items, so that the generated display background and the displayed items are more matched and can meet the preferences of different viewing groups.

[0077] See also Figure 8 In another embodiment of the present application, the first recognition module 702 is specifically used to: preprocess the image information of the object to be displayed, and identify the object body contained in the preprocessed image information of the object to be displayed; identify the object body based on the multimodal language model, generate a natural language description corresponding to the appearance of the object body, and obtain the first text description information.

[0078] In another embodiment of the present application, the second recognition module 703 is specifically used to: preprocess the image information of the person viewing the window display, and identify the character subject contained in the preprocessed character image information; identify the character subject based on the multimodal language model, generate a natural language description corresponding to the character characteristics of the character subject, and obtain second text description information.

[0079] In another embodiment of the present application, the display window includes a first screen and a second screen that is at an angle to the first screen; the background generation module 704 includes:

[0080] An image design unit, wherein the image design unit 7041 is specifically used to call an aesthetic design model, perform background design reasoning in combination with the first text description information and the second text description information, and obtain background prompt words;

[0081] The image generation unit 7042 inputs the background prompt words into the image generation model, and based on the positional relationship between the first screen and the second screen, generates and displays background images suitable for the first screen and the second screen respectively.

[0082] In another embodiment of the present application, the image acquisition module 701 includes a first acquisition unit 7011, which is specifically used to: receive first video information inside the smart window, and perform image sampling on the first video information at first preset time intervals; identify whether there are objects placed in the current smart window based on the internal image of the window obtained by image sampling; if there are objects placed, collect the object image of the target object on display.

[0083] In another embodiment of the present application, the image acquisition module 701 includes a second acquisition unit 7012, which is specifically used to: receive second video information in front of the smart window, and perform image sampling on the second video information every second preset time; identify whether there is someone viewing the window based on the image obtained by the image sampling; if there is someone viewing the window, collect the image of the audience currently viewing the window.

[0084] In another embodiment of the present application, the image acquisition module 701 also includes a preset time adjustment unit, and the preset time adjustment unit 7013 is specifically used to: determine whether the items displayed in the smart window are replaced; if replaced, record the replacement time of the items in the smart window; calculate the item replacement frequency based on the replacement time; generate a first preset time based on the item replacement frequency.

[0085] In another embodiment of the present application, the image acquisition module 701 also includes a preset time adjustment unit, and the preset time adjustment unit 7013 is specifically used to: determine whether the audience viewing the smart window has changed; if changed, record the audience change time; calculate the audience change frequency based on the audience change time; and generate a second preset time based on the audience change frequency.

[0086] In another embodiment of the present application, the background generation module 704 also includes: an image adjustment unit 7043, which is specifically used to determine the viewing position of the current audience based on the character image of the audience currently viewing the window display; according to the viewing position and the angle information between the first screen and the second screen, the background image suitable for the first screen and the background image suitable for the second screen are stretched and adjusted for display.

[0087] The specific description of the object background display system for smart window described in this embodiment can refer to the contents of the embodiment of the object background display method for smart window described above.

[0088] The object background display system for smart windows provided in the embodiment of the present invention enables the smart window to intelligently adjust the display background according to the displayed objects and the audience viewing the objects, and provide different backgrounds for different audiences in different scenes to display the goods, so that the generated background can meet the preferences of different viewing groups, and overall make the entire window more targeted to show the appearance style that the current audience is more likely to be interested in, so as to attract more audiences. And the generated display background can be adjusted according to the displayed objects, so that the display effect and the displayed objects are more matched. At the same time, the background can be adjusted according to the positional relationship between the screens used to display the background and the position of the audience, so that the display effect is more beautiful.

[0089] Based on the same inventive concept, an embodiment of this specification also provides a smart window, which includes: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the smart window executes the steps of the object background display method for the smart window described in the aforementioned embodiment.

[0090] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: Figure 1 , 2 Or the method shown in 6.

[0091] accomplish Figure 1 , 2 The computer program of the method shown in or 6 can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0092] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0093] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0094] In summary, the present invention can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that general data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0095] In addition, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for displaying object backgrounds in a smart window in the aforementioned embodiment.

[0096] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0097] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0098] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for displaying item background in a smart window, characterized in that: include: Obtain an object image of a target object on display and obtain a person image of a viewer currently viewing the window display; Identify the collected image information of the object to be displayed to obtain first text description information of the appearance of the object to be displayed; Recognize the collected image information of the person viewing the window display to obtain second text description information of the person's characteristics; A window background image is generated based on the first text description information and the second text description information, and the window background image is displayed.

2. The method for displaying item background in a smart window according to claim 1, characterized in that: The step of identifying the collected image information of the object to be displayed to obtain the first text description information of the appearance of the object to be displayed includes: Preprocessing the image information of the object to be displayed, and identifying the object body contained in the preprocessed image information of the object to be displayed; The object body is identified based on a multimodal language model, a natural language description corresponding to the appearance of the object body is generated, and first text description information is obtained.

3. The method for displaying item background in a smart window according to claim 1, characterized in that: The step of identifying the collected image information of the person viewing the window display to obtain the second text description information of the person's characteristics includes: Preprocessing the image information of the person viewing the window display, and identifying the person subject contained in the preprocessed image information of the person; The character subject is identified based on a multimodal language model, a natural language description corresponding to the character characteristics of the character subject is generated, and second text description information is obtained.

4. The method for displaying item background in a smart window according to claim 1, characterized in that: The display window includes a first screen and a second screen that is at an angle to the first screen; The generating of the window background image based on the first text description information and the second text description information, and displaying the window background image comprises: Calling the aesthetic design model, combining the first text description information and the second text description information to perform background design reasoning, and obtaining background prompt words; The background prompt words are input into the image generation model, and based on the positional relationship between the first screen and the second screen, background images suitable for the first screen and the second screen are generated and displayed respectively.

5. The method for displaying item background in a smart window according to claim 1, characterized in that: The obtaining of the displayed object image of the target object comprises: receiving first video information inside the smart window, and performing image sampling on the first video information at a first preset time interval; Based on the internal image of the window obtained by image sampling, it is recognized whether there are items placed in the current smart window; If there are objects placed, the object image of the displayed target object is collected.

6. The method for displaying item background in a smart window according to claim 1, characterized in that: The step of obtaining the image of the person currently viewing the window display comprises: receiving second video information in front of the smart window, and sampling the second video information at a second preset time interval; Based on the image sampling, the image is used to identify whether there is anyone viewing the window display; If someone is viewing the window display, the image of the person viewing the window display is collected.

7. An item background display system for a smart window, characterized in that: The object background display system for the smart window includes: An image acquisition module is used to acquire an image of a target object on display and an image of a person currently viewing the window display; A first recognition module is used to recognize the collected image information of the object to be displayed, and obtain first text description information of the appearance of the displayed object; A second recognition module is used to recognize the collected image information of the person viewing the window display, and obtain second text description information of the person's characteristics; The background generation module is used to input the first text description information and the second text description information into the image generation model to generate a window background image, and display the window background image.

8. A smart display window for displaying objects in the background, characterized in that: The smart window includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the smart window device to execute the steps of the object background display method for a smart window as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the program / instructions are executed by the processor, the steps of the method for displaying the background of items in a smart window as described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method for displaying item backgrounds in a smart window as described in any one of claims 1 to 6 are implemented.

Images