System and method for providing weather effects in images

Abstract

A system and method for providing a weather effect in an image includes selecting at least one weather texture image indicative of weather and providing a weather effect in the image by superimposing the selected weather texture image on the image. A method of providing a weather effect in an image includes receiving object identification information about a shape of an object in the image and spatial information about a depth of the object in the image, selecting at least one weather texture image indicative of weather, and providing a weather effect in the image.

Description

[0001] This application is a divisional application of the invention patent application with the application date of February 4, 2020, the application number of 202080015016.3, and the invention title of "Systems and Methods for Providing Weather Effects in Images". Technical Field

[0002] The present invention relates to a system and method for providing weather effects in images. Background Art

[0003] Currently, televisions or mobile devices display weather information by using text or pre-designed media. However, it may not be possible to effectively provide weather information to users by using text and icons. In order to provide weather information in the form of video data, time and resources are required to generate the video data. Summary of the Invention

[0004] Aspects of the present invention provide a system that can allow a device to simulate weather effects in an image by using very few computing resources.

[0005] Aspects of the present invention also provide a system that can provide 3D weather effects by using an artificial intelligence model to predict a three-dimensional (3D) space in a two-dimensional (2D) picture.

[0006] Aspects of the present invention also provide a system that can effectively utilize storage space by using weather texture images to real-time simulate weather effects in an image.

[0007] Aspects of the present invention also provide a method for a device to provide weather effects in an image, the method including: obtaining a first image to which a weather effect is to be applied; identifying an object in the first image; identifying a current time and a current weather condition corresponding to the device; and generating a second image with the applied weather effect based on a relationship with the object, according to the current time and the current weather condition.

[0008] Aspects of the present invention also provide a device for providing weather effects in an image, the device including: a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain a first image to which a weather effect is to be applied, identify an object in the first image, identify a current time and a current weather condition corresponding to the device, and generate a second image with the applied weather effect based on a relationship with the object, according to the current time and the current weather condition.

[0009] Aspects of the present invention also provide a computer-readable recording medium having recorded thereon a computer program for executing a method according to at least one embodiment of the present invention.

[0010] Additional aspects will be set forth in the following description and will be apparent from the description, or may be learned by practice of the presented embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other aspects, features, and advantages of certain embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is a schematic diagram showing an example of a system for providing a weather effect in a specific image according to an embodiment of the present invention;

[0013] Figure 2 is a schematic diagram showing an example of reflecting a weather effect in an image according to an embodiment of the present invention;

[0014] Figure 3 is a flowchart of a method for providing a weather effect in an image executed by a system according to an embodiment of the present invention;

[0015] Figure 4 is a flowchart of a method for selecting a weather texture image corresponding to the current weather executed by a device according to an embodiment of the present invention;

[0016] Figure 5 is a schematic diagram showing an example of a plurality of weather texture images corresponding to weather according to an embodiment of the present invention;

[0017] Figure 6 is a schematic diagram showing an example of object recognition information indicating an object recognized in an image according to an embodiment of the present invention;

[0018] Figure 7 is a schematic diagram showing an example of spatial information indicating a space analyzed in an image according to an embodiment of the present disclosure;

[0019] Figure 8 is a schematic diagram showing an example of weather information according to an embodiment of the present invention;

[0020] Figure 9 is a flowchart of a method for providing a weather effect in an image by a system based on criterion information for simulating a weather effect according to an embodiment of the present disclosure;

[0021] Figure 10 is a flowchart of a method for determining criteria for simulating a plurality of weather texture images executed by a server of a system according to an embodiment of the present invention;

[0022] Figure 11 is a flowchart of a method for determining criteria for simulating weather texture images for each depth range executed by a server according to an embodiment of the present disclosure;

[0023] Figure 12 It is a schematic diagram showing an example of the positions and intervals between image segments to be cut from a weather texture image according to an embodiment of the present invention;

[0024] Figure 13 It is a schematic diagram showing an example of the positions and intervals between image segments to be cut from a weather texture image according to an embodiment of the present invention;

[0025] Figure 14 It is a schematic diagram showing an example of cutting image segments from different positions of a weather texture image based on weather according to an embodiment of the present invention;

[0026] Figure 15 It is a schematic diagram showing an example of sequentially simulating image segments at a specific period according to an embodiment of the present invention;

[0027] Figure 16 It is a schematic diagram showing an example of simulating image segments in different periods based on a depth range according to an embodiment of the present invention;

[0028] Figure 17 It is a schematic diagram showing an example of cutting image segments in different shapes based on a depth range according to an embodiment of the present invention;

[0029] Figure 18 It is a schematic diagram showing an example of masking or adjusting the transparency of a part of an image segment based on a depth range according to an embodiment of the present disclosure;

[0030] Figure 19 It is a table showing an example of information provided to a device for simulating a weather effect;

[0031] Figure 20 It is an image of a graphical user interface (GUI) for setting criteria for simulating a weather effect according to an embodiment of the present disclosure;

[0032] Figure 21 It is a flowchart of a method for simulating a weather effect in an image by using a weather texture image, which is executed by a device according to an embodiment of the present disclosure;

[0033] Figure 22 It is a flowchart of a method for simulating a weather effect in an image by using a weather texture image received from a server, which is executed by a device of a system according to an embodiment of the present disclosure;

[0034] Figure 23 It is a flowchart of a method for simulating a weather effect in an image from which a weather object has been deleted, which is executed by a system according to an embodiment of the present disclosure;

[0035] Figure 24is a schematic diagram showing an example of an image from which a weather object has been removed according to an embodiment of the present invention;

[0036] Figure 25 is a flowchart of a method for simulating a weather effect in a reference image corresponding to the current time, performed by a system according to an embodiment of the present disclosure;

[0037] Figure 26 is a schematic diagram showing an example of a reference image corresponding to a preset time period according to an embodiment of the present invention;

[0038] Figure 27 is a flowchart of a method for changing the color of a reference image based on a color change path and simulating a weather effect in the color-changed reference image, performed by a system according to an embodiment of the present disclosure;

[0039] Figure 28 is a schematic diagram showing an example of a color change path between reference images according to an embodiment of the present invention;

[0040] Figure 29 is a schematic diagram showing an example of changing the color style of an image according to an embodiment of the present invention;

[0041] Figure 30 is a schematic diagram showing an example of an image reflecting a rain effect according to an embodiment of the present disclosure;

[0042] Figure 31 is a block diagram of a server according to an embodiment of the present invention;

[0043] Figure 32 is a block diagram of a device according to an embodiment of the present invention;

[0044] Figure 33 is a schematic diagram showing an example in which an external device sets criteria for simulating a weather effect, and the device receives information about the set criteria through an external database (DB) and provides a weather effect in an image according to an embodiment of the present disclosure;

[0045] Figure 34 is a schematic diagram showing an example in which an external device sets criteria for simulating a weather effect, and the device receives information about the set criteria through a server and provides a weather effect in an image according to an embodiment of the present disclosure; and

[0046] Figure 35 is a schematic diagram showing an example in which an external device sets criteria for simulating a weather effect through a server, and the device receives information about the set criteria through an external DB and provides a weather effect in an image according to an embodiment of the present disclosure. Detailed Description

[0047] According to an embodiment of the present invention, a method for a device to provide a weather effect in an image is provided, including obtaining an image to which a weather effect is to be applied; obtaining at least one weather texture image showing the weather; and providing a weather effect in the image based on the weather texture image by sequentially overlapping a plurality of image segments obtained from the obtained weather texture image on the image.

[0048] According to another embodiment of the present invention, a device for providing a weather effect in an image includes a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain an image to which a weather effect is to be applied, select at least one weather texture image showing the weather effect, and provide a weather effect in the image on the display based on the weather texture image by sequentially overlapping a plurality of image segments obtained from the weather texture image on the image.

[0049] According to another embodiment of the present invention, a computer-readable recording medium records a computer program for executing the above method.

[0050]

Invention Mode

[0051] Hereinafter, embodiments of the present disclosure will be described in detail by referring to the accompanying drawings. However, the present invention can be embodied in many different forms and should not be construed as limited to the embodiments of the present invention set forth herein. In the drawings, parts irrelevant to the present invention are not shown for clarity, and like reference numerals denote like elements.

[0052] It should be understood that when an element is referred to as "connected to" another element, it can be "directly connected to" the other element or "electrically connected to" the other element through an intermediate element. It will be further understood that when the terms "comprising" and / or "including" are used herein, the presence of the stated element is specified, but the presence or addition of one or more other elements is not excluded, unless the context clearly indicates otherwise.

[0053] Throughout the disclosure, "at least one of a, b, or c" means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variants thereof.

[0054] Reference will now be made in detail to embodiments of the present invention, which are illustrated in the accompanying drawings.

[0055] Figure 1 is a schematic diagram showing an example of a system for providing a weather effect in a specific image according to an embodiment of the present invention.

[0056] Reference Figure 1 , the system for providing a weather effect may include a device 1000 and a server 2000.

[0057] Device 1000 can select a specific image and display the image as including a three-dimensional (3D) weather effect. Device 1000 can select an image and receive information for simulating a weather effect in the selected image from server 2000. Device 1000 can provide the weather effect in the selected image by simulating a weather texture image as overlaid or otherwise included in the selected image. The weather texture image can be an image of an object indicating a specific weather and can include, for example, an image of raindrops, an image of snowflakes, or an image of fog, but the type of the weather effect is not limited thereto.

[0058] The weather effect in the image can be provided by cutting a plurality of image segments from the weather texture image and sequentially overlaying the plurality of image segments on the image. Device 1000 can analyze the depth of an object in the image and overlap an image segment reflecting the current weather on the image based on the analyzed depth. To provide the weather effect in the original picture or a picture converted from the original picture, Device 1000 can adjust the transparency of the image segment based on the depth and synthesize the transparency-adjusted image segment with the image.

[0059] Device 1000 can reflect the weather effect in a specific image in an ambient mode. The ambient mode can be an operation mode for providing only some functions of Device 1000 at low power. For example, in the ambient mode, most functions of Device 1000 can be deactivated, and only the input / output function and some preset functions of Device 1000 can be activated on the display. Optionally, Device 1000 can display the image in which the weather effect is reflected as a background image of Device 1000.

[0060] Device 1000 can be, for example, a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop computer, a media player, a micro server, a global positioning system (GPS) device, an e-book reader, a digital broadcast receiver, a navigation system, a kiosk, an MP3 player, a digital camera, a household appliance, or another mobile or non-mobile computing device, but is not limited thereto.

[0061] Figure 2 is a schematic diagram showing an example of reflecting a weather effect in an image according to an embodiment of the present invention.

[0062] Reference Figure 2, one or more artificial intelligence models can be used to analyze an image. For example, the image can be a two-dimensional (2D) or three-dimensional (3D) image. The image can be input into a first artificial intelligence model to detect and identify objects in the image, and thus object recognition information indicating the recognized objects in the image can be output from the first artificial intelligence model. The object recognition information can include information about the location and shape of the objects in the image. The image can be input into a second artificial intelligence model to analyze the space in the image, and thus spatial information about the depth of the space in the image can be output from the second artificial intelligence model. The second artificial intelligence model can be used to estimate the 3D space in a 2D image by analyzing the spatial depth in the 2D image. The image can be input into a third artificial intelligence model for removing weather objects in the image, and thus an image from which the weather objects are removed can be output from the third artificial intelligence model.

[0063] The first to third artificial intelligence models can be constructed considering the application field of the recognition model, the purpose of training, or the computing performance of the device. The first to third artificial intelligence models can be, for example, models based on artificial neural networks. The artificial neural network can include, for example, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network, but the type of the artificial neural network is not limited thereto.

[0064] A single artificial intelligence model for providing the functions of two or more of the above first to third artificial intelligence models can be used.

[0065] Device 1000 can obtain object recognition information, spatial information, and an image from which weather objects are removed, and obtain a weather texture image corresponding to the current weather. Device 1000 can cut a plurality of image segments from the weather texture image based on a preset criterion. Device 1000 can provide an image reflecting the weather effect related to the current weather by sequentially overlapping the plurality of image segments on the image from which the weather objects are removed.

[0066] Figure 3 is a flowchart of a method for providing a weather effect in an image executed by a system according to an embodiment of the present disclosure.

[0067] In operation S300, device 1000 can select an image. Device 1000 can display a graphical user interface (GUI) for selecting an image to be displayed on the screen in the environmental mode, and select an image based on a user input received through the displayed GUI. For example, device 1000 can select a picture stored in device 1000 or a picture taken in real time.

[0068] In operation S305, the device 1000 may send an image to the server 2000. The device 1000 may provide the image to the server 2000 and request the server 2000 to provide the data required to simulate weather effects in the image.

[0069] Although in operations S300 and S305, the device 1000 selects an image and sends it to the server 2000, the present disclosure is not limited thereto. The server 2000 may provide a list of images stored in the server 2000 to the device 1000, and the device 1000 may select a specific image from the image list.

[0070] In operation S310, the server 2000 may detect and identify the objects in the image by using a first artificial intelligence model. The server 2000 may obtain object recognition information indicating or identifying the objects in the image by inputting the image into the first artificial intelligence model for identifying the objects in the image. For example, the objects may include people, sky, buildings, and trees. The object recognition information is information indicating the objects identified in the image and may include, for example, information about the shape of the objects included in the image, information about the location of the objects, and the identification information of the objects, but the object recognition information is not limited thereto. The first artificial intelligence model may be a model pre-trained to identify the objects in the image and may be, for example, a model based on an artificial neural network.

[0071] In operation S315, the server 2000 may obtain spatial information about the depth of the objects in the image by using a second artificial intelligence model. The server 2000 may obtain spatial information about the depth of the space in the image by inputting the image into the second artificial intelligence model for analyzing the space in the image. The spatial information is information indicating the depth of the space in the image and may include, for example, information about the depth of the objects included in the image, information about the depth of the background in the image, and information about the depth relationship between the objects in the image, but the spatial information is not limited thereto. For example, when the object in the image is a tree and the background of the image is the sky, the spatial information may include information indicating the depth of the tree, information indicating the depth of the sky, and information indicating that the tree is placed at a closer reference position than the sky. The depth of the objects in the image may indicate whether each object is placed at a near position or a far position in the image. For example, when the depth range of the objects in the image is from 0 to 100, values from 0 to 40 may be set as a short distance, values from 40 to 70 may be set as a medium distance, and values from 70 to 100 may be set as a long distance. The second artificial intelligence model may be a model pre-trained to analyze the space in the image and may be, for example, a model based on an artificial neural network.

[0072] The first and second artificial intelligence models can be implemented by a single artificial intelligence model. In this case, an image can be input into the single artificial intelligence model to provide the functions of the first artificial intelligence model and the second artificial intelligence model, so that spatial information about the space in the image and object recognition information about the objects in the image can be output.

[0073] In operation S320, the server 2000 can send the object recognition information and the spatial information to the device 1000. The server 2000 can send the object recognition information and the spatial information obtained by analyzing the image selected by the device 1000 by using at least one artificial intelligence model to the device 1000.

[0074] In operation S330, the device 1000 can identify the current weather condition corresponding to the current position of the device 1000. The device 1000 can identify the current weather in the area where the device 1000 is located, or the current weather in an area different from the area where the device 1000 is located selected by the user. The device 1000 can receive weather information indicating the current weather from the server 2000 at a preset period or in real time according to the request of the device 1000. The weather information indicating the current weather can include, for example, information indicating clouds, snow, rain, fog, lightning, wind, precipitation, rainfall, fog density, cloud cover, wind direction, gust, wind speed, relative humidity, temperature, perceived temperature, air pressure, solar radiation, visibility, ultraviolet (UV) index, dew point, but the meteorological information is not limited thereto. The weather information can include, for example, information about weather forecasts, hourly weather forecasts, and weekly weather forecasts.

[0075] In operation S335, the device 1000 can obtain a weather texture image corresponding to the current weather. The device 1000 can pre-store a plurality of weather texture images related to various weather types and select at least one weather texture image suitable for the current weather from the plurality of pre-stored weather texture images. For example, the device 1000 can store in the memory a plurality of weather texture images indicating rainy weather, a plurality of weather texture images indicating snowy weather, and a plurality of weather texture images indicating foggy weather.

[0076] Considering the information about weather forecasts, hourly weather forecasts, and weekly weather forecasts, the device 1000 can pre-store weather texture images related to predicted weather. For example, the device 1000 can request the server 2000 to provide weather texture images related to a weekly weather forecast based on the weekly weather forecast and store the weather texture images received from the server 2000 in the memory. In this case, since the device 1000 only stores in the memory the weather texture images related to the weather predicted for a predetermined time, the memory of the device 1000 can be effectively managed.

[0077] Multiple weather texture images pre-registered according to weather types can respectively correspond to depth ranges. The multiple weather texture images pre-registered according to weather can be different from each other based on depth ranges. The depth range can be a value for defining the depth range of the space in the image. For example, when the depth range of the space in the image is from 0 to 100, the values from 0 to 40 can be set as the first depth range, the values from 40 to 70 can be set as the second depth range, and the values from 70 to 100 can be set as the third depth range. The details of weather expression may increase as the depth range increases. The resources for providing weather effects can be reduced inversely proportional to the number of depth ranges. The number of depth ranges can be, for example, from 2 to 10.

[0078] Device 1000 can select a weather texture image corresponding to the characteristics of the current weather from multiple weather texture images. Device 1000 can select a weather texture image based on the characteristics of the current weather and the depth of the space in the image. For example, device 1000 can select a weather texture image corresponding to a rainfall of 10 ml / h and a wind speed of 7 km / h from multiple weather texture images, and select a weather texture image corresponding to the first and second depth ranges of the space in the image from the selected weather texture image.

[0079] When a weather texture image suitable for the current weather is not stored in the memory, device 1000 can request server 2000 to provide a weather texture image suitable for the current weather, and receive the requested weather texture image from server 2000.

[0080] In operation S340, device 1000 can simulate a weather effect indicating the current weather in the image by using the obtained weather texture image. Device 1000 can select an image segment for replacement or modification from the selected weather texture image. The size of the weather texture image can be larger than the size of the image selected by device 1000, and device 1000 can select an image segment from the weather texture image to fit the size of the image selected by device 1000. Device 1000 can replace the image segment from the weather texture image by moving the cut position. The cut position can be determined differently according to weather characteristics. For example, when the rainfall is high, device 1000 can determine the cut position as the area of the rain texture image including a large number of raindrops, and when the rainfall is low, determine the cut position as the area of the rain texture image including a small number of raindrops. The degree of movement can be determined differently based on the depth to which the weather texture image will be applied. For example, since raindrops in the near space move fast, device 1000 can cut an image segment from the rain texture image by moving the cut position at large intervals to display raindrops in the near space in the image. For example, since raindrops in the far space move slowly, device 1000 can cut an image segment from the rain texture image by moving the cut position at small intervals to display raindrops in the far space in the image.

[0081] Device 1000 can provide weather effects in an image by successively overlapping, interlacing, or otherwise combining image segments on the image at a preset period. For example, device 1000 can overlap an image segment obtained from a weather texture image corresponding to a first depth range and an image segment obtained from a weather texture image corresponding to a second depth range together on the image. The period of successively reproducing the image segments obtained from the weather texture image corresponding to the first depth range can be different from the period of successively reproducing the image segments obtained from the weather texture image corresponding to the second depth range.

[0082] Device 1000 can simulate weather effects in an image, such as snow, rain, or fog effects, by using frames of a weather texture image and combining image segments cut from the weather texture image on the image.

[0083] Device 1000 can adjust the transparency of image segments differently based on depth and simulate weather effects by using the image segments with adjusted transparency. For example, when the spatial depth in the image is between 0 and 100, device 1000 can adjust the transparency of the image segments corresponding to a depth between 0 and 40 to 30%, the transparency of the image segments corresponding to a depth between 40 and 70 to 50%, and the transparency of the image segments corresponding to a depth from 70 to 100 to 70%.

[0084] Figure 4 is a flowchart of a method of selecting a weather texture image corresponding to the current weather, performed by device 1000 according to an embodiment of the present disclosure.

[0085] In operation S400, device 1000 can obtain a plurality of weather texture images indicating a first weather condition and corresponding to a plurality of depth ranges. The plurality of weather texture images indicating the first weather condition can respectively correspond to the plurality of depth ranges. For example, device 1000 can obtain a weather texture image corresponding to the first weather condition and the first depth range, a weather texture image corresponding to the first weather condition and the second depth range, and a weather texture image corresponding to the first weather condition and the third depth range from a memory. The first weather condition can correspond to at least one weather characteristic and its intensity or value. Weather characteristics can include, for example, information indicating clouds, snow, rain, fog, lightning, wind, precipitation, rainfall, fog density, cloud cover, wind direction, gust, wind speed, relative humidity, temperature, perceived temperature, atmospheric pressure, solar radiation, visibility, UV index, and dew point, but the weather characteristics are not limited thereto.

[0086] In operation S410, device 1000 may obtain a plurality of weather texture images indicating a second weather condition and corresponding to a plurality of depth ranges. The plurality of weather texture images indicating the second weather condition may respectively correspond to the plurality of depth ranges. For example, device 1000 may obtain from the memory a weather texture image corresponding to the second weather condition and a first depth range, a weather texture image corresponding to the second weather condition and a second depth range, and a weather texture image corresponding to the second weather condition and a third depth range. The second weather condition may correspond to at least one weather characteristic and its intensity or value. Weather characteristics may include, for example, information indicating clouds, snow, rain, fog, lightning, wind, precipitation, rainfall, fog density, cloud cover, wind direction, gust, wind speed, relative humidity, temperature, felt temperature, atmospheric pressure, solar radiation, visibility, UV index, and dew point, but the weather characteristics are not limited thereto.

[0087] In operation S420, device 1000 may select a weather texture image corresponding to the current weather. The current weather may be the weather at a location corresponding to the location of device 1000, or the weather at a location different from the current location of device 1000 selected by the user of device 1000. Device 1000 may select from the plurality of weather texture images a weather texture image corresponding to the characteristics of the current weather. Device 1000 may select the weather texture image based on the characteristics of the current weather and the depth of the space in the image. For example, device 1000 may select from the plurality of weather texture images a weather texture image corresponding to a rainfall of 10 ml / h and a wind speed of 7 km / h, and select from the selected weather texture image the weather texture image corresponding to the first and second depth ranges of the space in the image.

[0088] When a weather texture image suitable for the current weather is not stored in the memory, device 1000 may request server 2000 to provide a weather texture image suitable for the current weather, and receive the requested weather texture image from server 2000.

[0089] Although in Figure 4 device 1000 obtains a plurality of weather texture images indicating a first weather condition and a plurality of weather texture images indicating a second weather condition, the device 1000 that obtains the weather texture images is not limited thereto. Device 1000 may obtain weather texture images indicating various preset weather types. Device 1000 may receive in advance from server 2000 weather texture images indicating various preset weather types, and thus effectively use weather texture images suitable for the current weather even when the current weather changes.

[0090] Figure 5 is a schematic diagram showing an example of a plurality of weather texture images corresponding to weather according to an embodiment of the present invention.

[0091] ReferenceFigure 5 Among them, multiple weather texture images indicating snowy weather may include a first snow texture image 50, a second snow texture image 51, and a third snow texture image 52. The first snow texture image 50 may correspond to "snow" and a first depth range, the second snow texture image 51 may correspond to "snow" and a second depth range, and the third snow texture image 52 may correspond to "snow" and a third depth range.

[0092] The first depth range may be shallower than the second depth range, and the second depth range may be shallower than the third depth range. For example, when the depth range of a space or an object in an image is from 0 to 100, the value from 0 to 40 may be set as the first depth range, the value from 40 to 70 may be set as the second depth range, and the value from 70 to 100 may be set as the third depth range. The first snow texture image 50 corresponds to a depth shallower than that of the second and third snow texture images 51 and 52, so the size of the snowflakes included in the first snow texture image 50 may be larger than the size of the snowflakes included in the second and third snow texture images 51 and 52.

[0093] The second snow texture image 51 corresponds to a depth deeper than that of the first snow texture image 50 and shallower than that of the third snow texture image 52. Therefore, the snowflakes included in the second snow texture image 51 may appear smaller in size than the snowflakes included in the first snow texture image 50 and may be larger in size than the snowflakes included in the third snow texture image 52.

[0094] Optionally, for example, multiple weather texture images indicating rainy weather may include a first rain texture image 55, a second rain texture image 56, and a third rain texture image 57. The first rain texture image 55 may correspond to "rain" and a first depth range, the second rain texture image 56 may correspond to "rain" and a second depth range, and the third rain texture image 57 may correspond to "rain" and a third depth range.

[0095] The device 1000 may store the first to third snow texture images 50 to 52, the first to third rain texture images 55 to 57, etc. related to various weather characteristics and various depth ranges in the memory.

[0096] Although the weather texture image corresponds to Figure 5 one of the weather characteristics, the weather texture image is not limited thereto. The weather texture image may correspond to multiple weather characteristics. For example, the weather texture image may correspond to rain, rainfall, wind speed, and wind direction. In this case, the size, density, direction, etc. of the objects (such as raindrops) included in the weather texture image may vary based on the weather characteristics corresponding to the weather texture image.

[0097] The weather texture image can be an image in which weather objects are displayed on a transparent layer. In this way, when the weather texture image is overlaid on an image, only the weather objects can be displayed on the image.

[0098] Figure 6 It is a schematic diagram showing an example of object recognition information 62 indicating an object recognized in image 60 according to an embodiment of the present disclosure.

[0099] Reference Figure 6 , the server 2000 can identify the objects in the image 60 and obtain object recognition information 62 regarding the shape and position of the objects by inputting the image 60 into a first artificial intelligence model. Although the object recognition information 62 has Figure 6 the form of an image in, the object recognition information 62 is not limited thereto and may include various format data capable of recognizing the position, shape, etc. of the objects.

[0100] The object recognition information 62 is information indicating the objects recognized in the image 60, and may include, for example, information about the shape of the objects included in the image 60, information about the position of the objects, and the recognition information of the objects. However, the object recognition information is not limited thereto. The first artificial intelligence model can be a model pre-trained to detect and recognize the objects in the image 60, and can be, for example, a model based on an artificial neural network. The first artificial intelligence model can be, for example, an artificial intelligence model for semantic image segmentation. The first artificial intelligence model can detect and recognize the objects in the image 60 and the positions of the objects by estimating the categories of the pixels in the image 60.

[0101] Figure 7 It is a schematic diagram showing an example of space information 72 indicating the space analyzed in image 70 according to an embodiment of the present disclosure.

[0102] Reference Figure 7 , the server 2000 can analyze the space or region in the image 70 and obtain space information 72 indicating the depth of the space in the image 70 by inputting the image 70 into a second artificial intelligence model. Although the space information 72 has Figure 7 the form of an image in, the space information 72 is not limited thereto and may include various format data capable of recognizing the depth of the space in the image 70. The space information 72 is information indicating the depth of the space in the image 70, and may include, for example, information about the depth of the objects included in the image 70, information about the depth of the background in the image 70, and information about the depth relationship between the objects in the image 70, but is not limited thereto. The second artificial intelligence model can be a model pre-trained to analyze the space depth in the image 70, and can be, for example, a model based on an artificial neural network. The second artificial intelligence model can be, for example, an artificial intelligence model for depth prediction / estimation.

[0103] Figure 8 It is a schematic diagram showing an example of weather information according to an embodiment of the present invention.

[0104] Reference Figure 8 , the weather information may include, for example, information indicating location, time, clouds, snow, rain, relative humidity, temperature, perceived temperature, weather, atmospheric pressure, solar radiation, visibility, wind direction, gust, wind speed, UV index, and dew point.

[0105] The device 1000 may simulate a 3D image effect in the image by using a weather texture image indicating weather (such as snow, rain, sunlight, clouds, fog, lightning, or wind). The device 1000 may reflect a 3D image effect related to one or more of visibility, wind speed, and temperature in the original image. The device 1000 may reflect a 3D image effect in the image, for example, considering weather parameters such as wind intensity, rainfall, snowfall, fog density, air resistance, distance, and direction. Therefore, the intensity of the weather effect may be correspondingly reflected in the image according to the intensity of the weather effect at a specific location selected by the user or the location of the device 1000.

[0106] Figure 9 It is a flowchart of a method for providing a weather effect in an image by a system based on criterion information for simulating a weather effect according to an embodiment of the present disclosure.

[0107] Operations S900 to S915 correspond to Figure 3 operations S300 to S315 of , so redundant descriptions thereof are omitted.

[0108] In operation S920, the server 2000 may obtain criterion information for simulating a weather effect. The server 2000 may determine criteria for simulating a weather texture image. For example, the server 2000 may determine criteria for the weather texture image to be used based on the weather, criteria for a part of the weather texture image from which an image segment is to be replaced or modified, criteria for the interval between image segments, and criteria for the time for displaying an image segment based on a depth range. The server 2000 may obtain criterion information regarding the determined criteria. The criterion information for simulating a weather texture image may include, for example, information about the identifier of the weather texture image to be used based on the weather, criteria for replacing or modifying an image segment based on the weather texture image, the interval between image segments, and the time for displaying an image segment. The criterion information may be parameter type data for downloading data on specific criteria for simulating a weather texture image. The criteria and criterion information for simulating a weather texture image will be described in detail below.

[0109] In operation S925, the server 2000 may provide the criterion information, object identification information, and spatial information regarding the determined criterion to the device 1000. The criterion information, object identification information, and spatial information may be provided to the device 1000 in the form of parameter values.

[0110] In operation S930, the device 1000 may identify the current weather based on the location of the device 1000 or a location selected by the user, and in operation S935, obtain a weather texture image corresponding to the current weather. The device 1000 may obtain a weather texture image corresponding to the current weather based on the criterion information received from the server 2000. For example, the device 1000 may check the identifier of the weather texture image corresponding to the current weather according to the criterion information, and receive the weather texture image from the server 2000 based on the identifier of the weather texture image. Optionally, the device 1000 may extract the weather texture image from the memory based on the identifier of the weather texture image. The device 1000 may select the weather texture image based on the current weather and the depth of the object and space in the image.

[0111] In operation S940, the device 1000 may simulate a weather effect in the image based on the criterion information. The device 1000 may select a plurality of image segments from the weather texture image based on the criterion information, according to the current weather and the depth of the object and space in the image, and sequentially overlap the plurality of image segments on the image at a specific period. For example, the device 1000 may overlap the plurality of image segments on the image by using alpha blending. In this case, the device 1000 may apply a specific transparency to the image segment based on the depth range corresponding to the image segment, and overlap the image segment to which the specific transparency is applied on the image. The device 1000 may mask a part of the image segment corresponding to the long-distance area based on a specific criterion.

[0112] Figure 10 is a flowchart of a method for determining a criterion for simulating a plurality of weather texture images executed by the server 2000 of the system according to an embodiment of the present disclosure.

[0113] In operation S1000, the server 2000 may select a weather texture image corresponding to a specific depth range. The server 2000 may select a weather texture image corresponding to a specific weather and a specific depth range from the weather texture images stored in the database (DB).

[0114] In operation S1010, the server 2000 may determine the positions of image segments in the weather texture image to be used for simulating weather effects. The server 2000 may determine criteria for the portion of the weather texture image from which the image segments are obtained, as well as the intervals between the image segments, based on weather characteristics. For example, when the rainfall is large, the server 2000 may set the positions of the image segments in such a way that the image segments are cropped at large intervals from the portion of the rain texture image with dense raindrops.

[0115] In operation S1020, the server 2000 may determine the periods for simulating the image segments. The server 2000 may determine the periods for simulating the image segments based on the depth range. For example, the server 2000 may set a short simulation period for the image segments cropped from the weather texture image corresponding to the shallow depth range, and a long simulation period for the image segments cropped from the weather texture image corresponding to the deep depth range. Thus, the weather effects to be displayed at different depths in the image can be reproduced independently.

[0116] Figure 11 It is a flowchart of a method for determining criteria for simulating a weather texture image according to the depth range, performed by the server 2000 according to an embodiment of the present disclosure.

[0117] In operation S1100, the server 2000 may select a weather texture image corresponding to a first depth range. The first depth range may be a range in which the depth is shallower than the depth of a specific object in the image, and the weather texture image corresponding to the first depth range may include weather objects having a size greater than a preset value. The weather object may be an object indicating a specific weather, such as raindrops or snowflakes. For example, when the depth range of the space in the image is from 0 to 100 and the depth of the nearest object in the image is 40, the server 2000 may determine the depth range from 0 to 40 as the first depth range.

[0118] In operation S1110, the server 2000 may determine criteria for simulating a weather texture image corresponding to a first depth range. The server 2000 may determine criteria for the position of the image segments to be obtained from the weather texture image, the intervals between the image segments, and the simulation period of the image segments based on the characteristics of the weather and the depth range of the weather texture image. For example, when the weather texture image corresponding to the first depth range is a rain texture image corresponding to rainy weather, the server 2000 may set criteria for the portion of the rain texture image from which the image segments are to be obtained based on at least one of the precipitation amount, wind speed, or wind direction. The server 2000 may set the intervals between the image segments to be obtained based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set the simulation period for sequentially simulating the image segments based on the depth range of the weather texture image. For example, the first depth range may be a depth range shallower than the second depth range described below, and the image segments obtained from the weather texture image corresponding to the first depth range may be simulated in the image at a shorter period than the image segments cropped from the weather texture image corresponding to the second depth range described below.

[0119] In operation S1120, the server 2000 may select a weather texture image corresponding to a second depth range. The second depth range may be a range deeper than the depth of a specific object in the image, and the weather texture image corresponding to the second depth range may include weather objects having dimensions smaller than a preset value. For example, when the depth range of the space in the image is from 0 to 100 and the depth of the nearest object in the image is 40, the server 2000 may determine the depth range from 40 to 100 as the second depth range.

[0120] In operation S1130, the server 2000 may determine criteria for simulating the weather texture image corresponding to the second depth range. The second depth range may be a depth range deeper than the depth of a specific object in the image, and the image segments obtained from the weather texture image corresponding to the second depth range may be simulated as if they are shown behind the specific object. In this way, the server 2000 may determine the shape of the image segments to be cropped from the weather texture image corresponding to the second depth range so that the image segments do not overlap with the specific object in the image.

[0121] Optionally, the server 2000 may control the transparency of the cropped image segments or mask portions of the cropped image segments in such a way that the cropped image segments do not overlap with the specific object in the image. In this case, the server 2000 may determine that the area of the cropped image segments is transparent or masked based on the area occupied by an object closer than the cropped image segments. The transparency level may be controlled.

[0122] For example, when the weather texture image corresponding to the second depth range is a rain texture image corresponding to rainy weather, the server 2000 may set criteria for a part of the rain texture image from which an image segment is to be obtained, based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set an interval between the image segments to be obtained, based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set a simulation period for sequentially simulating the image segments, based on the depth range of the weather texture image. For example, the second depth range may be a deeper depth range than the first depth range, and the image segments obtained from the weather texture image corresponding to the second depth range may be simulated in the image at a longer period than the image segments obtained from the weather texture image corresponding to the first depth range.

[0123] Figure 12 is a schematic diagram showing an example of the positions and intervals between image segments to be cut from a weather texture image according to an embodiment of the present invention.

[0124] Figure 13 is a schematic diagram showing an example of the positions and intervals between image segments to be cut from a weather texture image according to an embodiment of the present invention.

[0125] Reference Figure 12 , the image segments 111, 112, and 113 may be cut from the rain texture image 110. Referring to Figure 13 , the image segments 114, 115, and 116 may be cut from the rain texture image 110. The cutting direction of the image segments to be cut from the rain texture image 110 and the interval between the image segments may be adjusted based on precipitation, wind direction, and wind speed. For example, when the precipitation, wind direction, and wind speed are low, the image segments 111, 112, and 113 may be selected at small intervals in a direction close to the vertical direction of the rain texture image 110. Otherwise, when the precipitation and wind speed are high, the image segments 114, 115, and 116 may be selected at large intervals in the diagonal direction of the rain texture image 110.

[0126] Figure 14 is a schematic diagram showing an example of selecting image segments from different positions of a weather texture image based on weather according to an embodiment of the present invention.

[0127] Reference Figure 14 , raindrops having different shapes may be placed at different densities in the rain texture image 130. For example, raindrops may be placed vertically on the left side of the rain texture image 130, and raindrops may be placed diagonally on the right side of the rain texture image 130. For example, raindrops may be placed at a low density at the top of the rain texture image 130, and raindrops may be placed at a high density at the bottom of the rain texture image 130.

[0128] Therefore, when the rainfall and wind speed are low, selection criteria can be set to select image segments 131, 132, and 133 from the upper left part of the rain texture image 130. Otherwise, when the rainfall and wind speed are high, selection criteria can be set to select image segments 134, 135, and 136 from the lower right part of the rain texture image 130.

[0129] Figure 15 is a schematic diagram showing an example of sequentially simulating image segments at a specific period according to an embodiment of the present invention.

[0130] Reference Figure 15 , image segments 141, 142, and 143 can be reproduced sequentially and repeatedly on the image 140. A period for covering image segments 141, 142, and 143 can be preset based on the depth range of image segments 141, 142, and 143. For example, image segment 141 can be covered on the image 140 from 0 seconds to 0.1 seconds, and then image segment 142 can be covered on the image 140 from 0.1 seconds to 0.2 seconds. Image segment 143 can be covered on the image 140 from 0.2 seconds to 0.3 seconds, and then image segment 141 can be covered on the image 140 from 0.3 seconds to 0.4 seconds. The transparency of covering image segments 141, 142, and 143 can be controlled, for example, based on the depth of an object in the image.

[0131] A snow effect or other weather effect can be provided in the image 140 by sequentially and repeatedly superimposing image segments 141, 142, and 143 on the image 140 at a specific period.

[0132] Figure 16 is a schematic diagram showing an example of simulating image segments at different periods based on the depth range according to an embodiment of the present invention.

[0133] Reference Figure 16 , one of the first image segments 1-1, 1-2, and 1-3 obtained from the weather texture image of the first depth range, one of the second image segments 2-1, 2-2, and 2-3 obtained from the weather texture image of the second depth range, and one of the third image segments 3-1, 3-2, and 3-3 obtained from the weather texture image of the third depth range can be merged on the image 150.

[0134] The first depth range can be a depth range shallower than the second depth range, and the second depth range can be a depth range shallower than the third depth range.

[0135] The first image segments 1-1, 1-2, and 1-3 obtained from the weather texture image of the first depth range can be sequentially overlapped on the image 150 at a period of 0.1 second. The second image segments 2-1, 2-2, and 2-3 obtained from the weather texture image of the second depth range can be sequentially overlapped on the image 150 at a period of 0.2 second. The third image segments 3-1, 3-2, and 3-3 obtained from the weather texture image of the third depth range can be sequentially overlapped on the image 150 at a period of 0.3 second.

[0136] In this case, based on the position and depth of the object in the image 150, at least part of the first image segments 1-1, 1-2, and 1-3, the second image segments 2-1, 2-2, and 2-3, and the third image segments 3-1, 3-2, and 3-3 can be transparent or occluded.

[0137] The first image segments 1-1, 1-2, and 1-3, the second image segments 2-1, 2-2, and 2-3, and the third image segments 3-1, 3-2, and 3-3 can be images on which weather objects of different sizes are displayed. The first image segments 1-1, 1-2, and 1-3, the second image segments 2-1, 2-2, and 2-3, and the third image segments 3-1, 3-2, and 3-3 can be overlapped at different periods, so a realistic 3D weather effect can be provided in the image 150.

[0138] Figure 17 is a schematic diagram showing an example of shearing image segments in different shapes based on depth ranges according to an embodiment of the present invention.

[0139] Reference Figure 17 , the first depth range can be a depth range shallower than the depths of the first object 161 and the second object 162 in the image 160. For example, when the spatial depth range in the image 160 is from 0 to 100, the first depth range can be a depth range from 0 to 40. An image segment 164 having the same size as the image 160 can be sheared from the weather texture image 163 corresponding to the first depth range.

[0140] The second depth range can be a depth range deeper than the depth of the first object 161 in the image 160 and shallower than the depth of the second object 162. For example, when the spatial depth range in the image 160 is from 0 to 100, the second depth range can be a depth range from 40 to 70. An image segment 166 having a shape that does not overlap with the first object 161 can be sheared from the weather texture image 165 corresponding to the second depth range.

[0141] The third depth range may be a depth range deeper than the depths of the first object 161 and the second object 162 in the image 160. For example, when the spatial depth range in the image 160 is from 0 to 100, the third depth range may be a depth range from 70 to 100. An image segment 168 having a shape that does not overlap with the first object 161 and the second object 162 may be cut out from the weather texture image 167 corresponding to the third depth range.

[0142] Figure 18 is a schematic diagram showing an example of masking or adjusting the transparency of a part of an image segment based on a depth range according to an embodiment of the present invention.

[0143] Reference Figure 18 , the image segments 170, 174, and 178 may be selected in a rectangular shape. The shapes of the image segments 170, 174, and 178 are not limited and may be any shape including a square, a circle, and an irregular shape. The shape of the image segment may correspond to the shape of the object in the image to which the image segment is to be overlaid.

[0144] In this case, since the image segment 170 corresponds to the third depth range, the region 172 of the image segment 170 that overlaps with an object shallower than the third depth range may be masked. Optionally, the region 172 of the image segment 170 may be made transparent.

[0145] Since the image segment 174 corresponds to the second depth range, the region 176 of the image segment 174 that overlaps with an object shallower than the second depth range may be masked. Optionally, the region 176 of the image segment 174 may be made transparent.

[0146] Figure 19 is a table showing an example of information provided to the device 1000 to simulate a weather effect according to an embodiment of the present disclosure.

[0147] Reference Figure 19 , object recognition information, spatial information, criterion information, etc. may be provided to the device 1000.

[0148] Request parameters are examples of criterion information required to simulate a weather effect and may include an identifier of the weather effect (e.g., effect_id), an identifier of the image (e.g., image_num), and information indicating the criterion for simulating the weather effect (e.g., effect-information). The request parameters may be fixed values and may be included in a file of the JavaScript Object Notation (Json) type.

[0149] The image may include an image to which a weather effect is to be applied. The image to which a weather effect is to be applied may include an original image, an image based on time conversion, and an image based on season conversion, but the image is not limited thereto.

[0150] A depth map is an example of spatial information indicating the depth of space in an image and can be a map file generated by performing depth prediction using a second artificial intelligence model. A depth map can be generated by recognizing 3D space from a 2D image and representing the distance values of pixels in the image in the form of a map.

[0151] A segmentation map is an example of object recognition information indicating objects recognized in an image and can be a map file generated by performing semantic image segmentation using a first artificial intelligence model.

[0152] The texture can indicate a weather texture image for providing a weather effect.

[0153] The server 2000 can provide request parameters, a depth map, and a segmentation map to the device 1000, and may not provide the image to which the weather effect is to be applied and the weather texture image to the device 1000. In this case, the server 2000 can provide the device 1000 with link information for downloading the image to which the weather effect is to be applied, and link information for downloading the weather texture image. The server 2000 can send a compressed file generated by compressing Figure 19 at least a part of the data to the device 1000.

[0154] Figure 20 is an image of the GUI 20 for setting criteria for simulating a weather effect according to an embodiment of the present disclosure.

[0155] Through Figure 20 the GUI 20, a user of the device 1000 or an operator of the server 2000 can set criteria for simulating a weather effect. Figure 20 the GUI 20 can be provided to the device 1000 through a web-based service. Referring to Figure 20 , the GUI 20 for setting criteria for simulating a weather effect may include an area 22 for selecting an image to which the weather effect is to be applied, an area 23 for displaying a preview image to which the weather effect is applied, an area 24 for selecting a weather type, and an area 25 for setting parameters of the weather effect.

[0156] The list of images stored in the device 1000 and the list of images stored in the server 2000 can be displayed in the area 22 for selecting an image to which the weather effect is to be applied.

[0157] When a list of images stored in server 2000 is displayed in area 22 for selecting an image to which a weather effect is to be applied, server 2000 may recommend images related to the current weather. Server 2000 may classify the images stored in server 2000 based on the weather. Server 2000 may receive weather information related to the current weather from a weather service provider server and recommend images corresponding to the current weather to device 1000 based on the received weather information.

[0158] Considering the time and area where device 1000 is located, server 2000 may recommend images corresponding to the current weather to device 1000. In this case, server 2000 may classify the images based on the location and time.

[0159] When a list of images stored in device 1000 is displayed in area 22 for selecting an image to which a weather effect is to be applied, device 1000 may recommend images related to the current weather. Device 1000 may classify the images stored in device 1000 based on the weather. Device 1000 may receive weather information related to the current weather from a weather service provider server and recommend images corresponding to the current weather to the user based on the received weather information.

[0160] GUI 20 may be provided to device 1000 through a specific application programming interface (API).

[0161] Identifiers of weather effects applicable to an image may be displayed in area 24 for selecting a weather type. For example, identifiers indicating rain, snow, and fog effects may be displayed in area 24.

[0162] Values indicating characteristics of the weather effect may be displayed in area 25 for setting parameters of the weather effect. For example, values for setting the speed, amount, and angle of a weather object may be displayed in area 25.

[0163] A preview image reflecting the weather effect may be displayed in area 23. A preview image showing a simulation result of the weather effect selected in area 24 in the image based on the characteristics selected in area 25 may be displayed in area 23 in real time.

[0164] Figure 21 is a flowchart of a method of simulating a weather effect in an image by using a weather texture image, which is executed by device 1000 according to an embodiment of the present disclosure.

[0165] In operation S2100, device 1000 may obtain a plurality of image segments from a weather texture image corresponding to a first depth range. Device 1000 may obtain weather information indicating the current weather and identify the characteristics of the current weather. Device 1000 may identify the positions of the image segments and the intervals between the image segments according to the characteristics of the current weather based on the criterion information for simulating weather effects. Device 1000 may obtain a plurality of image segments from the weather texture image based on the identified positions and intervals.

[0166] In operation S2110, device 1000 may identify the simulation period. Device 1000 may identify the simulation period according to the characteristics of the current weather and the first depth range based on the criterion information for simulating weather effects.

[0167] In operation S2120, device 1000 may obtain a plurality of image segments from a weather texture image corresponding to a second depth range. Device 1000 may identify the positions of the image segments and the intervals between the image segments according to the characteristics of the current weather based on the criterion information for simulating weather effects. Device 1000 may select a plurality of image segments from the weather texture image based on the identified shear positions and intervals. When an object exists in the image at a depth shallower than the second depth range, device 1000 may control the image segments not to overlap with the object in the image.

[0168] In operation S2130, device 1000 may identify the simulation period. Device 1000 may identify the simulation period according to the characteristics of the current weather and the second depth range based on the criterion information for simulating weather effects.

[0169] In operation S2140, device 1000 may simulate the image segments corresponding to the first depth range and the image segments corresponding to the second depth range in the image. Device 1000 may overlap one of the plurality of image segments corresponding to the first depth range and one of the plurality of image segments corresponding to the second depth range together on the image.

[0170] Figure 22 is a flowchart of a method for simulating a weather effect in an image by device 1000 of a system according to an embodiment of the present disclosure by using a weather texture image received from server 2000.

[0171] Figure 22 Operations S2200 to S2215 of Figure 3 correspond to operations S300 to S315 of , so redundant descriptions thereof are omitted.

[0172] In operation S2220, the server 2000 may obtain a plurality of weather texture images. The server 2000 may obtain the plurality of weather texture images pre-registered according to weather conditions from a database. The server 2000 may obtain, for example, a plurality of weather texture images indicating rainy weather, a plurality of weather texture images indicating snowy weather, and a plurality of weather texture images indicating foggy weather. The plurality of weather texture images pre-registered according to weather may each correspond to a depth range. The plurality of weather texture images pre-registered according to weather may be different from each other based on the depth range. The depth range may be a value for defining the depth range of space in an image.

[0173] In operation S2220, the server 2000 may obtain criterion information for simulating weather effects. For example, the server 2000 may determine criteria for the weather texture images to be used based on weather, criteria for a part of the weather texture image from which an image segment is to be cut, criteria for the interval between image segments, and criteria for the time for displaying the image segments based on the depth range. In operation S2230, the server 2000 may provide the device 1000 with the criterion information, object recognition information, space information, and the plurality of weather texture images regarding the determined criteria. The device 1000 may pre-store the plurality of weather texture images related to various weather types and select and use at least one weather texture image suitable for the current weather and the selected image from the plurality of pre-stored weather texture images.

[0174] In operation S2240, the device 1000 may simulate a weather effect in an image based on the criterion information. The device 1000 may select a weather texture image based on the current weather and the depth of the object and space in the image. The device 1000 may obtain a plurality of image segments from the selected weather texture image and sequentially merge the plurality of image segments on the image at a specific period.

[0175] Figure 23 It is a flowchart of a method for simulating a weather effect in an image from which a weather object has been removed, executed by a system according to an embodiment of the present disclosure.

[0176] Operations S2300 to S2315 and S2325 correspond to Figure 22 Operations S2200 to S2215 and S2225, and thus redundant descriptions thereof are omitted.

[0177] In operation S2320, the server 2000 may remove a weather object from an image by using a third artificial intelligence model. The server 2000 may obtain an image from which the weather object has been removed by inputting the image into the third artificial intelligence model for removing the weather object from the image. The third artificial intelligence model may be a model pre-trained to remove weather objects from images and may be, for example, a model based on an artificial neural network.

[0178] In operation S2330, the server 2000 may provide the device 1000 with an image from which a weather object is deleted, object recognition information, spatial information, and a plurality of weather texture images.

[0179] In operation S2340, the device 1000 may simulate a weather effect in the image from which the weather object is deleted. The device 1000 may select a weather texture image based on the current weather and the depth of the objects and the space in the image. The device 1000 may obtain a plurality of image segments from the selected weather texture image and sequentially overlap the plurality of image segments on the image from which the weather object is deleted at a specific period.

[0180] Figure 24 is a schematic diagram showing an example of an image from which a weather object is deleted according to an embodiment of the present disclosure.

[0181] Reference Figure 24 , the picture 190 taken on a rainy day includes raindrops. When the picture 190 including raindrops is input to the third artificial intelligence model, a picture 192 from which raindrops are deleted may be output from the third artificial intelligence model. A picture 191 including empty raindrops may be generated by deleting raindrops from the picture 190 including raindrops, and a picture 192 from which raindrops are deleted may be generated by completely omitting raindrops from the image.

[0182] Figure 25 is a flowchart of a method for simulating a weather effect in a reference image corresponding to the current time executed by a system according to an embodiment of the present disclosure.

[0183] Operations S2500 to S2520 correspond to Figure 9 operations S900 to S920 of , and thus redundant descriptions thereof are omitted.

[0184] In operation S2525, the server 2000 may generate a plurality of reference images corresponding to a preset time period. For example, the server 2000 may generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night by changing the color of the image or other objects in the image or features indicating a certain time of day (such as brightness, illumination), etc.

[0185] In operation S2530, the server 2000 may provide the device 1000 with a plurality of reference images, object recognition information, spatial information, and criterion information.

[0186] In operation S2535, the device 1000 may identify the current time.

[0187] In operation S2540, device 1000 may select a reference image corresponding to the current time. Device 1000 may select a reference image corresponding to the current time from the reference images received from server 2000. For example, when the current time is 13:00, device 1000 may select a reference image corresponding to the afternoon.

[0188] In operation S2545, device 1000 may simulate a weather effect in the selected reference image.

[0189] Figure 26 FIG. is a schematic diagram showing an example of a reference image corresponding to a preset time period according to an embodiment of the present invention.

[0190] Reference Figure 26 , it is possible to use an image to generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night.

[0191] Figure 27 FIG. is a flowchart of a method for changing the color of a reference image based on a color change path and simulating a weather effect in the color-changed reference image, which is executed by a system according to an embodiment of the present disclosure.

[0192] Operations S2700 to S2720 correspond to Figure 9 operations S900 to S920 of, and thus redundant descriptions thereof are omitted.

[0193] In operation S2725, server 2000 may generate a plurality of reference images corresponding to a preset time period. For example, server 2000 may generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night by changing the color of an image.

[0194] In operation S2730, server 2000 may generate color change path information indicating a color change path between a plurality of reference images. Server 2000 may change the first reference image to the second reference image in such a way that the color of the first reference image smoothly changes to the color of the second reference image. To this end, server 2000 may generate color change path information by determining a path for changing the color of a specific region of the first reference image to the color of a specific region of the second reference image. Server 2000 may obtain color change path information indicating a color change path between reference images by inputting a plurality of reference images into a fourth artificial intelligence model. The fourth artificial intelligence model may be a model that is pre-trained to naturally change colors between reference images in consideration of the characteristics of the reference images, and may be, for example, a model based on an artificial neural network.

[0195] In operation S2735, the server 2000 may provide the device 1000 with multiple reference images, color change path information, object recognition information, spatial information, and criterion information.

[0196] In operation S2740, the device 1000 may identify the current time.

[0197] In operation S2745, the device 1000 may select a reference image corresponding to the current time. The device 1000 may select a reference image corresponding to the current time from the reference images received from the server 2000. The device 1000 may also select a reference image after the selected reference image. For example, when the current time is 13:00, the device 1000 may select a reference image corresponding to the afternoon and a reference image corresponding to the evening.

[0198] In operation S2750, the device 1000 may change the color of the selected reference image based on the color change path information. For example, the server 2000 may gradually change the color of the reference image corresponding to the afternoon based on the color change path information between the reference image corresponding to the afternoon and the reference image corresponding to the evening.

[0199] In operation S2755, the device 1000 may simulate weather effects in the reference image with the color changed.

[0200] Figure 28 is a schematic diagram showing an example of a color change path between reference images according to an embodiment of the present invention.

[0201] Reference Figure 28 , color change path information on how the color of the afternoon reference image 230 needs to change from the color of the afternoon reference image 230 to the color of the evening reference image 231 can be generated. A color change path on how the color needs to change between respective regions of the reference image can be determined.

[0202] For example, a path 238 on how the color of a specific region 234 in the afternoon reference image 230 needs to change to the color of a specific region 235 in the evening reference image 231 can be determined on a specific color chart 237. The specific region 234 in the afternoon reference image 230 may be a region located at the same position as the specific region 235 in the evening reference image 231.

[0203] Although in the above description, the server 2000 obtains object recognition information by using a first artificial intelligence model, obtains spatial information by using a second artificial intelligence model, deletes weather objects in the image by using a third artificial intelligence model, and obtains color change path information between reference images by using a fourth artificial intelligence model, the present invention is not limited thereto.

[0204] The device 1000 may store at least one of the first to fourth artificial intelligence models received from the server 2000, and obtain required data by using at least one of the first to fourth artificial intelligence models. In this case, the first to fourth artificial intelligence models may be implemented as software.

[0205] Figure 29 It is a schematic diagram showing an example of changing the color style of an image according to an embodiment of the present invention.

[0206] Reference Figure 29 , the original image can be converted into images of various color styles. In this case, reference images related to specific color styles may be pre-registered, and when a registered reference image of a specific color style is selected, the original image can be converted based on the color style of the selected reference image.

[0207] Figure 30 It is a schematic diagram showing an example of an image reflecting a rain effect according to an embodiment of the present disclosure.

[0208] Reference Figure 30 , the device 1000 can provide a weather effect caused when a moving object such as a raindrop hits another object in the image by using the spatial information of the image. For example, the device 1000 can provide the effect of raindrops splashing on a person's shoulder and arm. The device 1000 can identify the position of the person shown by using the spatial information of the image, and provide the effect of raindrop splashing based on the depth of the space where the person is shown. The device 1000 can sequentially overlap the image segments of the rain texture image and the segments of the weather texture image including splashing raindrops at the moment when the raindrop hits the person's body. In this case, the spatial information of the image can be used to determine the position of the splashing raindrops.

[0209] The device 1000 can display thick fog on long-distance objects and light fog on short-distance objects. The device 1000 can provide a 3D image effect reflecting weather parameters (such as haze, mist, fog or fog patches), or provide a 3D image effect related to fine dust or dust clouds.

[0210] Figure 31 It is a block diagram of the server 2000 according to an embodiment of the present disclosure.

[0211] Reference Figure 31 , the server 2000 according to an embodiment of the present disclosure may include a communication interface 2100, a database (DB) 2200, an artificial intelligence model 2300, and a processor 2400. The DB 2200 may include an object recognition information DB 2210, a spatial information DB 2220, and a criterion information DB 2230.

[0212] The communication interface 2100 may include one or more elements for communicating with the device 1000. For example, the communication interface 2100 may include a short-range wireless communicator, a mobile communicator, and a wireless communicator. The communication interface 2100 may send to or receive from the device 1000 the information required for simulating weather effects in the analog image.

[0213] The device 1000 may store in the memory a program for processing and controlling the operation of the processor 2400, and store in the image the information required for simulating weather effects. The DB 2200 may store, for example, images, weather texture images, object recognition information, spatial information, simulation criterion information, reference images, and color change path information. The object recognition information DB 2210 may store the object recognition information output from the first artificial intelligence model. The spatial information DB 2220 may store the spatial information output from the second artificial intelligence model. The criterion information DB 2230 may store information on various criteria for simulating weather effects.

[0214] The artificial intelligence model 2300 may perform operations required for simulating weather effects in the image. For example, the artificial intelligence model 2300 may include a first artificial intelligence model for identifying objects in the image, a second artificial intelligence model for analyzing the space in the image, and a third artificial intelligence model for removing weather objects in the image. The artificial intelligence model 2300 may also include a fourth artificial intelligence model for generating color change path information indicating the color change path between reference images.

[0215] The functions related to artificial intelligence in the present invention may be executed using a processor and a memory. The processor 2400 may include one or more processors. In this case, the one or more processors may be general-purpose processors such as a central processing unit (CPU), an application processor (AP), and a digital signal processor (DSP), dedicated graphics processors such as a graphics processing unit (GPU) and a vision processing unit (VPU), or dedicated artificial intelligence processors such as a neural processing unit (NPU). The one or more processors control the processing of input data based on predefined operation rules or artificial intelligence models stored in the memory. Optionally, when the one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed as hardware structures dedicated to processing specific artificial intelligence models.

[0216] Generate a predefined operation rule or an artificial intelligence model through training. Here, achieving through training means training a basic artificial intelligence model based on multiple pieces of training data using a learning algorithm to form a predefined operation rule or configure an artificial intelligence model to achieve desired features (or purposes). The training can be performed by a device with the artificial intelligence function according to the present invention, or by a separate server and / or system. The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited thereto.

[0217] The artificial intelligence model can include multiple neural network layers. Each layer in the multiple neural network layers has multiple weights, and neural network calculations are performed through calculations between the calculation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers can be optimized through the results of training the artificial intelligence model. For example, the multiple weight values can be modified to reduce the loss value obtained by the artificial intelligence model during the training process or minimize the cost value. The artificial neural network can include, for example, CNN, DNN, RNN, RBM, DBN, BRDNN, or deep Q network, but is not limited thereto.

[0218] The processor 2400 controls the overall operation of the server 2000. The processor 2400 can control the communication interface 2100, the DB 2200, and the artificial intelligence model 2300 by executing programs stored in the device 1000. The processor 2400 can perform the operations of the server 2000 described herein by controlling the communication interface 2100, the DB 2200, and the artificial intelligence model 2300.

[0219] Specifically, the processor 2400 can obtain the image selected by the device 1000 and identify the object in the image by using the first artificial intelligence model.

[0220] The processor 2400 can obtain spatial information about the depth of the object in the image by using the second artificial intelligence model.

[0221] The processor 2400 can obtain multiple weather texture images. The processor 2400 can obtain multiple weather texture images pre-registered by weather from the DB 2200. The processor 2400 can obtain, for example, multiple weather texture images indicating rainy weather, multiple weather texture images indicating snowy weather, and multiple weather texture images indicating foggy weather. The multiple weather texture images pre-registered by weather can respectively correspond to depth ranges. The multiple weather texture images pre-registered by weather can be different from each other based on the depth ranges. The depth range can be a value used to define the depth range of the space in the image.

[0222] The processor 2400 can remove weather objects from the image by using the third artificial intelligence model.

[0223] The processor 2400 may determine criteria for simulating multiple weather texture images. For example, the processor 2400 may determine criteria for weather texture images to be used based on the weather, criteria for a part of the weather texture image from which an image segment is to be obtained, criteria for the interval between image segments, and criteria for the time to display the image segments based on the depth range.

[0224] The processor 2400 may generate multiple reference images corresponding to a preset time period. The time period may include an hour range of a day and may be classified as a general time period. For example, the processor 2400 may generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night by changing the color of the image. The processor 2400 may generate color change path information indicating the color change path between the multiple reference images.

[0225] The processor 2400 may provide the device 1000 with object recognition information, spatial information, multiple weather texture images, simulation criteria information, an image from which a weather object is removed, multiple reference images, and color change path information.

[0226] Figure 32 is a block diagram of the device 1000 according to an embodiment of the present invention.

[0227] As Figure 32 shown, the device 1000 may include a user inputter 1100, an outputter 1200, a processor 1300, a sensor 1400, a communicator 1500, an audio / video (A / V) inputter 1600, and a memory 1700.

[0228] The user inputter 1100 refers to a device by which a user inputs data for controlling the device 1000. For example, the user inputter 1100 may include a keyboard, a dome switch, a touchpad (e.g., a capacitive overlay, a resistive overlay, an infrared beam, a surface acoustic wave, an integral strain gauge, or a piezoelectric touchpad), a jog wheel, or a jog switch, but is not limited thereto.

[0229] The user inputter 1100 may receive a user input for simulating a weather effect in an image.

[0230] The outputter 1200 may output an audio signal, a video signal, or a vibration signal, and may include a display 1210, a sound outputter 1220, and a vibration motor 1230.

[0231] The display 1210 displays information processed by the device 1000. For example, the display 1210 may display a GUI for simulating a weather effect in an image.

[0232] When the display 1210 and the touchpad are layered to configure a touch screen, the display 1210 can be used as both an output device and an input device. The display 1210 may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), an organic light emitting diode (OLED), a flexible display, a 3D display, or an electrophoretic display.

[0233] The sound outputter 1220 outputs audio data received from the communicator 1500 or stored in the memory 1700. The vibration motor 1230 may output a vibration signal to generate a tactile effect.

[0234] The processor 1300 generally controls the overall operation of the device 1000. For example, the processor 1300 may control the user inputter 1100, the outputter 1200, the sensor 1400, the communicator 1500, and the A / V inputter 1600 by executing a program stored in the memory 1700. The processor 2400 may perform the operations of the device 1000 described herein by controlling the user inputter 1100, the outputter 1200, the sensor 1400, the communicator 1500, and the A / V inputter 1600.

[0235] Specifically, the processor 1300 may select an image. The processor 1300 may display a GUI for selecting an image to be displayed on the screen in the ambient mode, and select an image based on a user input received through the displayed GUI.

[0236] The processor 1300 may send an image to the server 2000. The processor 1300 may provide an image to the server 2000 and request the server 2000 to provide data required to simulate a weather effect in the image.

[0237] The processor 1300 may receive object recognition information, spatial information, multiple weather texture images, simulation criterion information, an image with weather objects removed therefrom, multiple reference images, and color change path information from the server 2000.

[0238] The processor 1300 may recognize the current weather and obtain a weather texture image corresponding to the current weather. The processor 1300 may select a weather texture image corresponding to the characteristics of the current weather from among the multiple weather texture images received from the server 2000. The processor 1300 may select a weather texture image based on the characteristics of the current weather and the spatial depth in the image. The processor 1300 may simulate a weather effect indicating the current weather in the image by using the selected weather texture image. The processor 1300 may obtain an image segment from the selected weather texture image. The processor 1300 may provide the weather effect in the image by sequentially and iteratively overlapping the image segments on the image in a preset period.

[0239] The processor 1300 may simulate weather effects in an image based on analog criterion information. The processor 1300 may identify the positions of image segments and the intervals between the image segments according to the characteristics of the current weather based on the criterion information for simulating weather effects. The processor 1300 may select multiple image segments from the weather texture image based on the identified cut positions and intervals. The processor 1300 may identify a simulation period according to the characteristics of the current weather and the depth range based on the criterion information for simulating weather effects. The processor 1300 may simulate the image segments corresponding to the first depth range and the image segments corresponding to the second depth range in the image. The processor 1300 may overlap one of the multiple image segments corresponding to the first depth range and one of the multiple image segments corresponding to the second depth range together on the image.

[0240] The processor 1300 may simulate weather effects in an image from which a weather object has been deleted.

[0241] The processor 1300 may simulate weather effects in a reference image corresponding to the current time. The processor 1300 may select a reference image corresponding to the current time from the reference images received from the server 2000. The processor 1300 may simulate weather effects in the selected reference image.

[0242] The processor 1300 may change the color of the selected reference image based on color change path information. For example, the processor 1300 may gradually change the color of the reference image corresponding to afternoon based on the color change path information between the reference image corresponding to afternoon and the reference image corresponding to evening. The processor 1300 may simulate weather effects in the reference image with the changed color.

[0243] The sensor 1400 may detect the state of the device 1000 or the state near the device 1000 and send the detected information to the processor 1300.

[0244] The sensor 1400 may include at least one of a magnetic sensor 1410, an acceleration sensor 1420, a temperature / humidity sensor 1430, an infrared sensor 1440, a gyro sensor 1450, a position sensor (such as GPS) 1460, a barometric pressure sensor 1470, a proximity sensor 1480, or an RGB sensor (or illuminance sensor) 1490, but is not limited thereto. Those of ordinary skill in the art can understand the functions of the sensors from their names, so detailed descriptions thereof are not provided herein.

[0245] The communicator 1500 may include one or more elements for communicating with the server 2000. For example, the communicator 1500 may include a short-range wireless communicator 1510, a mobile communicator 1520, and a broadcast receiver 1530.

[0246] The short-range wireless communicator 1510 may include, for example, a Bluetooth communicator, a Bluetooth Low Energy (BLE) communicator, a Near Field Communication (NFC) communicator, a Wireless Local Area Network (WLAN) (or Wi-Fi) communicator, a Zigbee communicator, an Infrared Data Association (IrDA) communicator, a Wi-Fi Direct (WFD) communicator, an Ultra-Wideband (UWB) communicator, and an Ant+ communicator, but is not limited thereto.

[0247] The mobile communicator 1520 transmits radio signals to and receives radio signals from at least one of a base station, an external terminal device, or a server in a mobile communication network. Here, the radio signals may include various types of data based on the transmission and reception of voice call signals, video call signals, or text / multimedia messages.

[0248] The broadcast receiver 1530 receives broadcast signals and / or broadcast-related information from an external source through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Depending on the implementation, the device 1000 may not include the broadcast receiver 1530.

[0249] The communicator 1500 may send information required to simulate weather effects in an analog image to and receive information from the server 2000.

[0250] The A / V inputter 1600 is used to input an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or an image capture mode. The image captured through the image sensor may be processed by the processor 1300 or a separate image processor.

[0251] The image frame processed by the camera 1610 may be stored in the memory 1700 or transmitted to the outside through the communicator 1500. Depending on the implementation of the device 1000, two or more cameras 1610 may be included.

[0252] The microphone 1620 receives an external sound signal and processes it into electrical voice data. For example, the microphone 1620 may receive a sound signal from an external device or a user. The microphone 1620 may use various noise cancellation algorithms to cancel noise that occurs when receiving the external sound signal.

[0253] The memory 1700 may store programs for processing and controlling the operation of the processor 1300, and store data input to or output from the device 1000.

[0254] The memory 1700 may include at least one type of storage medium such as flash memory, a hard disk, a multimedia card micro, a memory card (e.g., a Secure Digital (SD) or Extreme Digital (XD) card), a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0255] Programs stored in the memory 1700 may be classified into multiple modules based on their functions. For example, a user interface (UI) module 1710, a touch screen module 1720, and a notification module 1730.

[0256] The UI module 1710 may provide a dedicated UI or GUI connected to the device 1000 for each application, for example. The touch screen module 1720 may detect a touch gesture of a user on the touch screen and send information about the touch gesture to the processor 1300. The touch screen module 1720 according to an embodiment of the present invention may recognize and analyze a touch code. The touch screen module 1720 may be implemented as independent hardware including a controller.

[0257] The notification module 1730 may generate a signal for giving a notification of an event of the device 1000.

[0258] The device 1000 may perform some or all of the functions of the server 2000 described herein. For example, the device 1000 may perform a function of generating spatial information based on an image, a function of generating object recognition information based on an image, and a function of setting criteria for simulating a weather effect. In addition, the device 1000 may store various types of information required for simulating a weather effect, such as spatial information, object recognition information, criteria information, and weather texture images. In this case, the device 1000 may be a high-performance device.

[0259] Figure 33 is a schematic diagram showing an example in which an external device 3000 sets criteria for simulating a weather effect and the device 1000 receives information about the set criteria through an external DB 4000 and provides a weather effect in an image.

[0260] Reference Figure 33 , the external device 3000 may set criteria for simulating a weather effect in an image through the server 2000, and the device 1000 may receive information required for simulating a weather effect from the external DB 4000 to simulate a weather effect in an image.

[0261] The external device 3000 can provide an image for which a weather effect is to be applied to the server 2000 through a specific GUI displayed on the screen of the external device 3000 and set criteria for simulating the weather effect. For example, the external device 3000 can set the criteria for simulating the weather effect. The external device 3000 can not provide an image to the server 2000 and can select an image stored in the server 2000.

[0262] The server 2000 can generate information for simulating the weather effect based on the setting values input by the external device 3000 through the GUI. The information for simulating the weather effect can include, for example, an image, a processed image, a weather texture image, spatial information, object recognition information, and criteria information, but the information for simulating the weather effect is not limited thereto. The information for simulating the weather effect can include the above-mentioned parameter information regarding Figure 19 this.

[0263] The external device 3000 can provide an image to the server 2000 based on the web and receive information for simulating the weather effect in the image. The information for simulating the weather effect can be data in a metadata format.

[0264] The external device 3000 can provide the image and information for simulating the weather effect received from the server 2000 to the external DB 4000, and the external DB 4000 can store the image and information for simulating the weather effect.

[0265] The device 1000 can provide an image ID to the external DB 4000 and request information for simulating the weather effect, and receive the image and information for simulating the weather effect. The device 1000 can provide a weather effect in the image based on a preset criterion by using the information for simulating the weather effect.

[0266] The external device 3000 can be, for example, a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a PDA, a laptop computer, a media player, a micro server, a GPS device, an e-book reader, a digital broadcast receiver, a navigation system, a kiosk, an MP3 player, a digital camera, a household appliance, or another mobile or non-mobile computing device, but is not limited thereto. The external device 3000 can include Figure 32 components of this, but is not limited thereto.

[0267] The external DB 4000 can be a server for storing and managing information for simulating the weather effect.

[0268] Figure 34It is a schematic diagram showing that the external device 3000 sets criteria for simulating weather effects according to an embodiment of the present disclosure, and the device 1000 receives information about the set criteria through the server 2000 and provides an example of the weather effect in the image.

[0269] The external device 3000 can provide an image to which a weather effect is to be applied to the server 2000 through a specific GUI displayed on the screen of the external device 3000, and set criteria for simulating the weather effect. For example, the external device 3000 can set criteria for simulating the weather effect. The external device 3000 can not provide an image to the server 2000 and can select an image stored in the server 2000.

[0270] The server 2000 can generate information for simulating the weather effect based on the setting values input by the external device 3000 through the GUI. The information for simulating the weather effect can include, for example, an image, a processed image, a weather texture image, spatial information, object recognition information, and criteria information, but the information for simulating the weather effect is not limited thereto. The information for simulating the weather effect can include the above-mentioned parameter information regarding Figure 19 The server 2000 can store the image and the information for simulating the weather effect.

[0271] The device 1000 can provide an image ID to the server 2000 and request information for simulating the weather effect, and receive the image and information for simulating the weather effect. The device 1000 can provide the weather effect in the image based on preset criteria by using the information for simulating the weather effect.

[0272] Figure 35 It is a schematic diagram showing that the external device 3000 sets criteria for simulating weather effects through the server 2000 according to an embodiment of the present disclosure, and the device 1000 receives information about the criteria through the external DB 4000 and provides the weather effect in the image.

[0273] The external device 3000 can provide an image to which a weather effect is to be applied to the server 2000 through a specific GUI displayed on the screen of the external device 3000, and set criteria for simulating the weather effect. For example, the external device 3000 can set the criteria for simulating the weather effect through the Figure 20 GUI 20 shown in. The external device 3000 can not provide an image to the server 2000 and can select an image stored in the server 2000.

[0274] Server 2000 can generate information for simulating weather effects based on the setting values input by the external device 3000 through the GUI. The information for simulating weather effects may include, for example, images, processed images, weather texture images, spatial information, object recognition information, and criterion information, but is not limited thereto. The information for simulating weather effects may include the above-mentioned parameter information regarding Figure 19 of.

[0275] Server 2000 can provide the images and information for simulating weather effects to the external DB 4000, and the external DB 4000 can store the images and information for simulating weather effects.

[0276] Device 1000 can provide an image ID to the external DB 4000 and request information for simulating weather effects, and receive the images and information for simulating weather effects from the external DB 4000. Device 1000 can provide weather effects in the image based on a preset criterion by using the information for simulating weather effects.

[0277] Embodiments of the present invention can be implemented in the form of a computer-readable recording medium, which includes instructions executable by a computer, for example, program modules executable by a computer. The computer-readable recording medium can be any available medium accessible by a computer, and examples thereof include all volatile, non-volatile, detachable, and non-detachable media. The computer-readable recording medium can include a computer storage medium and a communication medium. Examples of the computer storage medium include all volatile, non-volatile, detachable, and non-detachable media implemented by using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Examples of the communication medium generally can include computer-readable instructions, data structures, and other data such as modulated data signals of program modules.

[0278] As used herein, the suffix “unit” or “-er” may indicate a hardware component such as a processor or a circuit and / or a software component executed by a hardware component such as a processor.

[0279] Throughout the disclosure, “at least one of a, b, or c” means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variants thereof.

[0280] The above description of the present invention is for illustrative purposes, and those of ordinary skill in the art will understand that various changes in form and detail can be made without departing from the scope of the present invention. Accordingly, it should be understood that the disclosed embodiments described herein are to be considered in a descriptive sense only and not for purposes of limitation. For example, each component described as a single type can be implemented in a distributed manner, and likewise, components described as distributed can be implemented in a combined manner.

[0281] The scope of the present invention is defined by the following claims rather than the detailed description, and it should be understood that all modifications from the claims and their equivalents are included within the scope of the present invention.

Claims

1. A method for a device to provide weather effects in an image, the method comprising: obtaining a first image to which the weather effect is to be applied; identifying an object in the first image; identifying a current time and a current weather condition corresponding to the device; and generating a second image with the weather effect applied based on a relationship with the object, according to the current time and the current weather condition.

2. The method according to claim 1, wherein identifying an object in the first image comprises: obtaining object recognition information about the shape of the object in the first image and spatial information about the depth of the object in the first image.

3. The method according to claim 1, wherein the current weather condition corresponds to the location of the device.

4. The method according to claim 1, wherein generating the second image comprises: using a first artificial intelligence model to generate the second image to include the weather effect in the first image.

5. The method according to claim 2, wherein a plurality of objects are in the first image, wherein the weather effect includes a visual effect caused when two objects among the plurality of objects collide by using the spatial information.

6. The method according to claim 2, further comprising: receiving the object recognition information and the spatial information from a server, wherein the object recognition information is generated by the server by applying the first image as an input to a second artificial intelligence model, and wherein the spatial information is generated by the server by applying the first image as an input to a third artificial intelligence model.

7. The method according to claim 6, wherein obtaining the first image comprises receiving the first image from the server, and the server deletes a weather object indicating a weather effect from the first image.

8. The method according to claim 6, wherein obtaining the first image comprises receiving a reference image corresponding to a preset time period from the server.

9. A device for providing weather effects in an image, the device comprising: a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain a first image to which the weather effect is to be applied, identify an object in the first image, identify a current time and a current weather condition corresponding to the device, and generate a second image with the weather effect applied based on a relationship with the object, according to the current time and the current weather condition.

10. The device according to claim 9, wherein the processor is further configured to execute the one or more instructions to: obtain object recognition information about the shape of the object in the first image and spatial information about the depth of the object in the first image.

11. The device according to claim 9, wherein the current weather condition corresponds to the location of the device.

12. The device according to claim 9, wherein The processor is further configured to execute the one or more instructions to: generate the second image using a first artificial intelligence model to include the weather effect in the first image.

13. The apparatus according to claim 10, wherein, a plurality of objects are in the first image, wherein the weather effect includes a visual effect caused by using the spatial information when two objects among the plurality of objects collide.

14. The apparatus according to claim 9, wherein, the processor is further configured to execute the one or more instructions to: receive object recognition information and spatial information from a server, wherein the object recognition information is generated by the server by applying the first image as an input to a second artificial intelligence model, and wherein the spatial information is generated by the server by applying the first image as an input to a third artificial intelligence model.

15. A computer-readable recording medium having recorded thereon a computer program for executing the method of claim 1.

Images