System and method for providing weather effects in images

Abstract

A system and method for providing a weather effect in an image includes selecting at least one weather texture image indicative of weather and providing a weather effect in the image by superimposing the selected weather texture image on the image. A method of providing a weather effect in an image includes receiving object identification information about a shape of an object in the image and spatial information about a depth of the object in the image, selecting at least one weather texture image indicative of weather, and providing a weather effect in the image.

Description

[0001] This application is a divisional application of the invention patent application with the application date of February 4, 2020, the application number of 202080015016.3, and the invention title of "System and Method for Providing Weather Effects in Images". Technical Field

[0002] The present invention relates to a system and method for providing weather effects in images. Background Art

[0003] Currently, televisions or mobile devices display weather information by using text or pre-designed media. However, it may not be possible to effectively provide weather information to users by using text and icons. To provide weather information in the form of video data, time and resources are required to generate the video data. Summary of the Invention

[0004] Aspects of the present invention provide a system that can allow a device to simulate weather effects in an image by using very few computing resources.

[0005] Aspects of the present invention also provide a system that can provide 3D weather effects by using an artificial intelligence model to predict a three-dimensional (3D space) in a two-dimensional (2D) picture.

[0006] Aspects of the present invention also provide a system that can effectively utilize storage space by using weather texture images to real-time simulate weather effects in an image.

[0007] Aspects of the present invention also provide a method for a device to provide weather effects in a first image, the method including: obtaining a plurality of weather texture images in different depth ranges to show weather effects based on the depth of an object in the first image; and sequentially overlapping a plurality of image segments of the weather texture image among the plurality of weather texture images on the object among the objects in the first image, wherein the intervals between the regions corresponding to the plurality of image segments in the weather texture image are determined based on the weather characteristics of the object among the objects in the first image.

[0008] Aspects of the present invention also provide a device for providing weather effects in a first image, the device including: a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain a plurality of weather texture images in different depth ranges to show weather effects based on the depth of an object in the first image, and sequentially overlapping a plurality of image segments of the weather texture image among the plurality of weather texture images on the object among the objects in the first image, wherein the intervals between the regions corresponding to the plurality of image segments in the weather texture image are determined based on the weather characteristics of the object among the objects in the first image.

[0009] Aspects of the present invention also provide a non-transitory computer-readable recording medium having recorded thereon a computer program which, when executed by at least one processor, causes the at least one processor to: obtain a plurality of weather texture images of different depth ranges to show a weather effect based on a depth range of an object in a first image; and sequentially overlap a plurality of image segments of the weather texture images among the plurality of weather texture images on the object among the objects in the first image; wherein an interval between regions corresponding to the plurality of image segments in the weather texture image is determined based on a weather characteristic of the object among the objects in the first image.

[0010] Additional aspects will be set forth in the following description and will be apparent from the description, or may be learned by practice of the presented embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other aspects, features, and advantages of certain embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is a schematic diagram showing an example of a system for providing a weather effect in a specific image according to an embodiment of the present invention;

[0013] Figure 2 is a schematic diagram showing an example of reflecting a weather effect in an image according to an embodiment of the present invention;

[0014] Figure 3 is a flowchart of a method for providing a weather effect in an image executed by a system according to an embodiment of the present invention;

[0015] Figure 4 is a flowchart of a method for selecting a weather texture image corresponding to a current weather executed by a device according to an embodiment of the present invention;

[0016] Figure 5 is a schematic diagram showing an example of a plurality of weather texture images corresponding to weather according to an embodiment of the present invention;

[0017] Figure 6 is a schematic diagram showing an example of object recognition information indicating an object recognized in an image according to an embodiment of the present invention;

[0018] Figure 7 is a schematic diagram showing an example of spatial information indicating a space analyzed in an image according to an embodiment of the present disclosure;

[0019] Figure 8 is a schematic diagram showing an example of weather information according to an embodiment of the present invention;

[0020] Figure 9A flowchart of a method for providing weather effects in an image based on criterion information for simulating weather effects, executed by a system according to an embodiment of the present disclosure;

[0021] Figure 10 A flowchart of a method for determining criteria for simulating multiple weather texture images, executed by a server of a system according to an embodiment of the present invention;

[0022] Figure 11 A flowchart of a method for determining criteria for simulating weather texture images for each depth range, executed by a server according to an embodiment of the present disclosure;

[0023] Figure 12 A schematic diagram showing an example of the positions and intervals between image segments to be cropped from a weather texture image according to an embodiment of the present invention;

[0024] Figure 13 A schematic diagram showing an example of the positions and intervals between image segments to be cropped from a weather texture image according to an embodiment of the present invention;

[0025] Figure 14 A schematic diagram showing an example of cropping image segments from different positions of a weather texture image based on weather according to an embodiment of the present invention;

[0026] Figure 15 A schematic diagram showing an example of sequentially simulating image segments at a specific period according to an embodiment of the present invention;

[0027] Figure 16 A schematic diagram showing an example of simulating image segments in different periods based on depth range according to an embodiment of the present invention;

[0028] Figure 17 A schematic diagram showing an example of cropping image segments in different shapes based on depth range according to an embodiment of the present invention;

[0029] Figure 18 A schematic diagram showing an example of masking or adjusting the transparency of parts of image segments based on depth range according to an embodiment of the present disclosure;

[0030] Figure 19 A table showing an example of information provided to a device for simulating weather effects according to an embodiment of the present disclosure;

[0031] Figure 20 An image of a graphical user interface (GUI) for setting criteria for simulating weather effects according to an embodiment of the present disclosure;

[0032] Figure 21A flowchart of a method for simulating weather effects in an image by a device according to an embodiment of the present disclosure;

[0033] Figure 22 A flowchart of a method for simulating weather effects in an image by a device of a system according to an embodiment of the present disclosure by using a weather texture image received from a server;

[0034] Figure 23 A flowchart of a method for simulating weather effects in an image from which a weather object has been removed by a system according to an embodiment of the present disclosure;

[0035] Figure 24 A schematic diagram showing an example of an image from which a weather object has been removed according to an embodiment of the present invention;

[0036] Figure 25 A flowchart of a method for simulating weather effects in a reference image corresponding to the current time by a system according to an embodiment of the present disclosure;

[0037] Figure 26 A schematic diagram showing an example of a reference image corresponding to a preset time period according to an embodiment of the present invention;

[0038] Figure 27 A flowchart of a method for changing the color of a reference image based on a color change path and simulating weather effects in the color-changed reference image by a system according to an embodiment of the present disclosure;

[0039] Figure 28 A schematic diagram showing an example of a color change path between reference images according to an embodiment of the present invention;

[0040] Figure 29 A schematic diagram showing an example of changing the color style of an image according to an embodiment of the present invention;

[0041] Figure 30 A schematic diagram showing an example of an image reflecting a rain effect according to an embodiment of the present disclosure;

[0042] Figure 31 A block diagram of a server according to an embodiment of the present invention;

[0043] Figure 32 A block diagram of a device according to an embodiment of the present invention;

[0044] Figure 33 A schematic diagram showing an example in which an external device sets criteria for simulating weather effects and the device receives information about the set criteria through an external database (DB) and provides weather effects in an image;

[0045] Figure 34 A schematic diagram showing that an external device sets criteria for simulating weather effects according to an embodiment of the present disclosure, and the device receives information about the set criteria through a server and provides an example of a weather effect in an image; and

[0046] Figure 35 A schematic diagram showing that an external device sets criteria for simulating weather effects through a server according to an embodiment of the present disclosure, and the device receives information about the set criteria through an external DB and provides an example of a weather effect in an image. Detailed implementation mode

[0047] According to an embodiment of the present invention, a method for a device to provide a weather effect in an image is provided, including obtaining an image to which a weather effect is to be applied; obtaining at least one weather texture image showing the weather; and providing a weather effect in the image based on the weather texture image by sequentially overlapping a plurality of image segments obtained from the obtained weather texture image on the image.

[0048] According to another embodiment of the present invention, a device for providing a weather effect in an image includes a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain an image to which a weather effect is to be applied, select at least one weather texture image showing the weather effect, and provide a weather effect in the image on the display based on the weather texture image by sequentially overlapping a plurality of image segments obtained from the weather texture image.

[0049] According to another embodiment of the present invention, a computer-readable recording medium records a computer program for executing the above method.

[0050]

Invention mode

[0051] Hereinafter, the present disclosure will be described in detail by explaining embodiments with reference to the accompanying drawings. However, the present invention can be embodied in many different forms and should not be construed as limited to the embodiments of the present invention set forth herein. In the drawings, parts unrelated to the present invention are not shown for clarity, and like reference numerals denote like elements.

[0052] It should be understood that when an element is referred to as being "connected to" another element, it can be "directly connected to" another element or "electrically connected to" another element through an intermediate element. It will be further understood that when the terms "comprises" and / or "comprising" are used herein, the presence of the stated element is specified, but the presence or addition of one or more other elements is not excluded, unless the context clearly indicates otherwise.

[0053] Throughout the disclosure, "at least one of a, b, or c" means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0054] Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

[0055] Figure 1 is a schematic diagram showing an example of a system for providing a weather effect in a specific image according to an embodiment of the present invention.

[0056] Reference Figure 1 , the system for providing a weather effect may include a device 1000 and a server 2000.

[0057] The device 1000 may select a specific image and display the image as including a three-dimensional (3D) weather effect. The device 1000 may select an image and receive information for simulating a weather effect in the selected image from the server 2000. The device 1000 may provide a weather effect in the selected image by simulating a weather texture image to cover or otherwise be included in the selected image. The weather texture image may be an image of an object indicating a specific weather and may include, for example, a raindrop image, a snowflake image, or a fog image, but the type of weather effect is not limited thereto.

[0058] The weather effect in the image may be provided by cutting a plurality of image segments from the weather texture image and sequentially superimposing the plurality of image segments on the image. The device 1000 may analyze the depth of an object in the image and overlap an image segment reflecting the current weather on the image based on the analyzed depth. To provide a weather effect in an original picture or a picture converted from the original picture, the device 1000 may adjust the transparency of the image segment based on the depth and synthesize the transparency-adjusted image segment with the image.

[0059] The device 1000 may reflect a weather effect in a specific image in an ambient mode. The ambient mode may be an operation mode for providing only some functions of the device 1000 at low power. For example, in the ambient mode, most functions of the device 1000 may not be activated, and only the input / output function and some preset functions of the device 1000 may be activated on the display. Optionally, the device 1000 may display the image in which the weather effect is reflected as a background image of the device 1000.

[0060] The device 1000 can be, for example, a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop computer, a media player, a micro server, a global positioning system (GPS) device, an e-book reader, a digital broadcast receiver, a navigation system, a kiosk, an MP3 player, a digital camera, a household appliance, or another mobile or non-mobile computing device, but is not limited thereto.

[0061] Figure 2 is a schematic diagram showing an example of reflecting a weather effect in an image according to an embodiment of the present invention.

[0062] Reference Figure 2 , one or more artificial intelligence models can be used to analyze the image. For example, the image can be a two-dimensional (2D) or three-dimensional (3D) image. The image can be input into a first artificial intelligence model to detect and identify the objects in the image, and thus object recognition information indicating the recognized objects in the image can be output from the first artificial intelligence model. The object recognition information can include information about the position and shape of the objects in the image. The image can be input into a second artificial intelligence model to analyze the space in the image, and thus spatial information about the depth of the space in the image can be output from the second artificial intelligence model. The second artificial intelligence model can be used to estimate the 3D space in the 2D image by analyzing the spatial depth in the 2D image. The image can be input into a third artificial intelligence model for removing weather objects from the image, and thus an image from which the weather objects are removed can be output from the third artificial intelligence model.

[0063] The first to third artificial intelligence models can be constructed considering the application field of the recognition model, the purpose of training, or the computing performance of the device. The first to third artificial intelligence models can be, for example, models based on artificial neural networks. The artificial neural network can include, for example, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network, but the type of the artificial neural network is not limited thereto.

[0064] A single artificial intelligence model can be used to provide the functions of two or more of the above first to third artificial intelligence models.

[0065] The device 1000 can obtain object recognition information, spatial information, and an image from which weather objects are removed, and obtain a weather texture image corresponding to the current weather. The device 1000 can cut a plurality of image segments from the weather texture image based on a preset criterion. The device 1000 can provide an image reflecting the weather effect related to the current weather by sequentially overlapping the plurality of image segments on the image from which the weather objects are removed.

[0066] Figure 3 It is a flowchart of a method for providing a weather effect in an image executed by a system according to an embodiment of the present disclosure.

[0067] In operation S300, the device 1000 may select an image. The device 1000 may display a graphical user interface (GUI) for selecting an image to be displayed on the screen in an ambient mode, and select an image based on user input received through the displayed GUI. For example, the device 1000 may select a picture stored in the device 1000 or a picture taken in real time.

[0068] In operation S305, the device 1000 may send the image to the server 2000. The device 1000 may provide the image to the server 2000 and request the server 2000 to provide data required to simulate a weather effect in the image.

[0069] Although in operations S300 and S305 the device 1000 selects an image and sends it to the server 2000, the present disclosure is not limited thereto. The server 2000 may provide a list of images stored in the server 2000 to the device 1000, and the device 1000 may select a specific image from the image list.

[0070] In operation S310, the server 2000 may detect and identify objects in the image by using a first artificial intelligence model. The server 2000 may obtain object recognition information indicating or identifying the objects in the image by inputting the image into the first artificial intelligence model for identifying objects in the image. For example, the objects may include people, sky, buildings, and trees. The object recognition information is information indicating the objects recognized in the image, and may include, for example, information about the shape of the objects included in the image, information about the location of the objects, and the recognition information of the objects, but the object recognition information is not limited thereto. The first artificial intelligence model may be a model pre-trained to recognize objects in the image, and may be, for example, a model based on an artificial neural network.

[0071] In operation S315, the server 2000 can obtain spatial information about the depth of an object in an image by using a second artificial intelligence model. The server 2000 can obtain spatial information about the spatial depth in the image by inputting the image into the second artificial intelligence model for analyzing the space in the image. The spatial information is information indicating the depth of the space in the image and can include, for example, information about the depth of the objects included in the image, information about the depth of the background in the image, and information about the depth relationship between the objects in the image, but the spatial information is not limited thereto. For example, when the object in the image is a tree and the background of the image is the sky, the spatial information can include information indicating the depth of the tree, information indicating the depth of the sky, and information indicating that the tree is placed at a reference position closer than the sky. The depth of an object in the image can indicate whether each object is placed at a near position or a far position in the image. For example, when the depth range of the objects in the image is from 0 to 100, values from 0 to 40 can be set as a short distance, values from 40 to 70 can be set as a medium distance, and values from 70 to 100 can be set as a long distance. The second artificial intelligence model can be a model pre-trained to analyze the space in the image and can be, for example, a model based on an artificial neural network.

[0072] The first and second artificial intelligence models can be implemented by a single artificial intelligence model. In this case, the image can be input into the single artificial intelligence model to provide the functions of the first artificial intelligence model and the second artificial intelligence model, so that spatial information about the space in the image and object recognition information about the objects in the image can be output.

[0073] In operation S320, the server 2000 can send the object recognition information and the spatial information to the device 1000. The server 2000 can send the object recognition information and the spatial information obtained by analyzing the image selected by the device 1000 by using at least one artificial intelligence model to the device 1000.

[0074] In operation S330, device 1000 may identify the current weather condition corresponding to the current location of device 1000. Device 1000 may identify the current weather of the area where device 1000 is located, or the current weather of an area different from the area where device 1000 is located selected by the user. Device 1000 may receive weather information indicating the current weather from server 2000 in accordance with a preset period or in real time according to a request from device 1000. The weather information indicating the current weather may include, for example, information indicating clouds, snow, rain, fog, lightning, wind, precipitation, rainfall, fog density, cloud cover, wind direction, gust, wind speed, relative humidity, temperature, felt temperature, air pressure, solar radiation, visibility, ultraviolet (UV) index, dew point, but the meteorological information is not limited thereto. The weather information may include, for example, information on weather forecasts, hourly weather forecasts, and weekly weather forecasts.

[0075] In operation S335, device 1000 may obtain a weather texture image corresponding to the current weather. Device 1000 may pre-store a plurality of weather texture images related to various weather types, and select at least one weather texture image suitable for the current weather from the plurality of pre-stored weather texture images. For example, device 1000 may store in the memory a plurality of weather texture images indicating rainy weather, a plurality of weather texture images indicating snowy weather, and a plurality of weather texture images indicating foggy weather.

[0076] Considering the information on weather forecasts, hourly weather forecasts, and weekly weather forecasts, device 1000 may pre-store weather texture images related to predicted weather. For example, device 1000 may request server 2000 to provide weather texture images related to the weekly weather forecast based on the weekly weather forecast, and store the weather texture images received from server 2000 in the memory. In this case, since device 1000 stores only weather texture images related to the weather predicted for a predetermined time in the memory, the memory of device 1000 can be effectively managed.

[0077] A plurality of weather texture images pre-registered according to weather types may respectively correspond to depth ranges. A plurality of weather texture images pre-registered according to weather may be different from each other based on the depth ranges. The depth range may be a value for defining the depth range of the space in the image. For example, when the depth range of the space in the image is from 0 to 100, the values from 0 to 40 may be set as the first depth range, the values from 40 to 70 may be set as the second depth range, and the values from 70 to 100 may be set as the third depth range. The details of the weather expression may increase as the depth range increases. The resources for providing the weather effect may decrease inversely with the number of depth ranges. The number of depth ranges may be, for example, from 2 to 10.

[0078] Device 1000 may select a weather texture image corresponding to the characteristics of the current weather from multiple weather texture images. Device 1000 may select a weather texture image based on the characteristics of the current weather and the depth of the space in the image. For example, device 1000 may select a weather texture image corresponding to a rainfall of 10 ml / h and a wind speed of 7 km / h from multiple weather texture images, and select a weather texture image corresponding to the first and second depth ranges of the space in the image from the selected weather texture image.

[0079] When a weather texture image suitable for the current weather is not stored in the memory, device 1000 may request server 2000 to provide a weather texture image suitable for the current weather, and receive the requested weather texture image from server 2000.

[0080] In operation S340, device 1000 may simulate a weather effect indicating the current weather in the image by using the obtained weather texture image. Device 1000 may select an image segment for replacement or modification from the selected weather texture image. The size of the weather texture image may be larger than the size of the image selected by device 1000, and device 1000 may select an image segment from the weather texture image to fit the size of the image selected by device 1000. Device 1000 may replace the image segment from the weather texture image by moving the cut position. The cut position may be determined differently according to weather characteristics. For example, when the rainfall is high, device 1000 may determine the cut position as the area of the rain texture image including a large number of raindrops, and when the rainfall is low, determine the cut position as the area of the rain texture image including a small number of raindrops. The degree of movement may be determined differently based on the depth to which the weather texture image will be applied. For example, since raindrops in the near space move fast, device 1000 may cut an image segment from the rain texture image by moving the cut position at large intervals to display raindrops in the near space in the image. For example, since raindrops in the far space move slowly, device 1000 may cut an image segment from the rain texture image by moving the cut position at small intervals to display raindrops in the far space in the image.

[0081] Device 1000 may provide the weather effect in the image by sequentially overlapping, interlacing, or otherwise combining the image segments on the image at a preset period. For example, device 1000 may overlap an image segment obtained from a weather texture image corresponding to a first depth range and an image segment obtained from a weather texture image corresponding to a second depth range on the image. The period for sequentially reproducing the image segment obtained from the weather texture image corresponding to the first depth range may be different from the period for sequentially reproducing the image segment obtained from the weather texture image corresponding to the second depth range.

[0082] Device 1000 can simulate weather effects in an image, such as snow, rain, or fog effects, by using frames of a weather texture image and merging image segments cut from the weather texture image onto the image.

[0083] Device 1000 can adjust the transparency of the image segments differently based on depth and simulate weather effects by using the image segments with adjusted transparency. For example, when the spatial depth in the image is between 0 and 100, device 1000 can adjust the transparency of the image segments corresponding to depths between 0 and 40 to 30%, the transparency of the image segments corresponding to depths between 40 and 70 to 50%, and the transparency of the image segments corresponding to depths from 70 to 100 to 70%.

[0084] Figure 4 is a flowchart of a method for device 1000 to select a weather texture image corresponding to the current weather according to an embodiment of the present disclosure.

[0085] In operation S400, device 1000 can obtain a plurality of weather texture images indicating a first weather condition and corresponding to a plurality of depth ranges. The plurality of weather texture images indicating the first weather condition can respectively correspond to the plurality of depth ranges. For example, device 1000 can obtain from the memory a weather texture image corresponding to the first weather condition and a first depth range, a weather texture image corresponding to the first weather condition and a second depth range, and a weather texture image corresponding to the first weather condition and a third depth range. The first weather condition can correspond to at least one weather characteristic and its intensity or value. Weather characteristics can include, for example, information indicating cloud, snow, rain, fog, lightning, wind, precipitation, rainfall, fog density, cloud cover, wind direction, gust, wind speed, relative humidity, temperature, felt temperature, atmospheric pressure, solar radiation, visibility, UV index, and dew point, but weather characteristics are not limited thereto.

[0086] In operation S410, device 1000 can obtain a plurality of weather texture images indicating a second weather condition and corresponding to a plurality of depth ranges. The plurality of weather texture images indicating the second weather condition can respectively correspond to the plurality of depth ranges. For example, device 1000 can obtain from the memory a weather texture image corresponding to the second weather condition and a first depth range, a weather texture image corresponding to the second weather condition and a second depth range, and a weather texture image corresponding to the second weather condition and a third depth range. The second weather condition can correspond to at least one weather characteristic and its intensity or value. Weather characteristics can include, for example, information indicating cloud, snow, rain, fog, lightning, wind, precipitation, rainfall, fog density, cloud cover, wind direction, gust, wind speed, relative humidity, temperature, felt temperature, atmospheric pressure, solar radiation, visibility, UV index, and dew point, but weather characteristics are not limited thereto.

[0087] In operation S420, the device 1000 may select a weather texture image corresponding to the current weather. The current weather may be the weather at a location corresponding to the location of the device 1000, or the weather at a location different from the current location of the device 1000 selected by the user of the device 1000. The device 1000 may select a weather texture image corresponding to the characteristics of the current weather from among a plurality of weather texture images. The device 1000 may select a weather texture image based on the characteristics of the current weather and the depth of the space in the image. For example, the device 1000 may select a weather texture image corresponding to a rainfall of 10 ml / h and a wind speed of 7 km / h from among a plurality of weather texture images, and select a weather texture image corresponding to the first and second depth ranges of the space in the image from the selected weather texture image.

[0088] When a weather texture image suitable for the current weather is not stored in the memory, the device 1000 may request the server 2000 to provide a weather texture image suitable for the current weather, and receive the requested weather texture image from the server 2000.

[0089] Although in Figure 4 the device 1000 obtains a plurality of weather texture images indicating a first weather condition and a plurality of weather texture images indicating a second weather condition, the device 1000 that obtains the weather texture images is not limited thereto. The device 1000 may obtain weather texture images indicating various preset weather types. The device 1000 may receive weather texture images indicating various preset weather types from the server 2000 in advance, and thus effectively use a weather texture image suitable for the current weather even when the current weather changes.

[0090] Figure 5 is a schematic diagram showing an example of a plurality of weather texture images corresponding to the weather according to an embodiment of the present invention.

[0091] Reference Figure 5 , the plurality of weather texture images indicating snowy weather may include a first snow texture image 50, a second snow texture image 51, and a third snow texture image 52. The first snow texture image 50 may correspond to "snow" and a first depth range, the second snow texture image 51 may correspond to "snow" and a second depth range, and the third snow texture image 52 may correspond to "snow" and a third depth range.

[0092] The first depth range may be shallower than the second depth range, and the second depth range may be shallower than the third depth range. For example, when the depth range of a space or an object in an image is from 0 to 100, values from 0 to 40 may be set as the first depth range, values from 40 to 70 may be set as the second depth range, and values from 70 to 100 may be set as the third depth range. The first snow texture image 50 corresponds to a depth shallower than the depths of the second and third snow texture images 51 and 52, and thus the size of the snowflakes included in the first snow texture image 50 may be larger than the size of the snowflakes included in the second and third snow texture images 51 and 52.

[0093] The second snow texture image 51 corresponds to a depth deeper than the depth of the first snow texture image 50 and shallower than the depth of the third snow texture image 52. Thus, the snowflakes included in the second snow texture image 51 may appear smaller in size than the snowflakes included in the first snow texture image 50 and may be larger in size than the snowflakes included in the third snow texture image 52.

[0094] Optionally, for example, a plurality of weather texture images indicating rainy weather may include a first rain texture image 55, a second rain texture image 56, and a third rain texture image 57. The first rain texture image 55 may correspond to "rain" and the first depth range, the second rain texture image 56 may correspond to "rain" and the second depth range, and the third rain texture image 57 may correspond to "rain" and the third depth range.

[0095] The device 1000 may store in a memory the first to third snow texture images 50 to 52, the first to third rain texture images 55 to 57, etc., related to various weather characteristics and various depth ranges.

[0096] Although the weather texture image corresponds to Figure 5 one of the weather characteristics, the weather texture image is not limited thereto. The weather texture image may correspond to a plurality of weather characteristics. For example, the weather texture image may correspond to rain, rainfall, wind speed, and wind direction. In this case, the size, density, direction, etc. of the objects (such as raindrops) included in the weather texture image may vary based on the weather characteristics corresponding to the weather texture image.

[0097] The weather texture image may be an image in which weather objects are displayed on a transparent layer. In this way, when the weather texture image is overlaid on an image, only the weather objects may be displayed on the image.

[0098] Figure 6 is a schematic diagram showing an example of object recognition information 62 indicating an object recognized in an image 60 according to an embodiment of the present disclosure.

[0099] Refer to Figure 6, the server 2000 can identify the objects in the image 60 by inputting the image 60 into the first artificial intelligence model and obtain object recognition information 62 regarding the shape and position of the objects. Although the object recognition information 62 has the form of an image in Figure 6 , the object recognition information 62 is not limited thereto and can include various format data capable of identifying the position, shape, etc. of the objects.

[0100] The object recognition information 62 is information indicating the objects recognized in the image 60, and can include, for example, information regarding the shape of the objects included in the image 60, information regarding the position of the objects, and the recognition information of the objects. However, the object recognition information is not limited thereto. The first artificial intelligence model can be a model pre-trained to detect and recognize the objects in the image 60, and can be, for example, a model based on an artificial neural network. The first artificial intelligence model can be, for example, an artificial intelligence model for semantic image segmentation. The first artificial intelligence model can detect and recognize the objects in the image 60 and the positions of the objects by estimating the categories of the pixels in the image 60.

[0101] Figure 7 is a schematic diagram showing an example of the spatial information 72 indicating the space analyzed in the image 70 according to an embodiment of the present disclosure.

[0102] Referring to Figure 7 , the server 2000 can analyze the space or region in the image 70 by inputting the image 70 into the second artificial intelligence model and obtain spatial information 72 indicating the depth of the space in the image 70. Although the spatial information 72 has the form of an image in Figure 7 , the spatial information 72 is not limited thereto and can include various format data capable of identifying the depth of the space in the image 70. The spatial information 72 is information indicating the depth of the space in the image 70, and can include, for example, information regarding the depth of the objects included in the image 70, information regarding the depth of the background in the image 70, and information regarding the depth relationship between the objects in the image 70, but is not limited thereto. The second artificial intelligence model can be a model pre-trained to analyze the spatial depth in the image 70, and can be, for example, a model based on an artificial neural network. The second artificial intelligence model can be, for example, an artificial intelligence model for depth prediction / estimation.

[0103] Figure 8 is a schematic diagram showing an example of the weather information according to an embodiment of the present invention.

[0104] Referring to Figure 8 , the weather information can include, for example, information indicating location, time, cloud, snow, rain, relative humidity, temperature, perceived temperature, weather, atmospheric pressure, solar radiation, visibility, wind direction, gust, wind speed, UV index, and dew point.

[0105] The device 1000 can simulate 3D image effects in an image by using a weather texture image indicating weather (such as snow, rain, sunlight, clouds, fog, lightning, or wind). The device 1000 can reflect 3D image effects related to one or more of visibility, wind speed, and temperature in the original image. The device 1000 can reflect 3D image effects in the image, for example, considering weather parameters such as the intensity of wind, the amount of rainfall, the amount of snow, the density of fog, air resistance, distance, and direction. Therefore, the intensity of the weather effect can be correspondingly reflected in the image according to the intensity of the weather effect at a specific location selected by the user or the location of the device 1000.

[0106] Figure 9 is a flowchart of a method for providing a weather effect in an image based on criterion information for simulating a weather effect, which is executed by a system according to an embodiment of the present disclosure.

[0107] Operations S900 to S915 correspond to Figure 3 operations S300 to S315 of , so redundant descriptions thereof are omitted.

[0108] In operation S920, the server 2000 can obtain criterion information for simulating a weather effect. The server 2000 can determine criteria for simulating a weather texture image. For example, the server 2000 can determine criteria for the weather texture image to be used based on the weather, criteria for a part of the weather texture image from which an image segment is to be replaced or modified, criteria for the interval between image segments, and criteria for the time for displaying an image segment based on a depth range. The server 2000 can obtain criterion information regarding the determined criteria. The criterion information for simulating a weather texture image can include, for example, information about the identifier of the weather texture image to be used based on the weather, criteria for replacing or modifying an image segment based on the weather texture image, the interval between image segments, and the time for displaying an image segment. The criterion information can be parameter type data for downloading data on specific criteria for simulating a weather texture image. The criteria and criterion information for simulating a weather texture image will be described in detail below.

[0109] In operation S925, the server 2000 can provide the criterion information, object recognition information, and spatial information regarding the determined criteria to the device 1000. The criterion information, object recognition information, and spatial information can be provided to the device 1000 in the form of parameter values.

[0110] In operation S930, device 1000 may identify the current weather based on the location of device 1000 or a location selected by a user, and in operation S935, obtain a weather texture image corresponding to the current weather. Device 1000 may obtain a weather texture image corresponding to the current weather based on the criterion information received from server 2000. For example, device 1000 may check the identifier of the weather texture image corresponding to the current weather according to the criterion information, and receive the weather texture image from server 2000 based on the identifier of the weather texture image. Optionally, device 1000 may extract the weather texture image from the memory based on the identifier of the weather texture image. Device 1000 may select a weather texture image based on the current weather and the depth of objects and space in the image.

[0111] In operation S940, device 1000 may simulate a weather effect in the image based on the criterion information. Device 1000 may, based on the criterion information, select multiple image segments from the weather texture image according to the current weather and the depth of objects and space in the image, and sequentially overlap the multiple image segments on the image at a specific period. For example, device 1000 may overlap multiple image segments on the image by using alpha blending. In this case, device 1000 may apply a specific transparency to the image segments based on the depth range corresponding to the image segments, and overlap the image segments to which the specific transparency has been applied on the image. Device 1000 may mask a part of the image segments corresponding to a long-distance area based on a specific criterion.

[0112] Figure 10 is a flowchart of a method for determining criteria for simulating multiple weather texture images, which is executed by server 2000 of a system according to an embodiment of the present disclosure.

[0113] In operation S1000, server 2000 may select a weather texture image corresponding to a specific depth range. Server 2000 may select a weather texture image corresponding to a specific weather and a specific depth range from the weather texture images stored in a database (DB).

[0114] In operation S1010, server 2000 may determine the positions of the image segments to be used for simulating the weather effect in the weather texture image. Server 2000 may determine the criterion for the part of the weather texture image from which the image segments are obtained and the interval between the image segments based on the weather characteristics. For example, when the rainfall is large, server 2000 may set the positions of the image segments in such a way that the image segments are cut from the part of the rain texture image with dense raindrops at large intervals.

[0115] In operation S1020, the server 2000 may determine a period for simulating an image segment. The server 2000 may determine the period for simulating an image segment based on the depth range. For example, the server 2000 may set a short simulation period for an image segment cropped from a weather texture image corresponding to a shallow depth range, and set a long simulation period for an image segment cropped from a weather texture image corresponding to a deep depth range. Accordingly, weather effects to be displayed at different depths of the image may be reproduced independently.

[0116] Figure 11 is a flowchart of a method for determining criteria for simulating a weather texture image according to a depth range, performed by the server 2000 according to an embodiment of the present disclosure.

[0117] In operation S1100, the server 2000 may select a weather texture image corresponding to a first depth range. The first depth range may be a range shallower than the depth of a specific object in the image, and the weather texture image corresponding to the first depth range may include a weather object having a size greater than a preset value. The weather object may be an object indicating a specific weather, such as raindrops or snowflakes. For example, when the depth range of a space in the image is from 0 to 100 and the depth of the nearest object in the image is 40, the server 2000 may determine the depth range from 0 to 40 as the first depth range.

[0118] In operation S1110, the server 2000 may determine criteria for simulating the weather texture image corresponding to the first depth range. The server 2000 may determine criteria for the position of an image segment to be obtained from the weather texture image, the interval between image segments, and the simulation period of the image segments based on the characteristics of the weather and the depth range of the weather texture image. For example, when the weather texture image corresponding to the first depth range is a rain texture image corresponding to rainy weather, the server 2000 may set criteria for a portion of the rain texture image from which an image segment is to be obtained based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set the interval between image segments to be obtained based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set a simulation period for sequentially simulating the image segments based on the depth range of the weather texture image. For example, the first depth range may be shallower than a second depth range described below, and the image segments obtained from the weather texture image corresponding to the first depth range may be simulated in the image at a shorter period than the image segments cropped from the weather texture image corresponding to the second depth range described below.

[0119] In operation S1120, the server 2000 may select a weather texture image corresponding to a second depth range. The second depth range may be a range deeper than the depth of a specific object in the image, and the weather texture image corresponding to the second depth range may include weather objects having a size less than a preset value. For example, when the depth range of the space in the image is from 0 to 100 and the depth of the nearest object in the image is 40, the server 2000 may determine the depth range from 40 to 100 as the second depth range.

[0120] In operation S1130, the server 2000 may determine criteria for simulating the weather texture image corresponding to the second depth range. The second depth range may be a depth range deeper than the depth of a specific object in the image, and an image segment obtained from the weather texture image corresponding to the second depth range may be simulated as if it were displayed behind the specific object. In this way, the server 2000 may determine the shape of the image segment to be cut from the weather texture image corresponding to the second depth range so that the image segment does not overlap with the specific object in the image.

[0121] Optionally, the server 2000 may control the transparency of the cut image segment or mask a part of the cut image segment in such a way that the cut image segment does not overlap with the specific object in the image. In this case, the server 2000 may determine that the area of the cut image segment is transparent or masked based on the area occupied by an object closer than the cut image segment. The transparency level may be controlled.

[0122] For example, when the weather texture image corresponding to the second depth range is a rain texture image corresponding to rainy weather, the server 2000 may set criteria for a part of the rain texture image from which the image segment is to be obtained based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set the interval between the image segments to be obtained based on at least one of precipitation, wind speed, or wind direction. The server 2000 may set a simulation period for sequentially simulating the image segments based on the depth range of the weather texture image. For example, the second depth range may be a depth range deeper than the first depth range, and the image segments obtained from the weather texture image corresponding to the second depth range may be simulated in the image at a period longer than that of the image segments obtained from the weather texture image corresponding to the first depth range.

[0123] Figure 12 is a schematic diagram showing an example of the positions and intervals between image segments to be cut from a weather texture image according to an embodiment of the present invention.

[0124] Figure 13 is a schematic diagram showing an example of the positions and intervals between image segments to be cut from a weather texture image according to an embodiment of the present invention.

[0125] Reference Figure 12 As shown in Figure 12 , the image segments 111, 112, and 113 can be cut from the rain texture image 110. Reference Figure 13 As shown in Figure 13 , the image segments 114, 115, and 116 can be cut from the rain texture image 110. The cutting direction of the image segments to be cut from the rain texture image 110 and the interval between the image segments can be adjusted based on the precipitation, wind direction, and wind speed. For example, when the precipitation, wind direction, and wind speed are low, the image segments 111, 112, and 113 can be selected at small intervals in a direction close to the vertical direction of the rain texture image 110. Otherwise, when the precipitation and wind speed are high, the image segments 114, 115, and 116 can be selected at large intervals in the diagonal direction of the rain texture image 110.

[0126] Figure 14 FIG. Figure 14 is a schematic diagram showing an example of selecting image segments from different positions of a weather texture image based on weather according to an embodiment of the present invention.

[0127] Reference Figure 14 As shown in Figure 14 , raindrops with different shapes can be placed at different densities in the rain texture image 130. For example, raindrops can be placed vertically on the left side of the rain texture image 130, and raindrops can be placed diagonally on the right side of the rain texture image 130. For example, raindrops can be placed at a low density at the top of the rain texture image 130, and raindrops can be placed at a high density at the bottom of the rain texture image 130.

[0128] Therefore, when the rainfall and wind speed are low, selection criteria can be set to select the image segments 131, 132, and 133 from the upper left part of the rain texture image 130. Otherwise, when the rainfall and wind speed are high, selection criteria can be set to select the image segments 134, 135, and 136 from the lower right part of the rain texture image 130.

[0129] Figure 15 FIG. Figure 15 is a schematic diagram showing an example of sequentially simulating image segments at a specific period according to an embodiment of the present invention.

[0130] Reference Figure 15 As shown in Figure 15 , the image segments 141, 142, and 143 can be reproduced sequentially and repeatedly on the image 140. A period for covering the image segments 141, 142, and 143 can be preset based on the depth range of the image segments 141, 142, and 143. For example, the image segment 141 can be covered on the image 140 from 0 seconds to 0.1 seconds, and then the image segment 142 can be covered on the image 140 from 0.1 seconds to 0.2 seconds. The image segment 143 can be covered on the image 140 from 0.2 seconds to 0.3 seconds, and then the image segment 141 can be covered on the image 140 from 0.3 seconds to 0.4 seconds. The transparency of covering the image segments 141, 142, and 143 can be controlled, for example, based on the depth of an object in the image.

[0131] A snow effect or other weather effect can be provided in the image 140 by sequentially and repeatedly superimposing the image segments 141, 142, and 143 on the image 140 at a specific period.

[0132] Figure 16 It is a schematic diagram showing an example of simulating image segments at different periods based on the depth range according to an embodiment of the present invention.

[0133] Reference Figure 16 , one of the first image segments 1-1, 1-2, and 1-3 obtained from the weather texture image of the first depth range, one of the second image segments 2-1, 2-2, and 2-3 obtained from the weather texture image of the second depth range, and one of the third image segments 3-1, 3-2, and 3-3 obtained from the weather texture image of the third depth range can be combined on the image 150.

[0134] The first depth range can be a depth range shallower than the second depth range, and the second depth range can be a depth range shallower than the third depth range.

[0135] The first image segments 1-1, 1-2, and 1-3 obtained from the weather texture image of the first depth range can be sequentially overlapped on the image 150 at a period of 0.1 seconds. The second image segments 2-1, 2-2, and 2-3 obtained from the weather texture image of the second depth range can be sequentially overlapped on the image 150 at a period of 0.2 seconds. The third image segments 3-1, 3-2, and 3-3 obtained from the weather texture image of the third depth range can be sequentially overlapped on the image 150 at a period of 0.3 seconds.

[0136] In this case, at least part of the first image segments 1-1, 1-2, and 1-3, the second image segments 2-1, 2-2, and 2-3, and the third image segments 3-1, 3-2, and 3-3 can be transparent or occluded based on the position and depth of the object in the image 150.

[0137] The first image segments 1-1, 1-2, and 1-3, the second image segments 2-1, 2-2, and 2-3, and the third image segments 3-1, 3-2, and 3-3 can be images on which weather objects of different sizes are displayed. The first image segments 1-1, 1-2, and 1-3, the second image segments 2-1, 2-2, and 2-3, and the third image segments 3-1, 3-2, and 3-3 can be overlapped at different periods, so a realistic 3D weather effect can be provided in the image 150.

[0138] Figure 17 It is a schematic diagram showing an example of shearing image segments in different shapes based on the depth range according to an embodiment of the present invention.

[0139] Reference Figure 17, the first depth range may be a depth range shallower than the depths of the first object 161 and the second object 162 in the image 160. For example, when the spatial depth range in the image 160 is from 0 to 100, the first depth range may be a depth range from 0 to 40. An image segment 164 having the same size as the image 160 may be cut from the weather texture image 163 corresponding to the first depth range.

[0140] The second depth range may be a depth range deeper than the depth of the first object 161 in the image 160 and shallower than the depth of the second object 162. For example, when the spatial depth range in the image 160 is from 0 to 100, the second depth range may be a depth range from 40 to 70. An image segment 166 having a shape non-overlapping with the first object 161 may be cut from the weather texture image 165 corresponding to the second depth range.

[0141] The third depth range may be a depth range deeper than the depths of the first object 161 and the second object 162 in the image 160. For example, when the spatial depth range in the image 160 is from 0 to 100, the third depth range may be a depth range from 70 to 100. An image segment 168 having a shape non-overlapping with the first object 161 and the second object 162 may be cut from the weather texture image 167 corresponding to the third depth range.

[0142] Figure 18 is a schematic diagram showing an example of masking or adjusting the transparency of a part of an image segment based on a depth range according to an embodiment of the present invention.

[0143] Reference Figure 18 , the image segments 170, 174, and 178 may be selected in a rectangular shape. The shapes of the image segments 170, 174, and 178 are not limited and may be any shape including a square, a circle, and an irregular shape. The shape of the image segment may correspond to the shape of the object in the image to which the image segment is to be overlaid.

[0144] In this case, since the image segment 170 corresponds to the third depth range, the region 172 of the image segment 170 overlapping with an object shallower than the third depth range may be masked. Optionally, the region 172 of the image segment 170 may be made transparent.

[0145] Because the image segment 174 corresponds to the second depth range, the region 176 of the image segment 174 overlapping with an object shallower than the second depth range may be masked. Optionally, the region 176 of the image segment 174 may be made transparent.

[0146] Figure 19 is a table showing an example of information provided to the device 1000 to simulate weather effects according to an embodiment of the present disclosure.

[0147] Reference Figure 19 , object recognition information, spatial information, criterion information, etc. can be provided to the device 1000.

[0148] The request parameters are examples of the criterion information required to simulate the weather effect, and can include an identifier of the weather effect (e.g., effect_id), an identifier of the image (e.g., image_num), and information indicating the criterion for simulating the weather effect (e.g., effect-information). The request parameters can be fixed values and can be included in a file of the JavaScript Object Notation (Json) type.

[0149] The image can include the image to which the weather effect is to be applied. The image to which the weather effect is to be applied can include the original image, the image converted based on time, and the image converted based on season, but the image is not limited thereto.

[0150] The depth map is an example of the spatial information indicating the depth of the space in the image, and can be a map file generated by performing depth prediction using a second artificial intelligence model. The depth map can be generated by recognizing the 3D space from the 2D image and representing the distance values of the pixels in the image in the form of a map.

[0151] The segmentation map is an example of the object recognition information indicating the objects recognized in the image, and can be a map file generated by performing semantic image segmentation using a first artificial intelligence model.

[0152] The texture can indicate the weather texture image for providing the weather effect.

[0153] The server 2000 can provide the request parameters, the depth map, and the segmentation map to the device 1000, and may not provide the image to which the weather effect is to be applied and the weather texture image to the device 1000. In this case, the server 2000 can provide the device 1000 with the link information for downloading the image to which the weather effect is to be applied, and the link information for downloading the weather texture image. The server 2000 can send a compressed file generated by compressing Figure 19 at least a part of the data.

[0154] Figure 20 is an image of the GUI 20 for setting the criterion for simulating the weather effect according to an embodiment of the present disclosure.

[0155] Through Figure 20 the GUI 20, the user of the device 1000 or the operator of the server 2000 can set the criterion for simulating the weather effect. Figure 20 the GUI 20 can be provided to the device 1000 through a web-based service. ReferenceFigure 20 The GUI 20 for setting criteria for simulating weather effects may include an area 22 for selecting an area of an image to which a weather effect is to be applied, an area 23 for displaying a preview image to which the weather effect has been applied, an area 24 for selecting a weather type, and an area 25 for setting parameters of the weather effect.

[0156] The list of images stored in the device 1000 and the list of images stored in the server 2000 may be displayed in the area 22 for selecting an image to which a weather effect is to be applied.

[0157] When the list of images stored in the server 2000 is displayed in the area 22 for selecting an image to which a weather effect is to be applied, the server 2000 may recommend images related to the current weather. The server 2000 may classify the images stored in the server 2000 based on the weather. The server 2000 may receive weather information related to the current weather from a weather service provider server and recommend images corresponding to the current weather to the device 1000 based on the received weather information.

[0158] Taking into account the time and area where the device 1000 is located, the server 2000 may recommend images corresponding to the current weather to the device 1000. In this case, the server 2000 may classify the images based on the location and time.

[0159] When the list of images stored in the device 1000 is displayed in the area 22 for selecting an image to which a weather effect is to be applied, the device 1000 may recommend images related to the current weather. The device 1000 may classify the images stored in the device 1000 based on the weather. The device 1000 may receive weather information related to the current weather from a weather service provider server and recommend images corresponding to the current weather to the user based on the received weather information.

[0160] The GUI 20 may be provided to the device 1000 through a specific application programming interface (API).

[0161] Identifiers of weather effects applicable to the image may be displayed in the area 24 for selecting a weather type. For example, identifiers indicating rain, snow, and fog effects may be displayed in the area 24.

[0162] Values indicating characteristics of the weather effect may be displayed in the area 25 for setting parameters of the weather effect. For example, values for setting the speed, amount, and angle of a weather object may be displayed in the area 25.

[0163] A preview image reflecting the weather effect can be displayed in region 23. A preview image showing the simulation result of the weather effect selected in region 24 can be displayed in real time in region 23 in an image based on the characteristics selected in region 25.

[0164] Figure 21 It is a flowchart of a method for simulating the weather effect in an image by device 1000 according to an embodiment of the present disclosure by using a weather texture image.

[0165] In operation S2100, device 1000 can obtain a plurality of image segments from a weather texture image corresponding to a first depth range. Device 1000 can obtain weather information indicating the current weather and identify the characteristics of the current weather. Device 1000 can identify the positions of the image segments and the intervals between the image segments according to the characteristics of the current weather based on the criterion information for simulating the weather effect. Device 1000 can obtain a plurality of image segments from the weather texture image based on the identified positions and intervals.

[0166] In operation S2110, device 1000 can identify the simulation period. Device 1000 can identify the simulation period according to the characteristics of the current weather and the first depth range based on the criterion information for simulating the weather effect.

[0167] In operation S2120, device 1000 can obtain a plurality of image segments from a weather texture image corresponding to a second depth range. Device 1000 can identify the positions of the image segments and the intervals between the image segments according to the characteristics of the current weather based on the criterion information for simulating the weather effect. Device 1000 can select a plurality of image segments from the weather texture image based on the identified shear positions and intervals. When an object exists in the image at a depth shallower than the second depth range, device 1000 can control the image segments not to overlap with the object in the image.

[0168] In operation S2130, device 1000 can identify the simulation period. Device 1000 can identify the simulation period according to the characteristics of the current weather and the second depth range based on the criterion information for simulating the weather effect.

[0169] In operation S2140, device 1000 can simulate the image segments corresponding to the first depth range and the image segments corresponding to the second depth range in the image. Device 1000 can overlap one of the plurality of image segments corresponding to the first depth range and one of the plurality of image segments corresponding to the second depth range on the image together.

[0170] Figure 22 It is a flowchart of a method for simulating the weather effect in an image by device 1000 of a system according to an embodiment of the present disclosure by using a weather texture image received from server 2000.

[0171] Figure 22 Operations S2200 to S2215 correspond to Figure 3 operations S300 to S315, and thus redundant descriptions thereof are omitted.

[0172] In operation S2220, server 2000 may obtain a plurality of weather texture images. Server 2000 may obtain a plurality of weather texture images pre-registered according to weather conditions from a database. Server 2000 may obtain, for example, a plurality of weather texture images indicating rainy weather, a plurality of weather texture images indicating snowy weather, and a plurality of weather texture images indicating foggy weather. The plurality of weather texture images pre-registered according to weather may respectively correspond to depth ranges. The plurality of weather texture images pre-registered according to weather may be different from each other based on depth ranges. A depth range may be a value for defining a depth range of space in an image.

[0173] In operation S2220, server 2000 may obtain criterion information for simulating weather effects. For example, server 2000 may determine criteria for weather texture images to be used based on weather, criteria for a part of a weather texture image from which an image segment is to be cut, criteria for an interval between image segments, and criteria for a time for displaying image segments based on depth ranges. In operation S2230, server 2000 may provide the device 1000 with criterion information, object recognition information, space information, and a plurality of weather texture images regarding the determined criteria. The device 1000 may pre-store a plurality of weather texture images related to various weather types and select and use at least one weather texture image suitable for the current weather and the selected image from the plurality of pre-stored weather texture images.

[0174] In operation S2240, the device 1000 may simulate weather effects in an image based on the criterion information. The device 1000 may select a weather texture image based on the current weather and the depth of objects and space in the image. The device 1000 may obtain a plurality of image segments from the selected weather texture image and sequentially combine the plurality of image segments on the image at a specific period.

[0175] Figure 23 is a flowchart of a method for simulating weather effects in an image from which a weather object has been deleted, performed by a system according to an embodiment of the present disclosure.

[0176] Operations S2300 to S2315 and S2325 correspond to Figure 22 operations S2200 to S2215 and S2225, and thus redundant descriptions thereof are omitted.

[0177] In operation S2320, the server 2000 may remove weather objects from an image by using a third artificial intelligence model. The server 2000 may obtain an image with weather objects removed therefrom by inputting the image into the third artificial intelligence model for removing weather objects from the image. The third artificial intelligence model may be a model pre-trained to remove weather objects from an image and may be, for example, a model based on an artificial neural network.

[0178] In operation S2330, the server 2000 may provide the device 1000 with an image with weather objects removed therefrom, object recognition information, spatial information, and a plurality of weather texture images.

[0179] In operation S2340, the device 1000 may simulate a weather effect in the image with weather objects removed therefrom. The device 1000 may select a weather texture image based on the current weather and the depth of the objects and space in the image. The device 1000 may obtain a plurality of image segments from the selected weather texture image and sequentially overlap the plurality of image segments on the image with weather objects removed therefrom at a specific period.

[0180] Figure 24 is a schematic diagram showing an example of an image with weather objects removed according to an embodiment of the present disclosure.

[0181] Reference Figure 24 , picture 190 taken on a rainy day includes raindrops. When picture 190 including raindrops is input into the third artificial intelligence model, picture 192 with raindrops removed therefrom may be output from the third artificial intelligence model. Picture 191 including empty raindrops may be generated by removing raindrops from picture 190 including raindrops, and picture 192 with raindrops removed therefrom may be generated by completely omitting raindrops from the image.

[0182] Figure 25 is a flowchart of a method for simulating a weather effect in a reference image corresponding to the current time executed by a system according to an embodiment of the present disclosure.

[0183] Operations S2500 to S2520 correspond to Figure 9 operations S900 to S920 thereof, and thus redundant descriptions thereof are omitted.

[0184] In operation S2525, the server 2000 may generate a plurality of reference images corresponding to a preset time period. The server 2000 may, for example, generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night by changing the color of the image or other objects in the image or features indicating a certain time of day in the image (such as brightness, illumination), etc.

[0185] In operation S2530, the server 2000 may provide the device 1000 with multiple reference images, object recognition information, spatial information, and criterion information.

[0186] In operation S2535, the device 1000 may identify the current time.

[0187] In operation S2540, the device 1000 may select a reference image corresponding to the current time. The device 1000 may select a reference image corresponding to the current time from the reference images received from the server 2000. For example, when the current time is 13:00, the device 1000 may select a reference image corresponding to the afternoon.

[0188] In operation S2545, the device 1000 may simulate weather effects in the selected reference image.

[0189] Figure 26 FIG. is a schematic diagram showing an example of a reference image corresponding to a preset time period according to an embodiment of the present invention.

[0190] Reference Figure 26 , it is possible to use an image to generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night.

[0191] Figure 27 FIG. is a flowchart of a method for changing the color of a reference image based on a color change path and simulating weather effects in the reference image with the color changed, which is executed by a system according to an embodiment of the present disclosure.

[0192] Operations S2700 to S2720 correspond to Figure 9 operations S900 to S920 of , and thus redundant descriptions thereof are omitted.

[0193] In operation S2725, the server 2000 may generate multiple reference images corresponding to a preset time period. For example, the server 2000 may generate a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night by changing the color of an image.

[0194] In operation S2730, the server 2000 may generate color change path information indicating the color change path between multiple reference images. The server 2000 may change the first reference image to the second reference image in such a way that the color of the first reference image smoothly changes to the color of the second reference image. To this end, the server 2000 may generate color change path information by determining the path for changing the color of a specific region of the first reference image to the color of a specific region of the second reference image. The server 2000 may obtain color change path information indicating the color change path between reference images by inputting multiple reference images into a fourth artificial intelligence model. The fourth artificial intelligence model may be a model that has been pre-trained to naturally change colors between reference images considering the characteristics of the reference images, and may be, for example, a model based on an artificial neural network.

[0195] In operation S2735, the server 2000 may provide the device 1000 with multiple reference images, color change path information, object recognition information, spatial information, and criterion information.

[0196] In operation S2740, the device 1000 may identify the current time.

[0197] In operation S2745, the device 1000 may select a reference image corresponding to the current time. The device 1000 may select a reference image corresponding to the current time from the reference images received from the server 2000. The device 1000 may also select a reference image after the selected reference image. For example, when the current time is 13:00, the device 1000 may select a reference image corresponding to the afternoon and a reference image corresponding to the evening.

[0198] In operation S2750, the device 1000 may change the color of the selected reference image based on the color change path information. For example, the server 2000 may gradually change the color of the reference image corresponding to the afternoon based on the color change path information between the reference image corresponding to the afternoon and the reference image corresponding to the evening.

[0199] In operation S2755, the device 1000 may simulate weather effects in the reference image with the color changed.

[0200] Figure 28 is a schematic diagram showing an example of the color change path between reference images according to an embodiment of the present invention.

[0201] Reference Figure 28 , color change path information on how the color of the afternoon reference image 230 needs to change from the afternoon reference image 230 to the color of the evening reference image 231 may be generated. The color change path on how the color needs to change between the respective regions of the reference image may be determined.

[0202] For example, a path 238 regarding how the color of a specific area 234 in the afternoon reference image 230 needs to be changed to the color of a specific area 235 in the evening reference image 231 can be determined on a specific color chart 237. The specific area 234 in the afternoon reference image 230 can be an area located at the same position as the specific area 235 in the evening reference image 231.

[0203] Although in the above description, the server 2000 obtains object recognition information by using a first artificial intelligence model, obtains spatial information by using a second artificial intelligence model, deletes weather objects in the image by using a third artificial intelligence model, and obtains color change path information between reference images by using a fourth artificial intelligence model, the present invention is not limited thereto.

[0204] The device 1000 can store at least one of the first to fourth artificial intelligence models received from the server 2000, and obtain required data by using at least one of the first to fourth artificial intelligence models. In this case, the first to fourth artificial intelligence models can be implemented as software.

[0205] Figure 29 It is a schematic diagram showing an example of changing the color style of an image according to an embodiment of the present invention.

[0206] Reference Figure 29 , the original image can be converted into an image with various color styles. In this case, reference images related to specific color styles can be pre-registered, and when a registered reference image of a specific color style is selected, the original image can be converted based on the color style of the selected reference image.

[0207] Figure 30 It is a schematic diagram showing an example of an image reflecting the rain effect according to an embodiment of the present disclosure.

[0208] Reference Figure 30 , the device 1000 can provide a weather effect caused when a moving object such as a raindrop hits another object in the image by using the spatial information of the image. For example, the device 1000 can provide the effect of raindrops splashing on a person's shoulder and arm. The device 1000 can identify the position of the person shown by using the spatial information of the image, and provide the effect of raindrop splashing based on the depth of the space showing the person. The device 1000 can sequentially overlap an image segment of a rain texture image and a segment of a weather texture image including splashing raindrops at the moment when the raindrop hits the person's body. In this case, the spatial information of the image can be used to determine the position of the splashing raindrops.

[0209] Device 1000 may display thick fog on long - distance objects and light fog on short - distance objects. Device 1000 may provide a 3D image effect reflecting weather parameters (e.g., haze, mist, fog, or fog patches), or provide a 3D image effect related to fine dust or dust clouds.

[0210] Figure 31 is a block diagram of server 2000 according to an embodiment of the present disclosure.

[0211] Reference Figure 31 , according to an embodiment of the present disclosure, server 2000 may include a communication interface 2100, a database (DB) 2200, an artificial intelligence model 2300, and a processor 2400. DB 2200 may include an object recognition information DB 2210, a space information DB 2220, and a criterion information DB 2230.

[0212] Communication interface 2100 may include one or more elements for communicating with device 1000. For example, communication interface 2100 may include a short - range wireless communicator, a mobile communicator, and a wireless communicator. Communication interface 2100 may send to or receive from device 1000 the information required to simulate weather effects in an analog image.

[0213] Device 1000 may store in a memory a program for processing and controlling the operation of processor 2400, and store in an image the information required to simulate weather effects. DB 2200 may store, for example, images, weather texture images, object recognition information, space information, simulation criterion information, reference images, and color change path information. Object recognition information DB 2210 may store the object recognition information output from the first artificial intelligence model. Space information DB 2220 may store the space information output from the second artificial intelligence model. Criterion information DB 2230 may store information on various criteria for simulating weather effects.

[0214] Artificial intelligence model 2300 may perform operations required to simulate weather effects in an image. For example, artificial intelligence model 2300 may include a first artificial intelligence model for identifying objects in an image, a second artificial intelligence model for analyzing the space in an image, and a third artificial intelligence model for removing weather objects in an image. Artificial intelligence model 2300 may also include a fourth artificial intelligence model for generating color change path information indicating the color change path between reference images.

[0215] A processor and a memory can be used to execute the functions related to artificial intelligence in the present invention. The processor 2400 may include one or more processors. In this case, the one or more processors may be general-purpose processors such as a central processing unit (CPU), an application processor (AP), and a digital signal processor (DSP), dedicated graphics processors such as a graphics processing unit (GPU) and a vision processing unit (VPU), or dedicated artificial intelligence processors such as a neural processing unit (NPU). The one or more processors control the processing of input data based on predefined operation rules or artificial intelligence models stored in the memory. Optionally, when the one or more processors are dedicated artificial intelligence processors, the dedicated artificial intelligence processors may be designed as hardware structures dedicated to processing specific artificial intelligence models.

[0216] Predefined operation rules or artificial intelligence models are generated through training. Here, achieving through training means forming predefined operation rules or artificial intelligence models configured to achieve desired features (or purposes) by training a basic artificial intelligence model based on multiple pieces of training data using a learning algorithm. The training can be performed by a device having the artificial intelligence function according to the present invention, or by a separate server and / or system. The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited thereto.

[0217] The artificial intelligence model may include multiple neural network layers. Each of the multiple neural network layers has multiple weights, and neural network calculations are performed through calculations between the calculation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers can be optimized through the results of training the artificial intelligence model. For example, the multiple weight values can be modified to reduce the loss value obtained by the artificial intelligence model during the training process or to minimize the cost value. The artificial neural network may include, for example, CNN, DNN, RNN, RBM, DBN, BRDNN, or deep Q-network, but is not limited thereto.

[0218] The processor 2400 controls the overall operation of the server 2000. The processor 2400 can control the communication interface 2100, the DB 2200, and the artificial intelligence model 2300 by executing a program stored in the device 1000. The processor 2400 can perform the operations of the server 2000 described herein by controlling the communication interface 2100, the DB 2200, and the artificial intelligence model 2300.

[0219] Specifically, the processor 2400 can obtain the image selected by the device 1000 and identify the object in the image by using a first artificial intelligence model.

[0220] The processor 2400 can obtain spatial information about the depth of the object in the image by using a second artificial intelligence model.

[0221] The processor 2400 may obtain a plurality of weather texture images. The processor 2400 may obtain, from the DB 2200, a plurality of weather texture images pre-registered according to the weather. The processor 2400 may obtain, for example, a plurality of weather texture images indicating rainy weather, a plurality of weather texture images indicating snowy weather, and a plurality of weather texture images indicating foggy weather. The plurality of weather texture images pre-registered according to the weather may respectively correspond to depth ranges. The plurality of weather texture images pre-registered according to the weather may be different from each other based on the depth ranges. The depth range may be a value for defining the depth range of space in the image.

[0222] The processor 2400 may remove weather objects from the image by using a third artificial intelligence model.

[0223] The processor 2400 may determine criteria for simulating a plurality of weather texture images. The processor 2400 may determine, for example, criteria for the weather texture images to be used based on the weather, a part of the weather texture image from which an image segment is to be obtained, criteria for the interval between image segments, and criteria for the time for displaying the image segments based on the depth range.

[0224] The processor 2400 may generate a plurality of reference images corresponding to a preset time period. The time period may include an hour range in a day and may be classified into a general time period. The processor 2400 may generate, for example, a reference image corresponding to the morning, a reference image corresponding to the afternoon, a reference image corresponding to the evening, and a reference image corresponding to the night by changing the color of the image. The processor 2400 may generate color change path information indicating the color change path between the plurality of reference images.

[0225] The processor 2400 may provide the device 1000 with object recognition information, spatial information, a plurality of weather texture images, simulation criteria information, the image from which weather objects are removed, a plurality of reference images, and color change path information.

[0226] Figure 32 is a block diagram of the device 1000 according to an embodiment of the present invention.

[0227] As Figure 32 shown, the device 1000 may include a user inputter 1100, an outputter 1200, a processor 1300, a sensor 1400, a communicator 1500, an audio / video (A / V) inputter 1600, and a memory 1700.

[0228] The user input device 1100 refers to a device used by a user to input data for controlling the device 1000. For example, the user input device 1100 may include a keyboard, a dome switch, a touchpad (e.g., a capacitive overlay, a resistive overlay, an infrared beam, surface acoustic wave, piezoresistive, or piezoelectric touchpad), a jog wheel, or a jog switch, but is not limited thereto.

[0229] The user input device 1100 can receive user input for simulating weather effects in an image.

[0230] The output device 1200 can output an audio signal, a video signal, or a vibration signal, and may include a display 1210, a sound output device 1220, and a vibration motor 1230.

[0231] The display 1210 displays information processed by the device 1000. For example, the display 1210 may display a GUI for simulating weather effects in an image.

[0232] When the display 1210 and the touchpad are layered to configure a touch screen, the display 1210 can be used as both an output device and an input device. The display 1210 may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), an organic light emitting diode (OLED), a flexible display, a 3D display, or an electrophoretic display.

[0233] The sound output device 1220 outputs audio data received from the communicator 1500 or stored in the memory 1700. The vibration motor 1230 can output a vibration signal to generate a tactile effect.

[0234] The processor 1300 generally controls the overall operation of the device 1000. For example, the processor 1300 can control the user input device 1100, the output device 1200, the sensor 1400, the communicator 1500, and the A / V input device 1600 by executing a program stored in the memory 1700. The processor 2400 can perform the operations of the device 1000 described herein by controlling the user input device 1100, the output device 1200, the sensor 1400, the communicator 1500, and the A / V input device 1600.

[0235] Specifically, the processor 1300 can select an image. The processor 1300 can display a GUI for selecting an image to be displayed on the screen in the ambient mode, and select an image based on user input received through the displayed GUI.

[0236] The processor 1300 can send an image to the server 2000. The processor 1300 can provide the image to the server 2000 and request the server 2000 to provide data required for simulating weather effects in the image.

[0237] The processor 1300 may receive object recognition information, spatial information, multiple weather texture images, simulation criterion information, an image with weather objects removed therefrom, multiple reference images, and color change path information from the server 2000.

[0238] The processor 1300 may recognize the current weather and obtain a weather texture image corresponding to the current weather. The processor 1300 may select a weather texture image corresponding to the characteristics of the current weather from among the multiple weather texture images received from the server 2000. The processor 1300 may select a weather texture image based on the characteristics of the current weather and the spatial depth in the image. The processor 1300 may simulate a weather effect indicating the current weather in the image by using the selected weather texture image. The processor 1300 may obtain an image segment from the selected weather texture image. The processor 1300 may provide a weather effect in the image by sequentially and iteratively overlapping the image segments on the image at a preset period.

[0239] The processor 1300 may simulate a weather effect in the image based on the simulation criterion information. The processor 1300 may identify the positions of the image segments and the intervals between the image segments according to the characteristics of the current weather based on the criterion information for simulating the weather effect. The processor 1300 may select multiple image segments from the weather texture image based on the identified cut positions and intervals. The processor 1300 may identify a simulation period according to the characteristics of the current weather and the depth range based on the criterion information for simulating the weather effect. The processor 1300 may simulate the image segments corresponding to the first depth range and the image segments corresponding to the second depth range in the image. The processor 1300 may overlap one of the multiple image segments corresponding to the first depth range and one of the multiple image segments corresponding to the second depth range on the image together.

[0240] The processor 1300 may simulate a weather effect in the image from which weather objects have been removed.

[0241] The processor 1300 may simulate a weather effect in the reference image corresponding to the current time. The processor 1300 may select a reference image corresponding to the current time from among the reference images received from the server 2000. The processor 1300 may simulate a weather effect in the selected reference image.

[0242] The processor 1300 may change the color of the selected reference image based on the color change path information. For example, the processor 1300 may gradually change the color of the reference image corresponding to afternoon based on the color change path information between the reference image corresponding to afternoon and the reference image corresponding to evening. The processor 1300 may simulate a weather effect in the reference image with the color changed.

[0243] The sensor 1400 can detect the state of the device 1000 or the state near the device 1000, and send the detected information to the processor 1300.

[0244] The sensor 1400 may include at least one of a magnetic sensor 1410, an acceleration sensor 1420, a temperature / humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, a position sensor (such as GPS) 1460, a barometric pressure sensor 1470, a proximity sensor 1480, or an RGB sensor (or illuminance sensor) 1490, but is not limited thereto. A person of ordinary skill in the art can understand the functions of the sensors from their names, and thus their detailed descriptions are not provided herein.

[0245] The communicator 1500 may include one or more elements for communicating with the server 2000. For example, the communicator 1500 may include a short-range wireless communicator 1510, a mobile communicator 1520, and a broadcast receiver 1530.

[0246] The short-range wireless communicator 1510 may include, for example, a Bluetooth communicator, a Bluetooth Low Energy (BLE) communicator, a Near Field Communication (NFC) communicator, a Wireless Local Area Network (WLAN) (or Wi-Fi) communicator, a Zigbee communicator, an Infrared Data Association (IrDA) communicator, a Wi-Fi Direct (WFD) communicator, an Ultra-Wideband (UWB) communicator, and an Ant+ communicator, but is not limited thereto.

[0247] The mobile communicator 1520 transmits radio signals to and receives radio signals from at least one of a base station, an external terminal device, or a server in a mobile communication network. Here, the radio signals may include various types of data based on the transmission and reception of voice call signals, video call signals, or text / multimedia messages.

[0248] The broadcast receiver 1530 receives a broadcast signal and / or broadcast-related information from an external source through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Depending on the implementation, the device 1000 may not include the broadcast receiver 1530.

[0249] The communicator 1500 can send to and receive from the server 2000 the information required to simulate weather effects in an image.

[0250] The A / V input unit 1600 is used to input an audio signal or a video signal, and may include a camera 1610 and a microphone 1620. The camera 1610 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or an image capture mode. The image captured through the image sensor may be processed by the processor 1300 or a separate image processor.

[0251] The image frame processed by the camera 1610 may be stored in the memory 1700 or transmitted to the outside through the communicator 1500. According to the implementation of the device 1000, two or more cameras 1610 may be included.

[0252] The microphone 1620 receives an external sound signal and processes it into electrical voice data. For example, the microphone 1620 may receive a sound signal from an external device or a user. The microphone 1620 may use various noise cancellation algorithms to cancel the noise that occurs when receiving the external sound signal.

[0253] The memory 1700 may store a program for processing and controlling the operation of the processor 1300, and store data input to or output from the device 1000.

[0254] The memory 1700 may include at least one type of storage medium such as a flash memory, a hard disk, a multimedia card micro, a memory card (e.g., a secure digital (SD) or extreme digital (XD) card), a random access memory (RAM), a static RAM (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0255] The program stored in the memory 1700 may be classified into a plurality of modules based on its functions, for example, a user interface (UI) module 1710, a touch screen module 1720, and a notification module 1730.

[0256] The UI module 1710 may provide a dedicated UI or GUI connected to the device 1000 for each application, for example. The touch screen module 1720 may detect a touch gesture of a user on the touch screen and send information about the touch gesture to the processor 1300. The touch screen module 1720 according to an embodiment of the present invention may recognize and analyze a touch code. The touch screen module 1720 may be implemented as independent hardware including a controller.

[0257] The notification module 1730 may generate a signal for giving a notification of an event of the device 1000.

[0258] Device 1000 may perform some or all of the functions of server 2000 described herein. For example, device 1000 may perform functions such as generating spatial information based on an image, generating object recognition information based on an image, and setting criteria for simulating weather effects. Additionally, device 1000 may store various types of information required for simulating weather effects, such as spatial information, object recognition information, criteria information, and weather texture images. In this case, device 1000 may be a high-performance device.

[0259] Figure 33 FIG. is a schematic diagram showing an example in which an external device 3000 sets criteria for simulating weather effects according to an embodiment of the present disclosure, and device 1000 receives information about the set criteria through an external DB 4000 and provides a weather effect in an image.

[0260] Reference Figure 33 , the external device 3000 may set criteria for simulating a weather effect in an image through the server 2000, and the device 1000 may receive information required for simulating the weather effect from the external DB 4000 to simulate the weather effect in the image.

[0261] The external device 3000 may provide an image to which a weather effect is to be applied to the server 2000 through a specific GUI displayed on the screen of the external device 3000, and set criteria for simulating the weather effect. For example, the external device 3000 may set criteria for simulating the weather effect, the external device 3000 may not provide an image to the server 2000, and may select an image stored in the server 2000.

[0262] The server 2000 may generate information for simulating the weather effect based on the setting values input by the external device 3000 through the GUI. The information for simulating the weather effect may include, for example, an image, a processed image, a weather texture image, spatial information, object recognition information, and criteria information, but the information for simulating the weather effect is not limited thereto. The information for simulating the weather effect may include the above-mentioned parameters regarding Figure 19 of.

[0263] The external device 3000 may provide an image to the server 2000 based on the web and receive information for simulating the weather effect in the image from the server 2000. The information for simulating the weather effect may be data in a metadata format.

[0264] The external device 3000 may provide the image and information for simulating the weather effect received from the server 2000 to the external DB 4000, and the external DB 4000 may store the image and information for simulating the weather effect.

[0265] Device 1000 can provide an image ID to external DB 4000 and request information for simulating weather effects, and receive images and information for simulating weather effects. Device 1000 can provide weather effects in the image based on preset criteria by using the information for simulating weather effects.

[0266] External device 3000 can be, for example, a smartphone, a tablet computer, a PC, a smart TV, a mobile phone, a PDA, a laptop computer, a media player, a micro server, a GPS device, an e-book reader, a digital broadcast receiver, a navigation system, a kiosk, an MP3 player, a digital camera, a household appliance, or another mobile or non-mobile computing device, but is not limited thereto. External device 3000 can include Figure 32 components of, but is not limited thereto.

[0267] External DB 4000 can be a server for storing and managing information for simulating weather effects.

[0268] Figure 34 is a schematic diagram showing an example in which external device 3000 sets criteria for simulating weather effects and device 1000 receives information about the set criteria through server 2000 and provides weather effects in the image.

[0269] External device 3000 can provide an image to which a weather effect is to be applied to server 2000 through a specific GUI displayed on the screen of external device 3000, and set criteria for simulating weather effects. For example, external device 3000 can set criteria for simulating weather effects. External device 3000 can not provide an image to server 2000 and can select an image stored in server 2000.

[0270] Server 2000 can generate information for simulating weather effects based on the set values input by external device 3000 through the GUI. The information for simulating weather effects can include, for example, images, processed images, weather texture images, spatial information, object recognition information, and criteria information, but the information for simulating weather effects is not limited thereto. The information for simulating weather effects can include the above-mentioned parameter information about Figure 19 Server 2000 can store images and information for simulating weather effects.

[0271] Device 1000 can provide an image ID to server 2000 and request information for simulating weather effects, and receive images and information for simulating weather effects. Device 1000 can provide weather effects in the image based on preset criteria by using the information for simulating weather effects.

[0272] Figure 35It is a schematic diagram showing that an external device 3000 sets criteria for simulating weather effects through a server 2000 according to an embodiment of the present disclosure, and a device 1000 receives information about the criteria through an external DB 4000 and provides a weather effect in an image as an example.

[0273] The external device 3000 can provide an image to which a weather effect is to be applied to the server 2000 through a specific GUI displayed on the screen of the external device 3000 and set criteria for simulating the weather effect. For example, the external device 3000 can set the criteria for simulating the weather effect through the Figure 20 GUI 20 shown in. The external device 3000 may not provide an image to the server 2000 and may select an image stored in the server 2000.

[0274] The server 2000 can generate information for simulating the weather effect based on the setting values input by the external device 3000 through the GUI. The information for simulating the weather effect may include, for example, an image, a processed image, a weather texture image, spatial information, object recognition information, and criteria information, but is not limited thereto. The information for simulating the weather effect may include the above-mentioned parameter information regarding Figure 19 this.

[0275] The server 2000 can provide an image and information for simulating the weather effect to the external DB 4000, and the external DB 4000 can store the image and information for simulating the weather effect.

[0276] The device 1000 can provide an image ID to the external DB 4000 and request information for simulating the weather effect, and receive the image and information for simulating the weather effect from the external DB 4000. The device 1000 can provide a weather effect in the image based on a preset criterion by using the information for simulating the weather effect.

[0277] Embodiments of the present invention can be implemented in the form of a computer-readable recording medium, which includes instructions executable by a computer, for example, program modules executable by a computer. The computer-readable recording medium can be any available medium accessible by a computer, and examples thereof include all volatile, non-volatile, separable, and inseparable media. The computer-readable recording medium can include a computer storage medium and a communication medium. Examples of the computer storage medium include all volatile, non-volatile, separable, and inseparable media implemented by using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Examples of the communication medium generally can include computer-readable instructions, data structures, and other data such as modulated data signals of program modules.

[0278] As used herein, the suffix "unit" or "-er" may indicate a hardware component such as a processor or a circuit and / or a software component executed by a hardware component such as a processor.

[0279] Throughout the disclosure, "at least one of a, b, or c" means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variants thereof.

[0280] The foregoing description of the present invention is for illustrative purposes, and those of ordinary skill in the art will understand that various changes can be made in form and detail without departing from the scope of the present invention. Accordingly, it should be understood that the embodiments of the disclosure described herein should be considered in a descriptive sense only and not for purposes of limitation. For example, each component described as a single type can be implemented in a distributed manner, and conversely, components described as distributed can be implemented in a combined manner.

[0281] The scope of the present invention is defined by the following claims rather than the detailed description, and it should be understood that all modifications from the claims and their equivalents are included within the scope of the present invention.

Claims

1. A method for a device to provide a weather effect in a first image, the method comprising: obtaining a plurality of weather texture images of different depth ranges to show the weather effect based on the depth of an object in the first image; and sequentially overlapping a plurality of image segments of a weather texture image among the plurality of weather texture images on the object among the objects in the first image, wherein the interval between regions corresponding to the plurality of image segments in the weather texture image is determined based on the weather characteristics of the object among the objects in the first image.

2. The method according to claim 1, further comprising: receiving spatial information about the depth of the object in the first image from a server, wherein the spatial information is generated by the server by applying the first image as an input to an artificial intelligence model.

3. The method according to claim 1, wherein the plurality of weather texture images include a first weather texture image and a second weather texture image, the first weather texture image being for showing a weather effect corresponding to a first depth range, the second weather texture image being for showing a weather effect corresponding to a second depth range, the first depth range corresponding to at least one of the depths of the object in the first image, and the second depth range corresponding to at least one of the depths of the object in the first image.

4. The method according to claim 3, wherein the overlapping includes sequentially overlapping a plurality of first image segments corresponding to the first depth range from the first weather texture image on the first image, and sequentially overlapping a plurality of second image segments corresponding to the second depth range from the second weather texture image on the first image.

5. The method according to claim 4, wherein a first period in which the plurality of first image segments are overlapped is different from a second period in which the plurality of second image segments are overlapped.

6. The method according to claim 4, wherein the positions of the plurality of first image segments and the plurality of second image segments are determined based on the weather characteristics of the object among the objects in the first image.

7. The method according to claim 3, wherein a first size of a weather object indicating a weather effect corresponding to the first depth range in the first weather texture image is different from a second size of a weather object indicating a weather effect corresponding to the second depth range in the second weather texture image.

8. A device for providing a weather effect in a first image, the device comprising: a display; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to obtain a plurality of weather texture images of different depth ranges to show the weather effect based on the depth of an object in the first image, and sequentially overlap a plurality of image segments of a weather texture image among the plurality of weather texture images on the object among the objects in the first image. Among them, the intervals between the regions corresponding to the multiple image segments in the weather texture image are determined based on the weather characteristics of the objects among the objects of the first image.

9. The apparatus according to claim 8, further comprising: a communication interface, wherein the processor is further configured to receive spatial information about the depth of the objects of the first image from a server, and wherein the spatial information is generated by the server by applying the first image as an input to an artificial intelligence model.

10. The apparatus according to claim 8, wherein, the multiple weather texture images include a first weather texture image and a second weather texture image. The first weather texture image is used to show a weather effect corresponding to a first depth range, and the second weather texture image is used to show a weather effect corresponding to a second depth range. The first depth range corresponds to at least one of the objects in the first image, and the second depth range corresponds to at least one of the depths of the objects in the first image.

11. The apparatus according to claim 10, wherein, the processor is further configured to execute the one or more instructions to sequentially overlap multiple first image segments corresponding to the first depth range from the first weather texture image on the first image, and sequentially overlap multiple second image segments corresponding to the second depth range from the second weather texture image on the first image.

12. The apparatus according to claim 11, wherein, a first period during which the multiple first image segments are overlapped is different from a second period during which the multiple second image segments are overlapped.

13. The apparatus according to claim 11, wherein, the positions of the multiple first image segments and the multiple second image segments are determined based on the weather characteristics of the objects among the objects of the first image.

14. The apparatus according to claim 10, wherein, a first size of a weather object indicating a weather effect corresponding to the first depth range in the first weather texture image is different from a second size of a weather object indicating a weather effect corresponding to the second depth range in the second weather texture image.

15. A non-transitory computer-readable recording medium having recorded thereon a computer program which, when executed by at least one processor, causes the at least one processor to: obtain multiple weather texture images of different depth ranges to show weather effects based on the depth ranges of the objects of a first image; and sequentially overlap multiple image segments of the weather texture images among the multiple weather texture images on the objects among the objects of the first image; wherein, the intervals between the regions corresponding to the multiple image segments in the weather texture image are determined based on the weather characteristics of the objects among the objects of the first image.

Images