Method and system for measuring fine dust on basis of image

Abstract

Disclosed are a method and a system for measuring fine dust on the basis of an image. The disclosed system for measuring fine dust, according to an embodiment, comprises: an image capturing apparatus that captures an image of a target location, converts all or a part of the captured image to generate a converted image or converted data, and transmits the converted image or the converted data; and a fine dust measurement apparatus that receives the converted image or the converted data from the image capturing apparatus, and inputs the received converted image or converted data into a deep learning model to train the deep learning model to output the concentration of fine dust in the target location.

Description

Image-based fine dust measurement method and system

[0001] An embodiment of the present invention relates to a fine dust measurement technology.

[0002] Fine dust is a type of dust whose particles are so small they are invisible to the naked eye. Research suggests that this fine dust overrides the body's defense system, causing various health problems, including cardiovascular, respiratory, and cerebrovascular diseases. The International Agency for Research on Cancer (IARC), a division of the World Health Organization (WHO), has designated fine dust as a Group 1 carcinogen. Due to these risks, fine dust is being analyzed as a contributing factor to the decline in economic activity among members of society.

[0003] Meanwhile, CCTV cameras are installed throughout buildings, roads, and other areas for crime prevention, security, and vehicle surveillance. CCTV footage allows for real-time monitoring of local conditions. However, CCTV footage is not currently being utilized to measure fine dust concentrations. RGB images captured by CCTV cameras are more sensitive to changes in lighting, humidity, and climate conditions than to fine dust concentration itself, making it difficult to accurately measure the impact of fine dust.

[0004] The disclosed embodiment is intended to provide a new technique for measuring fine dust and a system.

[0005] A fine dust measurement system according to one embodiment disclosed includes an image capturing device that captures a target location, converts all or part of the captured image to generate a converted image or converted data, and transmits the converted image or converted data; and a fine dust measurement device that receives the converted image or converted data from the image capturing device, inputs the received converted image or converted data into a deep learning model, and trains the deep learning model to output a fine dust concentration of the target location.

[0006] The above image capturing device can generate a converted image by performing a first type of conversion that converts all or part of the captured image into an image with different characteristics, or can generate converted data by performing a second type of conversion that converts all or part of the captured image into data of a different form.

[0007] The above image capturing device can determine which type of conversion to perform among the first type of conversion and the second type of conversion depending on the degree of network communication between the image capturing device and the fine dust measuring device.

[0008] The above video capturing device may determine to perform the first type of conversion when the network communication level is equal to or higher than a first reference level set in advance, and may determine to perform the second type of conversion when the network communication level is lower than a second reference level set lower than the first reference level.

[0009] The above image capturing device generates a plurality of converted images of different types according to a preset conversion method during the first type of conversion and transmits the converted images to the fine dust measuring device, and the fine dust measuring device inputs the plurality of converted images into the deep learning model to output fine dust concentration prediction values ​​for the plurality of converted images, and trains the deep learning model so that the difference between each fine dust concentration prediction value and the correct answer value is minimized.

[0010] The above fine dust measuring device extracts a predicted value closest to the correct answer value among the predicted values ​​of the fine dust concentration, matches the type of the converted image corresponding to the extracted predicted value with at least one of the environmental information and climate information at the time of shooting, and stores the matched and stored information, and can share the matched and stored information with the image shooting device.

[0011] The above image capturing device can obtain at least one of environmental information and climate information when capturing a target location, determine whether to convert all or part of a captured image into a certain type of image based on at least one of the acquired environmental information and climate information, generate a converted image by converting all or part of the captured image into the determined type of image, and transmit the converted image to the fine dust measuring device.

[0012] The above fine dust measuring device can input the converted image into a pre-trained deep learning model to output the fine dust concentration of the target location.

[0013] A method for measuring fine dust according to one embodiment of the present disclosure is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: photographing a target location; converting all or part of the photographed image to generate a converted image or converted data; and transmitting the converted image or the converted data to a fine dust measuring device having a deep learning model, wherein the fine dust measuring device inputs the converted image or the converted data into a deep learning model to train the deep learning model to output a fine dust concentration of the target location.

[0014] The above generating step may generate a converted image by performing a first type of conversion that converts all or part of the photographed image into an image with different characteristics, or may generate converted data by performing a second type of conversion that converts all or part of the photographed image into data of a different form.

[0015] The above generating step may include a step of determining to perform the first type of conversion when the network communication level with the fine dust measuring device is equal to or higher than a first reference level set in advance, and a step of determining to perform the second type of conversion when the network communication level is lower than a second reference level set lower than the first reference level.

[0016] The above fine dust measurement method may further include a step of acquiring at least one of environmental information and climate information when photographing a target location; and a step of determining whether to convert all or part of a photographed image into a certain type of image based on at least one of the acquired environmental information and climate information when performing the first type of conversion.

[0017] A method for measuring fine dust according to another embodiment disclosed is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: acquiring a photographed image of a target location; converting all or part of the photographed image to generate a converted image; and inputting the converted image into a deep learning model to train the deep learning model to output a fine dust concentration of the target location.

[0018] The step of generating the above-described converted image may generate a plurality of converted images of different types according to a preset conversion method, and the step of training may input the plurality of converted images into the deep learning model to output a fine dust concentration prediction value for each of the plurality of converted images, and train the deep learning model so that the difference between each fine dust concentration prediction value and the correct answer value is minimized.

[0019] The above fine dust measurement method may further include a step of extracting a predicted value closest to the correct answer value among the predicted fine dust concentration values; and a step of matching and storing the type of the converted image corresponding to the extracted predicted value with at least one of environmental information and climate information at the time of shooting the captured image.

[0020] According to the disclosed embodiment, by converting an image captured by an image capture device into a converted image or converted data having characteristics sensitive to fine dust and then inputting it into a deep learning model to output the fine dust concentration of a target location, the fine dust concentration of a specific location can be easily measured without expensive fine dust measurement equipment.

[0021] Furthermore, since fine dust concentration can be measured simply by capturing an image of the target location, fine dust concentration can be easily checked using CCTV cameras installed throughout the area. Furthermore, if users want to know the fine dust concentration at a specific location while moving around, they can simply provide an image of that location.

[0022] FIG. 1 is a diagram showing the configuration of an image-based fine dust measurement system according to one embodiment of the present invention.

[0023] Figure 2 is a block diagram showing the configuration of an image capturing device according to one embodiment of the present invention.

[0024] FIG. 3 is a diagram showing a state of extracting a region of interest from an image according to one embodiment of the present invention.

[0025] FIG. 4 is a diagram showing first to sixth transformed images generated by performing a first type of transformation on an image of a region of interest in one embodiment of the present invention.

[0026] Figure 5 is a diagram schematically showing a learning method of a deep learning model in one embodiment of the present invention.

[0027] Figure 6 is a block diagram showing the configuration of a fine dust measuring device according to another embodiment of the present invention.

[0028] Figure 7 is a flowchart for explaining the learning process of a deep learning model in one embodiment of the present invention.

[0029] Figure 8 is a flowchart for explaining the inference process of a deep learning model in one embodiment of the present invention.

[0030] FIG. 9 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.

[0031] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, these are merely examples and the present invention is not limited thereto.

[0032] In describing embodiments of the present invention, if a detailed description of a known technology related to the present invention is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of their functions in the present invention, and this may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification. The terminology used in the detailed description is only for the purpose of describing embodiments of the present invention and should not be limited in any way. Unless clearly used otherwise, the singular form includes the plural form. In this description, expressions such as "comprises" or "having" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other features, numbers, steps, operations, elements, parts or combinations thereof other than those described.

[0033] Additionally, while terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms may be used to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component."

[0034] FIG. 1 is a diagram showing the configuration of an image-based fine dust measurement system according to one embodiment of the present invention.

[0035] Referring to FIG. 1, a fine dust measurement system (100) may include an image capturing device (102) and a fine dust measurement device (104). The image capturing device (102) may be communicatively connected to the fine dust measurement device (104) via a communication network (150).

[0036] Here, the communication network (150) may include the Internet, one or more local area networks, wide area networks, a cellular network, a mobile network, other types of networks, or a combination of these networks.

[0037] The video recording device (102) can capture images of a location (target location) where the concentration of fine dust is to be measured. The video recording device (102) can also be fixedly installed at the location where the concentration of fine dust is to be measured. For example, the video recording device (102) can be fixedly installed on a building or road, like a CCTV camera.

[0038] However, this is not limited to this, and the video recording device (102) may be provided to be movable via a separate means of transportation. For example, the video recording device (102) may be installed on a vehicle and provided to move while filming a location where the concentration of fine dust is to be measured.

[0039] In addition, the video recording device (102) may be a mobile device (e.g., a smart phone, a tablet PC, smart glasses, a wearable device, etc.) that people carry around and have a camera, and may include other electronic devices capable of recording videos.

[0040] FIG. 2 is a block diagram showing the configuration of an image capturing device (102) according to one embodiment of the present invention. Referring to FIG. 2, the image capturing device (102) may include a capturing module (111), an area of ​​interest extraction module (113), a conversion module (115), and a communication module (117).

[0041] The photographing module (111) may include a camera. The photographing module (111) may photograph a location where the concentration of fine dust is to be measured through the camera. In one embodiment, the image captured by the photographing module (111) may be an RGB (Red, Green, Blue) image.

[0042] The region of interest extraction module (113) can extract a region of interest from a captured image. The region of interest extraction module (113) can extract a boundary region of each object included in the captured image, and if a change in pixel value within the boundary region exceeds a preset threshold value, the region including the object can be extracted as a region of interest. Since the technology for extracting the boundary region of an object within an image is already known, a detailed description thereof will be omitted.

[0043] FIG. 3 is a diagram illustrating a state of extracting a region of interest from an image according to one embodiment of the present invention. Referring to FIG. 3, the region of interest extraction module (113) can extract the boundary area of ​​each object included in the captured image, i.e., a mountain, a vinyl house, a tree, and a temporary building.

[0044] Here, pixels within the boundary area of ​​the mountain and the tree are generally displayed as dark, so the change in pixel value is small. On the other hand, pixels within the boundary area of ​​the vinyl house are brighter than the surrounding background, and the change in pixel value is large due to the internal steel frame that is revealed at regular intervals. In addition, pixels within the boundary area of ​​the temporary building (a building with a green roof in Fig. 2) show a large change in pixel value due to the uneven color and curvature of the roof. Therefore, the region of interest extraction module (113) can extract parts of the temporary building and the vinyl house from the captured image as regions of interest (ROI), respectively.

[0045] The transformation module (115) can perform transformation to use the image of the region of interest as input to the deep learning model (104a). That is, the transformation module (115) can transform the image of the region of interest (i.e., RGB image) into an image with different characteristics (first type transformation) or into a different form of data (second type transformation). The transformation module (115) can determine which type of transformation, the first type or the second type, to perform on the image of the region of interest depending on the network communication environment between the image capturing device (102) and the fine dust measuring device (104).

[0046] In one embodiment, the conversion module (115) may perform a first type of conversion on the image of the region of interest when the network communication level between the image capturing device (102) and the fine dust measuring device (104) is equal to or higher than a first preset reference level. That is, when the network communication level is equal to or higher than the first preset reference level, the conversion module (115) may convert the image of the region of interest into an image of a different characteristic. Here, the network communication level may be calculated based on one or more of the network communication delay, network bandwidth, and network communication speed between the image capturing device (102) and the fine dust measuring device (104).

[0047] Specifically, the conversion module (115) can convert an image of a region of interest into one or more images having characteristics sensitive to fine dust. That is, the conversion module (115) can convert an image of a region of interest into one or more images suitable for measuring the concentration of fine dust.

[0048] In one embodiment, the conversion module (115) can generate a first converted image by converting an RGB (Red, Green, Blue) image of a region of interest into an HSV (Hue, Saturation, Value) image. Here, since the HSV image has pure color information in H (Hue), it can classify colors more easily than an RGB image, and can be less affected by changes in illuminance or shade by adjusting the V (Value) value.

[0049] When the conversion module (115) converts the image of the region of interest into an HSV image, the HSV image itself may be used as the first conversion image, or the H (Hue) and S (Saturation) channels in the HSV image may be used as the first conversion image, or only the S (Saturation) channel in the HSV image may be used as the first conversion image. In this case, the H (Hue) or S (Saturation) value changes depending on the concentration of fine dust.

[0050] In addition, the conversion module (115) can generate a second converted image by converting the RGB image of the region of interest by applying the DCP (Dark Channel Prior) technique. By applying the DCP (Dark Channel Prior) technique to the RGB image of the region of interest, it is possible to remove haze (a phenomenon in which the object appears hazy due to the propagation of light being obstructed by materials existing between the object and the camera) present in the RGB image of the region of interest. At this time, the image conversion module (106) can extract the transmittance feature of the RGB image of the region of interest based on the DCP (Dark Channel Prior) technique and use it as the second converted image. The second converted image has different turbidity depending on the concentration of fine dust.

[0051] In addition, the transformation module (115) can generate a third transformed image by transforming the RGB image of the region of interest by applying a Gabor filter. When the Gabor filter is applied to the RGB image of the region of interest, an edge can be extracted from the region of interest, and the clarity of the edge varies depending on the amount of fine dust.

[0052] In addition, the transformation module (115) can generate a fourth transformed image by applying a Sobel filter to the RGB image of the region of interest to transform the image. The Sobel filter is a filter that filters according to the directionality of frequencies in an image, and can detect diagonal edges more sensitively than horizontal and vertical edges.

[0053] In addition, the transformation module (115) can generate a fifth transformed image by applying the LBP (Local Binary Pattern) technique to the RGB image of the region of interest to transform the image. The LBP (Local Binary Pattern) technique extracts the features of the image by converting the pixel values ​​around each pixel of the image into binary numbers (0 or 1), and the binary numbers are generated based on the relative brightness difference between the central pixel and the neighboring pixels. In other words, if the neighboring pixel is larger than the central pixel, it is processed as binary 1, and if it is smaller, it is processed as binary 0.

[0054] In addition, the transformation module (115) can generate a sixth transformed image by applying a Laplacian filter to the RGB image of the region of interest to transform the image. The Laplacian filter performs second-order differentiation in the horizontal and vertical directions of the image, thereby finding the center of the edge portion (i.e., the inflection point of the pixel value change).

[0055] FIG. 4 is a diagram illustrating first to sixth transformed images generated by performing a first type of transformation on an image of a region of interest in one embodiment of the present invention. The transformation module (115) can generate the first to sixth transformed images, respectively, based on the image of the region of interest during the learning stage of the deep learning model (104a).

[0056] Meanwhile, the transformation module (115) can determine which image transformation to perform when performing the first type of transformation on the image of the region of interest in the inference step for measuring the concentration of fine dust after the training of the deep learning model (104a) described below is completed, based on the input additional information. Here, the additional information may be environmental information or climate information of the location for measuring the concentration of fine dust. In one embodiment, the additional information may be, but is not limited to, temperature, humidity, illuminance, and wind speed of the location for measuring the concentration of fine dust.

[0057] The conversion module (115) can perform a second type of conversion on the image of the region of interest when the network communication level between the video capturing device (102) and the fine dust measuring device (104) is below a preset second reference level. Here, the second reference level is a level lower than the first reference level.

[0058] The transformation module (115) can transform an image of a region of interest into a data value through a second type of transformation. In one embodiment, the transformation module (115) can calculate the RMS (Root Mean Square) Contrast for the image of the region of interest and use it as the first transformation data. The RMS (Root Mean Square) Contrast can be defined as the standard deviation of image pixel intensities. The RMS Contrast can be expressed by the mathematical expression 1.

[0059] (Equation 1)

[0060]

[0061] I ij : Intensity of pixel (i,j) of an image of size M×N

[0062] avg(I): Average intensity of all pixels in the image

[0063] Additionally, the transformation module (115) can calculate entropy for an image of a region of interest and use it as second transformation data. Here, entropy quantifies information contained in an image and is related to image texture, and can be calculated using the following mathematical expression 2.

[0064] (Equation 2)

[0065]

[0066] p i : Probability that pixel intensity is equal to i

[0067] M: maximum intensity of the image

[0068] Here, since the RMS (Root Mean Square) Contrast or entropy is a single numerical value, even if the network communication level between the image capturing device (102) and the fine dust measuring device (104) is below the preset second reference level, the conversion data for the image of the area of ​​interest can be transmitted without difficulty.

[0069] Meanwhile, although the transformation module (115) is described here as performing the first type transformation or the second type transformation on the image of the region of interest, it is not limited thereto and the first type transformation or the second type transformation may also be performed on the captured image.

[0070] The communication module (117) can communicate with the fine dust measuring device (104). The communication module (117) can measure the degree of network communication between the image capturing device (102) and the fine dust measuring device (104). In addition, the communication module (117) can transmit a converted image or converted data for an image of a region of interest generated by the conversion module (115) to the fine dust measuring device (104).

[0071] The fine dust measuring device (104) can receive a converted image or converted data from the image capturing device (102) and measure the fine dust concentration of the corresponding location based on the converted image or converted data. Here, the converted image or converted data may be for a captured image or an area of ​​interest of the captured image. The fine dust measuring device (104) can measure the fine dust concentration based on deep learning technology. Accordingly, the fine dust measuring device (104) can include a deep learning model (104a).

[0072] The deep learning model (104a) may be trained to receive a converted image or converted data as input and predict the concentration of fine dust at a given location based on the input converted image or converted data. The deep learning model (104a) may be trained to minimize the difference between the predicted concentration of fine dust and the actually measured concentration of fine dust (i.e., the correct answer value).

[0073] Here, the correct answer value can be obtained from a device (e.g., a light scattering sensor) that actually measures the fine dust concentration at the target location (the location where fine dust is to be measured). The fine dust concentration measuring device can measure the fine dust concentration at the target location at the time of the target location capture. However, this is not limited to this, and the correct answer value can also be obtained from an external organization by receiving fine dust concentration data for the area including the target location.

[0074] When a converted image is received from an image capture device (102), a learning method of a deep learning model (104a) will be described with reference to FIG. 5. FIG. 5 is a diagram schematically illustrating a learning method of a deep learning model (104a) in one embodiment of the present invention.

[0075] Referring to FIG. 5, the fine dust measuring device (104) can receive the first to sixth converted images for the captured image (or image of the region of interest) from the image capturing device (102), respectively.

[0076] The fine dust measuring device (104) can input the first to sixth transformed images into the deep learning model (104a), respectively. The first to sixth transformed images are not input simultaneously, but may be input sequentially for each learning epoch. In one embodiment, the deep learning model (104a) may be an image-based classification model. For example, the deep learning model (104a) may be ResNet 50, but is not limited thereto, and various other deep learning models may be used.

[0077] The deep learning model (104a) can receive a first converted image as input and output a first fine dust concentration prediction value. The deep learning model (104a) can receive a second converted image as input and output a second fine dust concentration prediction value. Similarly, the deep learning model (104a) can receive a third to a sixth converted image as input and output a third to a sixth fine dust concentration prediction value, respectively. The deep learning model (104a) can be trained to compare the first to the sixth fine dust concentration prediction values ​​with the correct answer value (actually measured fine dust concentration value) and minimize the difference between the first to the sixth fine dust concentration prediction values.

[0078] In one embodiment, during the learning process, the fine dust measuring device (104) extracts a predicted value that is closest to the correct answer value among the first to sixth fine dust concentration predicted values, and stores the type of the converted image corresponding to the extracted predicted value by matching it with at least one of environmental information (e.g., illuminance of the target location, etc.) and climate information (e.g., temperature, humidity, and wind speed of the target location, etc.) at the time of shooting the image.

[0079] For example, if the sixth fine dust concentration prediction value is closest to the correct answer value, the fine dust measuring device (104) can match the type of converted image corresponding to the sixth fine dust concentration prediction value with environmental information and climate information at the time of image capture and store it. Through this, it is possible to establish a standard for which type of image (i.e., which image among the first converted image to the sixth converted image) the captured image is converted into based on which environmental information and climate information. The fine dust measuring device (104) can share the above-mentioned matched information with the image capture device (102).

[0080] The fine dust measuring device (104) can receive converted images of images taken at various locations and time periods when training a deep learning model (104a), and train the deep learning model (104a) using the received converted images as training data.

[0081] In addition, the fine dust measuring device (104) can receive conversion data (e.g., RMS Contrast value or entropy value for a captured image (or region of interest image)) from the image capturing device (102) and train a deep learning model (104a). The deep learning model (104a) can output a fine dust concentration prediction value by taking the conversion data as input. The deep learning model (104a) can be trained so that the difference between the fine dust concentration prediction value and the correct answer value is minimized.

[0082] Meanwhile, although one deep learning model (104a) is described here as an example, it is not limited thereto, and a deep learning model may be provided for each converted image or converted data. For example, a corresponding deep learning model may be provided for each converted image or converted data, such as a first deep learning model that takes a first converted image as input and outputs a first fine dust concentration prediction value, a second deep learning model that takes a second converted image as input and outputs a second fine dust concentration prediction value, etc.

[0083] According to the disclosed embodiment, by converting an image captured by an image capture device into a converted image or converted data having characteristics sensitive to fine dust and then inputting it into a deep learning model to output the fine dust concentration of a target location, the fine dust concentration of a specific location can be easily measured without expensive fine dust measurement equipment.

[0084] Furthermore, since fine dust concentration can be measured simply by capturing an image of the target location, fine dust concentration can be easily checked using CCTV cameras installed throughout the area. Furthermore, if users want to know the fine dust concentration at a specific location while moving around, they can simply provide an image of that location.

[0085] As used herein, the term "module" may refer to a functional and structural combination of hardware for implementing the technical concepts of the present invention and software for operating the hardware. For example, the term "module" may refer to a logical unit of a given code and hardware resources for executing the given code, and does not necessarily refer to physically connected code or a single type of hardware.

[0086] In addition, in FIGS. 1 and 2, it has been described that the region of interest of the captured image is extracted from the image capturing device (102) and the image of the region of interest is converted into an image of a different characteristic, but it is not limited thereto, and this process can also be performed in the fine dust measuring device (104). FIG. 6 is a block diagram showing the configuration of the fine dust measuring device (104) according to another embodiment of the present invention.

[0087] Referring to FIG. 6, the fine dust measuring device (104) may include an image acquisition module (121), an area of ​​interest extraction module (123), an image conversion module (125), and a fine dust measuring module (127).

[0088] The image acquisition module (121) can acquire an image of a location where the concentration of fine dust is to be measured. In one embodiment, the image acquisition module (121) can acquire an image from an image capturing device. The image capturing device may be a CCTV camera installed on a building or road for purposes such as crime prevention, security, or vehicle surveillance, or a mobile device equipped with a camera, but is not limited thereto.

[0089] The region of interest extraction module (123) can extract a region of interest from an acquired image. The region of interest extraction module (123) extracts a boundary region of each object included in the acquired image, and if a change in pixel value within the boundary region exceeds a preset threshold value, the region including the object can be extracted as a region of interest. The region of interest extraction module (123) is identical or similar to the region of interest extraction module (113) of FIG. 2, so a detailed description thereof will be omitted.

[0090] The image conversion module (125) can convert an image of a region of interest (i.e., an RGB image) into an image with different characteristics. The image conversion module (125) can convert the image of the region of interest into one or more images having characteristics sensitive to fine dust. For example, the image conversion module (125) can generate one or more of the first converted image to the sixth converted image through image conversion for the image of the region of interest.

[0091] The image transformation module (125) can determine, based on input additional information (such as environmental information or climate information), whether to perform image transformation on the image of the area of ​​interest in the inference step for measuring the concentration of fine dust after the learning of the deep learning model (127a) described later is completed.

[0092] The fine dust measurement module (127) receives a converted image from the image conversion module (125) and can measure the fine dust concentration of the location where the image was captured based on the converted image. The fine dust measurement module (127) may include a deep learning model (127a). The deep learning model (127a) receives a converted image as input and can be trained to predict the fine dust concentration of the location based on the input converted image. Since the deep learning model (127a) is the same as or similar to the deep learning model (104a) of FIG. 1, a detailed description thereof will be omitted.

[0093] Figure 7 is a flowchart illustrating the training process of a deep learning model according to one embodiment of the present invention. While the illustrated flowchart divides the method into multiple steps, at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into substeps, or accompanied by one or more additional steps not shown.

[0094] Referring to Fig. 7, the fine dust measuring device (104) can acquire an image of a location where the concentration of fine dust is to be measured (S 101). Next, the fine dust measuring device (104) can extract an area of ​​interest from the acquired image (S 103).

[0095] Next, the fine dust measuring device (104) can generate multiple converted images by converting the image of the region of interest according to a preset conversion method (S 105). In one embodiment, the fine dust measuring device (104) can perform image conversion on the image of the region of interest to generate the first converted image to the sixth converted image, respectively.

[0096] Next, the fine dust measuring device (104) can input a plurality of transformed images into the deep learning model (104a) to train the deep learning model (104a) (S 107). Here, the deep learning model (104a) can output a fine dust concentration prediction value for each of the plurality of transformed images, and can be trained to minimize the difference between the output fine dust concentration prediction value and the correct answer value.

[0097] Next, the fine dust measuring device (104) extracts a predicted value that is closest to the correct value among the predicted values ​​of fine dust concentration for each of a plurality of converted images (S 109), and stores the type of the converted image corresponding to the extracted predicted value by matching it with at least one of the environmental information and climate information at the time of shooting the acquired image (S 111).

[0098] Figure 8 is a flowchart illustrating the inference process of a deep learning model according to one embodiment of the present invention. While the illustrated flowchart depicts the method as divided into multiple steps, at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into substeps, or performed with one or more additional steps not depicted.

[0099] Referring to Fig. 8, the fine dust measuring device (104) can acquire an image of a location (target location) where the concentration of fine dust is to be measured (S 201). Next, the fine dust measuring device (104) can extract an area of ​​interest from the acquired image (S 203).

[0100] Next, the fine dust measuring device (104) can acquire one or more of environmental information and climate information when capturing the acquired image (S 205). Next, the fine dust measuring device (104) can determine what type of image to convert the image of the region of interest into based on one or more of the environmental information and climate information (S 207).

[0101] Next, the fine dust measuring device (104) can generate a converted image by converting the image of the region of interest into a predetermined type of image (S 209). Next, the fine dust measuring device (104) can input the converted image into a pre-trained deep learning model (104a) to output the fine dust concentration of the target location (S 211).

[0102] Meanwhile, here, it has been described that it is determined what type of image to convert the image of the region of interest into based on at least one of environmental information and climate information, and that the converted image of the determined type is generated, but it is not limited thereto, and the image of the region of interest is converted according to a preset conversion method to generate all of the first converted image to the sixth converted image, and the first converted image to the sixth converted image is input to a pre-trained deep learning model (104a) to output the fine dust concentration, and the fine dust concentration of the target location can be calculated by averaging the output fine dust concentration values.

[0103] FIG. 9 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

[0104] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be an image capturing device (102). Additionally, the computing device (12) may be a fine dust measuring device (104).

[0105] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.

[0106] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.

[0107] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

[0108] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0109] While representative embodiments of the present invention have been described in detail above, those skilled in the art will appreciate that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims set forth below but also by equivalents thereof.

Claims

1. An image capturing device that captures a target location, converts all or part of the captured image to generate a converted image or converted data, and transmits the converted image or converted data; and A fine dust measurement system including a fine dust measurement device that receives the converted image or converted data from the video shooting device, inputs the received converted image or converted data into a deep learning model, and trains the deep learning model to output the fine dust concentration of the target location.

2. In claim 1, The above video recording device, A fine dust measurement system that generates a converted image by performing a first type of conversion that converts all or part of the above-mentioned photographed image into an image having different characteristics, or generates converted data by performing a second type of conversion that converts all or part of the above-mentioned photographed image into data of a different form.

3. In claim 2, The above video recording device, A fine dust measurement system that determines which type of conversion to perform among the first type of conversion and the second type of conversion depending on the degree of network communication between the video shooting device and the fine dust measurement device.

4. In claim 3, The above video recording device, A fine dust measurement system, wherein it is determined to perform the first type of conversion when the network communication level is equal to or higher than a first reference level set above, and it is determined to perform the second type of conversion when the network communication level is lower than a second reference level set below the first reference level.

5. In claim 2, The above video recording device, In the conversion of the first type, a plurality of different types of conversion images are generated according to the preset conversion method and transmitted to the fine dust measuring device. The above fine dust measuring device, A fine dust measurement system which inputs each of the plurality of transformed images into the deep learning model to output a fine dust concentration prediction value for each of the plurality of transformed images, and trains the deep learning model to minimize the difference between each fine dust concentration prediction value and the correct answer value.

6. In claim 5, The above fine dust measuring device, A fine dust measurement system which extracts a predicted value closest to the correct answer value among the predicted values ​​of the fine dust concentration, matches the type of a converted image corresponding to the extracted predicted value with at least one of the environmental information and climate information at the time of shooting and stores it, and shares the matched and stored information with the image shooting device.

7. In claim 6, The above video recording device, A fine dust measurement system which acquires at least one of environmental information and climate information when shooting a target location, determines whether to convert all or part of a shot image into a certain type of image based on at least one of the acquired environmental information and climate information, converts all or part of the shot image into the determined type of image to generate a converted image, and transmits the converted image to the fine dust measurement device.

8. In claim 7, The above fine dust measuring device, A fine dust measurement system that inputs the above-mentioned converted image into a pre-trained deep learning model and outputs the fine dust concentration of the target location.

9. One or more processors, and A method performed on a computing device having a memory storing one or more programs executed by one or more processors, Step 1: Take a picture of the target location; A step of converting all or part of a photographed image to generate a converted image or converted data; and A step of transmitting the above-mentioned converted image or the above-mentioned converted data to a fine dust measuring device having a deep learning model, The above fine dust measuring device is a fine dust measuring method that inputs the converted image or converted data into a deep learning model and trains the deep learning model to output the fine dust concentration of the target location.

10. In claim 9, The above generating steps are: A method for measuring fine dust, wherein a first type of transformation is performed to convert all or part of the above-described photographed image into an image having different characteristics, thereby generating a transformed image, or a second type of transformation is performed to convert all or part of the above-described photographed image into data of a different form, thereby generating transformed data.

11. In claim 10, The above generating steps are: A method for measuring fine dust, comprising the step of determining to perform the first type of conversion when the level of network communication with the fine dust measuring device is equal to or higher than a first reference level set in advance, and determining to perform the second type of conversion when the level of network communication is lower than a second reference level set lower than the first reference level.

12. In claim 10, The above fine dust measurement method is, A step of acquiring at least one of environmental information and climate information when shooting a target location; and A method for measuring fine dust, further comprising a step of determining whether to convert all or part of a photographed image into a certain type of image based on at least one of the acquired environmental information and climate information during the conversion of the first type.

13. One or more processors, and A method performed on a computing device having a memory storing one or more programs executed by one or more processors, A step of acquiring a photographed image of a target location; A step of generating a converted image by converting all or part of the above-mentioned photographed image; and A method for measuring fine dust, comprising the step of inputting the above-mentioned converted image into a deep learning model and training the deep learning model to output the fine dust concentration of the target location.

14. In claim 13, The step of generating the above conversion image generates multiple conversion images of different types according to a preset conversion method, The above-mentioned training step is a method for measuring fine dust, wherein the deep learning model is trained to input each of the plurality of converted images into the deep learning model to output a fine dust concentration prediction value for each of the plurality of converted images, and the difference between each fine dust concentration prediction value and the correct answer value is minimized.

15. In claim 14, The above fine dust measurement method is, A step of extracting a predicted value closest to the correct answer value among the predicted values ​​of the fine dust concentration; and A method for measuring fine dust, further comprising a step of storing the type of a transformed image corresponding to the extracted predicted value by matching it with at least one of environmental information and climate information at the time of shooting the photographed image.

Images