Apparatus for controlling salt spray based on freezing detection using deep learning and method therefor

Abstract

The method for controlling brine spraying according to the present invention comprises the steps of: a collecting unit collecting weather information corresponding to a road surface, a road surface image captured of the road surface, and road surface noise generated on the road surface by a vehicle; a detection unit analyzing the weather information, the road surface image, and the road surface noise through a deep learning model to derive the condition of the road surface; and a control unit outputting a control command to spray brine onto the road surface according to the condition of the road surface.

Description

Technology Field

[0001] The present invention relates to brine spray control, and more specifically, to an apparatus for controlling brine spray based on freezing detection using deep learning and a method for the same. Background Technology

[0002] Black ice can form when the temperature in an area with water drops below freezing and the water freezes. In particular, black ice can occur when the temperature drops sharply after rain, causing standing rainwater to freeze before it can evaporate. Furthermore, black ice can form even without rain when the temperature drops rapidly in areas where water accumulates, such as around exposed outdoor drains. Prior art literature

[0003] Korean Patent Publication No. 2023-0083736 (Published June 12, 2023) The problem to be solved

[0004] The objective of the present invention is to provide an apparatus for controlling brine spray based on freezing detection using deep learning and a method for doing the same. means of solving the problem

[0005] The method for controlling brine spraying according to the present invention comprises the steps of: a collecting unit collecting weather information corresponding to a road surface, a road surface image captured of the road surface, and road surface noise generated on the road surface by a vehicle; a detection unit analyzing the weather information, the road surface image, and the road surface noise through a deep learning model to derive the condition of the road surface; and a control unit outputting a control command to spray brine onto the road surface according to the condition of the road surface.

[0006] The step of deriving the condition of the road surface includes the step of the deep learning model analyzing the weather information, the road surface image, and the road surface noise to derive an output vector representing a plurality of conditions of the road surface as a probability, and the step of the detection unit deriving one of a plurality of predefined conditions of the road surface as the condition of the road surface according to the probability of the output vector.

[0007] At least one of the plurality of states above includes a transient state which is an intermediate state between a wet state in which the road surface is wet and a frozen state in which black ice is formed on the road surface.

[0008] The step of outputting a control command to spray brine onto the road surface is characterized in that the control unit outputs a control command to spray brine onto the road surface through a brine spraying device if the condition of the road surface is in the transient state according to the probability of the output vector.

[0009] The step of deriving the condition of the road surface includes the step of the detection unit generating a merged input vector including the weather information, the road surface image, and the road surface noise, and inputting the generated merged input vector into a deep learning model; the step of the merged detection network of the deep learning model performing an operation in which weights learned for the merged input vector are applied to derive an output vector representing a plurality of predefined conditions of the road surface as probabilities; and the step of the detection unit deriving the condition of the road surface according to the probability of the output vector.

[0010] The step of deriving the state of the road surface comprises: a step in which the detection unit generates a first sub-input vector including the road surface noise and inputs the generated first sub-input vector into a deep learning model; a step in which the detection unit generates a second sub-input vector including the weather information and the road surface image and inputs the generated second sub-input vector into the deep learning model; a step in which the first sub-detection network of the deep learning model performs an operation in which weights learned for the first sub-input vector are applied to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities; a step in which the second sub-detection network of the deep learning model performs an operation in which weights learned for the second sub-input vector are applied to derive a second sub-output vector representing a plurality of predefined states of the road surface as probabilities; a step in which the merging module of the deep learning model merges the first sub-output vector and the first sub-output vector to generate an output vector representing a plurality of predefined states of the road surface as probabilities; and a step in which the detection unit [determines] the probability of the output vector Accordingly, it includes a step of deriving the condition of the road surface.

[0011] The device for controlling brine spraying according to the present invention includes a collection unit that collects weather information corresponding to a road surface, a road surface image taken of the road surface, and road surface noise generated on the road surface by a vehicle, a detection unit that analyzes the weather information, the road surface image, and the road surface noise through a deep learning model to derive the condition of the road surface, and a control unit that outputs a control command to spray brine onto the road surface according to the condition of the road surface.

[0012] The detection unit is characterized by the fact that when the deep learning model analyzes the weather information, the road surface image, and the road surface noise to derive an output vector representing a plurality of states of the road surface as a probability, it derives one of a plurality of predefined states of the road surface as the state of the road surface according to the probability of the output vector.

[0013] At least one of the above plurality of states includes a transient state which is an intermediate state between a wet state in which the road surface is wet and a frozen state in which black ice is formed on the road surface.

[0014] The control unit is characterized by outputting a control command to spray brine onto the road surface through a brine spraying device when the condition of the road surface is in the transient state according to the probability of the output vector.

[0015] The detection unit generates a merged input vector including the weather information, the road surface image, and the road surface noise, inputs the generated merged input vector into a deep learning model, and when the merged detection network of the deep learning model performs an operation in which weights learned for the merged input vector are applied to derive an output vector representing a plurality of predefined states of the road surface as probabilities, the state of the road surface is derived according to the probability of the output vector.

[0016] The detection unit generates a first sub-input vector including the road surface noise, inputs the generated first sub-input vector into a deep learning model, generates a second sub-input vector including the weather information and the road surface image, inputs the generated second sub-input vector into the deep learning model, the first sub-detection network of the deep learning model performs an operation in which weights learned for the first sub-input vector are applied to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities, the second sub-detection network of the deep learning model performs an operation in which weights learned for the second sub-input vector are applied to derive a second sub-output vector representing a plurality of predefined states of the road surface as probabilities, and the merging module of the deep learning model merges the first sub-output vector and the first sub-output vector to generate an output vector representing a plurality of predefined states of the road surface as probabilities, thereby deriving the state of the road surface according to the probability of the output vector. Effects of the invention

[0017] According to the present invention, by analyzing weather information, road surface images, and road surface noise to detect an intermediate state between a wet state and a frozen state, that is, a transient state immediately before black ice occurs, and by detecting the optimal time for brine injection and controlling the injection of brine, accidents caused by black ice can be prevented. Brief explanation of the drawing

[0018] FIG. 1 is a diagram illustrating a system for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a device for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the configuration of a deep learning model for detecting the condition of a road surface according to an embodiment of the present invention. FIG. 4 is a diagram illustrating the configuration of a deep learning model for detecting the condition of a road surface according to another embodiment of the present invention. FIG. 5 is a flowchart illustrating a method for training a deep learning model according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention. FIG. 7 is an example diagram of a hardware system for implementing a device for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention. Specific details for implementing the invention

[0019] In order to clarify the features and advantages of the means for solving the problem of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention illustrated in the attached drawings.

[0020] However, detailed descriptions of known functions or configurations that may obscure the essence of the invention are omitted in the following description and the attached drawings. Additionally, it should be noted that identical components throughout the drawings are indicated by the same reference numerals whenever possible.

[0021] Terms and words used in the following description and drawings should not be interpreted as being limited to their ordinary or dictionary meanings, but should be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor can appropriately define the concept of terms to best describe his invention. Accordingly, the embodiments described in this specification and the configurations illustrated in the drawings are merely the most preferred embodiments of the invention and do not represent all aspects of the technical spirit of the invention; therefore, it should be understood that various equivalents and modifications capable of replacing them may exist at the time of filing this application.

[0022] Furthermore, terms including ordinal numbers, such as first, second, etc., are used to describe various components and are used solely for the purpose of distinguishing one component from another, and are not used to limit said components. For example, without departing from the scope of the present invention, the second component may be named the first component, and similarly, the first component may be named the second component.

[0023] Furthermore, when it is stated that one component is "connected" or "joined" to another component, this implies that they may be connected or joined logically or physically. In other words, it should be understood that while a component may be directly connected or joined to another component, there may also be other components in between, or the connection may be indirect.

[0024] Furthermore, the terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. Additionally, terms such as “comprising” or “having” described in this specification are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0025] Additionally, terms such as “…part,” “…unit,” and “module” described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0026] Additionally, “one (a or an),” “one,” “the,” and similar terms may be used in the context describing the present invention (particularly in the context of the following claims) in a sense including both singular and plural, unless otherwise indicated in this specification or clearly contradicted by the context.

[0027] In addition, embodiments within the scope of the present invention include a computer-readable medium having or transmitting computer-executable instructions or data structures stored on a computer-readable medium. Such a computer-readable medium may be any available medium accessible by a general-purpose or special-purpose computer system. For example, such a computer-readable medium may include, but is not limited to, physical storage media such as RAM, ROM, EPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium accessible by a general-purpose or special-purpose computer system that can be used to store or transmit certain program code means in the form of computer-executable instructions, computer-readable instructions or data structures.

[0028] In the following description and claims, "network" is defined as one or more data links that enable the transmission of electronic data between computer systems and / or modules. When information is transmitted to or provided to a computer system through a network or other communication connection (wired, wireless, or a combination of wired and wireless), this connection may be understood as a computer-readable medium. Computer-readable instructions include instructions and data that, for example, cause a general-purpose computer system or a special-purpose computer system to perform a specific function or group of functions. Computer-executable instructions may be binary, intermediate format instructions, such as assembly language or even source code.

[0029] Furthermore, the present invention may be applied in a network computing environment having various types of computer system configurations, including personal computers, laptop computers, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers, etc. The present invention may also be executed in a distributed system environment in which both local and remote computer systems linked via a network via a wired data link, a wireless data link, or a combination of wired and wireless data links perform tasks. In a distributed system environment, program modules may be located in local and remote memory storage devices.

[0030] First, a system for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention will be described. FIG. 1 is a diagram illustrating a system for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a device for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the configuration of a deep learning model for detecting the condition of a road surface according to one embodiment of the present invention. FIG. 4 is a diagram illustrating the configuration of a deep learning model for detecting the condition of a road surface according to another embodiment of the present invention.

[0031] Referring to FIG. 1, a system according to an embodiment of the present invention includes a control server (10), a sensor device (20), a brine spraying device management server (30), and a brine spraying device (40).

[0032] The control server (10) collects weather information corresponding to the road surface (e.g., temperature, humidity, wind direction), a road surface image taken of the road surface, and road surface noise generated on the road surface by a vehicle from a sensor device (20) or a server (not shown) that provides external weather data, and analyzes the collected weather information, road surface image, and road surface noise through a deep learning model to detect the condition of the road surface. According to the present invention, the deep learning model can detect any one of a learned class, that is, a condition among a plurality of conditions of the road surface. Here, the plurality of conditions of the road surface include a dry condition (DRY) where the road surface is dry, a wet condition (WET) where the road surface is wet, a transient condition (ICY0) which is an intermediate state between the wet condition (WET) and the freezing condition (ICY1), and a freezing condition (ICY1) where black ice is formed on the road surface. Additionally, in the case of the freezing condition, it may be further divided into a plurality of classes depending on the degree of freezing.

[0033] The sensor device (20) may include a weather sensor for detecting weather information (e.g., temperature, humidity, wind direction), an image sensor for capturing a road surface image, and an audio sensor for recording road surface noise.

[0034] The brine spraying device (40) is a device for spraying brine onto the road surface, and the brine spraying device management server (30) is a server for controlling the brine spraying device (40).

[0035] Referring to FIG. 2, the control server (10) includes a learning unit (100), a collection unit (200), a detection unit (300), and a control unit (400).

[0036] The learning unit (100) is intended to generate a deep learning model through deep learning. Examples of deep learning models include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and Transformer. When the learning of the deep learning model is completed, the learning unit (100) provides the deep learning model to the detection unit (300).

[0037] A deep learning model includes multiple layers, and each of these layers performs multiple operations. Each result of a multiple operation in any given layer is weighted and passed to the next layer. This means that weights are applied to the result of the operation in the current layer and input into the operation of the next layer. Accordingly, the performance of multiple operations with weights applied between the multiple layers of a deep learning model is referred to as a weighted operation. Here, the multiple layers include one or more combinations of a fully-connected layer, a convolutional layer, a recurrent layer, a graph layer, and a pooling layer. Examples of the multiple operations include convolution operations, downsampling operations, upsampling operations, and operations based on activation functions. In particular, examples of activation functions include sigmoid, hyperbolic tangent (tanh), ELU (Exponential Linear Unit), ReLU (Rectified Linear Unit), Leakly ReLU, Maxout, Minout, and Softmax.

[0038] According to one embodiment, as illustrated in Fig. 3, the deep learning model includes a merged detection network (MDN). The merged detection network (MDN) is trained to generate a merged input vector including information, a road surface image, and road surface noise, receive the generated merged input vector, and perform weight operations on the merged input vector to derive an output vector representing a plurality of predefined states (DRY, WET, ICY0, ICY1…) of the road surface as probabilities.

[0039] According to another embodiment, as illustrated in FIG. 4, the deep learning model includes a first sub-detection network (SDN1), a second sub-detection network (SDN2), and a merging module (MM). The first sub-detection network (SDN1) receives a first sub-input vector containing road surface noise and performs a weighting operation on the first sub-input vector to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities. The second sub-detection network (SDN2) receives a second sub-input vector containing weather information and a road surface image and performs a weighting operation on the second sub-input vector to derive a second sub-output vector representing a plurality of predefined states of the road surface as probabilities. The merging module (MM) merges the first sub-output vector and the first sub-output vector to derive an output vector representing a plurality of predefined states of the road surface as probabilities.

[0040] The collection unit (200) is intended to collect weather information corresponding to the road surface (e.g., temperature, humidity, wind direction), a road surface image taken of the road surface, and road surface noise generated on the road surface by a vehicle from a sensor device (20) or a server (not shown) that provides external weather data.

[0041] The detection unit (300) is intended to derive the condition of the road surface by analyzing weather information, road surface images, and road surface noise through a deep learning model. At this time, when the deep learning model derives an output vector representing multiple conditions of the road surface as probabilities by analyzing weather information, road surface images, and road surface noise, the detection unit (300) derives one of the multiple conditions of the road surface as the condition of the road surface according to the probability corresponding to the multiple conditions (DRY, WET, ICY0, ICY1…) of the output vector.

[0042] The control unit (400) determines whether the condition of the road surface is excessive, and if the condition of the road surface is excessive, outputs a control command to spray brine onto the road surface through the brine spraying device (40). Accordingly, the output control command is transmitted to the brine spraying device management server (30), and the brine spraying device management server (30) controls the brine spraying device (40) to spray brine onto the road surface.

[0043] Next, a method for training a deep learning model according to an embodiment of the present invention will be described. FIG. 5 is a flowchart illustrating a method for training a deep learning model according to an embodiment of the present invention.

[0044] Referring to FIG. 5, the learning unit (100) loads training data to train a deep learning model in step S110. That is, the training data is generated and stored in advance. This training data is intended to enable the deep learning model to distinguish between multiple classes, that is, multiple states, that are pre-set. Here, the multiple states include a dry state (DRY) where the road surface is dry, a wet state (WET) where the road surface is wet, an intermediate state (ICY0) between the wet state (WET) and the freezing state (ICY1), and a freezing state (ICY1) where black ice is formed on the road surface. Additionally, in the case of the freezing state, it may be further divided into multiple classes depending on the degree of freezing.

[0045] The training data includes an input and a label corresponding to the input. The input is a vector comprising at least one of weather information corresponding to the road surface, a road surface image captured by the road surface, and road surface noise generated on the road surface by a vehicle, according to the network (merged detection network, first sub-detection network, second sub-detection network). The label represents one of a plurality of road surface states determined by actual measurement. In this case, a one-hot encoding vector may be used. For example, if the input is for a dry state, the label is "[Dry, Wet, Excessive, Freezing] = [1, 0, 0, 0]", if the input is for a wet state, the label is "[Dry, Wet, Excessive, Freezing] = [0, 1, 0, 0]", if the input is for an excess state, the label is "[Dry, Wet, Excessive, Freezing] = [0, 0, 1, 0]", and if the input is for a freezing state, the label is "[Dry, Wet, Excessive, Freezing] = [0, 0, 0, 1]".

[0046] When training data is prepared, the training unit (100) produces an output for the input through the deep learning model to be trained in step S120. Here, the input and output may be composed of vectors (or tensors).

[0047] According to one embodiment of step S120, when the deep learning model is composed of a merged detection network (MDN), the learning unit (100) generates a merged input vector including weather information, a road surface image, and road surface noise, and inputs the generated merged input vector to the merged detection network (MDN) of the deep learning model according to one embodiment. At this time, the driving noise is encoded through the Mel-Frequency Cepstral Coefficient (MFCC) to generate the merged input vector, the weather information is converted into time series data and encoded to generate the merged input vector, and the image is generated by extracting a feature map to generate the merged input vector. Accordingly, with reference to FIG. 3, the merged detection network (MDN) performs weighting operations on the merged input vector to derive an output vector representing a plurality of predefined states of the road surface as probabilities.

[0048] According to another embodiment of step S120, when the deep learning model is composed of a first sub-detection network (SDN1), a second sub-detection network (SDN2), and a merging module (MM), the learning unit (100) generates a first sub-input vector including road surface noise and inputs the generated first sub-input vector to the first sub-detection network (SDN1) of the deep learning model according to another embodiment. Additionally, the learning unit (100) generates a second sub-input vector including weather information and a road surface image and inputs the generated second sub-input vector to the second sub-detection network (SDN2) of the deep learning model according to another embodiment. Accordingly, with reference to FIG. 4, the first sub-detection network (SDN1) performs a weighting operation on the first sub-input vector to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities. Additionally, the second sub-detection network (SDN2) performs a weighting operation on the second sub-input vector to derive a second sub-output vector representing multiple predefined states of the road surface as probabilities. Subsequently, the merging module (MM) merges the first sub-output vector and the first sub-output vector to derive an output vector representing multiple predefined states of the road surface as probabilities.

[0049] Accordingly, the learning unit (100) can calculate a loss representing the difference between the output vector and the label through a loss function at step S130.

[0050] Next, the learning unit (100) performs optimization by modifying the weights of the deep learning model so that the loss derived through the loss function in step S140 is minimized.

[0051] The aforementioned steps S120 through S150 are performed repeatedly using multiple different training data, and the weights of the deep learning model are updated repeatedly according to these repetitions. This repetition is carried out until the loss becomes less than or equal to a preset target value. Accordingly, the learning unit (100) determines in step S150 whether the loss calculated earlier (S130) is less than or equal to a preset target value, and if the loss is less than or equal to a preset target value, the learning of the deep learning model is completed in step S160.

[0052] Next, a method for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention will be described. FIG. 6 is a flowchart illustrating a method for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention.

[0053] Referring to FIG. 6, the collection unit (200) collects weather information corresponding to the road surface (e.g., temperature, humidity, wind direction), a road surface image taken of the road surface, and road surface noise generated on the road surface by a vehicle from a sensor device (20) or a server (not shown) that provides external weather data in step S210.

[0054] Next, the detection unit (300) analyzes weather information, road surface images, and road surface noise through a deep learning model in step S220 to derive the condition of the road surface. At this time, when the deep learning model analyzes the weather information, road surface images, and road surface noise to derive an output vector representing multiple conditions of the road surface as probabilities, the detection unit (300) derives one of the multiple conditions of the road surface as the condition of the road surface according to the probability corresponding to the multiple conditions of the output vector. At this time, the detection unit (300) selects the condition that has the highest probability among the probabilities of the output vector that is greater than or equal to a threshold value. For example, the threshold value is assumed to be 80%. When the output vector is "[Dry, Wet, Excessive, Freezing] = [0.02, 0.02, 0.90, 0.06]", the probabilities of the dry condition, wet condition, excessive condition, and freezing condition, respectively, represent 2%, 2%, 90%, and 6%. Accordingly, the detection unit (300) can determine that the condition of the road surface is an overactive state based on the probability corresponding to the multiple states of the output vector. As another example, when the output vector is "[Dry, Wet, Overactive, Freezing] = [0.02, 0.90, 0.02, 0.06]", the probabilities of the dry state, wet state, overactive state, and freezing state, respectively, represent 2%, 90%, 2%, and 6%. Accordingly, the detection unit (300) can determine that the condition of the road surface is a wet state based on the probability corresponding to the multiple states of the output vector. The operation of this S220 step is explained in more detail as follows.

[0055] According to one embodiment of step S220, as illustrated in FIG. 3, when the deep learning model is composed of a merged detection network (MDN), the detection unit (300) generates a merged input vector including weather information, a road surface image, and road surface noise, and inputs the generated merged input vector to the merged detection network (MDN) of the deep learning model according to one embodiment. Accordingly, the merged detection network (MDN) performs a weighting operation on the merged input vector to derive an output vector representing a plurality of predefined states of the road surface as probabilities. Then, the detection unit (300) derives one of the plurality of states of the road surface as a state of the road surface according to the probability corresponding to the plurality of states of the output vector.

[0056] According to another embodiment of step S220, as illustrated in FIG. 4, when the deep learning model is composed of a first sub-detection network (SDN1), a second sub-detection network (SDN2), and a merging module (MM), the detection unit (300) generates a first sub-input vector including road surface noise and inputs the generated first sub-input vector to the first sub-detection network (SDN1) of the deep learning model according to another embodiment. Additionally, the detection unit (300) generates a second sub-input vector including weather information and a road surface image and inputs the generated second sub-input vector to the second sub-detection network (SDN2) of the deep learning model according to another embodiment. Accordingly, the first sub-detection network (SDN1) performs a weighting operation on the first sub-input vector to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities. Additionally, the second sub-detection network (SDN2) performs weighting operations on the second sub-input vector to derive a second sub-output vector representing multiple predefined states of the road surface as probabilities. Subsequently, the merging module (MM) merges the first sub-output vector and the first sub-output vector to derive an output vector representing multiple predefined states of the road surface as probabilities. At this time, the merging module (MM) applies a defined rule (e.g., linear merging, Ties-merging) to the first sub-output vector and the first sub-output vector and the weights to select one weight and derives the output vector through fine-tuning. Then, the detection unit (300) derives one of the multiple states of the road surface as the state of the road surface according to the probability corresponding to the multiple states of the output vector.

[0057] Next, the control unit (400) determines whether the condition of the road surface is an overactive state in step S230. That is, the detection unit (300) determines whether an overactive state has been selected based on the probability of the output vector. For example, assuming the threshold for determining the state is 80%, if the output vector is "[Dry, Wet, Overactive, Freezing] = [0.02, 0.02, 0.90, 0.06]", the probabilities of the dry state, wet state, overactive state, and freezing state, respectively, are 2%, 2%, 90%, and 6%, and accordingly, this refers to the case where an overactive state was selected in step S220.

[0058] If the result of determining in step S230 is that the road surface condition is not an excessive condition, the aforementioned steps S210 to S230 are repeated.

[0059] If, as a result of the determination in step S230, the road surface condition is in an excessive state, the control unit (400) outputs a control command to spray brine onto the road surface through the brine spraying device (40) in step S240. Accordingly, the output control command is transmitted to the brine spraying device management server (30), and the brine spraying device management server (30) controls the brine spraying device (40) to spray brine onto the road surface. In this way, the present invention can prevent accidents caused by black ice by detecting the excessive state, which is the state immediately before black ice occurs, as the optimal time for brine spraying and controlling the spraying of brine.

[0060] FIG. 7 is an example diagram of a hardware system for implementing a device for controlling brine spray based on freezing detection using deep learning according to an embodiment of the present invention.

[0061] As illustrated in FIG. 7, a hardware system (2000) according to one embodiment of the present invention may have a configuration including a processor unit (2100), a memory interface unit (2200), and a peripheral device interface unit (2300).

[0062] Each component within such a hardware system (2000) may be an individual component or integrated into one or more integrated circuits, and each of these components may be combined by a bus system (not shown).

[0063] Here, in the case of a bus system, it is an abstraction representing any one or more individual physical buses, communication lines / interfaces, and / or multi-drop or point-to-point connections connected by appropriate bridges, adapters, and / or controllers.

[0064] The processor unit (2100) performs the role of executing various software modules stored in the memory unit (2210) by communicating with the memory unit (2210) through the memory interface unit (2200) to perform various functions in the hardware system.

[0065] Here, in the memory unit (2210), each of the components including the learning unit (100), collection unit (200), detection unit (300), and control unit (400) described above with reference to FIG. 2 can be stored in the form of a software module, and an operating system (OS) can be additionally stored. The components including the learning unit (100), collection unit (200), detection unit (300), and control unit (400) can be loaded into the processor unit (2100) and executed.

[0066] Each component including the learning unit (100), collection unit (200), detection unit (300), and control unit (400) described above may be implemented in the form of a software module or a hardware module executed by a processor, or in a form that combines a software module and a hardware module.

[0067] In this way, software modules, hardware modules, or combinations of software and hardware modules executed by a processor can be implemented as actual hardware systems (e.g., computer systems).

[0068] In the case of an operating system (e.g., embedded operating systems such as I-OS, Android, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks), it includes various procedures, instruction sets, software components, and / or drivers that control and manage general system tasks (e.g., memory management, storage device control, power management, etc.) and plays a role in facilitating communication between various hardware modules and software modules.

[0069] For reference, the memory section (2210) may include a memory hierarchy that includes, but is not limited to, a cache, main memory, and secondary memory, and such a memory hierarchy may be implemented through any combination of, for example, RAM (e.g., SRAM, DRAM, DDRAM), ROM, FLASH, magnetic and / or optical storage devices [e.g., disk drive, magnetic tape, CD (compact disk) and DVD (digital video disc), etc.].

[0070] The peripheral device interface section (2300) performs the role of enabling communication between the processor section (2100) and the peripheral device.

[0071] Here, the peripheral device is intended to provide different functions to the hardware system (2000), and in one embodiment of the present invention, for example, a communication unit (2310) may be included.

[0072] Here, the communication unit (2310) performs the role of providing communication functions with other devices, and for this purpose, it may include, for example, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, and a memory, and may include known circuits that perform this function, which are not limited thereto.

[0073] Communication protocols supported by the communication unit (2310) include, for example, Wireless LAN (WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced), 5G communication system, Wireless Mobile Broadband Service (WMBS), Bluetooth, RFID (Radio Frequency Identification), infrared Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, Near Field Communication (NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, etc. may be included.In addition, wired communication networks may include wired LAN (Local Area Network), wired WAN (Wide Area Network), Power Line Communication (PLC), USB communication, Ethernet, serial communication, optical / coaxial cables, etc., and any protocol capable of providing a communication environment with other devices, which is not limited thereto, may be included.

[0074] In a hardware system (2000) according to one embodiment of the present invention, each component stored in the memory unit (2210) in the form of a software module performs an interface with the communication unit (2310) through the memory interface unit (2200) and the peripheral device interface unit (2300) in the form of instructions executed by the processor unit (2100).

[0075] As explained above, although this specification contains details of a number of specific embodiments, they should not be understood as limiting the scope of any invention or claimables, but rather as descriptions of features that may be characteristic of a specific embodiment of a specific invention. In the context of individual embodiments, specific features described in this specification may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. Furthermore, while features may operate in a specific combination and be described as initially claimed, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be changed to a sub-combination or a variation of the sub-combination.

[0076] Likewise, although operations are depicted in the drawings in a specific order, this should not be understood as requiring that such operations be performed in that specific or sequential order depicted to obtain a desirable result, or that all depicted operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components of the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together into a single software product or packaged into multiple software products.

[0077] Specific embodiments of the subject matter described herein have been explained. Other embodiments fall within the scope of the following claims. For example, the operations cited in the claims may be performed in different orders and still achieve desirable results. As an example, the process illustrated in the accompanying drawings does not necessarily require the specific illustrated order or sequential order to obtain desirable results. In specific embodiments, multitasking and parallel processing may be advantageous.

[0078] The description provided herein presents the best mode of the invention and offers examples to explain the invention and to enable those skilled in the art to manufacture and use the invention. The specification thus written is not intended to limit the invention to the specific terms presented. Accordingly, although the invention has been described in detail with reference to the examples above, those skilled in the art may make modifications, changes, and variations to these examples without departing from the scope of the invention.

[0079] Therefore, the scope of the present invention should not be determined by the described embodiments but by the claims. Industrial applicability

[0080] The present invention relates to an apparatus and a method for controlling brine spraying based on freezing detection using deep learning. The present invention analyzes weather information, road surface images, and road surface noise to detect an intermediate state between a wet state and a frozen state, and detects the optimal timing for brine spraying to control the spraying of brine, thereby preventing traffic accidents caused by black ice. Therefore, the present invention has industrial applicability as it is not only sufficiently feasible for commercial sale or business but also clearly implementable in reality. Explanation of the symbols

[0081] 10: Control Server 20: Sensor device 30: Salt sprayer management server 40: Salt sprayer 100: Learning Department 200: Collection Department 300: Detector 400: Control unit

Claims

Claim 1 A method for controlling brine spraying, characterized by comprising: a step in which a collection unit collects weather information corresponding to a road surface, a road surface image captured of the road surface, and road surface noise generated on the road surface by a vehicle; a step in which a detection unit analyzes the weather information, the road surface image, and the road surface noise through a deep learning model to derive the condition of the road surface; and a step in which a control unit outputs a control command to spray brine onto the road surface according to the condition of the road surface. Claim 2 A method for controlling brine spraying according to claim 1, wherein the step of deriving the condition of the road surface comprises: a step in which the deep learning model analyzes the weather information, the road surface image, and the road surface noise to derive an output vector representing a plurality of conditions of the road surface as probabilities; and a step in which the detection unit derives one of a plurality of predefined conditions of the road surface as the condition of the road surface according to the probability of the output vector. Claim 3 A method for controlling brine spraying according to paragraph 2, characterized in that at least one of the plurality of states includes a transient state which is an intermediate state between a wet state in which the road surface is wet and a frozen state in which black ice is formed on the road surface. Claim 4 A method for controlling brine spraying according to claim 3, wherein the step of outputting a control command to spray brine onto the road surface is characterized in that the control unit outputs a control command to spray brine onto the road surface through a brine spraying device when the condition of the road surface is in the transient state according to the probability of the output vector. Claim 5 A method for controlling brine spraying according to claim 1, wherein the step of deriving the condition of the road surface comprises: a step in which the detection unit generates a merged input vector including the weather information, the road surface image, and the road surface noise, and inputs the generated merged input vector to a deep learning model; a step in which the merged detection network of the deep learning model performs an operation in which weights learned for the merged input vector are applied to derive an output vector representing a plurality of predefined conditions of the road surface as probabilities; and a step in which the detection unit derives the condition of the road surface according to the probability of the output vector. Claim 6 In claim 1, the step of deriving the condition of the road surface comprises: a step in which the detection unit generates a first sub-input vector including the road surface noise and inputs the generated first sub-input vector to a deep learning model; a step in which the detection unit generates a second sub-input vector including the weather information and the road surface image and inputs the generated second sub-input vector to the deep learning model; a step in which the first sub-detection network of the deep learning model performs an operation to which learned weights are applied to the first sub-input vector to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities; a step in which the second sub-detection network of the deep learning model performs an operation to which learned weights are applied to the second sub-input vector to derive a second sub-output vector representing a plurality of predefined states of the road surface as probabilities; and a step in which the merging module of the deep learning model merges the first sub-output vector and the first sub-output vector to generate an output vector representing a plurality of predefined states of the road surface as probabilities. A method for controlling brine spraying, characterized by including the step of the detection unit deriving the condition of the road surface according to the probability of the output vector. Claim 7 An apparatus for controlling brine spraying, characterized by comprising: a collection unit that collects weather information corresponding to a road surface, a road surface image captured of the road surface, and road surface noise generated on the road surface by a vehicle; a detection unit that analyzes the weather information, the road surface image, and the road surface noise through a deep learning model to derive the condition of the road surface; and a control unit that outputs a control command to spray brine onto the road surface according to the condition of the road surface. Claim 8 An apparatus for controlling brine spraying according to claim 7, wherein, when the deep learning model analyzes the weather information, the road surface image, and the road surface noise to derive an output vector representing a plurality of states of the road surface as a probability, the detection unit derives one of a plurality of predefined states of the road surface as the state of the road surface according to the probability of the output vector. Claim 9 An apparatus for controlling brine spraying according to claim 8, characterized in that at least one of the plurality of states includes a transient state which is an intermediate state between a wet state in which the road surface is wet and a freezing state in which black ice is formed on the road surface. Claim 10 A device for controlling brine spraying according to claim 9, characterized in that the control unit outputs a control command to spray brine onto the road surface through a brine spraying device when the condition of the road surface is in the transient state according to the probability of the output vector. Claim 11 An apparatus for controlling brine spraying according to claim 7, wherein the detection unit generates a merged input vector including the weather information, the road surface image, and the road surface noise, inputs the generated merged input vector into a deep learning model, and when the merged detection network of the deep learning model performs an operation in which weights learned for the merged input vector are applied to derive an output vector representing a plurality of predefined states of the road surface as probabilities, the state of the road surface is derived according to the probability of the output vector. Claim 12 In claim 7, the detection unit generates a first sub-input vector including the road surface noise, inputs the generated first sub-input vector to a deep learning model, generates a second sub-input vector including the weather information and the road surface image, inputs the generated second sub-input vector to the deep learning model, the first sub-detection network of the deep learning model performs an operation in which weights learned for the first sub-input vector are applied to derive a first sub-output vector representing a plurality of predefined states of the road surface as probabilities, the second sub-detection network of the deep learning model performs an operation in which weights learned for the second sub-input vector are applied to derive a second sub-output vector representing a plurality of predefined states of the road surface as probabilities, and when the merging module of the deep learning model merges the first sub-output vector and the first sub-output vector to generate an output vector representing a plurality of predefined states of the road surface as probabilities, the state of the road surface is derived according to the probability of the output vector, characterized in that it controls brine spraying. Device.

Images