Apparatus and method for estimating and managing road surface type using sound wave signals

Abstract

According to various embodiments, an electronic device for classifying road surfaces using sonic signals and a method for classifying road surfaces using the same are disclosed. Also disclosed are devices and methods for managing road surfaces by road surface classification. Meanwhile, a method for providing infrastructure on a road for implementing the road surface classification method is disclosed. Various other embodiments are possible.

Description

[Technical Field]

[0001] The present disclosure relates to an apparatus for estimating the type of road surface using a sound wave signal and a method for classifying and managing road surfaces using the same, and more particularly to an apparatus for classifying sound wave signals reflected on a road surface using an artificial neural network and controlling the road surface or a moving object based on the classified type of road surface, and a method for managing road surfaces using the same. [Background technology]

[0002] Generally, a ground vehicle moving on the ground controls its acceleration and deceleration in accordance with the friction coefficient of the ground it is moving on, i.e., the road surface. Therefore, it is important to accurately estimate the road surface friction coefficient from the aspects of stability control and maximum maneuverability control.

[0003] The rapid increase in black ice accidents in winter is a good example of the need for road friction coefficient estimation technology, as they occur due to sudden changes in the road friction coefficient in situations where the ice is not readily apparent.

[0004] In addition, regenerative braking technology is essential for electric vehicles, which have been commercialized in recent years, to improve energy efficiency. However, there is a growing need for technology that can estimate the friction coefficient of the road surface in advance in order to ensure driving stability when regenerative braking is applied.

[0005] Conventionally, methods that utilize vehicle dynamic information and sensing information have been mainly used to estimate such road surface conditions or road surface friction coefficients based on the road surface conditions.

[0006] In the case of the method using vehicle dynamics information, the measurement information from various sensors mounted on the vehicle is substituted into a vehicle dynamics model for estimation. This method has the drawback of decreasing accuracy in situations that deviate from the predetermined modeling, and also has the limitation that the road friction coefficient cannot be estimated in advance because the measurement can be performed after passing over the road surface.

[0007] In addition, in the case of methods based on electromagnetic wave sensors such as video information, it is possible to estimate the friction coefficient of the road surface remotely, but this requires expensive sensor equipment and corresponding signal processing equipment, and there is a limit to the results, as they may vary depending on the sensor's installation position and direction.

[0008] Meanwhile, road surface estimation technology using acoustic information is also being actively discussed, but conventionally, the focus has been on technology that estimates road surface conditions based on the friction noise between the ground and tires, which not only lacks accuracy but also has the limitation of not being able to check the road surface conditions ahead before passing over the road.

[0009] Therefore, existing methods have limitations in that they are unable to predict road surface conditions in advance, or the prediction process is uneconomical and inaccurate, making it difficult to efficiently solve the above-mentioned problems. Summary of the Invention [Problem to be solved by the invention]

[0010] In order to solve the above problems, the present disclosure aims to provide an apparatus and method for estimating the type of road surface using a sound wave signal.

[0011] Another object of the present disclosure is to provide an apparatus and method for controlling and managing road surfaces in real time through estimation of road surface type.

[0012] On the other hand, the problems to be solved by the present disclosure are not limited to the above problems, and problems not mentioned will be clearly understood by a person having ordinary skill in the art to which the disclosure contained in the present disclosure belongs from this specification and the accompanying drawings. [Means for solving the problem]

[0013] According to various embodiments of the present disclosure, an electronic device for classifying road surfaces using sonic signals includes a transceiver configured to transmit and receive sonic signals toward a target road surface spaced a first distance from the electronic device, an atmospheric sensor, and at least one processor electrically connected to the transceiver and the atmospheric sensor, wherein the at least one processor is configured to: use the transceiver to transmit sonic signals toward a target road surface spaced a first distance from the electronic device; use the transceiver to receive reflections of the sonic signals from the target road surface; use the atmospheric sensor to obtain atmospheric information associated with the sonic signals; obtain first data for the received reflections; generate second data by correcting the first data based on the atmospheric information; obtain third data related to frequency-domain information of the second data based on the second data; and determine a type of the target road surface based on the third data and a road surface classification artificial neural network, wherein the road surface classification artificial neural network can be trained with a frequency-domain data set generated based on sonic signals reflected from a road surface at a second distance different from the first distance.

[0014] According to an embodiment of the present disclosure, the second data may be generated by correcting the first data based on the atmospheric information and the first distance.

[0015] The first distance may also be estimated based on the time at which the acoustic signal is transmitted and the time at which the reflected signal is received.

[0016] According to an embodiment of the present disclosure, the third data may be acquired by performing a Short-Time Fourier Transformation (STFT) on the second data.

[0017] According to one embodiment of the present disclosure, the at least one processor may be configured to generate a signal to control a road surface management device installed on the target road surface based on the determined type of the target road surface.

[0018] The road surface maintenance device according to various embodiments of the present disclosure may include a heat wire or a saline spray device.

[0019] According to one embodiment of the present disclosure, the at least one processor may be configured to determine whether a preset weather condition is met, and if the preset weather condition is met, generate a signal to control the road surface management device.

[0020] The at least one processor may also be configured to check whether the type of target road surface determined at a first time point has changed at a second time point, and if the first class determined as the type of the target road surface at the first time point is different from the second class determined as the type of the target road surface at the second time point, to determine whether or not to generate a control signal for a device installed on the target road surface based on the type of target road surface determined at a third time point.

[0021] According to one embodiment of the present disclosure, the type of the target road surface may be determined every first cycle, and the at least one processor may be configured to determine the type of the target road surface every second cycle if the type of the target road surface is determined to be a first class.

[0022] According to one embodiment of the present disclosure, the electronic device may further include at least one of an IR sensor that acquires temperature information of the target road surface or a vision sensor that acquires image information of the target road surface, and the at least one processor may be configured to determine the type of the road surface based on the temperature information or the image information.

[0023] According to various embodiments of the present disclosure, a method for classifying a road surface using a sonic signal performed by an electronic device includes transmitting a sonic signal toward a target road surface spaced a first distance from the electronic device; receiving a reflection of the sonic signal from the target road surface; acquiring atmospheric information associated with the sonic signal; acquiring first data for the received reflection; generating second data by correcting the first data based on the atmospheric information; acquiring third data related to frequency domain information of the second data based on the second data; and determining a type of the target road surface based on the third data and a road surface classification artificial neural network, wherein the road surface classification artificial neural network can be trained with a frequency domain data set generated based on a sonic signal reflected from a road surface at a second distance different from the first distance. [Effects of the Invention]

[0024] According to the present disclosure, by quickly and accurately classifying the type of road surface based on ultrasonic signals, accidents can be prevented before they occur by controlling the road surface or the vehicle according to the classified road surface.

[0025] Furthermore, according to the present disclosure, road surface management can be automatically controlled using road surface classification information, thereby enabling economical and efficient road surface management.

[0026] Furthermore, the present disclosure can provide users with more effective transportation network information by obtaining road surface information in real time.

[0027] On the other hand, the effects of the present disclosure are not limited to the effects described above, and unmentioned effects can be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from this specification and the accompanying drawings. [Brief explanation of the drawings]

[0028] [Figure 1]FIG. 1 illustrates a block diagram of a road surface classification device according to various embodiments of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating a road surface classification device according to an embodiment of the present disclosure installed in road infrastructure and operated. [Figure 3] 1 is a flowchart illustrating a method performed by a road surface classification device according to the present disclosure. [Figure 4] 1 is a diagram illustrating a time axis of a sound wave signal transmitted from a road surface classification device according to various embodiments of the present disclosure. [Figure 5] FIG. 2 illustrates a transmission interval of an acoustic signal and a reception interval of a reflected signal according to an embodiment of the present disclosure. [Figure 6] 1 illustrates an object on which a road surface classification device according to various embodiments of the present disclosure is provided; [Figure 7] FIG. 1 illustrates a method for obtaining a dataset for training a road surface classification artificial neural network according to various embodiments of the present disclosure. [Figure 8] 1 is a flowchart illustrating a process for pre-processing reflected signals received by a road surface classification device according to various embodiments of the present disclosure. [Figure 9] FIG. 1 illustrates a multi-modal artificial neural network according to one embodiment of the present disclosure. [Figure 10] 10 is a flowchart illustrating an operation of a road surface classification device according to an embodiment of the present disclosure, in which the road surface classification device changes a control action based on a predetermined control change trigger. [Figure 11] 10A-10C illustrate scenarios in which road surface classification results change according to one embodiment of the present disclosure. [Figure 12] FIG. 1 illustrates a road surface management method using a road surface classification device according to various embodiments of the present disclosure. [Figure 13] FIG. 1 illustrates a road surface classification device collecting traffic information according to an embodiment of the present disclosure. [Figure 14] 1 is a configuration diagram of a road surface type estimation device according to an embodiment of the present disclosure. [Figure 15]1 is a diagram illustrating a transmission signal and a reception signal in a road surface type estimation device using sound waves according to an embodiment of the present disclosure. FIG. [Figure 16] 1 is a diagram for exemplifying a signal converter in a road surface type estimation device using sound waves according to an embodiment of the present disclosure. FIG. [Figure 17] FIG. 1 is a diagram illustrating an artificial neural network in a road surface type estimation device using sound waves according to an embodiment of the present disclosure. [Figure 18] FIG. 10 is a diagram for explaining the operation of a convolution execution unit. [Figure 19] FIG. 10 is a diagram for explaining the code of the convolution execution unit of the road surface type estimation device using sound waves of the present disclosure. [Figure 20] 1 is a flowchart of a method for estimating road surface type using domain transformation of sound waves according to an embodiment of the present disclosure. [Figure 21] 1 is a flowchart illustrating an embodiment of a method for estimating road surface type using sound waves according to the present disclosure. [Figure 22] 1 is a flowchart illustrating a method for estimating a road surface type using sound waves corrected for atmospheric attenuation according to an embodiment of the present disclosure. [Figure 23] FIG. 1 is a diagram illustrating a road condition monitoring system including a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure. [Figure 24] FIG. 1 is a configuration diagram of a road condition monitoring system equipped with a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure. [Figure 25] 1A and 1B are diagrams illustrating an example of recognizing the state of a uniform road surface in a road condition monitoring system equipped with a vision sensor and an acoustic sensor according to an embodiment of the present disclosure. [Figure 26] 1A and 1B are diagrams illustrating an example of recognizing uneven road surface conditions in a road condition monitoring system equipped with a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure. [Figure 27]10A and 10B are diagrams for explaining a method for finding out what segmentation region the sensing region of the ultrasonic sensor is in a road condition monitoring system equipped with a vision sensor and an ultrasonic sensor according to an embodiment of the present disclosure. [Figure 28] FIG. 1 is a diagram illustrating an example of an artificial neural network of a road condition monitoring system including a vision sensor and an acoustic sensor according to an embodiment of the present disclosure. [Figure 29] FIG. 10 is a diagram illustrating an example of a segmentation processing unit of a road condition monitoring system including a vision sensor and an ultrasonic sensor according to an embodiment of the present disclosure. [Figure 30] 1 is a flowchart of an embodiment of a monitoring method in a road condition monitoring system including a vision sensor and an acoustic sensor according to the present disclosure. [Figure 31] 31 is a detailed flowchart of one embodiment of the fusion analysis step 3050 of FIG. 30. [Figure 32] FIG. 10 is a configuration diagram of a road condition monitoring system including a vision sensor and an acoustic wave sensor according to another embodiment of the present disclosure. [Figure 33] 10 is a flowchart of another embodiment of a monitoring method in a road condition monitoring system including a vision sensor and an acoustic wave sensor according to the present disclosure. [Figure 34] FIG. 2 is a diagram for explaining the operation of a control system for a road heating device according to an embodiment of the present disclosure. [Figure 35] 1 is a configuration diagram of a control system for a road anti-icing device according to an embodiment of the present disclosure. [Figure 36] FIG. 10 is a configuration diagram of a control system for a road anti-icing device according to another embodiment of the present disclosure. [Figures 37a-37c] FIG. 1 is a diagram illustrating an artificial intelligence analysis model used in a control system for a road anti-icing device according to an embodiment of the present disclosure. [Figure 38] 1 is a flowchart of an embodiment of a method for controlling a road anti-icing device according to the present disclosure. [Figure 39]38. FIG. 38 is a detailed flowchart of one embodiment of the control signal generation step 3850 of FIG. 38 when the road de-icing device according to the present disclosure is a hot wire device. [Figure 40] 38. FIG. 38 is a detailed flowchart of one embodiment of the control signal generation step 3850 of FIG. 38 when the road de-icing device according to the present disclosure is a saline sprayer. [Figure 41] 1 is a perspective view schematically illustrating a road infrastructure sensor construction structure according to an embodiment of the present disclosure. [Figure 42] 1 is a side view schematically illustrating a road infrastructure sensor construction structure according to an embodiment of the present disclosure. [Figure 43] 1A is a perspective view schematically showing an acoustic wave sensor unit according to an embodiment of the present disclosure, and FIG. 1B is a partially cross-sectional perspective view thereof. [Figure 44] FIG. 2 is a partial cross-sectional side view schematically illustrating an acoustic wave sensor unit according to one embodiment of the present disclosure. [Figure 45] FIG. 10 is a perspective view schematically illustrating a road infrastructure sensor construction structure according to another embodiment of the present disclosure. [Figure 46] FIG. 10 is a side view schematically illustrating a road infrastructure sensor construction structure according to another embodiment of the present disclosure. [Figure 47] 1A is a partially enlarged perspective view, FIG. 1B is a bottom perspective view, and FIG. 1C is a partially cross-sectional perspective view, each showing a schematic view of a road infrastructure sensor installation structure according to another embodiment of the present disclosure. [Figure 48] FIG. 10 is a partial cross-sectional side view schematically illustrating an acoustic wave sensor unit according to another embodiment of the present disclosure. [Figure 49] 1 is a flowchart illustrating a construction method for a road infrastructure sensor construction structure according to a preferred embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In describing the embodiments, technical content that is well known in the technical field to which the present disclosure pertains and is not directly related to the present disclosure will be omitted. This is to avoid obscuring the gist of the present disclosure by omitting unnecessary explanations, and to more clearly convey the gist of the present disclosure.

[0030] For the same reason, in the accompanying drawings, some components are exaggerated, omitted, or shown schematically, and the size of each component does not completely reflect the actual size. The same reference numerals are used to denote the same or corresponding components in each drawing.

[0031] The advantages and features of the present disclosure, as well as methods for achieving them, will become apparent from the following detailed description of the embodiments in conjunction with the accompanying drawings. However, the matters disclosed in the drawings are not intended to specify or limit the various embodiments, but should be understood to include all modifications, equivalents, or alternatives falling within the spirit and technical scope of the various embodiments. Specific structural or functional descriptions of the various embodiments are merely provided for the purpose of describing the various embodiments, and the embodiments of the present disclosure may be implemented in various forms and should not be construed as being limited to the embodiments expressly described in this specification or application.

[0032] That is, the embodiments of the present disclosure are provided so that this disclosure will be complete and will convey the scope of the disclosure to those skilled in the art, and the invention of the present disclosure is defined only by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

[0033] Terms such as "first" and / or "second" may be used to describe various components, but the components should not be limited by the terms. The terms are used solely to distinguish one component from another, e.g., a first component may be designated as a second component, and a second component may be designated as a first component, without departing from the scope of the concepts of the present disclosure.

[0034] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that it may be directly coupled or connected to the other component, but that there may be other components in between. On the other hand, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, such as "between" and "directly between," or "adjacent" and "directly adjacent to," should be interpreted similarly.

[0035] Each block of the process flowchart diagrams and combinations of flowchart diagrams in the figures can be implemented by computer program instructions. These computer program instructions can be loaded onto a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that the instructions, executed via the processor of the computer or other programmable data processing apparatus, create means for performing the functions described in the flowchart blocks. These computer program instructions can also be stored in a computer-usable or computer-readable memory that can direct the computer or other programmable data processing apparatus to implement functions in a particular manner, such that the instructions stored in the computer-usable or computer-readable memory can produce an article of manufacture that embodies instruction means for performing the functions described in the flowchart blocks. Computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are executed on the computer or other programmable data processing apparatus, such that the instructions to generate a computer-implemented process that causes the computer or other programmable data processing apparatus to provide steps for performing the functions described in the flowchart blocks.

[0036] Also, each block may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function. Also, it should be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, two blocks shown one after the other may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order according to their respective functions.

[0037] The term "unit" as used in this disclosure refers to software or hardware components such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC). A "unit" performs a specific function but is not limited to software or hardware. A "unit" may be configured to reside on an addressable storage medium or to implement one or more processors. Thus, according to some embodiments, a "unit" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within components and units may be combined into fewer components and units or further separated into additional components and units. Furthermore, components and units may be configured to implement one or more CPUs within a device or a security multimedia card. Also, according to various embodiments of the present disclosure, a "unit" may include one or more processors.

[0038] The operating principles of the present disclosure will be described in detail below with reference to the accompanying drawings. Hereinafter, if it is determined that a detailed description of a known function or configuration related to the present disclosure may unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. The terms used below are defined in consideration of the functions in the present disclosure, and may vary depending on the intentions or practices of users or operators. Therefore, the definitions should be based on the contents of the entire specification.

[0039] The present disclosure relates to a system for classifying road surfaces using acoustic signals and thereby managing road surfaces and vehicle operations.

[0040] A road surface classification device according to an embodiment of the present disclosure may include a device that is installed on road infrastructure or a mobile body, and determines the type and condition of a road surface.

[0041] A road surface classification device according to yet another embodiment of the present disclosure may include a server device that determines the type and condition of a road surface based on information received from a device installed in road infrastructure or a mobile object.

[0042] FIG. 1 illustrates a block diagram of a road surface classification device according to various embodiments of the present disclosure.

[0043] Referring to Fig. 1, a road surface classification device 100 according to one embodiment of the present disclosure may include a transceiver unit 110, a sensor unit 120, and a controller 130. Meanwhile, road surface classification devices according to various embodiments of the present disclosure may include additional components in addition to the above hardware components and are not limited to the components shown in Fig. 1. Fig. 1 is intended to illustrate the hardware components constituting the road surface classification device 100 of the present disclosure, and road surface classification devices according to other embodiments of the present disclosure may be configured by omitting some of the components shown in Fig. 1.

[0044] The transceiver unit 110 may include a transmitter (not shown) and a receiver (not shown) or a transceiver (not shown) as hardware components configured to transmit and receive sound signals. Each of the transmitter and receiver constituting the transceiver unit 110 will be described in detail below.

[0045] The transmitter is a device that generates and emits a sound wave signal and may be disposed in a direction that emits the sound wave signal toward the road surface. In this case, the emitted sound wave signal may include a high-frequency ultrasonic signal.

[0046] Meanwhile, the frequency of the generated sound wave signal may be fixed depending on the type of transmitter, or may be set or variable by user input. The sound wave signal may be transmitted singly according to user input, control by a control unit or server, or a predetermined rule, or one or more signals may be transmitted periodically within one period. In this case, the number of transmitted sound waves and the transmission period may be variable.

[0047] The receiver is a device that receives a sonic signal and may be configured to receive a sonic signal reflected from the road surface.

[0048] Meanwhile, in addition to the reflected signal, the receiver can also directly receive the sound wave signal transmitted from the adjacent transmitter. The signal received directly from the transmitter is a signal unrelated to the classification of the road surface to be determined by the road surface classification device of the present disclosure, and can be considered as noise, and such a noise signal is called crosstalk.

[0049] According to one embodiment of the present disclosure, the transmitter and receiver can be spaced apart to reduce crosstalk. According to yet another embodiment of the present disclosure, a structure, such as a sound-absorbing or sound-proofing material, may be placed between the transmitter and receiver to reduce signal disturbances (e.g., crosstalk) that may occur between the transmitter and receiver. The structure may be made of a material or structure that has physical properties that attenuate or absorb sound waves, or may be an electronic device configured to achieve such physical properties.

[0050] On the other hand, the transmitter and receiver do not necessarily need to be physically separated within the road surface classification device of the present disclosure, and may be realized in an integrated form, for example, as a transceiver. In the following description, the term "transmitter" includes all of the transmitter, receiver, and transceiver, and the term "transmitter" can refer to a hardware device in which the transmitter and receiver are integrated, or can refer to both the physically separated transmitter and receiver, or each of them.

[0051] The transceiver can transmit or receive sound waves within a range of view angles depending on the hardware capabilities. The road surface classification device 100 according to various embodiments of the present disclosure can use transceivers with different view angles depending on the object or environment in which the road surface classification device is installed. For example, the view angle of a transceiver used in a road surface classification device installed in road infrastructure may be smaller than the view angle of a transceiver used in a road surface classification device installed in a vehicle.

[0052] When a transmitter and a receiver are separately configured within a road surface classification device, the transmitters and receivers according to various embodiments of the present disclosure may be arranged taking into consideration the directivity angle between them. A receiver according to one embodiment of the present disclosure may be arranged outside the directivity angle range of the transmitter, thereby preventing the receiver from detecting sound signals at the outermost angles of the directivity angle emitted from the transmitter. A receiver according to yet another embodiment of the present disclosure may be arranged on the outer angle side relative to the center of the directivity angle of the transmitter so that the crosstalk signal detected by the receiver is below a reference value.

[0053] On the other hand, transceivers according to various embodiments of the present disclosure can be designed or arranged to respond only to specific frequency characteristics of waves reflected from the road surface.

[0054] The sensing unit 120 is a hardware component that acquires information necessary for road surface classification according to the present disclosure through measurements, and the sensing unit 120 according to the present disclosure may include an air sensor, a camera, and / or an IR sensor.

[0055] The sensing unit 120 according to various embodiments of the present disclosure may include an air sensor. The air sensor is a hardware device that acquires information about atmospheric conditions, and the air information measured or acquired by the air sensor may include at least one of temperature, humidity, and air pressure. The air information may further include information about wind. In this case, the information about wind may include physical quantities related to wind, such as wind speed, wind volume, or wind direction. In this specification, the air sensor may refer to a device that includes at least one of a temperature sensor, a humidity sensor, and an air pressure sensor. The air sensor may also refer to a device that can sense multiple different types of air information. The air sensor according to one embodiment of the present disclosure may measure the temperature, humidity, air pressure, and / or wind speed at a location where the road surface classification device is installed.

[0056] The sensing unit 120 according to various embodiments of the present disclosure may further include a camera and / or an IR sensor. The camera is a device for acquiring images and may acquire image information of the road surface, and the IR sensor may acquire temperature information of the road surface by sensing radiant heat emitted from the road surface. Since the temperature information acquired by the IR sensor is temperature information of the road surface and the temperature information acquired by the air sensor is temperature information of the atmosphere, the values ​​indicated by the temperature information acquired by different sensors may be different.

[0057] Various pieces of information measured or acquired by the sensing unit 120 of the present disclosure may be combined and utilized to improve the accuracy of road surface classification. That is, the road surface classification results output by the road surface classification device 100 according to various embodiments of the present disclosure may be generated based on multiple pieces of information, and specific embodiments thereof will be described later.

[0058] Meanwhile, the camera and / or IR sensor included in the sensing unit 120 according to the embodiment of the present disclosure is merely exemplary, and the sensing unit 120 may further include any sensing device other than the above-mentioned air sensor, camera, or IR sensor for obtaining information that can be used to classify road surfaces.

[0059] The control unit 130 is hardware configured to perform the method executed by the road surface classification device of the present disclosure, and may include at least one processor including logic circuits and arithmetic circuits. The control unit 130 may process data according to programs and / or instructions provided from a memory (not shown), and generate control signals according to the processing results.

[0060] According to various embodiments, the control unit 130 may, for example, control at least one other component (e.g., hardware or software component) of the road surface classification device 100 connected to the control unit 130, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculations, the control unit 130 may store instructions or data received from other components (e.g., the receiver 120 or the sensing unit 120) in a volatile memory (not shown), process the instructions or data stored in the volatile memory (not shown), and store the resulting data in a non-volatile memory (not shown). As an example, a signal acquired via the receiver 120 may be converted into a digital signal via an analog-to-digital converter (ADC) circuit included in the control unit 130 and processed. The converted digital signal may also be preprocessed with input data for input to an artificial neural network. Specific methods for processing received signals and / or data according to the present disclosure will be described later.

[0061] According to one embodiment, the control unit 130 may include a main processor (e.g., a central processing unit (CPU) or application processor (AP)) or an auxiliary processor (e.g., a graphics processing unit (GPU), a neural network processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor. For example, if the road surface classification device 100 includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use less power than the main processor or to be specialized for a designated function. The auxiliary processor may be implemented separately from or as part of the main processor.

[0062] The road surface classification device according to various embodiments of the present disclosure may include a communication unit (not shown). The communication unit refers to a hardware component that receives commands or data input from a user or other external devices, transmits commands or data generated by the road surface classification device to the outside, or transmits or receives commands or data from other components of the road surface classification device, and may include a wired / wireless communication module and / or an input / output interface. The road surface classification device according to various embodiments of the present disclosure may receive information from an external electronic device (e.g., a control box or management server installed outside the road surface classification device) via the communication unit, or transmit information acquired or generated by the road surface classification device to the external electronic device. Meanwhile, the communication unit may be implemented separately from the control unit 130 within the road surface classification device according to various embodiments of the present disclosure, or may be implemented by circuit elements included in the control unit 130 and included in the control unit 130. In other words, the road surface classification device according to various embodiments of the present disclosure may be a device that provides information necessary for classifying road surfaces in conjunction with an external electronic device.

[0063] According to one embodiment of the present disclosure, an artificial neural network (not shown) for classifying road surfaces of the present disclosure may be provided as software-on-chip (SOC) or microcontroller unit (MCU) included in the control unit 130 in the road surface classification device. Alternatively, the artificial neural network may be provided in the form of software operated by the control unit 130 and updated by communication from an external server or user input.

[0064] Meanwhile, the artificial neural network according to various embodiments of the present disclosure may be implemented in an external electronic device (e.g., the controller or the server), in which case data generated based on the sound wave signal in the control unit 130 of the road surface classification device and data necessary for road surface classification, such as atmospheric information, are transmitted to the external electronic device, and the external electronic device can classify the road surface based on the data received from the road surface classification device.

[0065] According to yet another embodiment of the present disclosure, the road surface classification device may be a server device. In this case, the road surface classification device may not include the acoustic wave transceiver unit 110 and the sensor unit 120, but may receive data necessary for road surface classification from an external electronic device via a communication unit (not shown), and classify the road surface based on the received data via the control unit 130. In addition, the classified road surface classification result and / or related control information may be transmitted to the external electronic device.

[0066] FIG. 2 is a diagram illustrating a road surface classification device according to an embodiment of the present disclosure installed in road infrastructure and operated.

[0067] Referring to FIG. 2, the road surface classification device 100 may be installed in a road infrastructure 200 so as to face a road surface 230 to be classified.

[0068] In the present disclosure, the road infrastructure 200 is a general term for traffic facilities including pole-type structures 210 such as traffic lights, street lights, road guide signs, or image information processing devices that are installed on or along roads, and refers to structures on which a road surface classification device can be installed, and is not limited to the above examples. According to an embodiment of the present disclosure, installing the road surface classification device 100 on the road infrastructure 200 may mean, but is not limited to, installing the road surface classification device 100 on the top end of the pole-type structure 210.

[0069] The road infrastructure 200 may include a controller 220 for controlling electronic devices installed on the pole-type structure 210. The electronic devices installed on the pole-type structure 210 may include light-emitting devices used in street lights and traffic lights, CCTV, traffic information collection cameras, or the road surface classification device of the present disclosure.

[0070] The controller 220 is a device that controls the electronic devices installed in the pole-type structure 210, and for example, if the pole-type structure is a street light, it may be a street light controller that controls the operation of the street light, or if the pole-type structure is a traffic light, it may be a traffic light controller that controls the signal of the traffic light.

[0071] The controller 220 according to various embodiments of the present disclosure can control the operation of the road surface classification device 100 of the present disclosure and can control the road surface on which the road infrastructure 200 is located based on road surface classification information or commands obtained from the road surface classification device 100.

[0072] The controller 220 according to various embodiments of the present disclosure can function as a gateway between the road surface classification device 100 and a management server (not shown). That is, the controller 220 can include a wired or wireless communication module and can transmit information acquired from the road surface classification device to the management server and receive commands and data for controlling the road surface classification device or roads from the management server.

[0073] The controller 220 according to various embodiments of the present disclosure may include an artificial neural network of the present disclosure, thereby classifying road surfaces directly based on information obtained from the road surface classification device. In this case, the processor and memory included in the controller 220 may have greater performance than the processor and memory of the road surface classification device, and therefore the artificial neural network provided to the controller 220 may have greater performance than the artificial neural network provided to the road surface classification device 100.

[0074] The controller 220 according to various embodiments of the present disclosure can control the road surface maintenance device 250 installed on the road based on the classified road surface for road surface maintenance.

[0075] The road surface maintenance device 250 according to various embodiments of the present disclosure may include snow removal devices such as heat rays or saline sprayers installed on roads, drainage facilities, etc. Specific details regarding the operation of the road surface maintenance device according to various embodiments of the present disclosure will be described later.

[0076] The method by which the road surface classification device according to the present disclosure classifies road surfaces will be described in detail below.

[0077] Generally, different materials have different acoustic impedances, so the reflected signal from the same incident sound wave varies depending on the material. Therefore, it is possible to distinguish materials by analyzing the reflected signal using these physical characteristics. In particular, because acoustic impedance is a physical quantity that has frequency characteristics, analyzing the reflected signal in the frequency domain allows for more precise classification of the material of the reflecting surface.

[0078] An artificial neural network can be used to implement the road surface classification method using acoustic reflection signals according to various embodiments of the present disclosure.

[0079] A neural network model of an artificial neural network according to various embodiments of the present disclosure may include multiple layers.

[0080] The neural network model may be configured in the form of a classifier that generates road surface classification information. The classifier may perform multiple classifications. For example, the neural network model may be a multiple classification model that classifies input data into multiple classes.

[0081] A neural network model according to one embodiment of the present disclosure may include a deep neural network (DNN) of a multi-layer perceptron algorithm that includes an input layer, multiple hidden layers, and an output layer.

[0082] According to still another embodiment of the present disclosure, the neural network model may include a convolutional neural network (CNN). The CNN structure may include at least one of AlexNet, LENET, NIN, VGGNet, ResNet, WideResnet, GoogleNet, FractaNet, DenseNet, FitNet, RitResNet, HighwayNet, MobileNet, and DeeplySupervisedNet. The neural network model may be implemented using multiple CNN structures.

[0083] For example, the neural network model may be implemented to include multiple VGGNet blocks. More specifically, the neural network model may be implemented by combining a first structure in which a CNN layer having 64 filters of 3x3 size, a batch normalization (BN) layer, and a ReLU layer are sequentially connected, and a second block in which a CNN layer having 128 filters of 3x3 size, a ReLU layer, and a BN layer are sequentially connected.

[0084] The neural network model may include a max pooling layer following each CNN block, and may include a global average pooling (GAP) layer, a fully connected (FC) layer, and an activation layer (e.g., sigmoid, softmax, etc.) at the end.

[0085] The artificial neural network according to various embodiments of the present disclosure refers to a neural network model for extracting characteristics from a frequency transformed signal of a sound wave signal to classify road surfaces, and is not limited to the above examples.

[0086] A road surface classification artificial neural network according to various embodiments of the present disclosure can be trained using frequency domain data of a reflected signal as input, and the trained artificial neural network can classify the road surface from which the target signal is reflected using frequency domain data of the target signal as input.

[0087] The frequency domain data may refer to data obtained by performing frequency domain conversion on a digital signal converted through ADC sampling of a reflected signal.

[0088] As a frequency domain transformation method according to various embodiments of the present disclosure, Short-Time Fourier Transform (STFT), Fast Fourier Transform (FFT), Cepstrum Transform, Wavelet Transform, Cross-Correlation Method, Convolution Transform, etc. The above frequency domain transformation methods are exemplary and are not limited to the listed transformation methods, and various transformation or analysis methods for analyzing a sound wave signal in the time domain in the frequency domain can be used.

[0089] An example of frequency domain data according to various embodiments of the present disclosure may include spectrogram data obtained through an STFT transform.

[0090] Yet another example of frequency-domain data according to various embodiments of the present disclosure may include data obtained by applying cross-correlation methods, where cross-correlation synthesis of input data may correspond to inputting the data into a convolutional layer, thereby enabling CNN-based learning and classification.

[0091] Meanwhile, the frequency domain data used for learning can be labeled with information necessary for road surface classification, including the type of road surface and / or atmospheric information.

[0092] According to one embodiment of the present disclosure, for training a road surface classification artificial neural network, the training data set may include a data set in which frequency domain data are labeled with the type of road surface on which each data point was obtained.

[0093] The types (classes) of road surfaces classified by a road surface classification device according to an embodiment of the present disclosure may include classes such as asphalt, cement, dirt, ice, marble, paint, slush (a mixture of water and ice), snow, and water. The listed types of classes are exemplary, and the number of classes or groups to be classified may vary depending on the situation in various embodiments of the present disclosure. Instead of using such direct labeling or group names, input data may be grouped in an arbitrary manner, such as a first class or a second class. Such arbitrary grouping may be, but is not limited to, a classification result obtained when using an unsupervised artificial neural network in which the training data does not include labels.

[0094] Figure 3 is a flowchart illustrating a method performed by a road surface classification device according to the present disclosure. According to various embodiments, the operations shown in Figure 3 are not limited to the order shown and may be performed in various orders. Also, according to various embodiments, more operations than those shown in Figure 3 may be performed, or at least one less operation may be performed. Figures 4 to 12 can be referenced as drawings for explaining the operations shown in Figure 3 in greater detail.

[0095] Referring to FIG. 3, a road surface classification device according to various embodiments of the present disclosure may transmit or emit a sonic signal toward a road surface to be classified using a transmitter in step 301. In step 301, the sonic signal may be transmitted at least once, and the number of times the signal is transmitted and the transmission period may be changed according to user input, preset conditions, or server control. When the sonic signal is transmitted multiple times within one determination period, multiple pieces of data for determining the classification or state of the road surface can be obtained, thereby improving the accuracy of road surface classification. Specific embodiments of the period for transmitting the sonic signal and the operation of transmitting multiple times within one period will be described below with reference to FIG. 4.

[0096] In step 302, the road surface classification device may receive a signal reflected from the target road surface using a receiver. Since the reflected signal is a reflected signal of the transmitted sonic signal, the sonic signal and the reflected signal may correspond to each other. If multiple sonic signals are transmitted, the corresponding reflected signals may be received multiple times.

[0097] A road surface classification device according to various embodiments of the present disclosure may acquire atmospheric information through the atmospheric sensor of the sensing unit 120 when a sound wave signal is transmitted. In this case, the time at which the atmospheric information is acquired does not necessarily have to coincide with the time at which the sound wave signal is transmitted, but rather there is a correspondence between the two within a certain time interval. That is, the road surface classification device may acquire atmospheric information corresponding to one sound wave signal, or one piece of atmospheric information corresponding to multiple sound wave signals. A road surface classification device according to various embodiments of the present disclosure may process a reflected signal corresponding to the emitted sound wave signal based on the atmospheric information corresponding to the emitted sound wave signal.

[0098] Meanwhile, the time from when a sound wave signal is transmitted from a transmitter to when it is reflected by a road surface and received by a receiver can be defined as the time of flight (ToF). Because the atmospheric propagation speed of sound waves can be determined under specific weather conditions, the distance between the road surface classification device and the target road surface can be measured based on the ToF and atmospheric information. Conversely, if the distance between the road surface classification device and the target road surface is known in advance, the ToF can be estimated. Therefore, road surface classification devices according to various embodiments of the present disclosure can distinguish between sound wave signals transmitted from a transmitter and corresponding received signals. That is, by defining a reception interval for the sound wave signals transmitted from the transmitter and corresponding received signals, signals received during the corresponding time interval can be determined as reflected signals of the transmitted sound wave signal, and signals received during other time intervals can be regarded as noise or reflected signals of other sound wave signals. A detailed embodiment of a control method for a road surface classification device that controls noise signals using this method will be described later with reference to FIG. 5.

[0099] In step 303, the road surface classification device 100 may preprocess the received reflected signals using a controller to obtain data for input to a road surface classification artificial neural network according to the present disclosure. In the present disclosure, signal preprocessing refers to the entire process of obtaining data for input to an artificial neural network based on the received reflected signals, and the preprocessing operation in step 303 may include sampling an analog signal to a digital signal, attenuation correction and ToF correction for the sampled signal, frequency domain conversion, etc. The preprocessing process for obtaining input data for a road surface classification artificial neural network according to various embodiments of the present disclosure will be described in detail with reference to Figures 6 to 8.

[0100] In step 304, the input data obtained by the pre-processing process can be input to a road surface classification artificial neural network. Meanwhile, the road surface classification artificial neural network according to various embodiments of the present disclosure can be trained with a training dataset consisting of multiple data obtained for various road surfaces to classify the road surfaces. The trained road surface classification artificial neural network can output results based on the input data.

[0101] The output result according to an embodiment of the present disclosure may include information about the probability value for each road surface classification class. If the artificial neural network model is trained to classify road surfaces into multiple types, the probability that the target road surface corresponds to each of the multiple road surface types may be expressed as a numerical value and output. In this case, the output road surface class may be one or more classes in descending order of probability.

[0102] According to still another embodiment of the present disclosure, an output result may be output by determining a specific class from among a plurality of classes. In this case, the specific class may be determined when the probability value of the class is equal to or greater than a threshold, or when the probability difference between the class and a second-ranked class is equal to or greater than a threshold.

[0103] Meanwhile, the output results of the road surface classification artificial neural network according to various embodiments of the present disclosure are information regarding the material or condition of the road surface, and are not limited to the above examples, and can be output in a form required by the user according to the design of the artificial neural network.

[0104] In step 305, the road surface classification device can perform various actions depending on the output result. By modifying or adding control actions based on the road surface classification result, the accuracy of the results or the efficiency of road surface management can be improved.

[0105] According to various embodiments of the present disclosure, if an output result differs from a previous output result, a process of comparing the output result with a subsequent output result can be performed prior to road surface control based on the output result to determine whether the change in road surface condition is due to a change in weather conditions or an output error. Specific embodiments relating to this will be described in detail with reference to Figures 10 and 11. In this case, the road surface classification device can change the transmission period or number of times of the sound wave signal depending on the output result.

[0106] According to various embodiments of the present disclosure, if the output result relates to a particular class (e.g., snow, ice, or slush), an instruction or signal may be generated and transmitted to control the road surface via a road surface management device. An embodiment for managing the road surface according to the output result is described in more detail with reference to FIG. 12.

[0107] FIG. 4 is a diagram illustrating, on a time axis, a sound wave signal transmitted from a road surface classification device according to various embodiments of the present disclosure.

[0108] 4, the sonic signal can be emitted multiple times within one transmission period. In this disclosure, a set of sonic signals transmitted within one transmission period to determine road surface conditions is referred to as a burst.

[0109] The number of sonic signals in a burst can be varied according to user setting or predetermined rules, the spacing between the sonic signals in a burst can be varied according to user setting or predetermined rules, the spacing between the sonic signals in a burst can be constant or variable, and the intensities of the sonic signals in a burst can be the same or different.

[0110] In this disclosure, the number, spacing, intensity, and duration of acoustic signals contained in a burst are referred to as the burst configuration. In this disclosure, different bursts may have the same or different burst configurations. The burst configuration for each burst can be changed according to user settings or predetermined rules.

[0111] The number of acoustic signals included in a burst according to one embodiment of the present disclosure may be one.

[0112] According to yet another embodiment of the present disclosure, the number of acoustic signals included in a burst may be multiple.

[0113] In this disclosure, the transmission period refers to the interval between burst transmissions by a road surface classification device to classify the condition or material of a target road surface. When a burst consists of one signal, i.e., when only a single signal is transmitted, the transmission period may refer to the time interval between adjacent regularly transmitted acoustic signals. Referring to FIG. 4, the transmission period may correspond to the time interval between the first signal 1a included in one burst (burst 1) and the first signal 2a included in the next burst (burst 2). The transmission period may be changed according to a user setting or a predetermined rule.

[0114] According to various embodiments of the present disclosure, the number of acoustic signals included in a burst and / or the transmission period can be changed according to the road surface classification results or weather conditions. For example, under certain weather conditions, such as when it is snowing or the temperature is below 0 degrees, the number of transmitted acoustic signals can be increased or the transmission period can be shortened to improve the accuracy of the road surface classification. Specific embodiments in this regard will be described in detail with reference to Figures 10 and 11.

[0115] Meanwhile, according to various embodiments of the present disclosure, the transmission period may be changed depending on the location or object where the road surface classification device is installed. This is to distinguish between a received signal reflected from the road surface and a crosstalk signal generated by a signal transmitted from a transmitter. The transmission period of a road surface classification device installed in road infrastructure may be longer than the transmission period of a road surface classification device installed in a vehicle. Therefore, the determination period of a road surface classification device installed in road infrastructure may be longer than the determination period of a road surface classification device installed in a vehicle.

[0116] According to various embodiments of the present disclosure, a road surface classification device can measure or determine the ToF for a target road surface or object. For example, the road surface classification device can transmit one or more acoustic signals and determine the ToF for the target road surface or object based on the corresponding received signals. Alternatively, the ToF can be determined based on the distance between the road surface classification device and the target road surface.

[0117] A road surface classification device according to various embodiments of the present disclosure may determine an appropriate transmission period and burst configuration based on the determined ToF, and transmit signals with the determined transmission period and burst configuration. The transmission period according to one embodiment of the present disclosure may be set to be longer than the ToF for the road surface. The duration of the burst according to one embodiment of the present disclosure may be set to be shorter than the transmission period.

[0118] According to various embodiments of the present disclosure, the road surface classification device can output a single road surface classification result for a target road surface corresponding to a single burst. Alternatively, the road surface classification device may display classification results for all sound signals included in a single burst. When a single result is output, the result may be output based on multiple classification results for each of the multiple sound signals included in the burst.

[0119] Referring to FIG. 4, the first result (result 1) is a road surface classification result obtained based on the signal of the first burst (burst 1) reflected off the road surface. In this case, the first result may be a result obtained based on the road surface classification results of the signals 1a, 1b, 1c, and 1d included in the first burst. For example, the most frequent value of the results 1a, 1b, 1c, and 1d may be output as the result. Alternatively, the road surface classification result for the first burst may be output based on the average value of the results 1a, 1b, 1c, and 1d.

[0120] In this disclosure, the time interval between road surface classification results of adjacent bursts, i.e., the time interval between the first result and the second result, can be referred to as the judgment period for road surface classification. The judgment period may coincide with the transmission period. However, in the judgment process, the output time may be irregular depending on the signal processing operation, so the judgment period does not need to be constant or coincide with the transmission period.

[0121] The road surface classification device according to various embodiments of the present disclosure can change the transmission period to change the judgment period. Alternatively, the judgment period may be changed according to a user setting or a predetermined rule. Specific embodiments for changing the judgment period will be described in detail with reference to FIGS. 10 and 11.

[0122] FIG. 5 is a diagram illustrating a transmission interval of an acoustic signal and a reception interval of a reflected signal according to an embodiment of the present disclosure.

[0123] Referring to Figure 5, the road surface classification device can transmit bursts or sonic signals in the transmission section. For convenience of explanation, Figure 5 illustrates the case of transmitting a single signal, but the present disclosure is not limited to this. That is, it should be understood that the transmission of sonic signals by the road surface classification device in the present disclosure includes not only the single emission of a single signal, but also the periodic transmission of bursts consisting of multiple signals.

[0124] As described above, the road surface classification device according to various embodiments of the present disclosure can determine the ToF of the transmitted acoustic signal relative to the road surface, and therefore can predetermine the corresponding reception interval for one transmission interval.

[0125] According to various embodiments of the present disclosure, if a signal is sensed at the receiver before the receiving interval, the road surface classification device can consider it a noise signal or crosstalk signal and can control the transmitter of the road surface classification device to reduce it.

[0126] 5, if the strength of a first signal received before the reception interval is greater than a first threshold, or if the difference between the strength of a second signal received in the reception interval and the strength of the first signal received before the reception interval is less than a second threshold, the power supplied to the transmitter can be changed to control the difference. The first threshold and / or the second threshold may be predetermined or may be set by a user input or an external device.

[0127] For example, if the strength of the first signal is greater than a first threshold, it may be determined that the influence of crosstalk is large, and the vibration of the transmitter may be controlled to be reduced. Alternatively, if the strength of the second signal is smaller than the strength of the first signal, it may be determined that noise from the external environment on the received signal is large, and the vibration of the transmitter may be controlled to be increased. According to one embodiment of the present disclosure, the vibration of the transmitter may be controlled by adjusting the magnitude of power supplied to the transmitter.

[0128] FIG. 6 is a diagram illustrating an object on which a road surface classification device according to various embodiments of the present disclosure may be mounted.

[0129] 6, a road surface classification device 100 according to various embodiments of the present disclosure may be installed on a moving body 610 or road infrastructure 620. In this case, the road surface classification device 100a installed on the moving body 610 such as a vehicle and the road surface classification device 100b installed on the road infrastructure 620 have different heights relative to the road surface, and therefore the ToFs of the transmitted sound waves are also different.

[0130] Since sound waves propagate through space, not only does their amplitude decrease with distance from the source, but also, when traveling through air, they are attenuated by the medium. Therefore, the characteristics of the reflected signals from road surfaces with different ToFs can be different from each other even when the road surfaces are in the same condition.

[0131] On the other hand, the road surface classification device of the present disclosure uses an artificial neural network to classify road surfaces based on the reflected signals from the road surface, and therefore requires a large number of data sets to train the artificial neural network.

[0132] FIG. 7 is a diagram illustrating a method for obtaining a dataset for training a road surface classification artificial neural network according to various embodiments of the present disclosure.

[0133] 7, training data sets for training road surface classification artificial neural networks according to various embodiments of the present disclosure may be acquired for various road surfaces for each road surface classification (class) using a transceiver included in a mobile measurement device 700. In general, to improve the classification performance of an artificial neural network, it is important to collect a large amount of data in a variety of terrains and environments, and therefore it is important to collect data using a device that is easily mobile.

[0134] The mobile measurement device 700 of the present disclosure refers to a sensor device mounted on a device that moves on a road, such as a bicycle, a car, or a scooter, and can include devices that can be moved by humans or mechanical devices.

[0135] Meanwhile, the ToF of the training data collected by the mobile measurement device 700 may be the same as the ToF of the road surface classification device 100a installed on a mobile object 610, such as the vehicle of FIG. 6. Alternatively, the position of the mobile measurement device relative to the ground may be set taking into account the position of the road surface classification device installed on the mobile object relative to the ground. In this case, the road surface classification artificial neural network trained with the training data set acquired by the mobile measurement device according to various embodiments of the present disclosure can be directly used in the road surface classification device 100a installed on the mobile object without any special correction for the reflected signals.

[0136] However, when the road surface classification device is installed at a different height from the moving object such as road infrastructure (100b) as shown in Figure 6, if the reflected signal acquired by the road surface classification device is directly input to the road surface classification artificial neural network, the classification accuracy for the target road surface may decrease.

[0137] 8 is a flowchart illustrating a process for pre-processing a received reflected signal by a road surface classification device according to various embodiments of the present disclosure. The pre-processing process of FIG. 8 is exemplary to illustrate the technical concepts of the present disclosure, and various embodiments may perform more operations than those shown in FIG. 8 or at least one less operation.

[0138] In step 801, a road surface classification device according to various embodiments of the present disclosure may obtain first data based on a received reflected signal.

[0139] As described above, since the reflected signal received through the receiver may be an analog signal, the road surface classification device of the present disclosure may convert the reflected signal into a digital signal through an ADC circuit included in the control unit 130. Alternatively, the transceiver included in the road surface classification device according to an embodiment of the present disclosure may process the reflected signal reflected through the road surface into a digital signal to obtain the first data.

[0140] In step 802, a road surface classification device according to various embodiments of the present disclosure may apply an atmospheric correction to the data converted into a digital signal to obtain second data.

[0141] Sound waves propagate through the atmosphere and are attenuated by the medium, and the amount of attenuation is determined by the propagation distance and the attenuation coefficient. Meanwhile, the attenuation coefficient is a value determined based on temperature, humidity, air pressure, and the frequency of the sound wave, so the road surface classification device according to various embodiments of the present disclosure can calculate the amount of attenuation of the sound wave based on the attenuation coefficient.

[0142] A road surface classification device according to various embodiments of the present disclosure can generate second data that corrects the amount of attenuation of the received reflected signal based on atmospheric information such as temperature, humidity, and air pressure obtained via an atmospheric sensor.

[0143] Meanwhile, the propagation distance of sound waves required for atmospheric correction may be input in advance by a user or may be acquired based on ToF. That is, distance information to the road surface may be input in advance depending on the location where the road surface classification device is installed, or distance information to the road surface may be acquired based on ToF information and atmospheric information acquired by the road surface classification device as described above.

[0144] In step 803, the road surface classification device according to various embodiments of the present disclosure may apply distance correction to the second data corrected for atmospheric attenuation to obtain third data.

[0145] 6 and 7, the sound wave signals that form the basis of the training data for the road surface classification artificial neural network according to various embodiments of the present disclosure may be signals acquired by reflection at a distance d1 from the road surface. Therefore, to improve the classification performance of the road surface classification device 100b installed at a height d2 different from d1, the sound wave signals acquired at d2 can be corrected to resemble the sound wave signals acquired at d1.

[0146] On the other hand, steps 802 and 803 can be performed in a single procedure. That is, according to various embodiments of the present disclosure, sound wave data can be acquired from the digital signal acquired in step 801, with the atmospheric attenuation and the distance to the road surface corrected based on the atmospheric information and distance information.

[0147] Additionally, depending on the installation location of the road surface classification device according to various embodiments of the present disclosure, the correction procedures of step 802 and / or step 803 may be omitted.

[0148] In step 804, the road surface classification device according to various embodiments of the present disclosure may perform a transformation to obtain frequency domain data for analyzing the corrected sound wave data in the frequency domain. The frequency domain transformation method according to various embodiments of the present disclosure has been described above. The obtained frequency domain data is input data to the road surface classification artificial neural network according to various embodiments of the present disclosure. Once the frequency domain data is input to the road surface classification artificial neural network, the road surface classification artificial neural network can output a road surface classification result for the target road surface.

[0149] A road surface classification device according to various embodiments of the present disclosure may output a result based on additional information other than the sound wave signal. Other information that can be acquired in addition to the sound wave signal may include image information acquired via a vision sensor (camera), road surface temperature information acquired via an IR sensor, and environmental information acquired via a communication unit.

[0150] According to one embodiment of the present disclosure, the road surface classification device can combine two or more different criteria.

[0151] Since the area that can be confirmed through sound waves may correspond to a portion of the road surface, image information that can confirm the state of a wider area can be used as a supplement to the road surface classification result. For example, only when the result confirmed through the image information matches the output value of the road surface classification artificial neural network, it can be determined that the road surface information is valid. Meanwhile, the road surface classification device according to various embodiments of the present disclosure may further include a separate image-based road surface classification artificial neural network for obtaining a road surface classification result for the image information.

[0152] Furthermore, according to various embodiments of the present disclosure, the road surface classification device may add a specific temperature condition to verify the result value for a specific road surface condition. For example, if the road surface temperature is higher than 0°C, ice cannot physically form under atmospheric pressure conditions, and therefore, if the road surface classification result is classified as ice under the corresponding temperature condition, it may be determined to be an error. Therefore, if the road surface or atmospheric temperature acquired through the IR sensor or atmospheric sensor of the sensing unit is confirmed to be above a specific temperature, if the road surface condition indicated by the road surface classification result is related to ice, an additional operation may be performed instead of outputting the result. Alternatively, if the road surface temperature is above or below a specific temperature, the device may be configured to output a result by further utilizing the result for the image information.

[0153] Furthermore, according to various embodiments of the present disclosure, the road surface classification device can output road surface classification results by further taking into account meteorological environment information. For example, when receiving meteorological environment information related to weather, such as when it is snowing or raining, the device can adjust the ranking of road surface classification results for classes that are likely to be classified under that weather.

[0154] Meanwhile, since the above-mentioned image information and temperature information are useful information for classifying road surface conditions, the road surface classification artificial neural network can be trained not only using data based on sound wave signals, but also by inputting related additional data to enhance its learning and classification performance.

[0155] FIG. 9 is a diagram illustrating a multi-modal artificial neural network according to one embodiment of the present disclosure.

[0156] Referring to FIG. 9, a road surface classification artificial neural network according to various embodiments of the present disclosure may include a multi-modal artificial neural network. The multi-modal artificial neural network can function as a single classifier through classifiers based on different information by inputting at least one of image information, atmospheric information, and road surface temperature information in addition to input data related to a sound wave signal. This correspondence learning allows multiple pieces of information related to a single road surface condition to be input together to obtain a more accurate road surface classification result. That is, the road surface classification device according to various embodiments of the present disclosure can output a road surface result by combining multiple pieces of information. The input data shown in FIG. 9 is merely exemplary, and only a portion of the image information, atmospheric information, and / or road surface temperature information may be utilized, or additional information may be utilized.

[0157] FIG. 10 is a flowchart showing an operation of the road surface classification device according to an embodiment of the present disclosure to change a control action based on a predetermined control change trigger.

[0158] In the present disclosure, a control change trigger refers to a situation or condition that changes the operation of a road surface classification device according to various embodiments of the present disclosure, and may be set in advance by a user or by a command from an external device.

[0159] On the other hand, when the control change trigger causes the control operation of the road surface classification device to be changed, it means that the methods set in the road surface classification device before the control change trigger occurs, such as the burst configuration, transmission period, judgment period, and road surface classification result output method, are changed.

[0160] The control change trigger may include, but is not limited to, a change in the road surface classification result (class) or the output of a particular class, weather conditions, time conditions, or geographic conditions.

[0161] As an example of a control change trigger, the operation of the road surface classification device may be changed when the road surface classification result changes.

[0162] FIG. 11 is a diagram illustrating scenarios in which road surface classification results change according to one embodiment of the present disclosure.

[0163] 11, the road surface classification result of the road surface classification device according to the embodiment of the present disclosure may be a first class R1 at a first time point t1, and may change to a second class R2 at a second time point t2, where the second class may be a class different from the first class.

[0164] According to one embodiment of the present disclosure, the second class may represent road conditions related to ice, and the first class may be classification results related to other road conditions.

[0165] If the road surface conditions change, the road surface classification device according to various embodiments of the present disclosure may need to change its control action.

[0166] For example, if the road surface classification result is different from the previous classification result, the road surface classification device may change the transmission period or the burst structure to determine whether there is an error in the road surface classification result. That is, the transmission period may be further shortened to increase the number of judgments, or the number of acoustic signals included in the burst may be increased to increase the number of judgments. Alternatively, instead of directly changing the transmission period or burst structure, it may be determined whether to change the transmission period or burst structure based on subsequent judgments.

[0167] 11, because the result at the second time point is different from the result at the first time point (R1≠R2), the road surface classification device according to an embodiment of the present disclosure can control the transmission period to be shorter and obtain more results during the short time interval (t3 to t6). Meanwhile, because the number of determinations for the second class is greater than the number of determinations for the first class during that time interval (t3 to t6), the road surface classification device determines that the determination for the second class is correct, changes the transmission period back to the original state, and outputs the result at the seventh time point t7, which is the time point according to the changed transmission period.

[0168] According to various embodiments of the present disclosure, in an example where a control operation is changed based on a subsequent judgment when a previous judgment is different, if a judgment is made to a first class R1 at a first time point t1 and a judgment is made to a second class R2 different from the first class at a second time point t2, instead of immediately changing the transmission period, the accuracy of the judgment for the second class and whether or not to change the transmission period may be determined based on the result of the third class at the next judgment time point, i.e., a third time point. That is, if the result of the third class is judged to be R2, the changed class at the second time point may be determined to be correct, and the transmission period may not be changed. If the result of the third class is not judged to be R2, the judgment of R2 may be determined to be an error, and the transmission period may be changed.

[0169] Meanwhile, the determination result at each time point may be a determination result corresponding to each burst according to various embodiments of the present disclosure.

[0170] A road surface classification device according to various embodiments of the present disclosure may further perform actions related to road surface management based on the results of the multiple determinations in the modified control action. For example, after determining the accuracy of a particular modified class, the determination for that class may be considered correct and a road surface management action related to the modified class may be performed. More details regarding road surface management actions are described with reference to FIG. 12.

[0171] Another example of a control change trigger could be to change the operation of the road surface classifier when weather or time conditions change.

[0172] As an example of a weather condition, if the temperature is below 0 degrees or is expected to be below 0 degrees, the road surface classification device according to various embodiments of the present disclosure may change its control operation by shortening the transmission cycle or increasing the number of transmissions in a burst in order to quickly determine whether black ice has occurred. Further examples of weather conditions include weather environments such as strong winds, rain, and heavy snow, and information about such weather environments may be obtained via an atmospheric sensor included in the road surface classification device or from an external device via a communication unit.

[0173] As an example of a time condition, black ice may occur more easily at night than during the day, so the transmission cycle or burst configuration may be changed at a specific time taking this into consideration. As another example of a time condition, road surface conditions may change less in summer than in winter, so the control operation may be changed by lengthening the transmission cycle or reducing the number of transmissions in a burst to reduce power consumption.

[0174] As another example of a control change trigger, the operation of the road surface classification device may be changed when geographical conditions change. If the road surface classification device is installed in road infrastructure, the geographical conditions cannot be changed, but the transmission period or burst configuration may be changed depending on the regional conditions. If the road surface classification device is installed in a mobile object such as a vehicle, the transmission period or burst configuration may be changed when entering a specific region. For example, if black ice enters a vulnerable section, the road surface classification device may change its control operation as described above when it receives the corresponding information.

[0175] On the other hand, the control change triggers related to the above-mentioned weather conditions, time conditions, or geographic conditions are exemplary and not limiting, and may be set by user input or a signal received from an external electronic device such as a server device.

[0176] FIG. 12 is a diagram illustrating a road surface management method using a road surface classification device according to various embodiments of the present disclosure.

[0177] 12, in step 1201, the road surface classification device may obtain information related to the control of the road surface. According to various embodiments of the present disclosure, the information related to the control of the road surface may include the result obtained in step 304 of FIG. 3 or the final result obtained by changing the control operation of FIG. 10. In addition, the information related to the control of the road surface may include weather information and / or road surface temperature information obtained by the road surface classification device.

[0178] In step 1202, a road surface classification device according to various embodiments of the present disclosure may determine whether operating conditions for road surface control are met based on the acquired information.

[0179] For example, if the road surface classification result obtained in step 1201 is related to black ice, i.e., if an ice-related class is obtained, the road surface classification device can determine that the conditions for operating a road surface management device installed on the road surface to eliminate or prevent icy conditions on the road surface have been met.

[0180] Alternatively, the acquired road surface classification results can be combined with meteorological information to determine the operating conditions for road surface control. For example, if the meteorological information satisfies certain conditions, it can be determined that there is a high risk of ice formation, and that the conditions for operating a road surface management device installed on the road to eliminate or prevent icy conditions on the road surface have been met.

[0181] Examples of weather information for operating the road surface maintenance device may include the following conditions: (1) When snow, rain, sleet, or frost is falling or is forecast to fall. (2) When the road surface temperature is below 0°C (3) When it is dawn (4) If the temperature drops suddenly (5) When strong winds are blowing

[0182] When at least one of the weather conditions is satisfied, it can be determined that road control is necessary if a road surface classification result is obtained in a particular class (e.g., water, slush, ice).

[0183] Road surface maintenance devices according to various embodiments of the present disclosure may include, but are not limited to, saline sprayers or hot wires.

[0184] In step 1203, the road surface classification device according to various embodiments of the present disclosure can generate a road surface control signal based on the acquired information and determination. The road surface control signal can include signals or commands necessary to control road surface maintenance devices installed on the road surface.

[0185] A road surface classification device according to various embodiments of the present disclosure may be linked to a road surface management device installed on a road. When the road surface classification device is directly linked to the road surface management device, the road surface classification device may generate a command signal for controlling the road surface management device and transmit it to the road surface management device. Alternatively, when the road surface classification device is indirectly linked to the road surface management device via an external server, the road surface classification device may generate a signal instructing control of the road surface management device and transmit it to the external server.

[0186] On the other hand, when the road surface maintenance device receives a road surface control signal, it can perform an operation to control the road surface based on the road surface control signal. For example, the road surface maintenance device can spray salt water or operate a heat wire based on the road surface control signal.

[0187] In another embodiment of the present disclosure for road surface management, the road surface classifier can determine the risk of road surface failure.

[0188] Asphalt can be damaged by repeated traffic of vehicles exceeding a certain weight. In particular, if water seeps into the asphalt in winter and freezes into ice, the volume expands, and when large vehicles such as trucks pass through, the road surface can be damaged.

[0189] A road surface classification device installed in road infrastructure according to various embodiments of the present disclosure can periodically sense the ToF of the road surface, and based on this, can measure traffic information such as passing vehicle information and traffic volume.

[0190] FIG. 13 is a diagram illustrating a road surface classification device according to an embodiment of the present disclosure collecting traffic information.

[0191] 13, the road surface classification device can collect traffic information about the road surface based on the ToF measurement. The traffic information collected by the road surface classification device according to various embodiments of the present disclosure can include information about the degree of road surface damage or traffic volume.

[0192] As described above, the road surface classification device can acquire information about the installation height of the road surface classification device, and therefore, the ToF corresponding to the installation height of the road surface classification device can be determined as the reference ToF. In other words, the ToF of the reflected signal reflected from the road surface can be set as the reference ToF.

[0193] Therefore, the road surface classification device according to an embodiment of the present disclosure can determine that there is no vehicle on the road surface if the ToF acquired by the road surface classification device is identified as corresponding to the reference ToF, and can determine that there is a vehicle on the road surface if the acquired ToF is identified as shorter than the reference ToF, and can obtain information on the size (height) of an object on the road surface estimated based on the acquired ToF.

[0194] Since a shorter ToF value indicates the presence of an object higher than the road surface, the road surface classification device can determine that a large vehicle has passed based on a signal with a short ToF value. The criteria for determining whether a large vehicle has passed may be preset by user input or a signal from an external device.

[0195] A road surface classification device according to various embodiments of the present disclosure can estimate the volume of traffic passing over a road surface during a predetermined time interval based on the ToF values ​​acquired during that time. In addition, the traffic volume information acquired by the road surface classification device of the present disclosure can further include information regarding the size of passing vehicles.

[0196] Referring to FIG. 13, ToF 1 when a large vehicle is passing (a) is measured to be smaller than ToF 2 when a small vehicle is passing (b).

[0197] The road surface classification device according to various embodiments of the present disclosure can acquire road surface condition information and / or weather information, and can combine the acquired information with traffic volume information to determine the risk of road surface damage and notify the outside. The road surface condition information can include road surface classification results and / or road surface temperature information.

[0198] For example, information on the number of large vehicles passing through during a period in which the road surface classification result is determined to be an ice-related class can be measured, and the related information can be provided to a user or an external device. The external device can include a server device of an agency that manages roads. Alternatively, information on the number of vehicles passing through during a period in which the road surface temperature is measured to be below a certain temperature can be obtained and transmitted to an external device. In this case, the degree of road surface damage can be estimated based on the specific road surface classification result or the number of large vehicles passing through under specific weather conditions. Alternatively, the risk of road surface damage can be managed by providing the obtained information on the number of large vehicles passing through to an external device.

[0199] A road surface classification device according to various embodiments of the present disclosure can determine whether to use the acquired received signal for road surface classification or for collecting traffic volume information based on the ToF of the acquired received signal. That is, if the ToF of the acquired received signal is within an error range with respect to a reference ToF, the signal can be determined to be a signal reflected from a road surface and used for road surface classification. If the ToF of the acquired received signal is shorter than the reference ToF, the signal can be determined to be acquired from a vehicle and traffic information can be acquired based on the signal.

[0200] In order to collect traffic volume information, the road surface classification device can transmit the sound signal at a shorter period than the period for road surface classification, i.e., the transmission period of the sound signal can be set in various ways according to the needs of the user, and the acquired signal can be processed in various ways according to the purpose.

[0201] Meanwhile, the road surface classification device of the present disclosure may include a road surface type estimation device. The road surface type estimation devices described below are various embodiments of the road surface classification device of the present disclosure, and it is clear that the operations performed by the road surface type estimation device according to one embodiment of the present disclosure can be performed by the road surface classification device according to various embodiments of the present disclosure.

[0202] FIG. 14 is a configuration diagram of a road surface type estimation device according to an embodiment of the present disclosure.

[0203] 14, a road surface type estimation device using sound waves according to an embodiment of the present disclosure may include a sound wave transceiver 1410, a signal converter 1420, an artificial neural network 1430, and a control unit (MCU) 1440. Meanwhile, the road surface type estimation device may further include an atmospheric attenuation correction unit (not shown) and an atmospheric information measurement unit (not shown).

[0204] The sound wave transmitting / receiving unit 1410 can transmit a sound wave signal to a road surface whose type is to be identified and then receive a reflected signal.

[0205] The sonic wave transceiver 1410 may include an sonic wave transmitter 1411 that outputs a transmission signal under the control of the control unit 1440, and an sonic wave receiver 1412 that receives a reflected signal that is the transmission signal reflected back from any surface.

[0206] The signal converter 1420 can perform a frequency transform on a predetermined region in the time domain of the received signal to obtain a frequency domain signal (eg, a spectrogram).

[0207] The signal transformer 1420 may include a Short-Time Fourier Transform (STFT), a Fast Fourier Transform (FFT), a Cepstrum transform, or a Wavelet transform, and the frequency domain signal (spectrogram) may be 2D or 3D.

[0208] The artificial neural network 1430 takes the frequency domain signal (spectrogram) as an input signal, extracts and classifies the characteristics of the input signal based on a trained road surface classification model, and can estimate the type of the road surface.

[0209] Meanwhile, the signal converter 1420 may include an analog-to-digital converter (ADC), which may convert the analog signal of the received signal into a digital signal.

[0210] An atmospheric attenuation correction unit (not shown) can calculate and correct the atmospheric attenuation of the digital signal.

[0211] The artificial neural network 1430 can use the converted signal or the corrected digital signal as an input signal, perform convolution on the input signal based on a trained road surface classification model, and classify the input signal to estimate the type of the road surface.

[0212] Meanwhile, the artificial neural network 1430 can perform classification and learning using at least one of decision trees, linear discriminant analysis, logistic regression classifiers, naive Bayes classifiers, support vector machines, nearest neighbor classifiers, and ensemble classifiers.

[0213] Decision trees include Fine tree, Medium tree, Coarse tree, All tree, and Optimizable tree; linear discriminant analysis includes Linear discriminant, Quadratic discriminant, All discriminants, and Optimizable discriminant; naive Bayes classifiers include Gaussian Naive Bayes, Kernel Naive Bayes, All Naive Bayes, and Optimizable Naive Bayes; support vector machines (SVMs) include Linear SVM, Quadratic SVM, Cubic SVM, Fine Gaussian SVM, Medium Gaussian SVM, Coarse Gaussian SVM, All SVM, and Optimizable SVM; and nearest neighbor classifiers include Fine KNN, Medium KNN, Coarse KNN, Cosine KNN, Cubic KNN, Weighted KNN, and All These include KNN and Optimizable KNN, and ensemble classifiers may include Boosted trees, Bagged trees, Subspace Discriminant, Subspace KNN, RUSBoosted trees, All Ensembles, and Optimizable Ensemble.

[0214] The control unit (MCU) 1440 can control the operation of the acoustic wave transceiver 1410, the signal converter 1420, and the artificial neural network 1430.

[0215] The signal converter 1420 and the artificial neural network 1430 are representations of program-implemented software components.

[0216] Meanwhile, although not shown in the drawings, the road surface type estimation device using sound waves according to various embodiments of the present disclosure includes a storage device (memory) that stores the learned road surface classification model and the software realized by the program. The storage device (memory) may be configured to be included in the control unit (MCU).

[0217] Meanwhile, the road surface type estimation device using sound waves according to the present disclosure may further include an atmospheric sensor (not shown) capable of measuring the temperature, humidity, and atmospheric pressure in the atmosphere.

[0218] Depending on the control of the control unit 1440, the atmospheric information including temperature, humidity and pressure may be used in the atmospheric attenuation correction unit or may be passed to the input of an artificial neural network 1430.

[0219] FIG. 15 is a diagram illustrating a transmission signal and a reception signal in a road surface type estimation device using sound waves according to an embodiment of the present disclosure.

[0220] As shown in FIG. 15, the control unit (MCU) 1440 can transmit a trigger signal having a predetermined magnitude (v: trigger voltage) and a preset transmission period (p: transmission period) to the acoustic wave transceiver unit 1410.

[0221] The sonic wave transmitter 1411 of the sonic wave transceiver 1410 can output a sonic signal 1501 having a specific frequency, for example, 40 kHz, to a corresponding road surface whose type is to be identified.

[0222] Then, the sonic receiver 1412 of the sonic transmitter / receiver 1410 can receive the reflected signal that is reflected back from the road surface.

[0223] Here, the signal 1502 received on the timeline like the acoustic signal 1501 may be a crosstalk signal of the acoustic signal transmitted by the acoustic transceiver 1410. Also, the control unit 1440 may determine a signal 1503 for a predetermined time from the point where the amplitude received after the transmission delay is the largest as the received signal.

[0224] For example, if the time when the amplitude of the signal received after the crosstalk signal is greatest is t_0, then a total of (a+b) ms from t_0-a [ms] to t_0+b [ms] can be observed, and in the received signal 203 of Figure 15, a is 0.2 and b is 5. a and b are variable values ​​that can be adjusted depending on the environment and conditions.

[0225] In the example of FIG. 15, one transmission cycle is 10 ms, which is the time it takes for the transmitted sound wave signal to fully disappear, and the sampling frequency of the sound wave transmitting / receiving unit 1410 is 1 MHz, which samples at 25 times the sound wave frequency of 40 kHz.

[0226] Meanwhile, as described above, the road surface condition can be sensed by sensing multiple received signals according to the transmission period, or the road surface condition can be sensed by transmitting a sound wave once and then processing a single reflected signal received.

[0227] FIG. 16 is a diagram for exemplifying a signal converter in a road surface type estimation device using sound waves according to an embodiment of the present disclosure.

[0228] In FIG. 16, an example will be described in which an STFT transformer is used as the signal transformer 1420.

[0229] As shown in FIG. 16, the STFT transformer can obtain a 2D spectrogram 1602 by performing a short-time Fourier transform on signals 1503 and 1601 received for a predetermined time after a transmission delay, excluding the crosstalk signal 1502 of the acoustic signal transmitted by the acoustic wave transmitting / receiving unit 1410 from the reflected signals received in FIG. 15.

[0230] The signal 1503 for one period may be Fourier transformed, or the received signal for multiple periods may be Fourier transformed.

[0231] In the present disclosure, materials can be distinguished using acoustic impedance and surface roughness information, etc. Acoustic impedance is not a constant and its value can change depending on the frequency at which sound waves vibrate. Therefore, analysis in the frequency domain can be useful. A time Fourier transform, which is one of several methods for converting a received time-domain signal into a frequency-domain signal, can be used. A short-time Fourier transform can also be used to confirm the FFT for each time (sampling time).

[0232] Furthermore, frequency analysis can be performed using not only Short-Time Fourier Transform but also Wavelet, and in one embodiment of the present disclosure, the STFT is used to reduce the amount of calculation and ensure sufficient data.

[0233] The short-time Fourier transform (STFT) is a method devised to take into account time variations that could not be resolved by existing Fourier transforms. The STFT divides a long signal that varies over time into short time units and then applies the Fourier transform.

[0234] Since the STFT separates a signal according to the window length, it shortens the length of the signal used for the Fourier transform, which can degrade the frequency resolution. On the other hand, increasing the window length to improve the frequency resolution can actually degrade the time resolution. To overcome the resolution limit due to this trade-off between frequency and time, the Wavelet Transform (WT) can be used.

[0235] If the window length in STFT is fixed, WT is to repeat STFT while changing the window length. Also, if the STFT uses sine curves that extend to infinity in time as the basis functions, wavelets are several types of functions that exist within a finite period. Wavelet functions include Morlet, Daubechies, Coiflets, Biorthogonal, Mexican Hat, and Symlets.

[0236] FIG. 17 is a diagram illustrating an artificial neural network in a road surface type estimation device using sound waves according to an embodiment of the present disclosure.

[0237] 17, the artificial neural network may include a deep neural network (DNN) of a multi-layer perceptron algorithm, including an input layer 1701, multiple hidden layers 1702, and an output layer 1703. Additionally, the artificial neural network according to various embodiments of the present disclosure may include a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm, further including a convolution performing unit (not shown).

[0238] The input layer 1701 can flatten the data of the spectrogram 1702 and input it in 1D.

[0239] Data input to the input layer 1701 can undergo feature extraction and classification via multiple hidden layers 1702.

[0240] The output layer 1703 can output a probability value for each type of learned road surface.

[0241] The artificial neural network can use softmax 1704 to determine and output the type of road surface that has the highest probability among the probability values ​​output from the output layer 1703.

[0242] Meanwhile, the artificial neural network can also receive atmospheric information (temperature, humidity, and barometric pressure information) and use it as input for the input layer 1701 .

[0243] The artificial neural network can also use the Fourier transform of the sound wave signal 1501 transmitted to the road surface as an input to the input layer 1701 .

[0244] Meanwhile, when the artificial neural network is a DCNN, the convolution execution unit may perform convolution operations on the transmitted digital input signal multiple times, perform batch normalization, an activation (ReLU) function, and a maxpooling function for each convolution operation, and output flattened data from the final convolution operation to the transfer layer.

[0245] In this case, the transfer layer is the input layer 1701 of the CNN, and the flattened output data of the convolution execution unit can be input in one dimension (1D), and the subsequent operations are the same as those described above.

[0246] FIG. 18 is a diagram for explaining the operation of the convolution execution unit.

[0247] The convolution execution unit may perform 1D convolution operations on the input signal multiple times (e.g., 5 times), and perform batch normalization, activation (ReLU) function, and maxpooling function for each convolution operation, and the output of the last convolution operation may be flattened data.

[0248] 18, for example, an input signal 1801 is approximately 7000 received signals over a period of 7 ms, a first convolution result 1802 is the result of performing 1D conv(64,16), BN, ReLU, and MP(8) on the input signal 1801, a second convolution result 1803 is the result of performing 1D conv(32,32), BN, ReLU, and MP(8) on the first convolution result 1802, a third convolution result 1804 is the result of performing 1D conv(16,64), BN, ReLU, and MP(8) on the second convolution result 1803, and a fourth convolution result 1805 is the result of performing 1D conv(16,64), BN, ReLU, and MP(8) on the third convolution result 1804. The fifth convolution execution result 1806 is the result of executing 1D conv(4,2568), BN, and ReLU on the fourth convolution execution result 1805.

[0249] FIG. 19 is a diagram for explaining the code of the convolution execution unit of the road surface type estimation device using sound waves according to the present disclosure.

[0250] FIG. 19 shows code that implements part of the convolution execution unit shown in FIG. 18 in software.

[0251] The code can include multiple alignment normalization (BatchNorm) functions and a max pooling function (MaxPool).

[0252] As an example that can be used according to one embodiment of the present disclosure, one-dimensional convolution is followed by one-dimensional alignment normalization, and then max pooling is performed, which is repeated four times as a set, and finally, a value equal to the number of road surfaces to be classified is output through a fully connected layer, and a probability is output for each road surface.

[0253] Meanwhile, in this disclosure, a method of performing one-dimensional (1D) convolution calculation has been described as an example, but convolution calculations can be performed not only in 1D but also in 2D and 3D.

[0254] FIG. 20 is a flowchart of a method for estimating road surface type using domain transformation of sound waves according to an embodiment of the present disclosure.

[0255] First, in order to execute the road surface type estimation method using domain transformation of sound waves according to the present disclosure, a learning step 2001 can be carried out first to generate a road surface classification model.

[0256] In the training step 2001, after transmitting sound wave signals to multiple types of road surfaces, the reflected signals are received and converted into frequency domain signals (e.g., spectrograms), and the frequency domain signals (spectrograms) can be input into the artificial neural network to train the road surface classification model.

[0257] Here, a Short-Time Fourier Transform (STFT), a Fast Fourier Transform (FFT), a Cepstrum transform, or a Wavelet transform can be used to convert the frequency domain signal into a frequency domain signal, which can be 2D or 3D.

[0258] Then, according to the control of the control unit, a sound wave signal is transmitted to the road surface whose type is to be identified, and the reflected signal is received (2002).

[0259] Then, under the control of the control unit, a signal transformation can be performed on a predetermined domain of the received signal to obtain a frequency domain signal (2003).

[0260] In the frequency domain signal acquisition step 2003, the crosstalk signal of the transmitted acoustic signal is removed from the received signal, and the frequency domain signal can be acquired by performing domain transformation on the signal received for a predetermined time after a transmission delay for each period of the acoustic signal.

[0261] Then, under the control of the control unit, the frequency domain signal is used as an input signal for an artificial neural network, and based on the learned road surface classification model, characteristics of the input signal are extracted and classified to determine the type of road surface (2004).

[0262] The artificial neural network may include a deep neural network (DNN) of a multi-layer perceptron algorithm, which includes an input layer 1701, multiple hidden layers 1702, and an output layer 1703. In this case, the output layer 1703 may output a probability value for each learned road surface type, and the artificial neural network may determine and output the road surface type with the highest probability using softmax 1704.

[0263] The artificial neural network receives atmospheric information (temperature, humidity, barometric pressure information) which can also be used as input for the input layer.

[0264] The artificial neural network can also use the Fourier transform of the sound wave signal transmitted to the road surface as the input of the input layer.

[0265] FIG. 21 is a flowchart illustrating an embodiment of a method for estimating road surface type using sound waves according to the present disclosure.

[0266] First, in order to execute the road surface type estimation method using sound waves according to the present disclosure, a learning step 2101 can be carried out first to generate a road surface classification model.

[0267] In the learning step 2101, after transmitting sound wave signals to multiple types of road surfaces, the reflected signals are received and converted into digital signals, and the converted digital signals are input into an artificial neural network to perform multiple convolution operations to learn the road surface classification model.

[0268] Thereafter, the sound wave signal is transmitted to the road surface whose type is to be identified according to the control of the control unit, and the reflected signal is received (2102).

[0269] Thereafter, analog signals can be converted to digital signals for a predetermined range of the received signals under the control of a control unit (2103).

[0270] In the signal conversion step 2103, the received signal can be converted into a digital signal for a predetermined time period, excluding the crosstalk signal of the transmitted sound wave signal, based on the point where the amplitude of the signal received after the transmission delay is the largest for each period of the sound wave signal.

[0271] For example, if the time when the amplitude of the signal received after the crosstalk signal is greatest is t_0, then the total (a+b) ms from t_0-a [ms] to t_0+b [ms] can be observed, where a and b can be variably adjusted depending on the environment and conditions.

[0272] Thereafter, the digital signal can be received and multiple convolution operations can be performed in the artificial neural network under the control of a control unit (2104).

[0273] In the convolution operation step 2104, the convolution operation is performed multiple times on the digital signal, and batch normalization, activation (ReLU) function, and maxpooling function can be performed for each convolution operation, and the output of the final convolution operation is flattened data.

[0274] Meanwhile, in this disclosure, a method of performing one-dimensional (1D) convolution calculation has been described as an example, but convolution calculations can be performed not only in 1D but also in 2D and 3D.

[0275] Thereafter, under the control of the control unit, the characteristics of the convoluted signal (flattened data) can be extracted and classified based on the road surface classification model trained by the artificial neural network to determine the type of road surface (2105).

[0276] The artificial neural network may include a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm, including a convolution execution unit that inputs the digital signal and executes multiple convolution operations, a transfer layer, multiple hidden layers, and an output layer. In this case, the output layer may output a probability value for each learned road surface type, and the artificial neural network may determine and output the road surface type with the highest probability using softmax.

[0277] The artificial neural network receives atmospheric information (temperature, humidity, barometric pressure information) which can also be used as input for the convolution performer.

[0278] The artificial neural network may also use the sound wave signal transmitted to the road surface as an input to the convolution performer.

[0279] FIG. 22 is a flowchart illustrating a method for estimating a road surface type using sound waves corrected for atmospheric attenuation according to an embodiment of the present disclosure.

[0280] First, in order to execute the road surface type estimation method using sound waves according to the present disclosure, a learning step 2201 can be carried out first to generate a road surface classification model.

[0281] In the learning step 2201, after transmitting sound wave signals to multiple types of road surfaces, the reflected signals are received, converted into digital signals, the converted digital signals are corrected for atmospheric attenuation, and then converted into frequency domain signals, and the frequency domain signals are input into a neural network to learn a road surface classification model.

[0282] Here, in the learning step 2201, a Short-Time Fourier Transform (STFT), a Fast Fourier Transform (FFT), a Cepstrum transform, or a Wavelet transform can be used to convert the frequency domain signal, which can be 2D or 3D.

[0283] Thereafter, according to the control of the control unit, a sound wave signal is transmitted to the road surface whose type is to be identified, and then the reflected signal can be received (2202).

[0284] Thereafter, analog signals can be converted to digital signals for a predetermined range of the received signals in response to control of the control unit (2203).

[0285] In the signal conversion step 2203, the received signal is converted into a digital signal for a predetermined time period, excluding the crosstalk signal of the transmitted sound wave signal, based on the point where the amplitude of the signal received after the transmission delay is the largest for each period of the sound wave signal.

[0286] For example, if the time when the amplitude of the signal received after the crosstalk signal is greatest is t_0, then the total (a+b) ms from t_0-a [ms] to t_0+b [ms] can be observed, where a and b can be adjusted depending on the environment and conditions.

[0287] Thereafter, the amount of atmospheric attenuation of the digital signal can be calculated and corrected according to the control of the control unit (2204).

[0288] In the atmospheric attenuation correction step 2204, atmospheric attenuation can be calculated and corrected using the following <Equation 1> to <Equation 8>. The atmospheric attenuation correction step may be performed by a control unit or an atmospheric attenuation correction unit which is software realized by a program.

[0289] First, the saturation pressure Psat can be calculated using the following mathematical formula 1.

[0290] [Mathematical formula 1] JPEG0007729657000001.jpg15170

[0291] where To1 is the atmospheric triple point [K] and T is the current temperature [K].

[0292] Absolute humidity (h) can be calculated using the following formula:

[0293] [Mathematical formula 2] JPEG0007729657000002.jpg11170

[0294] where hrin is the relative humidity [%], Psat is the saturation pressure [units], and Ps is the static pressure [atm].

[0295] Meanwhile, the scaled relaxation frequency for nitrogen (FrN), which accounts for 78% of the atmosphere, can be calculated using the following Equation 3:

[0296] [Mathematical formula 3] JPEG0007729657000003.jpg26170

[0297] Here, To is the reference temperature [K] and T is the current temperature [K].

[0298] Meanwhile, the scaled relaxation frequency for oxygen (FrO), which accounts for 21% of the atmosphere, can be calculated using the following Equation 4:

[0299] [Mathematical formula 4] JPEG0007729657000004.jpg19170

[0300] where h is the absolute humidity.

[0301] On the other hand, the attenuation coefficient (α: attenuation coefficient [nepers / m]) can be calculated using the following mathematical formula 5.

[0302] [Mathematical formula 5] JPEG0007729657000005.jpg38170

[0303] where Ps is the static pressure, F is the frequency of the acoustic signal (transmitted acoustic signal), T is the current temperature [K], To is the reference temperature [K], FrO is the extended relaxation frequency of oxygen, and FrN is the extended relaxation frequency of nitrogen.

[0304] Meanwhile, the attenuation ratio (A, unit: dB) of the sound wave signal can be calculated using the following <Equation 6>.

[0305] [Mathematical formula 6] JPEG0007729657000006.jpg8170

[0306] Here, α is the attenuation coefficient, and d is the distance between the sonic wave transceiver 100 and the road surface whose type is to be identified.

[0307] The above d can be calculated using the following Equation 7 using the time t (time of flight) required for the signal to be transmitted from the transmitter, reflected on the road surface, and detected by the receiver, and the speed of sound in the atmosphere (Vair).

[0308] [Mathematical formula 7] JPEG0007729657000007.jpg6170

[0309] Here, t is the time required and Vair is the speed of sound in the atmosphere [m / s].

[0310] On the other hand, the speed of sound in the atmosphere can be calculated using the following mathematical formula 8.

[0311] [Mathematical formula 8] JPEG0007729657000008.jpg15170

[0312] Here, Ks is the coefficient of stiffness of the object, and ρ is the density of the object (air).

[0313] In this case, if we assume that air (atmosphere) is an ideal gas, Ks = γP, where γ is the heat capacity ratio (1.4 for air), P is pressure, R is the ideal gas constant, and T is absolute temperature [K]. Since these are constants excluding temperature, they can also be expressed approximately.

[0314] The speed of sound in the atmosphere can also be corrected according to the temperature, pressure and humidity of the atmosphere and used for attenuation compensation.

[0315] Thereafter, a signal transformation may be performed on a predetermined domain of the corrected digital signal under the control of a controller to obtain a frequency domain signal (2205).

[0316] In the frequency domain signal acquisition step 2205, the crosstalk signal of the transmitted acoustic signal is removed from the corrected digital signal, and the frequency domain signal can be obtained by frequency converting the signal for a predetermined time received after the transmission delay for each period of the acoustic signal using a signal converter.

[0317] Then, under the control of the control unit, the frequency domain signal is used as an input signal for a neural network, and based on the learned road surface classification model, characteristics of the input signal are extracted and classified to determine the type of road surface (2206).

[0318] The neural network may include a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer, multiple hidden layers, and an output layer. In this case, the output layer outputs a probability value for each learned road surface type, and the neural network may determine and output the road surface type with the highest probability using softmax.

[0319] On the other hand, the structure of the neural network is not limited to the above-mentioned DNN.

[0320] The neural network receives atmospheric information (temperature, humidity, and pressure information) and can also use it as input for the input layer.

[0321] The neural network can also frequency-convert the sound wave signal transmitted to the road surface and use it as an input to the input layer.

[0322] On the other hand, instead of acquiring a frequency domain signal in the frequency domain signal acquisition step and inputting it into the artificial neural network, a digital signal corrected for atmospheric attenuation can be transmitted according to the control of the control unit, and multiple convolution operations can be performed in the artificial neural network.

[0323] In the convolution operation step, 1D convolution operations are performed multiple times on the atmospheric attenuation corrected digital signal, and batch normalization, activation (ReLU) function, and maxppooling function can be performed for each convolution operation, and the output of the final convolution operation is flattened data.

[0324] Then, under the control of the control unit, the characteristics of the convoluted signal (flattened data) can be extracted and classified based on the road surface classification model learned by the artificial neural network to determine the type of road surface.

[0325] In the case of performing convolution, the artificial neural network may include a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm, which further includes a convolution execution unit that receives the digital signal and performs multiple convolution operations. The output layer may output a probability value for each learned road surface type, and the artificial neural network may determine and output the road surface type with the highest probability using softmax.

[0326] Meanwhile, the structure of the artificial neural network is not limited to the above-mentioned DCNN.

[0327] The artificial neural network receives atmospheric information (temperature, humidity, barometric pressure information) which can also be used as input for the convolution performer.

[0328] The artificial neural network may also use the sound wave signal transmitted to the road surface as an input to the convolution performer.

[0329] A specific embodiment in which a road surface classification device according to an embodiment of the present disclosure is installed in road infrastructure and operated will be described in more detail below.

[0330] 23 is a diagram illustrating a road condition monitoring system including a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure, and is a specific embodiment of the road infrastructure shown in FIG.

[0331] As shown in FIG. 23, a structure 2301 is located on or near a road 2300, and the structure 2301 is provided with an acoustic sensor 2310 and a vision sensor 2320.

[0332] The ultrasonic sensor 2310 is located on the vehicle's path on the road 2300 and is mounted on the structure 2301 so as to be perpendicular to the road surface, and the vision sensor 2320 can be installed on the structure 2301 so as to capture the entire area of ​​the road.

[0333] Meanwhile, FIG. 23 shows a communication unit 2350 for transmitting data acquired from the sonic wave sensor 2310 and the vision sensor 2320 to a control unit (not shown).

[0334] With the development of artificial neural networks, solutions combining vision sensors with artificial intelligence models are spreading across all industries, and the vision sensor 2320 is one of the mainstream technologies in the fields of object recognition, detection, and segmentation. Advances in artificial intelligence technology have enabled the realization of algorithms that enable the vision sensor 2320 to operate in a similar way to how humans intuitively recognize objects and distinguish areas from photographs (images).

[0335] Meanwhile, object recognition using the ultrasonic sensor 2310 is possible by analyzing the waveform of the signal reflected after hitting the target surface using sound waves, and utilizes the principle that the reflected wave is determined according to the acoustic impedance and surface roughness of the target surface. In other words, by utilizing a wide range of sound spectrum, the ultrasonic sensor 2310 becomes resistant to external noise and can recognize black ice on the road surface.

[0336] One embodiment includes disclosing a method for accurately recognizing a wide range of road surfaces by combining the advantages of a vision sensor, which is capable of intuitively recognizing and classifying a wide area, with sonic sensor technology, which can accurately recognize objects without being affected by light sources by utilizing the physical properties of the target object.

[0337] FIG. 24 is a configuration diagram of a road condition monitoring system equipped with a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure.

[0338] As shown in FIG. 24, a road condition monitoring system equipped with a vision sensor and an ultrasonic sensor according to one embodiment of the present disclosure may include an ultrasonic sensor 2410, a vision sensor 2420, an artificial neural network 2430, a segmentation processing unit 2440, and a control unit 2470.

[0339] The sonic sensor 2410 can transmit sonic signals to predetermined points for road condition monitoring and then receive reflected signals.

[0340] The vision sensor 2420 can acquire an image of the road surface including the predetermined point.

[0341] The artificial neural network 2430 may classify the road surface condition of the predetermined point based on a road surface classification model trained using the reflected signal acquired by the sonic sensor 2410 as an input signal. The road surface conditions may include dry, water, black ice, snow, etc.

[0342] The segmentation processing unit 2440 may use an image acquired by the vision sensor 2420 as an input signal and divide the image into a plurality of segmented segmentation regions based on a segmentation model.

[0343] The control unit 2470 controls the operation of the sonic sensor 2410, the vision sensor 2420, the artificial neural network 2430, and the segmentation processing unit 2440, and can determine the road surface condition of the corresponding road by fusing the road surface condition of the preset point output from the artificial neural network 2430 and a plurality of segmented areas output from the segmentation processing unit 2440.

[0344] The process by which the control unit 2470 determines the road surface condition of the corresponding road will be described in detail with reference to FIG.

[0345] The control unit 2470 can calculate a segmentation area including the point where the sonic sensor hits the ground. However, at the time of installing the system according to the present disclosure, the ground impact point (sensing area) of the sonic sensor is preferably set in the normal wet and icy section of the road surface, and the position of the sensing area can be already known by the system.

[0346] Finally, image information can be output in which the classification class (type of road surface) of the waveform data is assigned to the segmentation area including the point where the sonic sensor hits the ground (sensing area).

[0347] FIG. 25 is a diagram illustrating an example of recognizing the state of a uniform road surface in a road condition monitoring system equipped with a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure.

[0348] Referring to Figure 25, (a) is an image captured by the vision sensor 2420, displaying the position of the sensing area (predetermined area) 2500 of the sonic sensor 2410. (b) shows the segmented area displayed after segmenting the captured image of (a). (c) classifies the type of road surface based on a learned road surface classification model (artificial intelligence model) and senses the road surface condition of the sensing area 2500. (d) shows an image in which a segmentation area including the area 2500 sensed in (c) is found from the segmentation area of ​​(b), and the final black ice area is displayed.

[0349] Here, in (b), the entire road area is divided into one segmentation area, and the sensing area 2500 that senses the road surface through the sonic sensor 2410 is sensed as black ice, so the final result can be output as in (d).

[0350] FIG. 26 is a diagram illustrating an example of recognizing the state of an uneven road surface in a road condition monitoring system including a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure.

[0351] Referring to Figure 26, (a) is an image captured by the vision sensor 2420, showing the position of the sensing area (predetermined area) 2600 of the sonic sensor 2410. (b) shows the segmented area displayed after segmenting the captured image of (a). (c) classifies the type of road surface based on a learned road surface classification model (artificial intelligence model) and senses the road surface condition of the sensing area 300. (d) shows an image in which a segmentation area including the area 2600 sensed in (c) is found from the segmentation area of ​​(b), and the final black ice area is displayed.

[0352] Here, (b) shows that the road is divided into multiple segmentation areas including wet and dry areas, and the sensing area 2600 that senses the road surface through the sonic sensor 2410 is sensed as black ice, so the final result can be output as shown in (d).

[0353] In other words, the present disclosure may include a technology that can reliably determine, via a vision sensor, the issue of to which portion / area of ​​the road surface image obtained by the vision sensor the road surface information accurately recognized by the acoustic sensor applies.

[0354] In the present disclosure, the operation of the road surface detection algorithm can be performed periodically (minutes, seconds) or asynchronously, and the risk of road slippage can be detected by acquiring data through an ultrasonic sensor and a vision sensor, detecting the type (condition) of the road surface based on the data acquired by the ultrasonic sensor, and detecting the area including the detected part of the ultrasonic sensor through image segmentation in the image acquired by the vision sensor.

[0355] In other words, if danger such as black ice is detected on the road surface, the danger information can be combined with the image divided into regions as a result of segmentation, and a notification of the dangerous section of the road surface can be sent to the administrator (control server).

[0356] FIG. 27 is a diagram for explaining a method for finding out which segmentation region is the sensing region of the ultrasonic sensor in a road condition monitoring system equipped with a vision sensor and an ultrasonic sensor according to an embodiment of the present disclosure.

[0357] The control unit can determine the road surface condition of the corresponding road by combining the road surface condition of the preset point output from the artificial neural network with the multiple segmented areas output from the segmentation processing unit 2440.

[0358] The control unit may calculate the position of the midpoint of each of the segmented areas, calculate a straight line equation for a plurality of line segments included in each of the segmented areas (a plurality of line segments forming each of the segmented areas), determine a first relative positive / negative relationship between the midpoint of the corresponding area and each of the plurality of line segments using the straight line equation for each of the segmented areas, determine a second relative positive / negative relationship between the predetermined point and each of the plurality of line segments using the straight line equation for each of the segmented areas, and determine a segmentation area where the second relative positive / negative relationship matches the first relative positive / negative relationship as an area including the predetermined point.

[0359] To explain this in more detail with reference to FIG. 27, it is assumed that the input RGB image is segmented and then divided into a plurality of regions, of which regions A and B are separated as shown in the figure.

[0360] The midpoint of area A is indicated by "2701", the midpoint of area B is indicated by "2702", and the sensing area of ​​the ultrasonic sensor (pre-set point) is indicated by "2700".

[0361] In FIG. 27, the positive and negative relationships are such that the right and bottom sides are in the (+) direction and the left and top sides are in the (-) direction, with the point (0,0) at the top left of the image as the reference.

[0362] Area A is formed into a pentagon and consists of segments 14, 45, 56, 67, and 71.

[0363] The positive / negative relationships between the midpoint 2701 of area A and each of the line segments 14, 45, 56, 67, and 71 of area A are (-), (-), (-), (+), and (+), respectively.

[0364] Area B is formed into a rectangle and consists of line segments 12, 23, 34, and 41.

[0365] The positive / negative relationships between the midpoint 2702 of area B and each of the line segments 12, 23, 34, and 41 of area B are (+), (-), (-), and (+), respectively.

[0366] In this case, the positive / negative relationship between the ultrasonic sensor's sensing area (predetermined point) 2700 and each line segment 14, 45, 56, 67, and 71 in area A is (+), (-), (-), (+), (+), respectively, and the positive / negative relationship between the ultrasonic sensor's sensing area (predetermined point) 2700 and each line segment 14, 45, 56, 67, and 71 in area A is (+), (-), (-), (+), (+), respectively.

[0367] Therefore, the sensing area (predetermined point) 2700 of the acoustic wave sensor is included in the B area.

[0368] FIG. 28 is a diagram illustrating an example of an artificial neural network of a road condition monitoring system including a vision sensor and an acoustic sensor according to an embodiment of the present disclosure.

[0369] Referring to FIG. 28, the artificial neural network can be formed by an artificial intelligence model implemented as either a 1D CNN (Conventional Neural Network) or an ANN (Artificial Neural Network).

[0370] The input of the artificial neural network may be the reflected waves received through the sonic sensor, and the output may be the type of road surface in the pre-defined sensing area of ​​the sonic sensor.

[0371] FIG. 29 is a diagram illustrating an example of a segmentation processing unit of a road condition monitoring system including a vision sensor and an acoustic wave sensor according to an embodiment of the present disclosure.

[0372] Referring to FIG. 29, the segmentation processing unit may be formed as an image segmentation model based on a convolutional artificial neural network (CNN) implemented using an auto-encoder or U-Net.

[0373] The input of the segmentation processing unit is an RGB image acquired through a vision sensor, and the output is a segmentation image that displays the areas segmented in the corresponding image.

[0374] FIG. 30 is a flowchart of one embodiment of a monitoring method in a road condition monitoring system equipped with a vision sensor and an acoustic sensor according to the present disclosure.

[0375] First, for road condition monitoring, a sonic sensor may transmit a sonic signal to a preset point and then receive a reflected signal (3010).

[0376] Thereafter, the road surface condition of the predetermined point can be classified based on a road surface classification model trained using the reflected signal acquired by the sonic wave sensor as an input signal (3020).

[0377] Meanwhile, while the sonic sensor receives the reflected signal after transmitting the sonic wave, the vision sensor can acquire an image of the road surface including the predetermined point (3030).

[0378] The image captured by the vision sensor may then be segmented 3040 into a plurality of distinct segmentation regions based on a segmentation model.

[0379] Thereafter, the road surface condition at the predetermined point may be fused with the plurality of divided segmented regions to analyze the road surface condition (3050).

[0380] Thereafter, the road surface condition of the road can be determined according to the analysis in the fusion analysis step 3050 (3060).

[0381] Then, it can be determined whether or not a danger is detected based on the road surface condition (3070).

[0382] As a result of the determination in the danger determination step 3070, since no danger is detected in the road surface condition of the road, the process can periodically proceed to steps 3010 and 3030.

[0383] Meanwhile, as a result of the determination in the danger determination step 3070, if a danger is detected in the road surface condition of the corresponding road, a signal informing the danger can be transmitted to the control server (3080).

[0384] Meanwhile, in the danger notification step 3080, a danger area is displayed on an image including the plurality of divided segmented areas, and can be transmitted to the control server.

[0385] FIG. 31 is a detailed flowchart of one embodiment of the fusion analysis step 3050 of FIG.

[0386] The fusion analysis step 3050 may include performing the following steps:

[0387] First, the position of the midpoint of each segmented region is calculated in the segmented image (3051).

[0388] Then, equations of lines are calculated for the multiple line segments included in each segmented region (multiple line segments forming each segmented region) (3052).

[0389] Then, for each of the segmented regions, a first relative positive / negative relationship between the midpoint of the region and each of the plurality of line segments is determined using the equation of the line (3053).

[0390] Then, for each of the segmented regions, a second relative positive / negative relationship between the predetermined point and each of the plurality of line segments is determined using the equation of the line (3054).

[0391] Thereafter, a segmentation area in which the second relative positive / negative relationship and the first relative positive / negative relationship match is determined as an area in which the predetermined point is included (3055).

[0392] FIG. 32 is a configuration diagram of a road condition monitoring system equipped with a vision sensor and an acoustic wave sensor according to yet another embodiment of the present disclosure.

[0393] As shown in FIG. 32, a road condition monitoring system equipped with a vision sensor and an ultrasonic sensor according to one embodiment of the present disclosure may include an ultrasonic sensor 3210, a vision sensor 3220, a first feature extraction unit 3281, a second feature extraction unit 3282, a joint classifier 3290, and a control unit 3270.

[0394] The sonic sensor 3210 can transmit sonic signals to predetermined points for road condition monitoring and then receive reflected signals.

[0395] The vision sensor 3220 can acquire an image of the road surface including the predetermined points.

[0396] The first feature extractor 3281 can extract a first feature from the reflected signal acquired by the acoustic wave sensor 3210 .

[0397] The second feature extraction unit 3282 can extract a second feature from the image acquired by the vision sensor 3220 .

[0398] The combination artificial neural network 3290 receives the first and second features as input and can classify the road surface condition of the corresponding road based on a road surface data combination classification model learned from the signals acquired by the sonic sensor 3210 and features extracted from the images acquired by the vision sensor 3220. The road surface conditions can include dry road, water, black ice, snow, etc.

[0399] The control unit 3270 can control the operation of the acoustic wave sensor 3210 , the vision sensor 3220 , the first feature extraction unit 3281 , the second feature extraction unit 3282 , and the combinational artificial neural network 3290 .

[0400] In the combination artificial neural network 3290, the first feature extracted from the reflected signal and the second feature extracted from the image can be trained and classified by a classification model (data combination classification model) with separate weights.

[0401] The values ​​of the first feature extracted from the reflected signal and the second feature extracted from the image can be used to learn object (road surface type) classification by input combination of image data and sound wave data using correlation. In addition, data characteristics can be analyzed and the weights and influences of the classifier based on image data and the weights and influences of the classifier based on sound wave data can be adjusted to train for final decision making (prediction).

[0402] FIG. 33 is a flowchart of another embodiment of a monitoring method in a road condition monitoring system including a vision sensor and an acoustic sensor according to the present disclosure.

[0403] First, for road condition monitoring, a sonic sensor may transmit a sonic signal to a preset point and then receive a reflected signal (3310).

[0404] A first feature of the reflected signal can then be extracted 3320 .

[0405] Meanwhile, while the sonic sensor receives the reflected signal after transmitting the sonic wave, the vision sensor can acquire an image of the road surface including the predetermined point (3330).

[0406] A second feature of the image can then be extracted (3340).

[0407] Then, the road surface condition of the road can be analyzed based on a classification model learned by combining the first feature extracted from the reflected signal and the second feature extracted from the image (3350).

[0408] Here, the first feature extracted from the reflected signal and the second feature extracted from the image can be trained and classified by a classification model having separate weights.

[0409] Thereafter, the road surface condition of the road in question can be determined (3360) according to the analysis in the road surface condition analysis step 3350.

[0410] Then, it can be determined whether a danger is detected in the road surface condition (3370).

[0411] As a result of the determination in the danger determination step 3370, since no danger is detected in the road surface condition of the road, the process can periodically proceed to steps 3310 and 3330.

[0412] Meanwhile, as a result of the determination in the danger determination step 3370, if a danger is detected in the road surface condition of the corresponding road, a signal informing the danger can be transmitted to the control server (3380).

[0413] Meanwhile, in the danger notification step 3380, a danger area is displayed on an image including the plurality of segmented areas, and can be transmitted to the control server.

[0414] Below, a specific embodiment for controlling a hot wire device or a saline spray device will be described in more detail as an example of a road surface management device according to an embodiment of the present disclosure.

[0415] 34 is a diagram illustrating the operation of a control system for a road hot-wire device according to an embodiment of the present disclosure, and is a specific embodiment of the road infrastructure shown in FIG.

[0416] As shown in FIG. 34, in a control system for a road heating device according to one embodiment of the present disclosure, a structure 3401 is located on a road, and the structure 3401 may include an ultrasonic sensor 3410 and a communication unit 3420, etc., and based on the sensing data of the ultrasonic sensor 3410, an automatic control box 3460 that controls the heating element 3470 in the anti-icing device 3400 can be controlled by the control server 3440.

[0417] The sonic sensor 3410 may be located on a vehicle path within the road and installed on the structure 3401 so as to be perpendicular to the road surface, but is not limited to this.

[0418] The structure 3401 means that a sonic sensor 3410 can be installed on the road, like a street light.

[0419] The sonic sensor 3410 can transmit a sonic signal to a predetermined point to sense road conditions and then receive a reflected signal.

[0420] Meanwhile, the communication unit 3420 can transmit data acquired through the sonic sensor 3410 to the control server 3440 .

[0421] In addition, when the control server 3440 detects whether or not the sonic sensor 3410 is broken (abnormal), it can notify the administrator terminal 3450 of the failure.

[0422] In anti-icing devices, the hot wire method can cause a fire on the asphalt if the hot wire is operated for longer than necessary, so it is important to consider how long the hot wire should be operated.

[0423] In the present disclosure, the temperature change of the road surface due to the heat ray heating is detected by the sonic sensor, and the anti-icing device can be precisely controlled.

[0424] Specifically, it is possible to generate an artificial intelligence model trained based on sonic sensing data accumulated in road environments with various temperatures, analyze the waveforms of the sonic sensor acquired based on the artificial intelligence model to detect changes in road surface temperature, and automatically control the operation of the heating element device.

[0425] For example, the system may be configured to start operating the heating wire after determining whether the road surface is dry or frozen through waveform analysis of the ultrasonic sensor, and stop operating the heating wire if the road surface temperature on the ultrasonic sensor output remains above 4 degrees, i.e., if the temperature is above the freezing point of water.

[0426] Furthermore, according to the present disclosure, it is possible to sense how much salt water has been sprayed onto the road surface while the salt water spraying device is operating.

[0427] Specifically, an artificial intelligence model is generated that is trained based on sonic sensing data accumulated in road surface environments with various distributions of salt water spray accuracy, and the waveform of the sonic sensor acquired based on the artificial intelligence model is analyzed to determine the accuracy (spray accuracy) of the spray (distribution) of salt water on the road surface. If the salt water is distributed over a predetermined range or more on the road surface, the system may be configured to stop spraying salt water.

[0428] The present disclosure relates to road surface detection and linked control technology that can timely activate installed / operating road hot wire devices or saline spray devices. Instead of the conventional method of acquiring road surface information using temperature / humidity sensors that attach sensors to the road surface, the technology recognizes road surface conditions and determines snow melting conditions based on sonic sensors, allowing for accurate control of the operation of the hot wire devices or saline spray devices.

[0429] It can be linked to existing snow melting equipment monitoring / control systems using a plug-in method without major modifications, and can provide a service that can improve the operating efficiency of snow melting equipment based on more accurate road hazard notifications than existing snow melting systems.

[0430] The algorithm for determining whether or not to melt snow may be constructed in the control server (service server) 3440 or in a control unit (MCU) provided together with the ultrasonic sensor 3410, and may be transmitted to the automatic control box 3460 of the anti-icing device 3400 via a communication unit.

[0431] The road surface condition detection is performed repeatedly periodically until the sonic sensor needs to be restored due to a malfunction, etc. After receiving a reflected wave (sensor value) from the sonic sensor 3410, it is transmitted to the control server 3440 (service server) via the communication unit. If the sensor status is normal, the control server 3440 analyzes the reflected wave using an artificial intelligence model in a big database, determines whether or not current snow melting work is necessary, and controls whether or not to operate the corresponding anti-icing device.

[0432] Meanwhile, the sonic sensor determines whether there is an abnormality based on the transmitted sensor value, and if the sensor status is abnormal, transmits a stop command to the automatic control box 3460 of the anti-icing device 3400, transmits a push alarm regarding the occurrence of the abnormality to the administrator terminal 3450, and transmits a history of operation stop due to the malfunction to the control server (not shown).

[0433] Fig. 35 is a configuration diagram of a control system for a road anti-icing device according to an embodiment of the present disclosure. The control system of Fig. 35 is a specific example of a road surface management device according to an embodiment of the present disclosure.

[0434] As shown in FIG. 35 , a control system for a road anti-icing device according to one embodiment of the present disclosure may include an acoustic sensor 3510, a control server 3540, a communication unit 3520, and an anti-icing device 3500.

[0435] The sonic sensor 3510 can transmit a sonic signal to a predetermined point to sense road conditions and then receive the reflected signal.

[0436] The control server 3540 can sense road surface condition data at the preset point based on an artificial intelligence analysis model trained using the reflected signal acquired by the ultrasonic sensor 3510 as an input signal, and generate a signal to control whether or not the anti-icing device 3500 operates according to the road surface condition data at the preset point.

[0437] The road surface condition data may include weather conditions, road surface type, road surface temperature, and salt injection amount (injection accuracy, distribution), etc.

[0438] The communication unit 3520 may transmit the reflected signal acquired by the sonic sensor 3510 to the control server 3540 .

[0439] The anti-icing device 3500 performs operations to prevent icing on the road under the control of the control server 3540. The anti-icing device 3500 may include at least one of a hot wire device or a saline spray device.

[0440] In detail, the control server 3540 can generate a control signal to operate the hot-wire device when the weather condition is "rain" or "snow", the classified road surface type is "wet road", "snowy road", or "icy road", and the detected road surface temperature is less than 4°C, and after the hot-wire device has operated, generate a control signal to stop the operation of the hot-wire device when the detected road surface temperature is 4°C or higher.

[0441] Meanwhile, the control server 3540 can generate a control signal to operate the salt water spray device when the weather condition is "rain" or "snow", the classified road surface type is "wet road", "snowy road", or "icy road", and the detected road surface temperature is less than 4°C, and can generate a control signal to stop the operation of the salt water spray device when the detected amount of salt water sprayed is 80% or more after the salt water spray device has started operating.

[0442] Meanwhile, when the control server 3540 detects an abnormality in the status of the sonic sensor 3510, it can transmit a notification message to the administrator terminal 3550 and transmit a status notification signal of the sonic sensor 3510 to the management server 3580.

[0443] FIG. 36 is a configuration diagram of a control system for a road anti-icing device according to another embodiment of the present disclosure.

[0444] As shown in FIG. 36, a control system for a road anti-icing device according to one embodiment of the present disclosure includes an acoustic sensor 3610, a control unit 3630, a communication unit 3620, and an anti-icing device 3600.

[0445] The sonic sensor 3610 can transmit a sonic signal to a predetermined point to sense road conditions and then receive the reflected signal.

[0446] The control unit 3630 can sense road surface condition data at the preset point based on an artificial intelligence analysis model learned using the reflected signal acquired by the ultrasonic sensor 3610 as an input signal, and generate a signal to control whether or not the anti-icing device 3600 operates based on the road surface condition data at the preset point.

[0447] The road surface condition data may include weather conditions, road surface type, road surface temperature, and salt injection amount (injection accuracy, distribution), etc.

[0448] The communication unit 3620 can transmit a control signal generated by the control unit 3630 to the anti-icing device 3600 .

[0449] The anti-icing device 3600 performs an operation to prevent icing on the road under the control of the control unit 3630. The anti-icing device 3600 may include at least one of a hot wire device or a saline spray device.

[0450] In detail, the control unit 3630 can generate a control signal to operate the heating element when the weather condition is "rain" or "snow", the classified road surface type is "wet road", "snowy road" or "icy road", and the detected road surface temperature is less than 4°C, and after the heating element is operated, generate a control signal to stop the operation of the heating element when the detected road surface temperature is 4°C or higher.

[0451] Meanwhile, the control unit 3630 can generate a control signal to operate the salt water spray device when the weather condition is "rain" or "snow", the classified road surface type is "wet road", "snowy road" or "icy road", and the detected road surface temperature is less than 4°C, and can generate a control signal to stop the operation of the salt water spray device when the detected amount of salt water sprayed is 80% or more after the salt water spray device has started operating.

[0452] Meanwhile, when the control unit 3630 detects an abnormality in the state of the sonic sensor 3610, it can transmit a notification message to the administrator terminal 3650 and transmit a status notification signal of the sonic sensor 3610 to the control server 3680.

[0453] 37a to 37c are diagrams for explaining an artificial intelligence analysis model used in a control system for a road anti-icing device according to an embodiment of the present disclosure.

[0454] The artificial intelligence analysis model may include a weather condition classification model that classifies weather conditions based on the reflected signals acquired by the sonic sensor, and a road surface type classification model that classifies the type of road surface based on the reflected signals acquired by the sonic sensor.

[0455] If the anti-icing device is a heating element, the artificial intelligence analysis model may further include a road surface temperature regression model that learns the reflected signal acquired by the sonic sensor together with the temperature of the road surface and outputs the temperature of the corresponding road surface based on the reflected signal acquired by the sonic sensor.

[0456] When the anti-icing device is the salt water spray device, the artificial intelligence analysis model may further include a road surface temperature regression model that learns the reflected signal acquired by the ultrasonic sensor together with the temperature of the road surface and outputs the temperature of the corresponding road surface based on the reflected signal acquired by the ultrasonic sensor, and a salt water spray amount regression model that learns the reflected signal acquired by the ultrasonic sensor together with the amount of salt water sprayed (distribution accuracy) and outputs the amount of salt water sprayed (distribution accuracy) based on the reflected signal acquired by the ultrasonic sensor.

[0457] That is, the artificial intelligence analysis model according to the present disclosure basically includes a weather condition classification model, a road surface type classification model, and a road surface temperature regression model, and if the anti-icing device includes the saline spray device, it may further include a saline spray amount regression model.

[0458] Figure 37a shows the structure of a road surface type classification model, which is constructed by sampling the signals (reflected waves) acquired by the ultrasonic sensor a total of T times over a certain period of time and learning the type information of the corresponding road surface together.

[0459] For example, the type of road surface corresponding to 1000 pieces of data (x1, x2, ..., x1000) sampled in 1 ms intervals for 1 second is learned.

[0460] As shown in Figure 37a, the acquired signal (reflected wave) of the sonic sensor can be input and the type of road surface can be classified into classes such as dry road (dry), wet road (wet), icy road (iced), and snow-covered road (snow).

[0461] On the other hand, although not shown in the drawings, the present invention may further include a weather condition classification model that classifies weather conditions based on the reflected signal acquired by the ultrasonic sensor, and further configured by learning the acquired signal of the ultrasonic sensor and weather information together.

[0462] Figure 37b is a diagram explaining a road surface temperature regression model that outputs the temperature of the corresponding road surface based on the reflected signal acquired by the ultrasonic sensor in order to control the hot wire device, and Figure 37c is a diagram explaining a salt water injection amount regression model that outputs the distribution amount (distribution accuracy) (%) of the corresponding salt water based on the reflected signal acquired by the ultrasonic sensor in order to control the salt water control device.

[0463] In FIG. 37b, the relationship between the data X acquired by the sonic sensor and the road surface temperature is not a two-dimensional (planar) graph, but rather X is a dataset formed using the concept of a hyperplane, which is a collection of multiple values.

[0464] Accordingly, the trained road surface temperature regression model outputs the temperature of the corresponding road surface based on the reflected signal acquired by the sonic wave sensor.

[0465] In Figure 37c, as in Figure 37b, the relationship between the data X acquired by the ultrasonic sensor and the amount of salt water sprayed (distribution accuracy) is not a two-dimensional (planar) graph, but rather X is a dataset formed using the concept of a hyperplane, which is a collection of multiple values.

[0466] Accordingly, the learned saline solution injection amount regression model outputs the corresponding saline solution injection amount (distribution accuracy) based on the reflected signal acquired by the ultrasonic wave sensor.

[0467] FIG. 38 is a flowchart of one embodiment of a method for controlling a road anti-icing device according to the present disclosure.

[0468] First, measurement data from the sonic sensor is collected (3810).

[0469] An artificial intelligence analysis model is generated based on the collected data (3820).

[0470] The artificial intelligence analysis model generation step 3820 generates a weather condition classification model that classifies weather conditions based on the reflected signals acquired by the ultrasonic sensor, generates a road surface type classification model that classifies the type of road surface based on the reflected signals acquired by the ultrasonic sensor, and generates a road surface temperature regression model that learns the reflected signals acquired by the ultrasonic sensor together with the road surface temperature and outputs the corresponding road surface temperature based on the reflected signals acquired by the ultrasonic sensor.

[0471] On the other hand, if the anti-icing device is a salt water spraying device, the artificial intelligence analysis model generation step 3820 learns the reflected signal acquired by the ultrasonic sensor together with the amount of salt water sprayed (distribution accuracy), and generates a salt water spray amount regression model that outputs the corresponding amount of salt water sprayed (distribution accuracy) based on the reflected signal acquired by the ultrasonic sensor.

[0472] That is, the artificial intelligence analysis model according to the present disclosure basically generates and includes a weather condition classification model, a road surface type classification model, and a road surface temperature regression model, and if the anti-icing device includes the saline solution injection device, it further generates a saline solution injection amount regression model.

[0473] Of course, the generated artificial intelligence analysis model must be installed in the control server or control unit.

[0474] Thereafter, the sonic sensor is used to transmit sonic signals to predetermined points for road condition monitoring, and then receives reflected signals (3830).

[0475] Then, the control server or the control unit senses road surface condition data of the preset point based on the artificial intelligence analysis model using the reflected signal acquired by the transmitted sonic sensor as an input signal.

[0476] The road surface condition data sensing step 3840 senses weather conditions, road surface type, road surface temperature, and salt spray amount (distribution accuracy), etc.

[0477] Thereafter, the control server or the control unit generates a control signal for controlling whether or not the anti-icing device is operated based on the road surface condition data (3850).

[0478] Thereafter, when the control server or the control unit detects an abnormality in the state of the sonic sensor, it transmits a notification message to the administrator terminal and transmits a notification signal of the state of the sonic sensor to the management server (3860).

[0479] FIG. 39 is a detailed flowchart of one embodiment of the control signal generation step 3850 of FIG. 38 when the road de-icing device according to the present disclosure is a hot wire device.

[0480] If the anti-icing device is a heating element, the control signal generating step 3850 first determines whether the sensed weather condition is "rain" or "snow" (3910).

[0481] If the weather condition is not "rain" or "snow" as determined in the determination step 3910, the process proceeds to step 3840, where road surface condition data is sensed.

[0482] On the other hand, if the weather condition is determined to be "rain" or "snow" in the determination step 3910, it is determined whether the type of the classified road surface is "wet," "snow," or "icy" (3920).

[0483] If the weather condition is determined to be "rain" or "snow" in the determination step 3920, and the classified road surface type is not "wet," "snow," or "icy," the process proceeds to step 3840, where road surface condition data is sensed.

[0484] On the other hand, if the weather condition is determined to be "rain" or "snow" in the determination step 3920, and the type of the classified road surface is "wet," "snow," or "iced," it is determined whether the sensed road surface temperature is below 4°C (3930).

[0485] If the result of the determination in the determination step 3930 is that the road surface temperature is not less than 4° C., the process proceeds to step S540, where road surface condition data is sensed.

[0486] On the other hand, if the result of the determination in decision step 3930 is that the road surface temperature is less than 4° C., a control signal for operating the hot wire device is generated (3940).

[0487] Thereafter, the process proceeds to road surface condition data sensing step 3940, where road surface condition data is sensed.

[0488] After the hot wire device is activated, it is determined whether the detected road surface temperature is 4° C. or higher (3950).

[0489] If the result of the determination in the determination step 3950 is that the road surface temperature is not 4° C. or higher, the process proceeds to step S540, where road surface condition data is sensed.

[0490] On the other hand, if the result of the determination in the determination step 3950 is that the road surface temperature is 4° C. or higher, a control signal to stop the operation of the heating element is generated (3960).

[0491] Thereafter, the process proceeds to road surface condition data sensing step 3840, where road surface condition data is sensed.

[0492] That is, when the weather condition is "rain" or "snow", the classified road surface type is "wet road", "snowy road" or "icy road", and the detected road surface temperature is less than 4°C, the heating element is activated, and when the detected road surface temperature is 4°C or higher after the heating element is activated, the heating element is deactivated.

[0493] FIG. 40 is a detailed flowchart of one embodiment of the control signal generation step 3850 of FIG. 38 when the road de-icing device according to the present disclosure is a saline sprayer.

[0494] If the road de-icing device is a saline sprayer, the control signal generating step 3850 first determines whether the sensed weather condition is "rain" or "snow" (4010).

[0495] If the weather condition is not "rain" or "snow" as a result of the determination in the determination step 4010, the process proceeds to step S540, where road surface condition data is sensed.

[0496] On the other hand, if the weather condition is determined to be "rain" or "snow" in the determination step 4010, it is determined whether the type of the classified road surface is "wet", "snow", or "icy" (4020).

[0497] If the weather condition is determined to be "rain" or "snow" in the determination step 4020, and the classified road surface type is not "wet," "snow," or "icy," the process proceeds to step 3840, where road surface condition data is sensed.

[0498] On the other hand, if the weather condition is determined to be "rain" or "snow" in the determination step 4020, and the type of the classified road surface is "wet," "snow," or "iced," it is determined whether the sensed road surface temperature is below 4°C (4030).

[0499] If the result of the determination in the determination step 4030 is that the road surface temperature is not less than 4° C., the process proceeds to step 3840, where road surface condition data is sensed.

[0500] On the other hand, if the result of the determination in the determination step 4030 is that the road surface temperature is less than 4° C., a control signal for operating the salt water spray device is generated (4040).

[0501] Thereafter, the process proceeds to road surface condition data sensing step 3840, where road surface condition data is sensed.

[0502] After the saline spraying device is operated, it is determined whether the amount of saline sprayed (spraying accuracy) is 80% or more (4050).

[0503] If it is determined in the determination step 4050 that the amount of salt water sprayed (spray accuracy) is not 80% or more, the process proceeds to step 3840, where road surface condition data is sensed.

[0504] On the other hand, if the result of the determination in the determination step 4050 is that the amount of saline spray (spray accuracy) is 80% or more, a control signal for stopping the operation of the saline spray device is generated (4060).

[0505] Thereafter, the process proceeds to road surface condition data sensing step 3840, where road surface condition data is sensed.

[0506] That is, when the weather condition is "rain" or "snow", the classified road surface type is "wet road", "snowy road" or "icy road", and the detected road surface temperature is less than 4°C, the saline spray device is operated, and after the saline spray device is operated, the operation of the saline spray device is stopped when the detected amount of salt water sprayed is 80% or more.

[0507] Meanwhile, although it has been described that the amount of salt water spray (spray accuracy, distribution) that causes the operation of the salt water spraying device to stop is 80% or more, the present invention is not limited to this.

[0508] On the other hand, although an example has been given in which both the road surface temperature regression model and the salt water injection amount regression model are used in combination to control the operation of the salt water injection device, it is also possible to control the operation of the salt water injection device using only the salt water injection amount regression model.

[0509] Although the above examples have been given in which a road anti-icing device includes a hot wire device or a saline spray device, the present disclosure is not limited to this, and it is also possible to control a system that includes both a hot wire device and a saline spray device.

[0510] It goes without saying that the methods according to the embodiments of the present disclosure can be realized by a computer-readable recording medium having a program for realizing the method stored therein and / or a program for realizing the method stored in a computer-readable recording medium.

[0511] That is, it will be readily understood by those skilled in the art that a program of instructions for implementing the method according to the embodiment of the present disclosure may be tangibly embodied and provided in a computer-readable recording medium. In other words, it may be embodied in the form of program instructions that can be executed by various computer means and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., alone or in combination.

[0512] When the methods of the present disclosure are implemented in software, a computer-readable storage medium storing one or more programs (software modules) can be provided. The one or more programs stored on the computer-readable storage medium are configured for execution by one or more processors in an electronic device. The one or more programs include instructions that cause the electronic device to perform the methods according to the embodiments described in the claims or specification of the present disclosure.

[0513] Such programs (software modules, software) can be stored in random access memory, non-volatile memory including flash memory, ROM (Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM), magnetic disc storage device, compact disc-ROM (CD-ROM), digital versatile discs (DVDs), other forms of optical storage, magnetic cassette, or in memory consisting of a combination of some or all of these. Furthermore, each type of memory may be included in multiple locations.

[0514] The program may also be stored in an attachable storage device accessible via a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a combination thereof. Such a storage device may be connected to a device that executes an embodiment of the present disclosure via an external port. Alternatively, a separate storage device on the communication network may be connected to a device that executes an embodiment of the present disclosure.

[0515] The following describes an exemplary method for installing the road infrastructure of the present disclosure on a road. More specifically, the structures in the following description are the structures shown in Figures 2, 23, and 34, and relate to the structure and installation method of the structures of the present disclosure.

[0516] The following will specifically explain the present disclosure with reference to FIGS. 41 to 49 showing the embodiment.

[0517] According to one embodiment of the present disclosure, a structure 4100, which is a main component of the road infrastructure sensor installation structure of the present disclosure, includes a vertical frame 4110 erected on a road or the edge of the road, and a horizontal frame 4120 installed on top of the vertical frame 4110 in the width direction of the road. For example, the structure 4100 may be a U-shaped structure for installing U-shaped streetlights or electronic billboards, or a high-pass IC structure. Any structure may be used as long as the horizontal frame 4120 is located above the road, and an ultrasonic sensor unit 4200, which will be described in detail later, may be installed below the horizontal frame 4120 and positioned above the road. As described above, the location of the ultrasonic sensor unit of the present disclosure is exemplary and is not limited to the location described in the present disclosure.

[0518] According to one embodiment of the present disclosure, the sonic sensor unit 4200, which is a main component of the road infrastructure sensor construction structure of the present disclosure, is installed at the bottom of the horizontal frame 4120 of the structure 4100, and is installed so as to be located above the road, irradiates sound waves onto the road surface, receives sound waves reflected from the road surface, and generates sonic information. The received sonic information is then transmitted to the control unit 4300, which will be described later, and the control unit 4300 converts the sonic information into a frequency, thereby enabling the condition of the road surface to be grasped.

[0519] Such an ultrasonic sensor unit 4200 of the present disclosure can be configured to include a transmitter that transmits an ultrasonic signal onto the road surface, receives the reflected ultrasonic signal, and outputs the transmitted signal under the control of the control unit 4300, and a receiver that receives a reflected signal that is the transmitted signal reflected back from any road surface.

[0520] In the ultrasonic sensor unit 4200 of the present disclosure, the transmitter and receiver are arranged so that ultrasonic waves are transmitted in a linear manner and reflected ultrasonic waves are received, but the ultrasonic sensor 4240 described below is expressed as being configured with the transmitter and receiver integrated.

[0521] On the other hand, in the method using sound waves to grasp the condition of the road surface, the transmitter and receiver are fixedly installed at a certain angle, so that the condition of only a very small area of ​​the road surface can be grasped. Therefore, in the present disclosure, through one embodiment consisting of a plurality of sound wave sensor units 4200 and another embodiment of the sound wave sensor unit 4200 consisting of a first transmitting / receiving member 4250 and a second transmitting / receiving member 4260 that can mutually receive emitted and reflected sound wave signals, it is possible to grasp the road surface condition of a relatively wider area than conventional methods, and to estimate the road condition with high reliability.

[0522] Specifically, in one embodiment, the ultrasonic sensor unit 4200 is installed at the bottom of the horizontal frame 4120 of the structure 4100, and multiple units are installed in the width direction of the road, thereby enabling the state of a wider road surface to be grasped than before, thereby achieving the effect of improving the reliability of grasping the state of the road surface.

[0523] In this case, the multiple sensor units 4200 provided in one embodiment are characterized in that the sound wave transmission time is set so that the sound waves irradiated onto and reflected from the road surface all arrive at the same time. This is because the arrival times of the reflected sound waves can all differ not only due to differences in the height of the road surface but also due to variables such as damage, and by setting the sound wave transmission time to be the same so that the sound wave characteristics of a normal road surface, which serves as the initial standard, can be extracted and compared in a relatively similar manner, the condition of the road surface can be quickly grasped.

[0524] The reason for setting the sound wave transmission times for each sonic wave sensor unit 4200 to have the same arrival time is that when generating sound wave information using the sound wave transmission times and arrival times, if multiple sound wave sensor units 4200 of the same device are installed, the sound waves have the same speed, so if the transmission times are all the same depending on variables such as the height of the road surface, the arrival times of the reflected sound waves will all be different and the sound wave information generated by each sound wave sensor unit 4200 will have to be analyzed.However, in order to make it easy to generate sound wave information using only the arrival times of the sound waves, by adjusting the sound wave transmission times of each sound wave sensor unit 4200 so that the arrival times of the sound waves are all the same depending on variables such as the height of the road surface, the difference between sound wave information with different arrival times can be quickly grasped and the road surface condition can be easily grasped quickly.

[0525] As an example, if the sound wave transmission times of the multiple sound wave sensor units 4200 are set according to the height of the road surface so that the arrival times are the same, when the road surface is in a normal state, the arrival times of the sound waves will all be the same, but when the road surface is in a damaged state or has black ice, the arrival times of the sound waves will differ from the preset arrival times of the sound waves, making it easy to determine that the condition of the entire road surface is different.However, if the arrival times of the sound waves of some of the multiple sound wave sensor units 4200 differ from the preset arrival times of the sound waves, it becomes possible to more quickly determine that the condition of only a part of the entire road surface is different.

[0526] In addition, in one embodiment, the multiple ultrasonic sensor units 4200 are characterized in that, when the time it takes for a sound wave to be transmitted from any one of the ultrasonic sensor units 4200 to be reflected on the road surface and return, i.e., the sound wave flight period, is t, n sensors sequentially transmit sound waves to the road surface a total of n or more times within t / 2 time, and receive the reflected sound waves during the remaining t / 2 time, thereby sampling road surface information and traffic volume information at least n times as much comprehensive road information within a specified period.

[0527] That is, the sonic wave sensor unit 4200 of one embodiment achieves the effect of detecting road surface conditions with improved reliability by sampling and generating a larger amount of sonic information such as road surface information and traffic volume information through a plurality of sonic wave sensors 4200. It will be apparent that the sonic wave sensor units 4200 of other embodiments described in detail later also sample and generate sonic information such as road surface information and traffic volume information through sound waves.

[0528] In addition, the sonic sensor unit 4200 in one embodiment includes a connecting beam 4210 having an upper portion connected to a lower portion of the horizontal frame 4120 of the structure 4100, a connecting rod 4220 having an upper portion hingedly connected to a lower portion of the connecting beam 4210 so as to be rotatable in the front-rear and left-right directions, a main body 4230 connected to an end of a lower portion of the connecting rod 4220, and a sonic sensor 4240 installed at a lower portion of the main body 4230. Here, the front-rear direction refers to the length direction of the road, and the left-right direction refers to the width direction of the road, and the front-rear and left-right directions (lateral directions) described below also refer to these directions.

[0529] The coupling member 4210 includes a lower receiving groove 4212 formed at the bottom and a ball bearing installed inside the lower receiving groove 4212, and the connecting rod 4220 includes an upper sphere 4224 at the end of its upper part that is inserted into and coupled to the inside of the lower receiving groove 4212 of the coupling member 4210, so that they are coupled in the form of a ball hinge that can rotate in the forward / backward and left / right directions.

[0530] At this time, it is obvious that the lower receiving groove 4212 is formed in a shape corresponding to the upper sphere 4224 of the connecting rod 4220, and the upper part of the connecting rod 4220 is connected to the lower part of the connecting rod 4210 with a ball hinge structure, so that the connecting rod 4220 can rotate in the forward / backward and left / right directions, and therefore, when vibrations occur in the structure 4100 due to vehicle traffic or external vibrations, shaking is prevented.

[0531] That is, the main body 4230 connected to the lower end of the connecting rod 4220 and the sonic sensor 4240 installed at the lower part of the main body 4230 are prevented from shaking by preventing the connecting rod 4220 from shaking, thereby achieving the effect of enabling the sonic sensor 4240 to stably irradiate sound waves onto the road surface set thereon.

[0532] The main body 4230 stably mounts the sonic sensor 4240 in the sonic sensor unit 4200 of one embodiment and in the sonic sensor unit 4200 of another embodiment described in detail later, and is connected to and controlled by the control unit 4300, which will be described in detail later. The sonic sensor 4240 is composed of a transmitter and a receiver, as described above, and generates sonic information by irradiating a sound wave onto a road surface and receiving the reflected sound wave, and transmits the generated sonic information to the control unit 4300.

[0533] Next, in another embodiment, the ultrasonic sensor unit 4200 includes a first transmitting / receiving member 4250 that is provided on one side of the lower part of the horizontal frame 4120 of the structure 4100 and that irradiates or receives ultrasonic waves onto or from the road surface, and a second transmitting / receiving member 4260 that is provided on the other side of the lower part of the horizontal frame 4120 of the structure 4100 and that irradiates or receives ultrasonic waves onto or from the road surface.

[0534] In this case, the first transmitting / receiving member 4250 is arranged at an angle that allows the second transmitting / receiving member 4260 to receive sound waves that are irradiated onto the road and reflected therefrom, and the second transmitting / receiving member 4260 is arranged at an angle that allows the first transmitting / receiving member 4250 to receive sound waves that are irradiated onto the road and reflected therefrom.

[0535] In other words, in another embodiment, the ultrasonic sensor unit 4200 transmits ultrasonic waves to the road surface from either the first transmitting / receiving member 4250 or the second transmitting / receiving member 4260, and the reflected ultrasonic waves are received by either the second transmitting / receiving member 4260 or the first transmitting / receiving member 4250, thereby achieving the effect that if the ultrasonic wave transmitting unit of either the first transmitting / receiving member 4250 or the second transmitting / receiving member 4260 fails, the road surface condition can be smoothly grasped using the other ultrasonic wave transmitting unit.

[0536] In addition, in another embodiment, the ultrasonic sensor unit 4200 can adjust the angle of either the first transmitting / receiving member 4250 or the second transmitting / receiving member 4260, and move the second transmitting / receiving member 4260 or the first transmitting / receiving member 4250 so that it can receive ultrasonic waves according to the adjusted angle, thereby enabling the condition of a wider road surface to be grasped than conventional methods.

[0537] Explaining in more detail, as Example 1 of another embodiment, the horizontal frame 4120 of the structure 4100 on which the ultrasonic sensor unit 4200 is installed includes a first rail 4122 installed longitudinally at the front lower part, and a rotary motor 4126 coupled to the first rail 4122 and controlled by the control unit 4300 to rotate the first rail 4122.

[0538] The ultrasonic sensor unit 4200 of Example 1 of another embodiment, i.e., the first transmitting and receiving member 4250 and the second transmitting and receiving member 4260, are configured to include a connecting bundle 4210 that is provided on one side and the other side of the first rail 4122 and moves in opposite directions in accordance with the rotation of the first rail 4122, a connecting rod 4220 whose upper part is connected to the lower part of the connecting bundle 4210 so as to be rotatable in the forward / backward and left / right directions, a main body 4230 connected to the lower end of the connecting rod 4220, a driving motor 4270 provided at the lower part of the main body 4230 and controlled by the control unit 4300, and an ultrasonic sensor 4240 provided at the lower part of the driving motor 4270 and rotated in the width direction of the road by the driving motor 4270 to adjust its angle.

[0539] That is, the ultrasonic sensor unit 4200 of Example 1 of another embodiment receives ultrasonic waves from the road surface area that was initially set, and by adjusting the angle of the first transmitting / receiving member 4250 and the second transmitting / receiving member 4260 and the widthwise position of the road by the control unit 4300, it is possible to receive ultrasonic waves from road surface areas other than the road surface area that was initially set, thereby obtaining the effect of being able to grasp the condition of a wider road surface than conventionally and improving the reliability of grasping the road surface condition.

[0540] In this case, the connection relationship between the first rail 4122 and the connecting bundle 4210 is such that threads are formed in different directions on the outer peripheral edges of one side and the other side of the first rail 4122, and the connecting bundle 4210 penetrates one side and the other side of the first rail 4122 and is screw-connected to the connecting bundle 4210, so that the connecting bundle 4210 moves laterally, i.e., toward one side or the other side, along the first rail 4122 rotated by the rotation motor 4126, and the pair of connecting bundles 4210 move in opposite directions along the threads formed on the outer peripheral edges of both sides of the first rail 4122.

[0541] Furthermore, to explain Example 2 of another embodiment in more detail, the horizontal frame 4120 of the structure 4100 on which the ultrasonic sensor unit 4200 is installed includes a first rail 4122 installed lengthwise at the front lower part, a second rail 4124 installed lengthwise at the rear lower part, and a rotary motor 4126 connected to the first rail 4122 and the second rail 4124, respectively, and controlled by the control unit 4300 described later to rotate the first rail 4122 and the second rail 4124.

[0542] The ultrasonic sensor unit 4200 of Example 2 of another embodiment, i.e., the first transmitting / receiving member 4250 and the second transmitting / receiving member 4260, includes a connecting bundle 4210 that is respectively provided on either the first rail 4122 or the second rail 4124 and moves in accordance with the rotation of the first rail 4122 or the second rail 4124, a main body 4230 that is provided below the connecting bundle 4210, a drive motor 4270 that is provided below the main body 4230 and controlled by the control unit 4300, and an ultrasonic sensor 4240 that is provided below the drive motor 4270 and rotates in the width direction of the road by the drive motor 4270 to adjust its angle.

[0543] That is, the ultrasonic sensor unit 4200 of Example 2 of another embodiment, like Example 1 of another embodiment, receives ultrasonic waves from the road surface area that was initially set, and by adjusting the angle and road width direction position of the first transmitting / receiving member 4250 and the second transmitting / receiving member 4260 by the control unit 4300, it is possible to receive ultrasonic waves from road surface areas other than the road surface area that was initially set, thereby obtaining the effect of being able to grasp the condition of a wider road surface than conventionally and improving the reliability of grasping the road surface condition.

[0544] At this time, the coupling relationship between the first rail 4122, the second rail 4124 and the coupling bundle 4210 is such that threads are formed on the outer peripheries of the first rail 4122 and the second rail 4124, and the first rail 4122 and the second rail 4124 penetrate and are screwed into the coupling bundle 4210, so that the coupling bundle 4210 moves laterally, i.e., to one side or the other, along the first rail 4122 and the second rail 4124 rotated by the rotation motor 4126.

[0545] In addition, in other embodiments, Example 1 and Example 2, the coupling relationship between the drive motor 4270 and the ultrasonic sensor 4240 is such that the drive motor 4270, which rotates left and right when activated, and the ultrasonic sensor 4240 are coupled via a gearbox 4280, and the drive motor 4270 is controlled by a control unit 4300, which will be described in detail later, thereby adjusting the installation angle of the ultrasonic sensor 4240.

[0546] In addition, the first transmitting / receiving member 4250 and the second transmitting / receiving member 4260 of Examples 1 and 2 of other embodiments can be configured to include a connecting rod 4220 that is hingedly connected to the connecting bundle 4210 so as to be rotatable in the forward / backward and left / right directions, similar to the ultrasonic sensor unit 4200 of the above-mentioned embodiment.

[0547] More specifically, the first transmitting and receiving member 4250 and the second transmitting and receiving member 4260 of Examples 1 and 2 of the other embodiment include a connecting rod 4220 whose upper portion is connected to the lower portion of the connecting bundle 4210 so as to be rotatable in the front-rear and left-right directions, and the main body 4230 of the first transmitting and receiving member 4250 and the second transmitting and receiving member 4260 is connected to the lower end of the connecting rod 4220, thereby having the same structure as the connecting bundle 4210, connecting rod 4220, and main body of the ultrasonic sensor unit 4200 of the previous embodiment. In this case, the connecting bundle 4210 and connecting rod 4220 are configured in the same manner as in the ultrasonic sensor unit 4200 of the previous embodiment.

[0548] In other words, in the ultrasonic sensor unit 4200 of Example 1 and Example 2 of other embodiments, i.e., the first transmitting / receiving member 4250 and the second transmitting / receiving member 4260, when vibrations occur in the structure 4100 due to the connecting rod 4220, the connecting rod 4220, the main body, and the ultrasonic sensor 4240 are prevented from shaking, thereby achieving the effect of more stable irradiation and reception of ultrasonic waves.

[0549] In addition, the main body 4230 of Examples 1 and 2, which are different from the first embodiment, includes an upper receiving groove 4234 formed on the upper part and a ball bearing installed inside the upper receiving groove 4234, and the connecting rod 4220 may include a lower sphere 4226 at the end of the lower part that is inserted into the inside of the upper receiving groove 4234 of the main body 4230 and connected thereto. As a result, the main body 4230 is connected to the connecting rod 4220 in a ball hinge shape so as to be rotatable in the forward / backward and left / right directions, thereby preventing shaking secondarily and enabling the ultrasonic sensor 4240 to transmit and receive ultrasonic waves more stably.

[0550] In addition, the connecting rod 4220 of Example 1 and Example 2, which are embodiments different from one embodiment, is connected to the upper outer periphery and connected so that the center is penetrated, and is located above the ultrasonic sensor 4240, and is configured to include a protective member 4222 which is formed in the center as a circular shape with a convex curved surface upward and is open at the bottom, while a reflective layer 4222a capable of reflecting ultrasonic waves is formed on the inner surface.

[0551] That is, the protective member 4222 is connected to the connecting rod 4220 by passing through the center, but is connected to be located above the sonic sensor 4240, so that even if the sonic signal reflected from the road surface is not directly received by the sonic sensor 4240, it is reflected and received by the reflective layer 4222a, thereby allowing as many sonic signals as possible to be received.

[0552] This usually has the effect of improving the reliability of measurements using sound waves by receiving as many sound waves as possible that move in a straight line in the line of sight, thereby improving the reliability of understanding the condition of the road surface.

[0553] In addition, the protective member 4222 protects the upper part of the sonic sensor 4240, thereby minimizing damage to the sonic sensor 4240 caused by ultraviolet rays or rainwater, or interference with reception of sound waves caused by bird droppings.

[0554] In this case, since the protective member 4222 is included, the main body 4230 is configured to include a plurality of auxiliary receiving sensors 4232 installed on the upper part, and the auxiliary receiving sensors 4232 have the effect of enabling more stable reception of reflected sound waves, since sound waves reflected by the reflective layer of the protective member 4222 are less likely to be received by the main body 4230 located on the upper part of the sound wave sensor 4240.

[0555] In addition, the rotary motor 4126 of the first and second embodiments has a rotary rod connected to the first rail 4122 and the second rail 4124 , respectively, and its upper portion is fixedly connected to the lower portion of the horizontal frame 4120 .

[0556] On the other hand, the ultrasonic sensor unit 4200 of an embodiment different from one embodiment of the present disclosure is characterized in that when irradiating the road surface with sound waves, the frequency of the sound waves is changed at preset time intervals via the control unit 4300. This makes it possible to generate ultrasonic information that can determine the road surface condition in more detail using other frequencies, thereby achieving the effect of more reliably grasping the road surface condition.

[0557] As an example, the ultrasonic sensor unit 4200 of an embodiment different from one embodiment of the present disclosure normally irradiates a 40 kHz sound wave onto the road surface, and when a preset time is reached, the control unit 4300 irradiates an 80 kHz sound wave onto the road surface, and when the preset time is reached again, the control unit 4300 irradiates a 120 kHz sound wave onto the road surface, thereby generating ultrasonic information that can determine the road surface condition in more detail.

[0558] That is, the ultrasonic sensor unit 4200 of an embodiment different from one embodiment of the present disclosure irradiates the road surface with sound waves at different frequencies at preset time intervals, but by irradiating the sound waves a preset number of times, the condition of the road surface can be determined in more detail, thereby achieving the effect of further improving the reliability of understanding the condition of the road surface.

[0559] The control unit 4300, which is a main component of the road infrastructure sensor construction structure of the present disclosure, is installed on the vertical frame 4110 of the structure 4100, and receives sound wave information from the sound wave sensor unit 4200 and transmits it to a central management server. Specifically, it extracts and classifies the characteristics of the sound wave signal from the sound wave information transmitted from the sound wave sensor 4240 to grasp the condition of the road surface, estimates the condition of the road surface, and transmits the estimated condition of the road surface to the central management server, thereby enabling measures to be taken to prevent accidents caused by the current condition of the road surface.

[0560] The control unit 4300 of the present disclosure not only controls the above-mentioned ultrasonic sensor 4240, but also automatically controls the rotation motor 4126 and the drive motor 4270 based on preset input values ​​or based on input information from a central management server.

[0561] In addition, the control unit 4300 of the present disclosure may include and control a signal converter that performs frequency conversion on a predetermined region on the domain of the sound wave information transmitted from the sound wave sensor 4240 to obtain a frequency domain signal, and an artificial neural network that extracts and classifies the characteristics of the input signal based on a road surface classification model learned using the frequency domain signal as an input signal to estimate the state of the road surface, etc.

[0562] That is, the control unit 4300 of the present disclosure can be included in a distribution board or terminal box and installed together in the vertical frame 4110 of the structure 4100, and can be configured to include a wired or wireless communication unit so as to transmit acoustic information to a central management server.

[0563] The method of constructing the road infrastructure sensor installation structure of the present disclosure as described above will be specifically described below.

[0564] The construction method for road infrastructure sensor construction structures disclosed herein includes a structure installation step 4S10 for installing a structure 4100 on a road, a sensor installation step 4S20 for installing an ultrasonic sensor unit 4200 in the structure 4100, and a terminal box installation step 4S30 for installing a control unit 4300 in the structure 4100.

[0565] The structure installation step 4S10 involves erecting a vertical frame 4110 on the road or the edge of the road, and connecting the horizontal frame 4120 to the top of the vertical frame 4110 in the width direction of the road, thereby installing a structure 4100 including the vertical frame 4110 and the horizontal frame 4120.

[0566] In this case, if the structure 4100 is "_" shaped like the street lights shown in Figures 41 to 44, it is installed in the same manner as above, and if it is "U" shaped like the advertising signboard structures shown in Figures 45 to 48, a pair of vertical frames 4110 are erected and a horizontal frame 4120 is provided to connect the tops of the pair of vertical frames 4110.

[0567] Furthermore, when the sonic wave sensor unit 4200 according to the other embodiment described above is installed, it will be apparent that a first rail 4122 and a second rail 4124 are provided on the lower part of the horizontal frame 4120 .

[0568] On the other hand, the structure installation step 4910 of the present disclosure can be omitted if the structure 4100 such as a streetlight or an electronic signboard structure has already been installed, since it has been performed in advance.

[0569] The sensor installation step 4920 is performed after the structure installation step 4910 by installing an acoustic wave sensor unit 4200, which irradiates acoustic waves onto the road surface, receives reflected acoustic waves, and generates acoustic wave information, on the horizontal frame 4120 of the structure 4100, and then installs the above-mentioned connecting bundle 4210 at the bottom of the horizontal frame 4120.

[0570] In this case, when the ultrasonic sensor unit 4200 of one embodiment is installed, the connecting bundle 4210 is fixedly connected to the lower part of the horizontal frame 4120, and when the ultrasonic sensor unit 4200 of another embodiment is installed, the connecting bundle 4210 is movably installed on the first rail 4122 and the second rail 4124 installed on the lower part of the horizontal frame 4120, respectively.

[0571] The coupling bundle 4210 includes a connecting rod 4220 connected to a lower portion when the ultrasonic sensor unit 4200 of one embodiment is installed, a protective member 4222 installed on the outer periphery of the connecting rod 4220, a main body 4230 connected to a lower portion of the connecting rod 4220, a plurality of auxiliary receiving sensors 4232 connected to an upper portion of the main body 4230, and an ultrasonic sensor 4240 connected to a lower portion of the main body 4230, or when the ultrasonic sensor unit 4200 of another embodiment is installed. In this case, an ultrasonic sensor unit 4200 is already connected, which includes a connecting rod 4220 connected to the lower part, a protective member 4222 installed on the outer periphery of the connecting rod 4220, a main body 4230 connected to the lower part of the connecting rod 4220, a plurality of auxiliary receiving sensors 4232 connected to the upper part of the main body 4230, a driving motor 4270 connected to the lower part of the main body 4230, and an ultrasonic sensor 4240 connected to the lower part of the driving motor 4270.

[0572] The terminal box installation step 4930 is performed after the sensor installation step 4920 by installing a control unit 4300 that is connected to the ultrasonic sensor unit 4200 on the vertical frame 4110 of the structure 4100 and is connected to a central management server via wire or wirelessly, so that the ultrasonic sensor unit 4200 is electrically connected so that it can be controlled by the control unit 4300.

[0573] In this case, the terminal box installation step 4930 further includes an initial transmission setting step in which, when an ultrasonic wave sensor unit 4200 of one embodiment is installed, the ultrasonic wave transmission time is set so that the arrival times of the ultrasonic waves reflected and received by the multiple ultrasonic wave sensor units 4200 are all the same, or, when an ultrasonic wave sensor unit 4200 of another embodiment is installed, the first transmitting / receiving member 4250 and the second transmitting / receiving member 4260, which are installed first, are configured to be able to receive the ultrasonic waves transmitted from each other.

[0574] As a result, the road infrastructure sensor system construction structure and construction method disclosed herein can not only smoothly grasp the condition of the road surface without contact using sound waves, but also improve the reliability of road surface condition estimation according to the grasp of the road surface condition through a wider measurement range than conventional methods. Due to the characteristics of sound waves, which have a tendency to travel in a straight line, the problem of difficulty in receiving even small disturbances can be overcome by allowing many sound waves to be received through the protective member 4222 on which the reflective layer 4222a is formed, thereby improving the reliability of measurement using sound waves. Furthermore, by minimizing the shaking of the sound wave sensor unit 4200 through the ball hinge structure when vibrations occur in the structure 4100 due to vehicle traffic or disturbances, and minimizing the reception of the natural frequency generated by the vibration of the structure 4100 through the protective member 4222, it is possible to obtain the effect of minimizing measurement errors and improving the reliability of measurement.

[0575] In the specific embodiments of the present disclosure described above, elements included in the disclosure are expressed as singular or plural in accordance with the specific embodiments presented. However, the expressions singular or plural are selected to fit the presented context for the convenience of explanation, and the present disclosure is not limited to singular or plural elements, and elements expressed in plural may be composed of singular elements, or elements expressed in singular may be composed of plural elements.

[0576] Meanwhile, the embodiments of the present disclosure disclosed in the specification and drawings merely present specific examples to easily explain the technical content of the present disclosure and to facilitate understanding of the present disclosure, and are not intended to limit the scope of the present disclosure. In other words, it is obvious to those skilled in the art that other modifications based on the technical concept of the present disclosure are possible. Furthermore, the embodiments disclosed in the present specification can be implemented in combination with each other as necessary. For example, parts of an embodiment different from one embodiment of the present disclosure can be combined with each other to implement an embodiment not described in the present specification.

[0577] However, in describing the methods of the present disclosure, the order of description does not necessarily correspond to the order of execution, and the methods may be reordered or executed in parallel.

[0578] Alternatively, the drawings illustrating the method of the present disclosure may include only some of the components, omitting some of the components within the scope that does not impair the essence of the present disclosure.

[0579] Furthermore, the method of the present disclosure can be implemented by combining part or all of the contents included in each embodiment within the scope that does not impair the essence of the disclosure.

Claims

1. 1. An electronic device for classifying road surfaces using acoustic signals, comprising: a transceiver configured to transmit and receive acoustic signals; an atmospheric sensor; at least one processor electrically connected to the transceiver and the air sensor; The at least one processor transmitting a first acoustic signal using the transceiver toward a target road surface spaced a first distance from the electronic device; receiving a reflected signal of the first acoustic signal from the target road surface using the transceiver; using the atmospheric sensor to obtain atmospheric information associated with the first acoustic signal; obtaining first data for the received reflected signal; generating second data by correcting the first data based on the atmospheric information; obtaining third data related to frequency domain information of the second data based on the second data; configured to determine the condition of the target road surface based on the third data and a road surface classification artificial neural network; the road surface classification artificial neural network is trained with a frequency domain data set generated based on a second sound signal transmitted from an electronic device installed at a location different from the location of the electronic device and reflected by a road surface at a second distance different from the first distance; The electronic device, wherein the at least one processor corrects the attenuation of the first sound wave signal based on a difference between a propagation distance of the first sound wave signal and a propagation distance of the second sound wave signal.

2. The electronic device of claim 1 , wherein the first distance is estimated based on a time point at which the first acoustic signal is transmitted and a time point at which the reflected signal is received.

3. The electronic device according to claim 1 , wherein the third data is obtained by performing a short-time Fourier transform on the second data.

4. The at least one processor a signal for controlling a road surface management device provided on the target road surface is generated based on the determined state of the target road surface; The electronic device according to claim 1 , wherein the road surface maintenance device includes a heat ray or a saline spray device.

5. The at least one processor Determine whether pre-set weather conditions are met, generating a signal to control the road surface management device when the preset weather conditions are met; determining whether the condition of the target road surface determined at the first time point has changed at the second time point; The electronic device described in claim 4, characterized in that when the first class determined as the state of the target road surface at the first time point is different from the second class determined as the state of the target road surface at the second time point, it is configured to determine whether or not to generate a control signal for a device installed on the target road surface based on the state of the target road surface determined at a third time point.

6. The condition of the target road surface is determined every first period; The at least one processor 2. The electronic device according to claim 1, wherein if the condition of the target road surface is determined to be of a first class, the electronic device is configured to determine the condition of the target road surface every second period.

7. The electronic device further includes at least one of an IR sensor that acquires temperature information of the target road surface or a vision sensor that acquires image information of the target road surface; The electronic device of claim 1 , wherein the at least one processor is configured to determine the condition of the target road surface based on the temperature information or the video information.

8. 1. A method for classifying road surfaces using acoustic signals implemented by an electronic device, comprising: transmitting a first acoustic signal toward a target road surface a first distance away from the electronic device; receiving a reflection signal of the first acoustic signal from the target road surface; obtaining atmospheric information associated with the first acoustic signal; acquiring first data for the received reflected signal; generating second data by correcting the first data based on the atmospheric information; obtaining third data related to frequency domain information of the second data based on the second data; determining the condition of the target road surface based on the third data and a road surface classification artificial neural network; the road surface classification artificial neural network is trained with a frequency domain data set generated based on a second sound signal transmitted from an electronic device installed at a location different from the location of the electronic device and reflected by a road surface at a second distance different from the first distance; The method, wherein the step of generating second data includes a step of correcting an attenuation amount of the first acoustic signal based on a difference between a propagation distance of the first acoustic signal and a propagation distance of the second acoustic signal.

9. 9. The method of claim 8, further comprising estimating the first distance based on a time point at which the first acoustic signal is transmitted and a time point at which the reflected signal is received.

10. 9. The method of claim 8, wherein the third data is obtained by a short-time Fourier transform of the second data.

11. The method further includes generating a signal for controlling a road surface management device provided on the target road surface based on the determined state of the target road surface; The method of claim 8, wherein the road surface maintenance device comprises a heat wire or a saline spray device.

12. The step of generating a signal to control the road surface management device includes: determining whether a preset weather condition is met; If the predetermined weather condition is satisfied, determining whether the condition of the target road surface determined at a first time point has changed at a second time point; The method of claim 11, further comprising: if the first class determined as the state of the target road surface at the first time point and the second class determined as the state of the target road surface at the second time point are different from each other, determining whether or not to generate a control signal for a device installed on the target road surface based on the state of the target road surface determined at a third time point.

13. the step of determining the condition of the target road surface further includes a step of determining the condition of the target road surface based on temperature information of the target road surface or image information of the target road surface, The condition of the target road surface is determined every first period; 9. The method according to claim 8, wherein if the condition of the target road surface is determined to be of the first class, the condition of the target road surface is determined every second period.

Images