Road-surface-type estimation device and method thereof

Abstract

The present invention relates to a road-surface-type estimation device and a method thereof and, more specifically, provides a road-surface-type estimation device and a method thereof, which use a soundwave sensor and learn, by using a an artificial neural network, a signal acquired by filtering only a signal reflected from a road surface among signals received from the sensor so as to estimate the type of the corresponding road surface.

Description

Road surface type estimation device and method thereof

[0001] The present invention relates to a road surface type estimation device and method thereof, and more particularly, to a road surface type estimation device and method thereof that transmits an acoustic signal, receives a reflected signal, and uses an artificial neural network to learn about the received signal to estimate the type of the corresponding road surface.

[0002] In the United States, approximately 6 million traffic accidents occur annually. In particular, slippery roads alone cause an average of over 1.2 million accidents annually in the United States, resulting in over 5,000 deaths. Consequently, slippery roads are emerging as a significant social problem, posing a significant burden to the government and insurance companies.

[0003] Korea is no exception. Black ice accidents occur every winter, with 6,500 incidents occurring in the past five years, resulting in over 200 deaths.

[0004] In Korea, snow melting occurs in four stages, all of which are manually relayed by humans, taking several hours to reach the final stage. The recent black ice accident on the Sangju-Yeongcheon Expressway was also a man-made disaster, as the official in charge checked the forecast but failed to request advance snow melting.

[0005] These accidents are due to the lack of sensors that can detect road conditions in real time and a notification system that uses these sensors.

[0006] Meanwhile, there are two main methods proposed so far to estimate the type of road surface or the friction coefficient of the road surface.

[0007] One is a direct method that involves causing physical friction, and the other is an indirect method that uses radio waves or sound waves.

[0008] First, direct methods include wheel-based methods and vehicle dynamics-based methods.

[0009] What these two methods have in common is that they estimate the coefficient of friction of the road surface by applying the brakes regardless of the driver's intention and the degree of deceleration of the vehicle.

[0010] This method requires the vehicle to step on the road surface where the coefficient of friction is to be determined, and it essentially requires a vehicle dynamics model. Furthermore, because it requires the brakes to be applied, it has limitations that affect ride comfort and fuel efficiency.

[0011] Meanwhile, indirect methods include those using electromagnetic waves and sound waves. Since both of these methods do not require physical friction, they can be used independently for vehicle dynamics models and have the advantage of knowing the condition of the road surface in advance without contact with the road surface.

[0012] These electromagnetic wave-based methods utilize cameras, lidar, radar, and other sensors. While capable of detecting objects from a considerable distance, they are relatively inaccurate and expensive, costing hundreds to thousands of dollars. Furthermore, the sheer volume of data makes it difficult to create datasets.

[0013] There is also a method that uses an image sensor, which is one of the electromagnetic wave methods, and distinguishes the type of road surface using a camera, which is an image sensor. However, image sensors are greatly affected by illuminance, and black ice is not easily visible to the human eye, so vision sensors cannot detect it. Due to these limitations, the average performance of methods using image sensors is less than 70%. In addition, since it requires not only an image sensor but also a high-performance image processing device, the cost is relatively high.

[0014] Just as black ice is invisible to the human eye, it cannot be detected by image sensors. In other words, image sensors have limitations in distinguishing between different types of road surfaces.

[0015] Meanwhile, as for the method using the above sound waves, there are passive sensing method and active sensing method. In the passive sensing method, there is a method that records the noise from the tire with a microphone for a certain period of time and then classifies it using an artificial neural network. In this method, the type of road surface can only be identified for the road surface that the tire has passed, and there is a limitation that the classification takes a long time. Meanwhile, among the active sensing methods, there is research that distinguishes the reflected road surface using only distance and reflection intensity information, but this has the disadvantage of being vulnerable to surrounding external disturbances.

[0016] Meanwhile, electric and hydrogen vehicles with sky-blue license plates are increasingly seen on the roads these days. Bloomberg predicts that by 2040, more than half of all vehicles sold will be electric, and one-third of all vehicles on the road will be electric.

[0017] To increase the mileage of electric vehicles, automakers are actively utilizing regenerative braking. However, it's not uncommon to hear of accidents caused by regenerative braking, such as loss of traction or wheel locking in the rear wheels.

[0018] Tesla's current regenerative braking settings allow you to change between Standard and Low, but there's no option to disable them. These accidents occur because the regenerative braking module's control cycle is 100ms, which is longer than other systems. Manufacturers don't allow for the regenerative braking feature to be disabled because it drastically reduces the vehicle's range.

[0019] It would be best to use regenerative braking depending on the condition of the road surface on which the vehicle is driving, but this problem can be said to occur because the type of road surface cannot be known in advance before the vehicle passes.

[0020] Meanwhile, when a road surface classification device is installed on a vehicle (mobile object) to perform road surface classification while driving, the Doppler effect increases the frequency difference between the original transmission frequency and the received signal as the vehicle (mobile object) speed increases. Consequently, the received signal distortion becomes more severe, necessitating the consideration of the vehicle's speed when classifying the road surface.

[0021] Furthermore, the method of transmitting a single acoustic signal toward the road surface and using the reflected signal to estimate the road surface type requires waiting for the time of flight (ToF: Time of Flight) for the acoustic signal to reach the road surface and return. Furthermore, the entire time, including the flight time, must be recorded (saved) to enable road surface classification. Consequently, the data size for a single road surface classification process is large, and signal processing time also increases.

[0022] Additionally, when the distance (interval) between transmitting and receiving sound waves is long, the transmitted and received sound signals are transmitted through the atmosphere (air) and naturally attenuated. Therefore, utilizing attenuated received sound signals can reduce the accuracy of systems used to determine road surface conditions.

[0023] Meanwhile, when actually installing or replacing sonic sensors on roads, workers must perform the work while exposed to the risk of accidents on the road with vehicles in motion, making installation and replacement difficult. Furthermore, the road must be controlled during this process, which is a major inconvenience. Therefore, the applicant needs to further develop the ease of installing and replacing sensors on roads. This requirement is also applicable to support posts installed around roads to accommodate objects other than sensors.

[0024] Meanwhile, conventional methods for assessing road conditions are uneconomical and inaccurate, making it difficult to secure sufficient road condition data and develop business models for utilizing such data. Furthermore, data utilization remains insufficiently sophisticated.

[0025] Korean Patent No. 10-1716270 discloses a device and method for estimating the condition of a forward ground.

[0026] Korean Patent No. 10-2136576 discloses a road surface condition measuring device and a road surface condition diagnosis system.

[0027] Korean Patent No. 10-2156295 discloses a method and device for estimating road surface types using ultrasonic signals.

[0028] Korean Patent No. 10-2207816 discloses a method and device for measuring road surface conditions using sensor data.

[0029] The present invention has been devised to solve the above-mentioned problems, and the purpose of the present invention is to provide a road surface type estimation device and method that estimate the type of a corresponding road surface by learning using an artificial neural network.

[0030] The problems to be solved by the present invention are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0031] According to one embodiment of the present invention, a road surface type estimation device using a vehicle-mounted sensor includes a vehicle-mounted sensor (100) that transmits a sound wave signal and then receives a reflected signal; an analog-to-digital converter (200) for converting an analog signal into a digital signal with respect to a preset region of a signal received from the vehicle-mounted sensor; a signal filter unit (300) for filtering and separating only a signal reflected from a road surface from the digital signal; an artificial neural network (400) that uses the filtered digital signal as an input signal and classifies the input signal based on a learned road surface classification model to estimate the type of the road surface; and a control unit (MCU) (500) that controls the operations of the vehicle-mounted sensor, the analog-to-digital converter, the signal filter unit, and the artificial neural network.

[0032] The above control unit (500) is characterized in that it determines the time of flight (ToF) through height information (h) from the ground for the vehicle's on-board sensor, and the signal filter unit (300) is characterized in that it filters a signal received for a preset time after the time of flight (ToF) under the control of the control unit (500).

[0033] The above control unit (500) is characterized in that it controls the operation of the analog-to-digital converter, the signal filter unit, and the artificial neural network to estimate the road surface type only when the driving speed is a km / h or more or the acceleration is b ㎨ or more (a and b are preset threshold values).

[0034] The artificial neural network (400) includes a frequency converter (410) for performing frequency conversion on a preset region in the time domain of the filtered digital signal received from the signal filter unit to obtain a frequency domain signal; and a neural network (420) for taking the frequency domain signal as an input signal, extracting and classifying the characteristics of the input signal based on the learned road surface classification model, and estimating the type of the road surface, wherein the neural network (420) is characterized in that it is a deep neural network of a multi-layer perceptron algorithm including an input layer, a plurality of hidden layers, and an output layer.

[0035] The above artificial neural network (400) is characterized by being a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the filtered digital signal transmitted from the signal filter unit and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504).

[0036] The above convolution performing unit (501) is characterized in that it performs multiple 1D convolution operations on an input signal, performs batch normalization, a Relu function, and a MaxPooling function for each convolution operation, and the output of the last convolution operation is flattened data, the output layer is characterized in that it outputs a probability value for each type of learned road surface, and the artificial neural network is characterized in that it determines and outputs the type of road surface with the highest probability using softmax.

[0037] In addition, a method for estimating a road surface type using a vehicle-mounted sensor according to an embodiment of the present invention for achieving the above-described object, the method for estimating a road surface type using a vehicle-mounted sensor, comprises: a reflection signal receiving step (S120) of transmitting an acoustic signal from a vehicle-mounted sensor and then receiving a signal that is reflected and received; a signal conversion step (S130) of converting an analog signal into a digital signal for a preset area of ​​the received signal under the control of the control unit; a signal filtering step (S140) of filtering and separating only a signal reflected from the road surface from the digital signal under the control of the control unit; a frequency domain signal acquisition step (S150) of performing frequency conversion on the filtered digital signal under the control of the control unit to obtain a frequency domain signal; And, according to the control of the control unit, the frequency domain signal is used as an input signal of a neural network, and, based on the learned road surface classification model, the characteristics of the input signal are extracted and classified to determine the type of the road surface, and a first road surface type determination step (S160) is included.

[0038] In addition, according to another embodiment of the present invention for achieving the above-described purpose, a method for estimating a road surface type using a vehicle-mounted sensor is characterized by including a reflection signal receiving step (S120) of transmitting an acoustic signal from a vehicle-mounted sensor and then receiving a signal that is reflected and received; a signal conversion step (S130) of converting an analog signal into a digital signal with respect to a preset area of ​​the received signal under the control of the control unit; a signal filtering step (S140) of filtering and separating only a signal reflected from a road surface from the digital signal under the control of the control unit; a convolution operation step (S170) of receiving the filtered digital signal from an artificial neural network under the control of the control unit and performing multiple convolution operations; and a second road surface type determination step (S180) of extracting and classifying characteristics of the convolution-operated signal based on a road surface classification model learned in the artificial neural network under the control of the control unit to determine the type of the road surface.

[0039] The above method for estimating a road surface type using a vehicle-mounted sensor further includes a setup step (S115) in which the control unit determines a time of flight (ToF) and a preset time (region of interest) to be filtered through height information (h) from the ground for the vehicle-mounted sensor, and the signal filtering step (S140) is characterized in that it filters a signal received during the preset time after the time of flight (ToF).

[0040] The method for estimating a road surface type using a vehicle-mounted sensor is characterized in that, under the control of the control unit, the road surface type is estimated only when the vehicle's driving speed is a km / h or more or its acceleration is b ㎨ or more (a and b are preset threshold values).

[0041] In addition, according to one embodiment of the present invention, a computer-readable recording medium storing a program for implementing a method for estimating a road surface type using a vehicle-mounted sensor is provided.

[0042] In addition, according to one embodiment of the present invention, a program stored in a computer-readable recording medium is provided to implement a method for estimating a road surface type using a vehicle-mounted sensor.

[0043] The solutions to the problems of the present invention are not limited to the solutions described above, and solutions that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0044] According to the present invention, it is possible to estimate the type of road surface by learning using an artificial neural network, so there is an effect of being able to determine the type of road surface with high accuracy through a simple configuration using existing sensors without additional sensors.

[0045] In addition, according to the present invention, since the type of road surface or the condition of the road surface can be determined with high accuracy, it is possible to quickly check whether black ice, which is a thin layer of ice, is formed, thereby providing an effect of providing advance warning to drivers driving in the relevant section.

[0046] The effects of the present invention are not limited to the effects described above, and effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0047] FIG. 1 is a configuration diagram of a road surface type estimation device using a vehicle-mounted sensor according to one embodiment of the present invention.

[0048] Figure 2 is a road-oriented sensor, and Figure 3 is a drawing for explaining transmission and reception signals in the road-oriented sensor.

[0049] FIG. 4 is a diagram illustrating a vehicle-mounted sensor used in the present invention, and FIG. 5 is a diagram for explaining a transmission signal and a reception signal from a vehicle-mounted sensor.

[0050] FIG. 6 is a configuration diagram of a road surface type estimation device using speed estimation of a moving object according to one embodiment of the present invention.

[0051] FIG. 7 is a drawing for explaining the Doppler effect in a road surface type estimation device using speed estimation of a moving object according to one embodiment of the present invention, and FIG. 8 is a graph for explaining the frequency change of an acoustic wave transmission / reception signal according to the speed of a moving object.

[0052] Figures 9 to 12 are drawings for explaining learning data for vehicle speed.

[0053] Fig. 13 is a configuration diagram of a road surface type estimation device using continuous signals according to one embodiment of the present invention.

[0054] FIG. 14 is a drawing for explaining a transmission signal and a reception signal stored (recorded) in a conventional road surface type estimation device, and FIG. 15 is a drawing for explaining a transmission signal and a reception signal stored (recorded) in a road surface type estimation device using a continuous signal according to one embodiment of the present invention.

[0055] FIG. 16 is a diagram for explaining a transmission signal and a reception signal in a road surface type estimation device using continuous signals according to one embodiment of the present invention.

[0056] Figure 17 is a drawing for explaining a sampling sensing method in a road surface type estimation device.

[0057] Fig. 18 is a configuration diagram of a road surface type estimation device considering vehicle speed and three-axis acceleration according to one embodiment of the present invention.

[0058] Fig. 19 is a configuration diagram of a road surface type estimation device considering vehicle speed and three-axis acceleration according to another embodiment of the present invention.

[0059] Fig. 20 is a drawing for explaining the sound wave transmitter / receiver (sensor) and 3-axis accelerometer of Fig. 18.

[0060] FIG. 21 is a drawing for explaining the mounted sensor and 3-axis accelerometer of the vehicle of FIG. 19.

[0061] Figure 22 is a configuration diagram of an example of an artificial neural network in a road surface type estimation device according to the present invention.

[0062] FIG. 23 is a drawing for explaining a signal converter in a road surface type estimation device according to one embodiment of the present invention.

[0063] Figure 24 is a configuration diagram of another embodiment of an artificial neural network in a road surface type estimation device according to the present invention.

[0064] Fig. 25 is a drawing for explaining the operation of the convolution performing unit of Fig. 24.

[0065] Figure 26 is a flowchart of an embodiment of a method for estimating road surface type using a vehicle-mounted sensor according to the present invention.

[0066] Figure 27 is a flowchart of another embodiment of a road surface type estimation method using a vehicle-mounted sensor according to the present invention.

[0067] Figure 28 is a flowchart of an embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

[0068] Figure 29 is a flowchart of another embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

[0069] Figure 30 is a flowchart of an embodiment of a road surface type estimation method using continuous signals according to the present invention.

[0070] Figure 31 is a flowchart of another embodiment of a road surface type estimation method using continuous signals according to the present invention.

[0071] FIG. 32 is a flowchart of an embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 18.

[0072] FIG. 33 is a flowchart of another embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 18.

[0073] FIG. 34 is a flowchart of an embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 19.

[0074] FIG. 35 is a flowchart of another embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 19.

[0075] Figure 36 is a perspective view of a support (TB) including horizontal supports and vertical supports installed on a road.

[0076] Figure 37 is an enlarged view of the AH section of Figure 36.

[0077] Figure 38 is a schematic drawing showing a state in which a horizontal support is folded by rotation relative to a vertical support.

[0078] Figure 39 is an enlarged view of the AJ portion of Figure 38.

[0079] Figure 40 is an enlarged view of the AK portion of Figure 38.

[0080] Figure 41 is a block diagram of an acoustic wave-based road use detection AI-IoT platform according to a preferred embodiment of the present invention.

[0081] FIG. 42 is a diagram illustrating a road information providing system according to various embodiments of the present disclosure.

[0082] FIG. 43 is a block diagram illustrating a configuration of a road information provision server according to one embodiment of the present disclosure.

[0083] FIG. 44 is a diagram illustrating a type of road classification device that provides information to a road information provision server according to the present disclosure.

[0084] FIG. 45 is a diagram showing the overall flowchart of the operation of a road information provision system according to one embodiment of the present disclosure.

[0085] FIG. 46 is a diagram illustrating a method for processing and generating data in a road information provision server according to various embodiments of the present disclosure.

[0086] FIG. 47 is a drawing showing an example of a screen displayed on UI / UX when providing information according to one embodiment of the present disclosure.

[0087] FIG. 48 is a flowchart illustrating a method by which a server according to one embodiment of the present disclosure provides a notification according to a road condition to a user terminal.

[0088] Figure 49 is a flowchart illustrating a method for providing information in response to a user terminal request according to one embodiment of the present disclosure.

[0089] FIG. 50 is a flowchart of a method for improving data accuracy when providing information at the request of a user terminal according to one embodiment of the present disclosure.

[0090] FIG. 51 is a flowchart of a method for compensating information providers according to one embodiment of the present disclosure.

[0091] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0092] When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0093] On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0094] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present invention. The singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like specify that a feature, number, process, operation, component, part or combination thereof described in the specification is present, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, processes, operations, components, parts or combinations thereof.

[0095] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning within the context of the relevant technology, and unless explicitly defined herein, they shall not be construed in an idealized or overly formal sense.

[0096] Hereinafter, the present invention will be described in more detail with reference to the attached drawings. Prior to this, it should be noted that terms and words used in this specification and claims should not be interpreted as limited to their conventional or dictionary meanings. Based on the principle that the inventor can appropriately define the concept of a term to best explain his or her invention, they should be interpreted with meanings and concepts consistent with the technical spirit of the present invention. In addition, unless otherwise defined, technical and scientific terms used have meanings commonly understood by those of ordinary skill in the art to which this invention pertains. In the following description and the attached drawings, descriptions of well-known functions and configurations that may unnecessarily obscure the gist of the present invention will be omitted. The drawings introduced below are provided as examples to ensure that the spirit of the present invention can be sufficiently conveyed to those skilled in the art. Therefore, the present invention is not limited to the drawings presented below and may be embodied in other forms. In addition, like reference numerals represent like elements throughout the specification. It should be noted that like reference numerals represent like elements throughout the drawings, wherever possible.

[0097]

[0098] The present disclosure relates to a system for classifying a road surface using an acoustic signal and managing the road surface or vehicle operation through the classification.

[0099] A road surface classification device according to one embodiment of the present disclosure may include a device installed on a drawing infrastructure or a mobile device (e.g., a vehicle) to determine the type or condition of a road surface. Meanwhile, the road surface classification device may be referred to by any name, such as a road surface type estimation device.

[0100] A road surface classification device according to another embodiment of the present disclosure may include a server device that determines the type or condition of a road surface based on information received from a device installed in road infrastructure or a moving vehicle, etc.

[0101]

[0102] Below, with reference to the drawings, the road surface type estimation device will be described in detail.

[0103] FIG. 1, FIG. 6, FIG. 13, FIG. 18, and FIG. 19 are configuration diagrams of a road surface type estimation device according to one embodiment of the present invention.

[0104] Referring to FIGS. 1, 6, 13, 18, and 19, the road surface type estimation device may include a sound wave transceiver (sensor) (100), an analog-to-digital converter (ADC) (200), an artificial neural network (400), and a control unit (MCU) (500). More specifically, the road surface type estimation device according to one embodiment of the present disclosure may further include at least one of a signal filter unit (300), a speed estimation unit (301), a buffer unit (302), and an atmospheric attenuation correction unit (303).

[0105] That is, the following description will describe each of the embodiments including one of the signal filter unit (300), the speed estimation unit (301), the buffer unit (302), and the atmospheric attenuation correction unit (303), but redundant descriptions of components that may overlap in each embodiment (e.g., the sensor (100), the analog-to-digital converter (200), the artificial neural network (400), the control unit (500), etc.) may be omitted.

[0106]

[0107] FIG. 1 is a configuration diagram of a road surface type estimation device using a vehicle-mounted sensor according to one embodiment of the present invention.

[0108] As illustrated in FIG. 1, a road surface type estimation device using a vehicle-mounted sensor according to one embodiment of the present invention includes a vehicle-mounted sensor (100), an analog-digital converter (ADC) (200), a signal filter unit (300), an artificial neural network (400), and a control unit (MCU) (500).

[0109] The sensor (100) mounted on the vehicle is a general ultrasonic sensor equipped in the vehicle, which transmits a sound wave signal and then receives the reflected signal.

[0110] The sensor (100) mounted on the vehicle includes a sound wave transmitter (not shown) that outputs a transmission signal according to the control of a control unit (not shown) of the vehicle, and a sound wave receiver (not shown) that receives a reflection signal that is returned when the transmission signal is reflected on an arbitrary surface.

[0111] Meanwhile, a sensor (100) according to one embodiment of the present disclosure may be installed in a vehicle by additional installation, rather than being pre-installed in the vehicle. In this case, the sound wave transmitting and receiving unit (sensor) (100) may include a sound wave transmitter (101) that outputs a transmission signal under the control of a control unit (500), and a sound wave receiver (102) that receives a reflection signal that is returned when the transmission signal is reflected on an arbitrary surface.

[0112] The above analog-to-digital converter (200) receives a signal (raw signal) received by the sound wave receiver from a sensor (100) installed in the vehicle, and converts the received signal from an analog signal to a digital signal.

[0113] The above signal filter unit (300) filters only the signal reflected from the road surface from the digital signal and outputs it to the artificial neural network (400).

[0114] The filtering method of the above signal filter unit (300) will be described in detail below with reference to FIGS. 2 and 3.

[0115] The above artificial neural network (400) uses the filtered digital signal as an input signal and classifies the input signal based on the learned road surface classification model to estimate the type of the road surface.

[0116] Meanwhile, in the artificial neural network (400), classification and learning are performed using at least one of decision trees, discriminant analysis, logistic regression classifiers, naive Bayes classifiers, support vector machines, nearest neighbor classifiers, and ensemble classifiers.

[0117] Decision trees include Fine tree, Medium tree, Coarse tree, All tree, and Optimizable tree; discriminant analysis includes Linear discriminant, Quadratic discriminant, All discriminants, and Optimizable discriminant; Naive Bayes classifier includes Gaussian Naive Bayes, Kernel Naive Bayes, All Naive Bayes, and Optimizable Naive Bayes; support vector machine (SVM) includes Linear SVM, Quadratic SVM, Cubic SVM, Fine Gaussian SVM, Medium Gaussian SVM, Coarse Gaussian SVM, All SVM, and Optimizable SVM; nearest neighbor classifiers include Fine KNN, Medium KNN, Coarse KNN, Cosine KNN, Cubic KNN, Weighted KNN, All KNN, and Optimizable KNN; and ensemble classifiers include Includes Boosted trees, Bagged trees, Subspace Discriminant, Subspace KNN, RUSBoosted trees, All Ensembles, and Optimizable Ensemble.

[0118] The control unit (MCU) (500) controls the operation of the sensor (100), the analog-to-digital converter (200), the signal filter unit (300), and the artificial neural network (400) installed in the vehicle.

[0119] The above analog-to-digital converter (200), signal filter unit (300), and artificial neural network (400) are expressed as components of software implemented as a program.

[0120] Meanwhile, although not illustrated in the drawing, the road surface type estimation device using a vehicle-mounted sensor according to the present invention includes a storage (memory) in which the learned road surface classification model and software implemented with the program are stored. The storage (memory) may be configured to be included in the control unit (MCU) (500). Meanwhile, the control unit (MCU) (500) may include a communication unit.

[0121]

[0122] Figure 2 is a road-oriented sensor, and Figure 3 is a drawing for explaining transmission and reception signals in the road-oriented sensor.

[0123] As shown in Figures 2 and 3, the road-oriented sensor is mounted facing the road surface, so the transmitted signal is incident vertically on the road surface, and the signals reflected multiple times are received with their magnitudes gradually attenuated.

[0124] Among the numerous sensors installed in vehicles, many ultrasonic sensors are parking sensors. However, these ultrasonic sensors are not installed facing the road surface, but rather are installed perpendicular to the road surface, such as at the front, rear, or side of the vehicle.

[0125]

[0126] FIG. 4 is a diagram illustrating a vehicle-mounted sensor used in the present invention, and FIG. 5 is a diagram for explaining a transmission signal and a reception signal from a vehicle-mounted sensor.

[0127] As shown in FIGS. 4 and 5, the vehicle-mounted sensor (100) is not installed facing the road surface, but is mounted facing the outside of the vehicle (the wall side), so that it is perpendicular to the road surface.

[0128] Referring to FIG. 5, the vehicle-mounted sensor (100) outputs a transmission signal in the same manner as in FIG. 3, but the signal (202) reflected on the road surface has a much smaller amplitude due to the FoV (Field of View), and the signal reflected by the wall and returned has a similar form to the transmission signal, but is received later than the signal (202) reflected on the road surface due to its long flight time.

[0129] Meanwhile, since the vehicle's on-board sensor (100) can know the height from the road surface, it is possible to know the flight time between the road surface and the vehicle's on-board sensor (100), and thus filter the signal for the corresponding portion.

[0130] That is, when the height information (h) of the vehicle's on-board sensor (100) from the ground (road surface) is known, it can be known that ToF is approximately after a certain point, and a preset region of interest (Region of Interest) is set after ToF and filtered.

[0131] Meanwhile, when material classification is performed on the filtered signal for the relevant area of ​​interest, if it is determined to be a material that cannot be obtained from a vehicle (steel) or a road surface, it is not processed as a signal obtained from the road surface.

[0132] In addition, since the presence of another object at the same (similar) distance as the distance between the vehicle-mounted sensor (100) and the road surface occurs only when the vehicle is parked, road surface classification is not performed because the amount of acceleration and deceleration is small during parking. In this case, the vehicle-mounted sensor (100) simply operates as a distance sensor using ToF as preset.

[0133] That is, in the control unit (500), the road surface type estimation device according to the present invention can be controlled to operate only when the driving speed is a km / h or more or the acceleration is b ㎨ or more (a and b are preset threshold values).

[0134] To this end, the control unit (500) can receive speed information and acceleration information from the vehicle control unit (not shown).

[0135] Then, the sound wave transmitter (101) of the sensor (100) pre-mounted on the vehicle outputs a sound wave signal (201) having a preset frequency (e.g., 40 kHz). Thereafter, the sound wave receiver (not shown) of the sensor (100) pre-mounted on the vehicle receives the reflected signal that is reflected back.

[0136] The above control unit (500) determines the time of flight (ToF) through the height information (h) from the ground (road surface) of the vehicle's on-board sensor (100), and the signal filter unit (300) filters the signal (202) received during a preset time (area of ​​interest) after the time of flight, i.e., a total of x ms from t_0 [mx] to t_0 + x [ms], according to the control of the control unit (500).

[0137] In Fig. 5, t_0 is 3 ms, t_0 + x is 5 ms, and a total of 2 ms is set as the region of interest, and filtering is explained as an example. Of course, t_0 and x can be adjusted depending on the environment or conditions.

[0138]

[0139] FIG. 6 is a configuration diagram of a road surface type estimation device using speed estimation of a moving object according to one embodiment of the present invention.

[0140] As illustrated in FIG. 6, a road surface type estimation device using speed estimation of a moving object according to one embodiment of the present invention includes a sound wave transceiver (sensor) (100), an analog-to-digital converter (ADC), a speed estimation unit (300), an artificial neural network (400), and a control unit (MCU) (500).

[0141] The above sound wave transmitter / receiver (sensor) (100) transmits a sound wave signal to the road surface of which the type is to be known, and then receives the reflected signal.

[0142] The above sound wave transceiver (sensor) (100) includes a sound wave transmitter (101) that outputs a transmission signal under the control of the control unit (500), and a sound wave receiver (102) that receives a reflection signal that is returned when the transmission signal is reflected on an arbitrary surface.

[0143] The above analog-to-digital converter (200) converts the analog signal of the received signal into a digital signal. The above speed estimation unit (301) can estimate the speed of the moving object by processing the converted digital signal.

[0144] The above artificial neural network (400) uses the corrected digital signal and the estimated speed of the moving object as input signals, and classifies the input signal based on the learned road surface classification model to estimate the type of the road surface.

[0145] The above control unit (MCU) (500) controls the operation of the sound wave transmitter / receiver (100), the analog-to-digital converter (200), the speed estimation unit (301), and the artificial neural network (400).

[0146] The above analog-to-digital converter (200), speed estimation unit (301), and artificial neural network (400) are expressed as components of software implemented as a program.

[0147] Meanwhile, although not illustrated in the drawing, the road surface type estimation device using the velocity estimation of a moving object according to the present invention includes a storage (memory) in which the learned road surface classification model and software implemented with the program are stored. The storage (memory) may be configured to be included in the control unit (MCU) (500).

[0148] Meanwhile, the road surface type estimation device using the speed estimation of a moving object according to the present invention may further include an atmospheric information measurement unit (600) capable of measuring temperature, humidity, and air pressure in the atmosphere.

[0149] According to the control of the above control unit (500), the atmospheric information including the temperature, humidity and atmospheric pressure measured by the above atmospheric information measurement unit (600) can be used in the speed estimation unit (301) and can also be transmitted as an input to the above artificial neural network (400).

[0150] FIG. 7 is a drawing for explaining the Doppler effect in a road surface type estimation device using speed estimation of a moving object according to one embodiment of the present invention, and FIG. 8 is a graph for explaining the frequency change of an acoustic wave transmission / reception signal according to the speed of a moving object.

[0151] As illustrated in FIG. 7, a road classification device is mounted on a moving object (vehicle), and a sensor (indicated as T / R in the drawing) is depicted as being installed at the front of the vehicle.

[0152] When a sensor is installed to classify the road surface while a vehicle is driving, the speed of the vehicle (v) c ) increases, the original transmission frequency (f) is shifted by the Doppler effect. t ) and the frequencies of the received signal (f r1 and f r2 ) the frequency difference between them becomes larger.

[0153] Referring to the graph shown in Fig. 8, f in the middle t is the frequency of the transmitted sound wave, and f r1 is the frequency of the reflected wave that hits the road surface behind the center of the sensor and returns based on the driving direction of the above-mentioned moving object, and f r2 is the frequency of the reflected wave that hits the road surface behind the center of the sensor and returns based on the driving direction of the moving object. Here, f t This is a graph when the frequency is set to 40kHz, and f r1 is about 38 kHz, f r2 can be seen as approximately 42 KHz.

[0154] Here, f r1 and f r2The reason we have two frequencies containing is because sound waves have a directivity.

[0155] In other words, as the speed of a moving object increases, the distortion of the received signal becomes more severe, making it necessary to consider the object's speed when classifying the road surface. However, in cases where the object's speed cannot be directly determined, the Doppler effect can be used to estimate the object's speed.

[0156] Hereinafter, with reference to <Mathematical Formula 1> to <Mathematical Formula 4> below, the process of calculating (estimating) the speed of a moving object using the sound wave signal received by the speed estimation unit (301) will be described.

[0157] First, you can set the variables as follows:

[0158] c: speed of sound

[0159] v c : Speed ​​of moving object

[0160] f t : Frequency of sound signal transmitted from sensor (sound wave transmitter / receiver)

[0161] f r1 : Since the acoustic signal has a directivity, the frequency of the reflected wave that hits the road surface behind the center of the sensor and returns based on the moving object

[0162] f r2 : Since the sound wave signal has a directivity, the frequency of the reflected wave that hits the road surface ahead of the center of the sensor and returns based on the moving object

[0163] ToF: Time of Flight of sound signals transmitted from a sensor (sound transceiver)

[0164] DoF: Distance of Flight of the sound signal transmitted from the sensor (sound transceiver)

[0165] θ: signal receiving angle of the received sound signal (reflected wave)

[0166] The frequency (f) of the reflected wave that hits the road surface behind the center of the above sensor and returns r1 ) is calculated using the following <Mathematical Formula 1>.

[0167] <Mathematical Formula 1>

[0168]

[0169] Here, 'c - v c ' is the speed of sound waves reflected from the road surface behind the center of the sensor perceived from the moving object side, and 'c + v c ' is the speed of sound waves transmitted from the sensor, perceived on the road surface behind the center of the sensor.

[0170] Meanwhile, the frequency (f) of the reflected wave that hits the road surface ahead of the center of the sensor and returns r2 ) is calculated using the following <Mathematical Formula 2>.

[0171] <Mathematical Formula 2>

[0172]

[0173] Here, 'c + v c ' is the speed of sound waves reflected from the road surface ahead of the center of the sensor sensed from the moving object, and 'c - v c ' is the speed of sound waves transmitted from the sensor, perceived on the road surface ahead of the center of the sensor.

[0174] Then, the speed of sound (v c ) can be calculated using the following <Mathematical Formula 3>.

[0175] <Mathematical Formula 3>

[0176]

[0177] Here, f t is the frequency of the sound signal transmitted from the sensor (sound transceiver) that is already known, and f r1 and f r2 It is obtained by performing a Fourier transform (FFT: Fast Fourier Transform) on the received sound signal.

[0178] Therefore, the reception angle θ of the reflected wave can be calculated using the following <Mathematical Formula 4>.

[0179] <Mathematical Formula 4>

[0180]

[0181] Here, ToF is the time of flight (Time of Flight) of the sound signal transmitted from the sensor (sound transceiver), and DoF is the distance of flight (Distance of Flight) of the sound signal transmitted from the sensor (sound transceiver).

[0182]

[0183] Figures 9 to 12 are drawings for explaining learning data for vehicle speed.

[0184] Since the present invention uses the speed of a moving object as input to an artificial neural network, even when transmitting the same transmission signal, the received signal varies depending on the speed of the moving object (vehicle), resulting in a different type of signal being received. Therefore, to achieve this, the received signal, based on the speed of the moving object (vehicle), must be digitized and used to train the neural network using that signal.

[0185] For example, FIG. 9 is learning data when the speed of a moving object (vehicle) is 10 km / h or less, FIG. 10 is learning data when the speed of a moving object (vehicle) is more than 10 km / h and less than 30 km / h, FIG. 11 is learning data when the speed of a moving object (vehicle) is more than 30 km / h and less than 50 km / h, and FIG. 12 is learning data when the speed of a moving object (vehicle) is more than 50 km / h.

[0186] In the learning step of the road surface type estimation method using the speed estimation of a moving object according to the present invention, a plurality of reception signals can be learned for each material and each speed.

[0187]

[0188] Fig. 13 is a configuration diagram of a road surface type estimation device using continuous signals according to one embodiment of the present invention.

[0189] As illustrated in FIG. 13, a road surface type estimation device using a continuous signal according to one embodiment of the present invention includes a sound wave transceiver (100), an analog-digital converter (ADC) (200), a buffer unit (302), an artificial neural network (400), and a control unit (MCU) (500).

[0190] The above buffer unit (302) includes a plurality of buffers that continuously receive signals for a preset period of time from a digital signal converted by an analog-to-digital converter (200).

[0191] The above buffer unit (302) includes at least two buffers, and the at least two buffers receive signals for the preset time in a preset order.

[0192] For example, if the analog-to-digital converter (200) samples the received continuous signal at 1 MHz, it is converted into 1,000,000 signals per second, and 8,000 signals (8,000 signals) having a size of 12 bits (0 to 4,096) can be continuously (sequentially) transmitted to each buffer for 8 ms.

[0193] Then, the control unit (500) controls the data output from the at least two buffers to be converted in a serial manner and transmitted to the artificial neural network (400).

[0194] Here, the form of parallel data can be converted into serial form using a universal asynchronous receiver / transmitter (UART) included in the control unit (500).

[0195] The above artificial neural network (400) uses the digital signal sequentially received from the buffer unit (302) as an input signal, and classifies the input signal based on the learned road surface classification model to estimate the type of the road surface.

[0196] The above control unit (MCU) (500) controls the operation of the sound wave transmitter / receiver (100), the analog-to-digital converter (200), the buffer unit (302), and the artificial neural network (400).

[0197] The above analog-to-digital converter (200), buffer unit (302), and artificial neural network (400) are expressed as components of software implemented as a program.

[0198] Meanwhile, although not illustrated in the drawing, the road surface type estimation device using continuous signals according to the present invention includes a storage (memory) in which the learned road surface classification model and software implemented with the program are stored. The storage (memory) may be configured to be included in the control unit (MCU) (500).

[0199] FIG. 14 is a drawing for explaining a transmission signal and a reception signal stored (recorded) in a conventional road surface type estimation device, and FIG. 15 is a drawing for explaining a transmission signal and a reception signal stored (recorded) in a road surface type estimation device using a continuous signal according to one embodiment of the present invention.

[0200] Referring to Figure 14, if the height information of the sound wave transmitter / receiver (sound wave sensor) from the ground is h, the distance that the sound wave flies is h, and the time it takes to fly the distance between the sound wave sensor and the road surface to receive one reflected signal is '2h / speed of sound', and 12,000 bits of data are wasted per 1ms.

[0201] That is, in the method of estimating the type of road surface by transmitting a single sound signal toward the road surface and using the reflected signal, the time of flight (ToF: Time of Flight) for the sound signal to reach the road surface and return must be waited for, and the entire time (2h / c) including the flight time must be recorded (saved) to enable road surface classification.

[0202] Meanwhile, referring to FIG. 14, since the sound wave transmitter / receiver (sound wave sensor) continuously transmits continuous signals toward the road surface and receives reflected continuous signals, there is no need to wait for or record the entire time of flight (ToF: Time of Flight) for the sound wave signal to travel to the road surface and back.

[0203] FIG. 16 is a diagram for explaining a transmission signal and a reception signal in a road surface type estimation device using continuous signals according to one embodiment of the present invention.

[0204] Referring to FIG. 3, under the control of the control unit (MCU) (500), the sound wave transmitter of the sound wave transceiver outputs one transmission signal (sound wave signal) having a preset frequency (e.g., 40 kHz) to the corresponding road surface whose type is to be known.

[0205] Afterwards, the sound wave receiver of the sound wave transmitter and receiver received the reflected signal that was reflected from the road surface and returned.

[0206] However, as illustrated in FIG. 16, in a road surface type estimation device using continuous signals according to one embodiment of the present invention, a sound wave transmitter (101) of a sound wave transceiver (100) outputs continuous transmission signals to the road surface, and a sound wave receiver (102) of the sound wave transceiver (100) receives continuously reflected signals and continuously senses and processes the signals.

[0207] In general, the speed of sound of a sound signal is sufficiently fast, as 343 m / s is 34.3 cm / ms under atmospheric conditions of 20℃.

[0208] In a situation where the sound wave transmitter continuously scans the road surface and the sound wave receiver receives the reflected signal, the sound wave that started from the sound wave transmitter (TX sensor) arrives at the sound wave receiver (RX sensor) approximately 1.5 ms later. Therefore, when continuously collecting data for the RX recorded at a point in time 2 ms after the TX scan, if the vehicle is driving at a speed of 100 km / h (approximately 2.78 cm forward per ms), the gap considering the delay is calculated to be 3 to 6 cm.

[0209] That is, since this level of gap is considered negligible for a moving vehicle to perceive the road surface, the delay is not taken into account.

[0210] FIG. 17 is a diagram for explaining a sampling sensing method in a road surface type estimation device. More specifically, FIG. 17 (a) is a diagram for explaining a sampling sensing method in a conventional road surface type estimation device, and FIG. 17 (b) is a diagram for explaining a continuous sensing method in a road surface type estimation device using continuous signals according to an embodiment of the present invention.

[0211] As shown in (a) of Fig. 17, in the past, when a vehicle was moving and transmitting a signal, a portion of the signal reflected and received was sampled to estimate the type of road surface, and the signal reflected from the road surface was received at regular intervals.

[0212] Meanwhile, as illustrated in (b) of FIG. 17, a road surface type estimation device using continuous signals according to one embodiment of the present invention outputs continuous signals toward the road surface from a sound wave transmitter (101) of a sound wave transceiver (100), and continuously senses a signal reflected from the output signal from a sound wave receiver (101) of the sound wave transceiver (100), thereby enabling analysis of continuous signals reflected without exception from the road surface on which the vehicle is driving.

[0213]

[0214] Fig. 18 is a configuration diagram of a road surface type estimation device considering vehicle speed and three-axis acceleration according to one embodiment of the present invention.

[0215] As illustrated in FIG. 18, a road surface type estimation device considering vehicle speed and three-axis acceleration according to one embodiment of the present invention includes a sound wave transceiver (100), an analog-digital converter (ADC) (200), an atmospheric attenuation correction unit (303), a three-axis accelerometer (700), an artificial neural network (400), and a control unit (MCU) (500).

[0216] The above atmospheric attenuation correction unit (303) calculates and corrects the atmospheric attenuation of the digital signal from the analog-to-digital converter (200).

[0217] A three-axis accelerometer (700) measures the three-axis acceleration of the vehicle and transmits it to the control unit (500).

[0218] The above measured three-axis acceleration changes while the vehicle is driving, and the control unit (500) transmits the three-axis acceleration that changes while driving, received from the three-axis accelerometer (700), as input to the artificial neural network (400).

[0219] In addition, the control unit (500) transmits the vehicle speed received from the vehicle and the time of flight (TOF: Time of Flight) of the sound wave signal output from the sound wave transceiver (100) as input signals of the artificial neural network (400).

[0220] The artificial neural network (400) uses the speed of the vehicle, the flight time of the sound signal, the three-axis acceleration, and the digital signal corrected for the atmospheric attenuation as input signals, and estimates the type of the road surface by classifying the input signal based on the learned road surface classification model.

[0221] The above control unit (MCU) (500) controls the operation of the sound wave transceiver (100), the analog-to-digital converter (200), the atmospheric attenuation correction unit (303), the three-axis accelerometer (700), and the artificial neural network (400).

[0222] The above analog-to-digital converter (200), the atmospheric attenuation correction unit (303), and the above artificial neural network (400) are expressed as components of software implemented as a program.

[0223] Meanwhile, although not illustrated in the drawing, the road surface type estimation device according to the present invention, considering the speed and three-axis acceleration of a vehicle, includes a storage (memory) in which the learned road surface classification model and software implemented with the program are stored. The storage (memory) may be configured to be included in the control unit (MCU) (500).

[0224] Meanwhile, in Fig. 18, it is described that the atmospheric attenuation correction unit (303) is included to correct the attenuation of atmospheric sound waves and then process the signal for which the attenuation has been corrected, but it is also possible to process without correcting the attenuation of atmospheric sound waves.

[0225] Meanwhile, the road surface type estimation device considering the speed and three-axis acceleration of the vehicle according to the present invention further includes an atmospheric information measurement unit (600) capable of measuring temperature, humidity, and air pressure in the atmosphere.

[0226] According to the control of the above control unit (500), the atmospheric information including the temperature, humidity and atmospheric pressure measured by the above atmospheric information measurement unit (600) can be used by the atmospheric attenuation correction unit (303) and can also be transmitted as input to the above artificial neural network (400).

[0227]

[0228] Fig. 19 is a configuration diagram of a road surface type estimation device considering vehicle speed and three-axis acceleration according to another embodiment of the present invention.

[0229] As illustrated in FIG. 19, a road surface type estimation device considering vehicle speed and 3-axis acceleration according to an embodiment of the present invention includes a sensor (800) pre-mounted on a vehicle, an analog-to-digital converter (ADC) (200), a signal filter unit (900), a 3-axis accelerometer (700), an artificial neural network (400), and a control unit (MCU) (500). Meanwhile, the sensor (800) pre-mounted on the vehicle may be substantially the same as the sensor (100) pre-mounted on the vehicle described above, and the signal filter unit (900) may also be substantially the same as the signal filter unit (300) described above. This is set for the convenience of explanation in each embodiment, and the explanation of the above-described embodiment may be equally applied. Therefore, the following description will focus on different contents from the above-described example.

[0230] The above signal filter unit (900) filters and outputs only the signal reflected from the road surface from the above digital signal.

[0231] The above atmospheric attenuation correction unit (303) calculates and corrects the atmospheric attenuation of the filtered digital signal and outputs it to the artificial neural network (400).

[0232] The filtering method of the signal filter unit (900) will be described in detail with reference to FIGS. 2, 4, 5, 20, and 21. The three-axis accelerometer (700) measures the three-axis acceleration of the vehicle and transmits it to the control unit (500).

[0233] The above measured three-axis acceleration changes while the vehicle is driving, and the control unit (500) transmits the three-axis acceleration that changes while driving, received from the three-axis accelerometer (700), as an input signal to the artificial neural network (400).

[0234] In addition, the control unit (500) transmits the vehicle speed received from the vehicle and the time of flight (TOF) of the sound wave signal output from the mounted sensor (800) as input signals of the artificial neural network (400).

[0235] The artificial neural network (400) uses the speed of the vehicle, the flight time of the sound signal, the three-axis acceleration, and the digital signal corrected for the amount of attenuation in the atmosphere as input signals, and estimates the type of the road surface by classifying the input signals based on the learned road surface classification model.

[0236] The above control unit (MCU) (500) controls the operation of the sensor (800), the analog-to-digital converter (200), the signal filter unit (900), the atmospheric attenuation correction unit (303), the three-axis accelerometer (700), and the artificial neural network (400) installed in the vehicle.

[0237] The above analog-to-digital converter (200), signal filter unit (900), atmospheric attenuation correction unit (303), and artificial neural network (400) are expressed as components of software implemented as a program.

[0238] Meanwhile, in Fig. 19, it is described that the atmospheric attenuation correction unit (303) is included to correct the attenuation of atmospheric sound waves and then process the signal for which the attenuation has been corrected, but it is also possible to process without correcting the attenuation of atmospheric sound waves.

[0239] Meanwhile, although not shown in the drawing, the road surface type estimation device considering the speed and three-axis acceleration of the vehicle according to the present invention includes a storage (memory) in which the learned road surface classification model and the software implemented with the program are stored.

[0240] The above storage (memory) may be configured to be included in the control unit (MCU) (500). Meanwhile, the control unit (MCU) (500) may include a communication unit.

[0241] FIG. 2 may be a road-oriented sensor, which is a sound wave transceiver (100) of FIG. 18, and FIG. 20 is a drawing for explaining the sound wave transceiver (sensor) (100) and a three-axis accelerometer (700) of FIG. 18.

[0242] As shown in FIGS. 2, 3 and 20, the road-oriented sensor is mounted facing the road surface, so the transmitted signal is incident vertically on the road surface, and the signals reflected multiple times are received with their magnitudes gradually reduced.

[0243] As illustrated in FIG. 2, when a trigger signal having a preset size (v: trigger voltage) and a preset transmission period (p: transmission period) is transmitted from the control unit (MCU) (500) to the sound wave transceiver (100), the sound wave transmitter (101) of the sound wave transceiver (100) outputs a sound wave signal (201) having 40 kHz to the corresponding road surface whose type is to be known.

[0244] Thereafter, the sound wave receiver (102) of the sound wave transceiver (100) receives the reflected signal that is reflected back from the road surface.

[0245] Here, the crosstalk signal received on the same time line as the sound signal (201) is not shown. In addition, the control unit (500) determines the signal (202) for a preset time period as the received signal based on the point where the amplitude of the signal received after the transmission delay is the largest.

[0246] For example, when the amplitude of the signal received after the transmission delay is the largest at t_0, a total of (a+b) ms can be observed from t_0 - a [ms] to t_0 + b [ms], and the received signal (202) of Fig. 3 has a of 0.5 and b of 2.5. Of course, a and b can be adjusted depending on the environment or conditions.

[0247] Here, the sampling frequency of the sound wave transmitter and receiver (100) was set to 1 MHz, which samples 25 times the 40 kHz sound wave frequency.

[0248] In addition, the 3-axis accelerometer (700) measures acceleration of the vehicle along the x-axis, y-axis, and z-axis. The x-axis acceleration is the same as the direction of travel of the vehicle and can be regarded as acceleration of the vehicle. The y-axis acceleration can indicate how much the vehicle vibrates left and right, and the z-axis acceleration can indicate how much the vehicle vibrates up and down.

[0249] Meanwhile, even if the road-oriented sensor transmits sound waves vertically toward the road surface while the vehicle is driving, the received reflected waves (203) form a certain angle with the ground.

[0250] Therefore, the received reflected wave (203) is in the form of a combination of vertical and horizontal components, as in <Mathematical Formula 5> below.

[0251] <Mathematical Formula 5>

[0252] Sr = Sv + Sr

[0253]

[0254]

[0255] Here, Sr is the received signal, Sv is the vertical component of the received signal, Sh is the horizontal component of the received signal, and θ is as shown in <Mathematical Formula 6> below.

[0256] <Mathematical Formula 6>

[0257]

[0258] Here, vc is the velocity of the moving object, ToF is the time of flight (Time of Flight) of the sound signal transmitted from the sensor (sound transceiver), and DoF is the distance of flight (Distance of Flight) of the sound signal transmitted from the sensor (sound transceiver).

[0259] Accordingly, Sv can be affected by z-axis acceleration, and Sh can be affected by x-axis acceleration.

[0260]

[0261] FIG. 21 is a drawing for explaining the mounted sensor (800) and 3-axis accelerometer (700) of the vehicle of FIG. 19.

[0262] Meanwhile, the vehicle-mounted sensor (800) disclosed in FIG. 21 can be applied in the same manner as the description of the vehicle-mounted sensor (100) regarding FIGS. 2 to 5 in the above-described examples.

[0263] A three-axis accelerometer (700) according to one embodiment of the present disclosure measures acceleration of a vehicle along the x-axis, y-axis, and z-axis. The x-axis acceleration is the same as the direction in which the vehicle is traveling and can be regarded as acceleration of the vehicle. The y-axis acceleration can indicate how much the vehicle vibrates left and right, and the z-axis acceleration can indicate how much the vehicle vibrates up and down.

[0264] Referring to Figures 9 to 12, the present invention uses the speed of the vehicle as input to the artificial neural network. Therefore, even if the same transmission signal is transmitted, the received signal varies depending on the speed of the moving object (vehicle), resulting in the reception of a different type of signal. Therefore, to achieve this, the received signal must be converted into data according to the speed of the moving object (vehicle) and the neural network must be trained with the signal.

[0265] For example, FIG. 9 is learning data when the speed of a moving object (vehicle) is 10 km / h or less, FIG. 10 is learning data when the speed of a moving object (vehicle) is more than 10 km / h and less than 30 km / h, FIG. 11 is learning data when the speed of a moving object (vehicle) is more than 30 km / h and less than 50 km / h, and FIG. 12 is learning data when the speed of a moving object (vehicle) is more than 50 km / h.

[0266]

[0267] In the learning step of the road surface type estimation method considering the speed and three-axis acceleration of the vehicle according to the present invention, a plurality of reception signals can be learned for each material and each speed.

[0268] Hereinafter, the process of calculating the attenuation of the sound wave signal in the atmospheric attenuation correction unit (303) will be described with reference to <Mathematical Formula 7> to <Mathematical Formula 14> below.

[0269] First, the saturation pressure (Psat) is calculated using the following <Mathematical Formula 7>.

[0270] <Mathematical Formula 7>

[0271]

[0272] Here, To1 is the triple point of the atmosphere [K], and T is the current temperature [K]. The absolute humidity (h) is calculated using the following <Mathematical Equation 8>.

[0273] <Mathematical Formula 8>

[0274]

[0275] Here, hrin is relative humidity [%], Psat is saturation pressure [unit], and Ps is static pressure [atm].

[0276] Meanwhile, the scaled relaxation frequency for nitrogen (FrN), which accounts for 78% of the atmosphere, is calculated using the following <Mathematical Equation 9>.

[0277] <Equation 9>

[0278]

[0279] Here, To is the reference temperature [K] and T is the current temperature [K].

[0280] Meanwhile, the scaled relaxation frequency for oxygen (FrO), which accounts for 21% of the atmosphere, is calculated using the following <Mathematical Formula 10>.

[0281] <Mathematical Formula 10>

[0282]

[0283] Here, h is the absolute humidity.

[0284] Meanwhile, the attenuation coefficient (α: attenuation coefficient [nepers / m]) is calculated using the following <Mathematical Formula 11>.

[0285] <Mathematical Formula 11>

[0286]

[0287] Here, Ps is the static pressure, F is the frequency of the sound signal (transmitted sound signal), T is the current temperature [K], To is the reference temperature [K], FrO is the extended relaxation frequency of oxygen, and FrN is the extended relaxation frequency of nitrogen.

[0288] Meanwhile, the attenuation ratio (A, unit: dB) of the sound signal is calculated using the following <Mathematical Formula 12>.

[0289] <Mathematical formula 12>

[0290]

[0291] Here, α is an attenuation coefficient, and d is the distance between the sound wave transmitter / receiver (100) and the road surface of which the type is to be known.

[0292] The above d is calculated using the following <Mathematical Formula 13> using the time t (time of flight) required for the signal to be transmitted from the transmitter and reflected on the road surface until it is detected by the receiver and the speed of sound in the atmosphere (Vair).

[0293] <Mathematical formula 13>

[0294]

[0295] Here, t is the time required, and Vair is the speed of sound in the air [m / s]. Meanwhile, the speed of sound in the air is calculated using the following <Mathematical Equation 14>.

[0296] <Mathematical Formula 14>

[0297]

[0298] Here, Ks is the coefficient of stiffness of the object, and ρ is the density of the object (atmosphere).

[0299] At this time, if air (atmosphere) is assumed to be an ideal gas, Ks = γP, where γ is the heat capacity ratio (1.4 for air), P is the pressure, R is the ideal gas constant, and T is the absolute temperature [K]. Since it is a constant except for the temperature, it can be expressed approximately.

[0300] The above atmospheric sound speed can be corrected for attenuation compensation according to the temperature, pressure, and humidity of the atmosphere.

[0301]

[0302] Figure 22 is a configuration diagram of an example of an artificial neural network in a road surface type estimation device according to the present invention.

[0303] As illustrated in Fig. 22, the artificial neural network (400) in the road surface type estimation device according to the present invention includes a signal converter (410) and a neural network (420).

[0304] The signal converter (410) performs frequency conversion on a preset region in the time domain of the received signal to obtain a frequency domain signal.

[0305] The signal converter (410) may be a Short-Time Fourier Transform (STFT) converter, a Fast Fourier Transform (FFT), a cepstrum, or a wavelet transform. The frequency domain signal may be 2D or 3D.

[0306] The above neural network (420) takes the frequency domain signal as an input signal, extracts the characteristics of the input signal based on the learned road surface classification model, classifies the input signal, and estimates the type of the road surface.

[0307] The above neural network (420) is a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer (401), multiple hidden layers (402), and an output layer (403).

[0308] Meanwhile, the structure of the above neural network (420) is not limited to the above-mentioned DNN.

[0309] The above input layer (401) flattens the data of the frequency domain signal and receives it as 1D.

[0310] Data input to the above input layer (401) is characterized and classified through multiple hidden layers (402). The above output layer (403) outputs a probability value for each type of learned road surface.

[0311] The above neural network (420) uses softmax (404) to determine and output the type of road surface with the highest probability among the probability values ​​output from the output layer (403).

[0312] In addition, the neural network (420) estimates the velocity (v) of the moving object estimated by the velocity estimation unit (301). c ) can be used as input to the above input layer (401).

[0313] Meanwhile, the neural network (420) can receive atmospheric information (temperature, humidity, and pressure information) and use it as input to the input layer (401).

[0314] In addition, the neural network (420) can also convert the frequency of the sound wave signal (201) transmitted to the road surface and use it as input to the input layer (401).

[0315] Since it takes less than about 1 ms to perform road surface classification for one input signal in the above artificial neural network (400), there is no problem in estimating the type of road surface by receiving continuously received signals.

[0316] In detail, it takes about 1ms from the time of receiving the reflected and continuously received data (RX), to preprocessing the neural network input data, and transmitting or controlling the artificial intelligence inference and inference results.

[0317] Compared to infrastructure sensors installed in devices such as streetlights, vehicle sensors have a much smaller size of sound wave data input to the artificial neural network (because the recording length is very short), so the neural network size and the size of the data to be preprocessed are very small. Therefore, even if there is some transmission delay, when the vehicle speed is 100 km / h, the sensing distance felt by the vehicle is at most about 6 cm, so it does not need to be considered.

[0318]

[0319] FIG. 23 is a drawing for explaining a signal converter in a road surface type estimation device according to one embodiment of the present invention.

[0320] In Fig. 23, an example of using an STFT converter as a signal converter (410) is explained.

[0321] As illustrated in FIG. 23, the STFT converter performs a short-time Fourier transform on a signal (202) of a preset area of ​​interest received by a sensor (100) mounted on a vehicle after the flight time to obtain a spectrogram (412).

[0322] In another embodiment, the STFT converter obtains a spectrogram (412) by performing a short-time Fourier transform on a signal (202, 411) received after a transmission delay for a preset time, excluding the crosstalk signal of the sound wave signal transmitted by the sound wave transceiver (100) among the reflected signals received in FIG. 3.

[0323] In another embodiment, the STFT converter obtains a spectrogram (412) by performing a short-time Fourier transform on each digital signal received from the buffer unit (302) for a preset time (e.g., 8 ms) from the continuous reflected signals received as in FIG. 16.

[0324] Fourier transform can be performed on a signal for one period, or on a received signal for multiple periods.

[0325] In the present invention, materials are distinguished using information such as acoustic impedance and surface roughness. However, acoustic impedance is not a constant, but its value changes depending on the frequency at which the sound wave vibrates. Therefore, analysis in the frequency domain is required. Among the various methods for converting a time domain received signal into a frequency domain signal, the Time Fourier Transform can be used, and the Short-Time Fourier Transform is used to check the FFT for each time (sampling time).

[0326] In addition, frequency analysis is possible using not only the Short-Time Fourier Transform but also Wavelet, and in the present invention, it was explained that STFT is used to secure sufficient data while reducing the amount of calculation.

[0327] The short-time Fourier transform (STFT) is a method designed to account for temporal variations that conventional Fourier transforms fail to address. The STFT divides a long, time-varying signal into short time units and then applies the Fourier transform.

[0328] However, the STFT also has its drawbacks. Because it separates signals based on window length, it reduces the length of the signal used in the Fourier transform, thereby degrading frequency resolution. However, even if frequency resolution is improved by increasing the window length, time resolution conversely deteriorates. To overcome the resolution limitations caused by this frequency-time tradeoff, the wavelet transform emerged.

[0329] If the window length in STFT is fixed, WT is performing STFT multiple times while changing the window length. In addition, if the basic function in STFT is a sinusoid that extends infinitely in time, there are various types of wavelet functions that exist for a finite period. Wavelet functions include Morlet, Daubechies, Coiflets, Biorthogonal, Mexican Hat, and Symlets.

[0330]

[0331] Figure 24 is a configuration diagram of another embodiment of an artificial neural network in a road surface type estimation device according to the present invention.

[0332] As illustrated in FIG. 24, the artificial neural network (400) is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501), a transmission layer (502), multiple hidden layers (503), and an output layer (504).

[0333] The above convolution performing unit (501) receives a signal (202) of a preset region of interest received after the flight time and performs 1D convolution operations multiple times, and performs batch normalization, Relu function, and MaxPooling function for each convolution operation, and outputs flattened data as the output of the last convolution operation to the transfer layer (502).

[0334] The above transfer layer (502) receives the flattened output data of the above convolution performing unit (501) in 1D. The data input to the transfer layer (502) is characterized and classified through a number of hidden layers (503). The output layer (504) outputs a probability value for each type of learned road surface.

[0335] The above artificial neural network (400) uses softmax (505) to determine and output the type of road surface with the highest probability among the probability values ​​output from the output layer (504).

[0336] Meanwhile, the artificial neural network (400) can receive atmospheric information (temperature, humidity, and air pressure information) and use it as input to the convolution performing unit (501).

[0337] In addition, the artificial neural network (400) can use the velocity (vc) of the moving object estimated by the velocity estimation unit (301) as an input to the convolution performing unit (501) or the transfer layer (502).

[0338] In addition, the artificial neural network (400) can also use the sound wave signal (201) transmitted to the road surface as an input to the convolution performing unit (501).

[0339] In addition, the artificial neural network (400) can also use the vehicle speed, flight time of sound signal, and three-axis acceleration received through the control unit (500) as input signals.

[0340] Since it takes less than about 1 ms to perform road surface classification for one input signal in the above artificial neural network (400), there is no problem in estimating the type of road surface by receiving continuously received signals.

[0341]

[0342] Fig. 25 is a drawing for explaining the operation of the convolution performing unit of Fig. 24.

[0343] As illustrated in FIG. 25, the convolution performing unit (501) of FIG. 24 performs 1D convolution operations five times on the input signal, and performs batch normalization, Relu function, and MaxPooling function for each convolution operation, and the output of the last convolution operation is flattened data.

[0344] The first convolution result (602) for the input signal (601) is the result of performing 1D conv (64,16), BN, ReLU, and MP (8) on the input signal (601), the second convolution result (603) is the result of performing 1D conv (32,32), BN, ReLU, and MP (8) on the first convolution result (602), the third convolution result (604) is the result of performing 1D conv (16,64), BN, ReLU, and MP (8) on the second convolution result (603), the fourth convolution result (605) is the result of performing 1D conv (8,128), BN, and ReLU on the third convolution result (604), and the fifth convolution result The result (606) is the result of performing 1D conv (4,2568), BN, and ReLU on the result of performing the fourth convolution (605).

[0345] Meanwhile, in the present invention, a method for performing a one-dimensional (1D) convolution operation is described as an example, but convolution operations are possible not only in 1D but also in 2D and 3D.

[0346]

[0347] Hereinafter, a road surface type estimation method according to the present invention will be described with reference to FIGS. 26 to 35. The methods illustrated in FIGS. 26 to 35 can be performed by the road surface type estimation device described above, and even if omitted below, the contents described with respect to the road surface type estimation device can be equally applied to the method described below.

[0348]

[0349] Figure 26 is a flowchart of an embodiment of a method for estimating road surface type using a vehicle-mounted sensor according to the present invention.

[0350] First, in order to perform the road surface type estimation method using sound waves according to the present invention, a first learning step (S110) must be performed first to create a road surface classification model.

[0351] In the first learning step (S110), in an environment where a sensor (a vehicle-mounted sensor) and a road surface are perpendicular to each other, a sound wave signal is transmitted to a plurality of types of road surfaces, a reflected signal is received, the signal is converted into a digital signal, the converted digital signal is converted into a frequency domain signal, and the frequency domain signal is input into a neural network (420) to train a road surface classification model.

[0352] Here, in the first learning step (S110), a short-time Fourier transform (STFT) converter, a fast Fourier transform (FFT), a cepstrum, or a wavelet transform may be used to convert into the frequency domain signal. The frequency domain signal may be 2D or 3D.

[0353] In addition, the control unit (500) determines the time of flight (ToF) and the preset time (area of ​​interest) to be filtered through the height information (h) from the ground for the vehicle's on-board sensor (S115).

[0354] In general, the sensor (100) mounted on the vehicle transmits a sound wave signal and then receives the reflected signal.

[0355] Under the control of the above control unit (500), the analog-to-digital converter (200) transmits a sound wave signal from the vehicle's on-board sensor (100), and then receives the reflected signal (S120).

[0356] Thereafter, under the control of the control unit, the analog-to-digital converter (200) converts an analog signal into a digital signal for a preset area of ​​the received signal (S130).

[0357] Afterwards, only the signal reflected from the road surface is filtered and separated from the above digital signal (S140).

[0358] In the above signal filtering step (S140), under the control of the control unit (500), the signal (202) received during the preset time (region of interest) after the time of flight (ToF), i.e., a total of x ms from t_0 [mx] to t_0 + x [ms], is filtered.

[0359] For example, if t_0 is 3 ms and t_0 + x is 5 ms, a total of 2 ms can be set as the region of interest for filtering. Of course, t_0 and x can be adjusted depending on the environment or conditions.

[0360] Thereafter, under the control of the control unit (500), signal conversion is performed on the filtered digital signal to obtain a frequency domain signal (S150).

[0361] Thereafter, under the control of the control unit (500), the frequency domain signal is used as an input signal of the neural network (420), and based on the learned road surface classification model, the characteristics of the input signal are extracted and classified to determine the type of the road surface (S160).

[0362] The above neural network (420) is a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer (401), multiple hidden layers (402), and an output layer (403). The output layer (403) outputs a probability value for each type of learned road surface, and the neural network determines and outputs the type of road surface with the highest probability using softmax (404).

[0363] Meanwhile, the structure of the neural network (420) is not limited to the above-mentioned DNN.

[0364] The above neural network (420) can also receive atmospheric information (temperature, humidity, and pressure information) and use it as input to the input layer (401).

[0365] In addition, the neural network (420) can also convert the frequency of the sound wave signal transmitted to the road surface and use it as input to the input layer (401).

[0366] Meanwhile, under the control of the control unit (500), the road surface type is estimated only when the driving speed of the vehicle is a km / h or more or the acceleration is b ㎨ or more (a and b are preset threshold values).

[0367] Figure 27 is a flowchart of another embodiment of a road surface type estimation method using a vehicle-mounted sensor according to the present invention.

[0368] First, in order to perform a road surface type estimation method using a vehicle-mounted sensor according to the present invention, a second learning step (S190) must be performed first to create a road surface classification model.

[0369] In the second learning step (S190), in an environment where the sensor (vehicle-mounted sensor) and the road surface are perpendicular, a sound wave signal is transmitted to a plurality of types of road surfaces, the reflected signal is received, and the signal is input as a digital signal into an artificial neural network (400) to perform multiple convolution operations to train the road surface classification model.

[0370] In addition, the control unit (500) determines the time of flight (ToF) and the preset time (area of ​​interest) to be filtered through the height information (h) from the ground for the vehicle's on-board sensor (S115).

[0371] In general, the sensor (100) mounted on the vehicle transmits a sound wave signal and then receives the reflected signal.

[0372] Under the control of the above control unit (500), the analog-to-digital converter (200) transmits a sound wave signal from the vehicle's pre-mounted sensor (100), and then receives the reflected signal (S120).

[0373] Thereafter, under the control of the control unit, the analog-to-digital converter (200) converts an analog signal into a digital signal for a preset area of ​​the received signal (S130).

[0374] Afterwards, only the signal reflected from the road surface is filtered and separated from the above digital signal (S140).

[0375] In the above signal filtering step (S140), under the control of the control unit (500), the signal (202) received during a preset time (region of interest) after the time of flight (ToF), i.e., a total of x ms from t_0 [mx] to t_0 + x [ms], is filtered.

[0376] For example, if t_0 is 3 ms and t_0 + x is 5 ms, a total of 2 ms can be set as the region of interest for filtering. Of course, t_0 and x can be adjusted depending on the environment or conditions.

[0377] Thereafter, under the control of the control unit (500), the filtered digital signal is received from the artificial neural network (400) and multiple convolution operations are performed (S170).

[0378] In the above convolution operation step (S170), a 1D convolution operation is performed multiple times on the filtered digital signal, and batch normalization, a Relu function, and a MaxPooling function are performed for each convolution operation, and the output of the last convolution operation is flattened data.

[0379] Thereafter, under the control of the control unit (500), the characteristics of the convolution-operated signal (flattened data) are extracted and classified based on the road surface classification model learned in the artificial neural network (400) to determine the type of the road surface (S180).

[0380] The above artificial neural network (400) is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the digital signal and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504), and the output layer (504) outputs a probability value for each type of learned road surface, and the artificial neural network (400) determines and outputs the type of road surface with the highest probability using softmax (505).

[0381] Meanwhile, the structure of the artificial neural network (400) is not limited to the above-mentioned DCNN.

[0382] The above artificial neural network (400) can also receive atmospheric information (temperature, humidity, and air pressure information) and use it as input to the convolution performing unit (501).

[0383] In addition, the artificial neural network (400) can also use the sound wave signal transmitted to the road surface as an input to the convolution performing unit (501).

[0384] Meanwhile, under the control of the control unit (500), the road surface type is estimated only when the driving speed of the vehicle is a km / h or more or the acceleration is b ㎨ or more (a and b are preset threshold values).

[0385]

[0386] Figure 28 is a flowchart of an embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

[0387] First, in order to perform the road surface type estimation method using sound waves according to the present invention, a first learning step (S210) must be performed first to create a road surface classification model.

[0388] In the first learning step (S210), for a plurality of types of road surfaces and a plurality of vehicle speeds, a sound wave signal is transmitted, a reflected signal is received, the signal is converted into a digital signal, the converted digital signal is converted into a frequency domain signal, and the frequency domain signal is input into a neural network (420) to train a road surface classification model.

[0389] Here, in the first learning step (S210), a short-time Fourier transform (STFT) converter, a fast Fourier transform (FFT), a cepstrum, or a wavelet transform may be used to convert into the frequency domain signal. The frequency domain signal may be 2D or 3D.

[0390] Afterwards, under the control of the control unit (500), a sound wave signal is transmitted to the corresponding road surface whose type is to be known, and the reflected signal is received (S220).

[0391] Thereafter, under the control of the control unit (500), the analog signal is converted into a digital signal for the preset area of ​​the received signal (S230).

[0392] In the signal conversion step (S230), from the received signal, the crosstalk signal of the transmitted sound wave signal is excluded, and the signal for a preset time period is converted into a digital signal based on the point where the amplitude of the signal received after the transmission delay is the largest for each cycle of the sound wave signal.

[0393] For example, when the amplitude of the signal received after the transmission delay is the largest at t_0, a total of (a+b) ms can be observed from t_0 - a [ms] to t_0 + b [ms], and a and b can be adjusted depending on the environment or conditions.

[0394] Thereafter, according to the control of the control unit, the speed of the moving object is estimated using the digital signal (S240). In the moving object speed estimation step (S240), the speed of the moving object can be estimated using the above-described <Mathematical Formula 1> to <Mathematical Formula 4>.

[0395] Thereafter, under the control of the control unit (500), signal conversion is performed on the digital signal to obtain a frequency domain signal (S250).

[0396] In the above frequency domain signal acquisition step (S250), the digital signal is frequency converted in a signal converter (410) to acquire the frequency domain signal.

[0397] Thereafter, under the control of the control unit (500), the frequency domain signal and the estimated velocity (v) of the moving object c ) as an input signal of the neural network (420), and based on the learned road surface classification model, the characteristics of the input signal are extracted and classified to determine the type of the road surface (S260).

[0398] The above neural network (420) is a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer (401), multiple hidden layers (402), and an output layer (403). The output layer (403) outputs a probability value for each type of learned road surface, and the neural network determines and outputs the type of road surface with the highest probability using softmax (404).

[0399] Meanwhile, the structure of the neural network (420) is not limited to the above-mentioned DNN.

[0400] The above neural network (420) can also receive atmospheric information (temperature, humidity, and pressure information) and use it as input to the input layer (401).

[0401] In addition, the neural network (420) can also convert the frequency of the sound wave signal transmitted to the road surface and use it as input to the input layer (401).

[0402]

[0403] Figure 29 is a flowchart of another embodiment of a road surface type estimation method using speed estimation of a moving object according to the present invention.

[0404] First, in order to perform a road surface type estimation method using speed estimation of a moving object according to the present invention, a second learning step (S290) must be performed first to create a road surface classification model.

[0405] In the second learning step (S290), for a plurality of types of road surfaces and a plurality of vehicle speeds, an acoustic signal is transmitted, the reflected signal is received, the signal is converted into a digital signal, and the converted digital signal is input into an artificial neural network (400) to perform multiple convolution operations to train the road surface classification model.

[0406] Afterwards, under the control of the control unit (500), a sound wave signal is transmitted to the corresponding road surface whose type is to be known, and then the reflected signal is received (S220).

[0407] Thereafter, under the control of the control unit (500), the analog signal is converted into a digital signal for the preset area of ​​the received signal (S230).

[0408] In the signal conversion step (S230), from the received signal, the crosstalk signal of the transmitted sound wave signal is excluded, and the signal for a preset time period is converted into a digital signal based on the point where the amplitude of the signal received after the transmission delay is the largest for each cycle of the sound wave signal.

[0409] Thereafter, according to the control of the above control unit, the speed of the moving object is estimated using the above digital signal (S240).

[0410] In the above moving object speed estimation step (S240), the speed of the moving object is estimated using <Mathematical Formula 1> to <Mathematical Formula 4> as described in FIG. 28.

[0411] Thereafter, under the control of the control unit (500), the artificial neural network (400) receives the digital signal and performs multiple convolution operations (S270).

[0412] In the above convolution operation step (S270), a 1D convolution operation is performed multiple times on the digital signal, and batch normalization, a Relu function, and a MaxPooling function are performed for each convolution operation, and the output of the last convolution operation is flattened data.

[0413] Thereafter, under the control of the control unit (500), the characteristics of the convolution-operated signal (flattened data) and the estimated speed of the moving object are extracted and classified based on the road surface classification model learned in the artificial neural network, thereby determining the type of the road surface (S280).

[0414] The above artificial neural network (400) is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the digital signal and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504). The output layer (504) outputs a probability value for each type of learned road surface, and the artificial neural network (400) determines and outputs the type of road surface with the highest probability using softmax (505).

[0415] Meanwhile, the structure of the artificial neural network (400) is not limited to the above-mentioned DCNN.

[0416] The above artificial neural network (400) can also receive atmospheric information (temperature, humidity, and air pressure information) and use it as input to the convolution performing unit (501).

[0417] In addition, the artificial neural network (400) can also use the sound wave signal transmitted to the road surface as an input to the convolution performing unit (501).

[0418]

[0419] Figure 30 is a flowchart of an embodiment of a road surface type estimation method using continuous signals according to the present invention.

[0420] First, in order to perform the road surface type estimation method using sound waves according to the present invention, a learning step (S310) must be performed first to create a road surface classification model.

[0421] In the above learning step (S310), for a plurality of types of road surfaces, after transmitting a continuous sound wave signal, the reflected continuous signal is received, the received continuous signal is converted into a digital signal, and a signal for a preset time is continuously transmitted from the converted digital signal, each signal is converted into a frequency domain signal, and each converted frequency domain signal is input into a neural network (420) to train a road surface classification model.

[0422] Here, in the learning step (S310), a short-time Fourier transform (STFT), a fast Fourier transform (FFT), a cepstrum, or a wavelet transform may be used to convert into the frequency domain signal. The frequency domain signal may be 2D or 3D.

[0423] Afterwards, under the control of the control unit (500), a continuous sound wave signal is transmitted to the corresponding road surface whose type is to be known, and the reflected continuous signal is received (S320).

[0424] Thereafter, under the control of the control unit (500), the analog signal of the received continuous signal is converted into a digital signal (S330).

[0425] Thereafter, under the control of the control unit, at least two buffers continuously receive signals for a preset time from the converted digital signal (S340).

[0426] Meanwhile, the received continuous signal can be compensated for attenuation by being corrected according to the temperature, pressure, and humidity of the atmosphere.

[0427] Here, the control unit (500) controls the data output from the at least two buffers to be converted in a serial manner and transmitted to the artificial neural network (400).

[0428] Thereafter, under the control of the control unit (500), signal conversion (frequency conversion) is performed on each of the continuously transmitted digital signals to obtain a frequency domain signal (S350).

[0429] In the above frequency domain signal acquisition step (S350), for each of the continuously transmitted digital signals, the frequency domain signal is acquired by converting the frequency in the signal converter (410).

[0430] Thereafter, under the control of the control unit (500), the frequency domain signal is used as an input signal of the neural network (420), and based on the learned road surface classification model, the characteristics of the input signal are extracted and classified to determine the type of the road surface (S360).

[0431] The above neural network (420) is a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer (401), multiple hidden layers (402), and an output layer (403). The output layer (403) outputs a probability value for each type of learned road surface, and the neural network determines and outputs the type of road surface with the highest probability using softmax (404).

[0432] Meanwhile, the structure of the neural network (420) is not limited to the above-mentioned DNN.

[0433] The above neural network (420) can also receive atmospheric information (temperature, humidity, and pressure information) and use it as input to the input layer (401).

[0434] In addition, the neural network (420) can also convert the frequency of the sound wave signal transmitted to the road surface and use it as input to the input layer (401).

[0435] Figure 31 is a flowchart of another embodiment of a road surface type estimation method using continuous signals according to the present invention.

[0436] First, in order to perform the road surface type estimation method using continuous signals according to the present invention, the second learning step (S390) must be performed first to create a road surface classification model.

[0437] In the second learning step (S390), for a plurality of types of road surfaces, a continuous sound wave signal is transmitted, the reflected continuous signal is received, the received continuous signal is converted into a digital signal, and a signal for a preset time is continuously transmitted from the converted digital signal, and the signal is input into an artificial neural network (400) to perform multiple convolution operations to train the road surface classification model.

[0438] Afterwards, under the control of the control unit (500), a continuous sound wave signal is transmitted to the corresponding road surface whose type is to be known, and the reflected continuous signal is received (S320).

[0439] Thereafter, under the control of the control unit (500), the analog signal of the received continuous signal is converted into a digital signal (S330).

[0440] Thereafter, under the control of the control unit, at least two buffers continuously receive signals for a preset time from the converted digital signal (S340).

[0441] Meanwhile, the received continuous signal can be compensated for attenuation by being corrected according to the temperature, pressure, and humidity of the atmosphere.

[0442] Thereafter, under the control of the control unit (500), multiple convolution operations are performed on each of the digital signals continuously transmitted from the artificial neural network (400) (S370).

[0443] In the above convolution operation step (S370), a 1D convolution operation is performed multiple times for each of the continuously transmitted digital signals, and batch normalization, Relu function, and MaxPooling function are performed for each convolution operation, and the output of the last convolution operation is flattened data.

[0444] Thereafter, under the control of the control unit (500), the characteristics of the convolution-operated signal (flattened data) are extracted and classified based on the road surface classification model learned in the artificial neural network (400) to determine the type of the road surface (S380).

[0445] The above artificial neural network (400) is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the digital signal and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504). The output layer (504) outputs a probability value for each type of learned road surface, and the artificial neural network (400) determines and outputs the type of road surface with the highest probability using softmax (505).

[0446] Meanwhile, the structure of the artificial neural network (400) is not limited to the above-mentioned DCNN.

[0447] The above artificial neural network (400) can also receive atmospheric information (temperature, humidity, and air pressure information) and use it as input to the convolution performing unit (501).

[0448] In addition, the artificial neural network (400) can also use the sound wave signal transmitted to the road surface as an input to the convolution performing unit (501).

[0449]

[0450] FIG. 32 is a flowchart of an embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 18.

[0451] First, in order to perform the road surface type estimation method using sound waves according to the present invention, a learning step (S410) must be performed first to create a road surface classification model.

[0452] In the above learning step (S410), for a plurality of types of road surfaces and a plurality of vehicle speeds, an acoustic signal is transmitted, a reflected signal is received, the signal is converted into a digital signal, the atmospheric attenuation is corrected for the converted digital signal, and then it is converted into a frequency domain signal, and the frequency domain signal is input into a neural network (420) to train a road surface classification model.

[0453] Here, in the learning step (S410), a short-time Fourier transform (STFT), a fast Fourier transform (FFT), a cepstrum, or a wavelet transform may be used to convert into the frequency domain signal. The frequency domain signal may be 2D or 3D.

[0454] Thereafter, under the control of the control unit (500), a sound wave signal is transmitted to the corresponding road surface whose type is to be known, and the reflected signal is received (S420).

[0455] Thereafter, under the control of the control unit (500), the analog signal is converted into a digital signal for the preset area of ​​the received signal (S430).

[0456] In the signal conversion step (S430), from the received signal, the crosstalk signal of the transmitted sound wave signal is excluded, and the signal for a preset time period is converted into a digital signal based on the point where the amplitude of the signal received after the transmission delay is the largest for each cycle of the sound wave signal.

[0457] For example, when the amplitude of the signal received after the transmission delay is the largest at t_0, a total of (a+b) ms can be observed from t_0 - a [ms] to t_0 + b [ms], and the received signal (202) of Fig. 3b has a of 0.5 and b of 2.5. Of course, a and b can be adjusted depending on the environment or conditions.

[0458] Thereafter, according to the control of the above control unit, the attenuation amount of the digital signal in the air is calculated and corrected (S440).

[0459] In the above atmospheric attenuation correction step (S440), the atmospheric attenuation is calculated using the above-described <Mathematical Formula 7> to <Mathematical Formula 14> and corrected.

[0460] Meanwhile, the three-axis accelerometer (700) continuously measures the three-axis acceleration of the vehicle and transmits it to the control unit (500), and the control unit (500) receives the speed of the vehicle from the vehicle control unit (not shown), and the control unit (500) transmits the speed of the vehicle, the time of flight (ToF) of the sound wave signal, and the measured three-axis acceleration as input signals of the neural network (420) (S445).

[0461] Thereafter, under the control of the control unit (500), signal conversion is performed on the digital signal for which the atmospheric attenuation has been corrected to obtain a frequency domain signal (S450).

[0462] In the above frequency domain signal acquisition step (S450), the digital signal for which the atmospheric attenuation has been corrected is frequency converted in a signal converter (410) to obtain the frequency domain signal.

[0463] Thereafter, under the control of the control unit (500), the frequency domain signal is used as an input signal of the neural network (420), and based on the road surface classification model learned in the first learning step (S410), the characteristics of the input signals are extracted and classified to determine the type of the road surface (S460).

[0464] The above neural network (420) is a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer (401), multiple hidden layers (402), and an output layer (403). The output layer (403) outputs a probability value for each type of learned road surface, and the neural network determines and outputs the type of road surface with the highest probability using softmax (404).

[0465] Meanwhile, the structure of the neural network (420) is not limited to the above-mentioned DNN.

[0466] The above neural network (420) can also receive atmospheric information (temperature, humidity, and pressure information) and use it as input to the input layer (401).

[0467] In addition, the neural network (420) can also convert the frequency of the sound wave signal transmitted to the road surface and use it as input to the input layer (401).

[0468] FIG. 33 is a flowchart of another embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 18.

[0469] First, in order to perform a road surface type estimation method considering the speed and three-axis acceleration of a vehicle according to the present invention, a second learning step (S490) must be performed first to create a road surface classification model.

[0470] In the second learning step (S490), for a plurality of types of road surfaces and a plurality of vehicle speeds, an acoustic signal is transmitted, the reflected signal is received, the signal is converted into a digital signal, the atmospheric attenuation is corrected for the converted digital signal, and then the signal is input into an artificial neural network (400) to perform multiple convolution operations to train the road surface classification model.

[0471] Afterwards, under the control of the control unit (500), a sound wave signal is transmitted to the corresponding road surface whose type is to be known, and the reflected signal is received (S420).

[0472] Thereafter, under the control of the control unit (500), the analog signal is converted into a digital signal for the preset area of ​​the received signal (S430).

[0473] In the signal conversion step (S430), from the received signal, the crosstalk signal of the transmitted sound wave signal is excluded, and the signal for a preset time period is converted into a digital signal based on the point where the amplitude of the signal received after the transmission delay is the largest for each cycle of the sound wave signal.

[0474] For example, when the amplitude of the signal received after the transmission delay is the largest at the time point t_0, a total of (a+b) ms can be observed from t_0 - a [ms] to t_0 + b [ms], and the received signal (202) of Fig. 3b has a of 0.5 and b of 2.5. Of course, a and b can be adjusted depending on the environment or conditions.

[0475] Thereafter, according to the control of the above control unit, the attenuation amount of the digital signal in the air is calculated and corrected (S440).

[0476] In the above atmospheric attenuation correction step (S440), the atmospheric attenuation is calculated and corrected using <Mathematical Formula 4> to <Mathematical Formula 14> as described in FIG. 32.

[0477] Meanwhile, the three-axis accelerometer (700) continuously measures the three-axis acceleration of the vehicle and transmits it to the control unit (500), and the control unit (500) receives the speed of the vehicle from the vehicle control unit (not shown), and the control unit (500) transmits the speed of the vehicle, the time of flight (ToF) of the sound wave signal, and the measured three-axis acceleration as input signals of the artificial neural network (420) (S445).

[0478] Thereafter, under the control of the control unit (500), a digital signal with the atmospheric attenuation corrected is received from the artificial neural network (400) and multiple convolution operations are performed (S470).

[0479] In the above convolution operation step (S470), a 1D convolution operation is performed multiple times on the digital signal for which the atmospheric attenuation has been corrected, and batch normalization, a Relu function, and a MaxPooling function are performed for each convolution operation, and the output of the last convolution operation is flattened data.

[0480] Thereafter, under the control of the control unit (500), based on the road surface classification model learned in the second learning step (S490), the speed of the vehicle, the flight time of the sound signal, the three-axis acceleration, and the characteristics of the convolution operation signal (flattened data) are extracted and classified to determine the type of the road surface (S480).

[0481] The above artificial neural network (400) is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the digital signal and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504). The output layer (504) outputs a probability value for each type of learned road surface, and the artificial neural network (400) determines and outputs the type of road surface with the highest probability using softmax (505).

[0482] Meanwhile, the structure of the artificial neural network (400) is not limited to the above-mentioned DCNN.

[0483] The above artificial neural network (400) can also receive atmospheric information (temperature, humidity, and air pressure information) and use it as input to the convolution performing unit (501).

[0484] In addition, the artificial neural network (400) can also use the sound wave signal transmitted to the road surface as an input to the convolution performing unit (501).

[0485] FIG. 34 is a flowchart of an embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 19.

[0486] First, in order to perform a road surface type estimation method considering the speed and three-axis acceleration of a vehicle according to the present invention, a third learning step (S500) must be performed first to create a road surface classification model.

[0487] In the third learning step (S500), in an environment where a sensor (a vehicle-mounted sensor) and a road surface are perpendicular to each other, a sound wave signal is transmitted for a plurality of types of road surfaces and a plurality of vehicle speeds, a reflected signal is received, the signal is converted into a digital signal, the atmospheric attenuation is corrected for the converted digital signal, and then the signal is converted into a frequency domain signal, and the frequency domain signal is input to the neural network to train the road surface classification model.

[0488] Here, in the third learning step (S4500), a short-time Fourier transform (STFT), a fast Fourier transform (FFT), a cepstrum, or a wavelet transform may be used to convert into the frequency domain signal. The frequency domain signal may be 2D or 3D.

[0489] In addition, the control unit (500) determines the time of flight (ToF) and the preset time (area of ​​interest) to be filtered through the height information (h) from the ground for the vehicle's on-board sensor (S415).

[0490] In general, the sensor (800) mounted on the vehicle transmits a sound wave signal and then receives the acquired reflected signal.

[0491] Thereafter, under the control of the control unit (500), the analog-to-digital converter (200) transmits a sound wave signal from the vehicle's pre-mounted sensor (100), and then receives the reflected signal (S420).

[0492] Thereafter, under the control of the control unit, the analog-to-digital converter (200) converts an analog signal into a digital signal for a preset area of ​​the received signal (S430).

[0493] Afterwards, only the signal reflected from the road surface is filtered and separated from the digital signal (S435).

[0494] In the above signal filtering step (S435), under the control of the control unit (500), the signal (202) received during a preset time (region of interest) after the time of flight (ToF), i.e., a total of x ms from t_1 [mx] to t_1 + x [ms], is filtered.

[0495] For example, if t_1 is 3 ms and t_1 + x is 5 ms, a total of 2 ms can be set as the region of interest for filtering. Of course, t_1 and x can be adjusted depending on the environment or conditions.

[0496] Thereafter, under the control of the control unit (500), the attenuation amount of the filtered digital signal in the atmosphere is calculated and corrected (S440).

[0497] In the above atmospheric attenuation correction step (S440), the atmospheric attenuation is calculated and corrected using <Mathematical Formula 7> to <Mathematical Formula 14> as described in FIG. 32.

[0498] Meanwhile, the speed of sound in the atmosphere can be corrected for attenuation compensation according to the temperature, pressure, and humidity of the atmosphere.

[0499] Meanwhile, the three-axis accelerometer (700) continuously measures the three-axis acceleration of the vehicle and transmits it to the control unit (500), and the control unit (500) receives the speed of the vehicle from the vehicle control unit (not shown), and the control unit (500) transmits the speed of the vehicle, the time of flight (ToF) of the sound wave signal, and the measured three-axis acceleration as input signals of the neural network (420) (S445).

[0500] Thereafter, under the control of the control unit (500), signal conversion is performed on the digital signal for which the atmospheric attenuation has been corrected to obtain a frequency domain signal (S450).

[0501] Thereafter, under the control of the control unit (500), the frequency domain signal is used as an input signal of the neural network (420), and based on the road surface classification model learned in the third learning step (S4500), the characteristics of the input signals are extracted and classified to determine the type of the road surface (S510).

[0502] The above neural network (420) is a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer (401), multiple hidden layers (402), and an output layer (403). The output layer (403) outputs a probability value for each type of learned road surface, and the neural network determines and outputs the type of road surface with the highest probability using softmax (404).

[0503] Meanwhile, the structure of the neural network (420) is not limited to the above-mentioned DNN.

[0504] The above neural network (420) can also receive atmospheric information (temperature, humidity, and pressure information) and use it as input to the input layer (401).

[0505] In addition, the neural network (420) can also convert the frequency of the sound wave signal transmitted to the road surface and use it as input to the input layer (401).

[0506] FIG. 35 is a flowchart of another embodiment of a road surface type estimation method in a road surface type estimation device considering the speed and three-axis acceleration of a vehicle according to the present invention of FIG. 19.

[0507] First, in order to perform a road surface type estimation method considering the speed and three-axis acceleration of a vehicle according to the present invention, a fourth learning step (S520) must be performed first to create a road surface classification model.

[0508] In the fourth learning step (S520), in an environment where a sensor (a vehicle-mounted sensor) and a road surface are perpendicular to each other, a sound wave signal is transmitted for multiple types of road surfaces and multiple vehicle speeds, a reflected signal is received, the signal is converted into a digital signal, and the atmospheric attenuation is corrected for the converted digital signal, and then the signal is input into the artificial neural network to perform multiple convolution operations to train the road surface classification model.

[0509] Here, in the third learning step (S4500), a short-time Fourier transform (STFT), a fast Fourier transform (FFT), a cepstrum, or a wavelet transform may be used to convert into the frequency domain signal. The frequency domain signal may be 2D or 3D.

[0510] In addition, the control unit (500) determines the time of flight (ToF) and the preset time (area of ​​interest) to be filtered through the height information (h) from the ground for the vehicle's on-board sensor (S415).

[0511] In general, the sensor (800) mounted on the vehicle transmits a sound wave signal and then receives the acquired reflected signal.

[0512] Thereafter, under the control of the control unit (500), the analog-to-digital converter (200) transmits a sound wave signal from the vehicle's pre-mounted sensor (100), and then receives the reflected signal (S420).

[0513] Thereafter, under the control of the control unit, the analog-to-digital converter (200) converts an analog signal into a digital signal for a preset area of ​​the received signal (S430).

[0514] Afterwards, only the signal reflected from the road surface is filtered and separated from the digital signal (S435).

[0515] In the above signal filtering step (S435), under the control of the control unit (500), the signal (202) received during a preset time (region of interest) after the time of flight (ToF), i.e., a total of x ms from t_1 [mx] to t_1 + x [ms], is filtered.

[0516] For example, if t_1 is 3 ms and t_1 + x is 5 ms, a total of 2 ms can be set as the region of interest for filtering. Of course, t_1 and x can be adjusted depending on the environment or conditions.

[0517] Thereafter, under the control of the control unit (500), the attenuation amount of the filtered digital signal in the atmosphere is calculated and corrected (S440).

[0518] In the above atmospheric attenuation correction step (S440), the atmospheric attenuation is calculated and corrected using <Mathematical Formula 7> to <Mathematical Formula 14> as described in FIG. 32.

[0519] Meanwhile, the speed of sound in the atmosphere can be corrected for attenuation compensation according to the temperature, pressure, and humidity of the atmosphere.

[0520] Meanwhile, the three-axis accelerometer (700) continuously measures the three-axis acceleration of the vehicle and transmits it to the control unit (500), and the control unit (500) receives the speed of the vehicle from the vehicle control unit (not shown), and the control unit (500) transmits the speed of the vehicle, the time of flight (ToF) of the sound wave signal, and the measured three-axis acceleration as input signals of the neural network (420) (S445).

[0521] Thereafter, under the control of the control unit (500), a digital signal with the atmospheric attenuation corrected is received from the artificial neural network (400) and multiple convolution operations are performed (S470).

[0522] In the above convolution operation step (S470), a 1D convolution operation is performed multiple times on the digital signal for which the atmospheric attenuation has been corrected, and batch normalization, a Relu function, and a MaxPooling function are performed for each convolution operation, and the output of the last convolution operation is flattened data.

[0523] Thereafter, under the control of the control unit (500), based on the road surface classification model learned in the fourth learning step (S120), the speed of the vehicle, the flight time of the sound signal, the three-axis acceleration, and the characteristics of the convolution operation signal (flattened data) are extracted and classified to determine the type of the road surface (S530).

[0524] The above artificial neural network (400) is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the digital signal and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504). The output layer (504) outputs a probability value for each type of learned road surface, and the artificial neural network (400) determines and outputs the type of road surface with the highest probability using softmax (505).

[0525] Meanwhile, the structure of the artificial neural network (400) is not limited to the above-mentioned DCNN.

[0526] The above artificial neural network (400) can also receive atmospheric information (temperature, humidity, and air pressure information) and use it as input to the convolution performing unit (501).

[0527] In addition, the artificial neural network (400) can also use the sound wave signal transmitted to the road surface as an input to the convolution performing unit (501).

[0528]

[0529] Below, we will describe in detail a support structure that can easily install and replace objects such as sensing units (e.g., road surface classification devices) on actual roads, and an AI-IoT platform for sound wave-based road use detection using the same.

[0530]

[0531] The width direction of the support (TB) including the horizontal support (20) and the vertical support (10) described below is the ±x direction indicated in the drawing, the length direction of the support (TB) is the ±y direction indicated in the drawing, and the elevation direction of the support (TB) is the ±z direction indicated in the drawing.

[0532] FIG. 36 is a perspective view of a support (TB) including a horizontal support (20) and a vertical support (10) installed on a road, FIG. 37 is an enlarged view of the AH portion of FIG. 36, FIG. 38 is a diagram schematically illustrating a state in which the horizontal support (20) is rotated and folded relative to the vertical support (10), FIG. 39 is an enlarged view of the AJ portion of FIG. 38, FIG. 40 is an enlarged view of the AK portion of FIG. 38, and FIG. 41 is a block diagram of an acoustic wave-based road use detection AI-IoT platform according to a preferred embodiment of the present invention.

[0533] As illustrated in FIGS. 36 to 41, the sound wave-based road use detection AI-IoT platform comprises a vertical support (10) having a fixed part (11) fixedly installed on a road and a rotating part (12) that can rotate relative to the fixed part (11), a horizontal support (20) coupled to the upper portion of the rotating part (12), a link (50) installed between the vertical support (10) and the horizontal support (20) to enable the horizontal support (20) to rotate relative to the vertical support (10), a rotation drive part (40) that rotates the rotating part (12) of the vertical support (10) and an elevation drive part (30) that raises and lowers the link (50) to rotate the horizontal support (20) relative to the vertical support (10), a sensing part (1100) installed on the horizontal support (20) to detect in real time any one of the road surface condition, traffic volume, and weather environment around the road, and a signal detected by the sensing part (1100). It is configured to include a control unit (1000) that receives sensing information, and a communication unit (1200) that receives sensing information from the control unit (1000) and transmits it to the outside.

[0534] Referring to Fig. 36, a support (TB) including a vertical support (10) and a horizontal support (20) coupled to the upper portion of the vertical support (10) is installed on a road.

[0535] The support beam (TB) is fixedly installed on the road through a fixing member (11) of a vertical support beam (10).

[0536] Specifically, the vertical support (10) is fixedly installed on the road through a fixing member (11), and has a rotating member (12) that can rotate with respect to the fixing member (11) on the upper part of the fixing member (11).

[0537] Referring to Fig. 38, the rotating part (12) can be provided to be rotatable in all directions on the xy plane with respect to the fixed part (11). The arrow shown on the rotating part (12) in Fig. 38 indicates the rotation direction of the rotating part (12) on the xy plane.

[0538] The rotational motion of the rotating part (12) is performed through the rotational driving part (40).

[0539] Referring to Fig. 37, the vertical support (10) has a first fastening portion (14) to which a rotation means (e.g., a drill or driver) for operating a rotation drive portion (40) is fastened on one surface of the fixed portion (11).

[0540] Referring to FIGS. 38 to 40, the rotation drive unit (40) is installed on the fixed unit (11) of the vertical support (10). Specifically, the rotation drive unit (40) is installed between the fixed unit (11) and the rotation unit (12).

[0541] The rotary drive unit (40) includes a first worm (41) that can be rotated, a first worm wheel (42) that is installed so as to be rotatable according to the first worm (41), and a rotary cover (44) that is coupled to the first worm wheel (42) at the lower part and to the rotary unit (12) at the upper part, and that rotates together with the rotary unit (12) according to the rotation of the first worm wheel (42).

[0542] Referring to FIGS. 37 and 38, the first worm (41) is installed on the inside of the fixed part (11). The first worm (41) is provided at a position corresponding to the first fastening part (14) provided on one surface of the fixed part (11). Accordingly, the first worm (41) can be rotated by rotating the rotation axis by a rotation means that is fastened through the first fastening part (14).

[0543] The first worm (41) is installed so that its central axis is perpendicular to the central axis of the first worm wheel (42). More specifically, the first worm (41) is installed so that its teeth formed on the outer surface are engaged with teeth formed along the circumference of a portion of the outer surface of the first worm wheel (42), and its central axis is perpendicular to the central axis of the first worm wheel (42).

[0544] The first worm wheel (42) is installed on the inside of the fixed part (11) so that the teeth on the outer surface mesh with the teeth of the first worm (41) and its central axis is perpendicular to the central axis of the first worm (41). Accordingly, the first worm wheel (42) can rotate according to the rotation of the first worm (41). The fixed part (11) has an upper cover (13) coupled to the upper opening. The upper cover (13) has a first worm wheel insertion hole (WI1) in the central portion. The first worm wheel (42) is installed on the inside of the fixed part (11) so that its upper portion passes through the first worm wheel insertion hole (WI1) of the upper cover (13). An upper bearing (UB) is provided on the upper portion of the first wheel contact portion (43) where teeth are formed in the upward and downward direction (±z direction), and a lower bearing (LB) is provided on the lower portion in the upward and downward direction (±z direction).

[0545] The rotary cover (44) is coupled to the upper part of the first worm wheel (42) through its lower part and is coupled to the rotary part (12) through its upper part. In other words, the rotary drive part (40) is provided between the rotary part (12) and the fixed part (11) through the rotary cover (44) and is coupled to each of the rotary part (12) and the fixed part (11).

[0546] The rotating part (12) rotates as follows through the rotating drive part (40). A rotating means is connected to the fixed part (11) through the first fastening part (14). The rotating means is connected to the rotation axis of the first worm (41) through the first fastening part (14) and rotates the first worm (41). The first worm wheel (42) rotates according to the rotation of the first worm (41). The rotating cover (44) is fixedly connected to the first worm wheel (42) through its lower part and fixedly connected to the rotating part (12) through its upper part. Therefore, the rotating cover (44) rotates together with the rotating part (12) according to the rotation of the first worm wheel (42).

[0547] Referring to FIGS. 36 and 38, the horizontal support (20) is coupled to the upper portion of the rotating part (12).

[0548] Specifically, the horizontal support (20) is hingedly connected to the upper portion of the rotating member (12) via a first hinge axis (HG1). Accordingly, the horizontal support (20) is provided to be rotatable in the xz plane with respect to the vertical support (10). The arrow shown on the horizontal support (20) in Fig. 38 indicates the rotational direction of the horizontal support (20) with respect to the vertical support (10).

[0549] More specifically, a link (50) is provided between a horizontal support (20) and a vertical support (10). The link (50) can be raised and lowered through a lifting drive unit (30) and rotate the horizontal support (20) relative to the vertical support (10).

[0550] The horizontal support (20) has a second hinge axis joint (HP) on the lower surface to enable hinge joint with a link (50).

[0551] One end of the link (50) is hingedly connected to the second hinge axis coupling portion (HP) via the second hinge axis (HG2), and the other end is hingedly connected to the upper portion of the lifting drive unit (30) via the third hinge axis (HG3). The lifting drive unit (30) is installed on the inside of the rotating unit (12).

[0552] Accordingly, at least a portion of the link (50) (one end including a lower portion provided with a hole to which the third hinge axis (HG3) is coupled) is located on the inside of the rotating portion (12), and the remaining portion (the other end including an upper portion provided with a hole to which the second hinge axis (HG2) is coupled) is located on the outside of the rotating portion (12). Specifically, the remaining portion of the link (50) passes through an elevation guide portion (18) formed as an opening portion on one surface of the rotating portion (12) and is positioned to protrude on the outside.

[0553] Referring to FIGS. 38 and 39, the lifting drive unit (30) is installed on the inside of the rotating unit (12).

[0554] Specifically, the lifting drive unit (30) includes a rotatable second worm (31), a second worm wheel (32) installed rotatable along the second worm (31), a screw (34) installed rotatable along the second worm wheel (32), a screw guide (35) that is fastened to the outside of the screw (34) and moves up and down according to the rotation of the screw (34), and a lifting member (36) provided on the outside of the screw.

[0555] Referring to Fig. 39, the second worm wheel (32) is installed on the upper part of the rotary cover (44). The second worm wheel (32) is installed on the upper part of the rotary cover (44) with its upper part positioned in the second worm wheel insertion hole (WI2) provided in the central part of the support member (SP). At this time, a lower bearing (LB) is provided between the rotary cover (44) and the second worm wheel (32), and an upper bearing (UB) is provided above the second wheel contact portion (33) in the elevation direction (±z direction) where the teeth of the second worm wheel (32) are formed. The upper bearing (UB) is located between the second wheel contact portion (33) and the support member (SP).

[0556] The second worm wheel (32) is installed so that the teeth formed along the circumferential direction on the outer surface engage with the teeth of the second worm (31). The second worm wheel (32) is installed so that its central axis is perpendicular to the central axis of the second worm (31).

[0557] The second worm (31) is provided at a position corresponding to the second fastening portion (15) provided on one side of the rotating portion (12). Accordingly, the second worm (31) can be rotated by rotating the rotation axis by a rotation means that is fastened through the second fastening portion (15). Here, the rotation means may be the same as the rotation means that is fastened through the first fastening portion (14).

[0558] The second fastening portion (15) may preferably be provided on one side of the rotating portion (12). In this case, the first fastening portion (14) described above may be provided on one side of the fixed portion (11).

[0559] Alternatively, the second fastening portion (15) may be provided on the fixed portion (11) equipped with the first fastening portion (14). In this case, the second fastening portion (15) may be provided on the same side of the fixed portion (11) equipped with the first fastening portion (14) so ​​that its central axis is positioned on the same line. Alternatively, the second fastening portion (15) may be provided on the fixed portion (11) equipped with the first fastening portion (14), but may be provided on a different side from the side equipped with the first fastening portion (14).

[0560] The second worm (31) is installed so that the teeth formed on the outer surface engage with the teeth of the second worm wheel (32), and its central axis is perpendicular to the central axis of the second worm wheel (32). Accordingly, the second worm wheel (32) can rotate along with the rotation of the second worm (31).

[0561] The screw (34) is provided on the upper part of the second worm wheel (32) so that its central axis aligns with the central axis of the second worm wheel (32). The screw (34) has screw threads on its outer surface. The screw guide (35) is fastened to the outer side of the screw (34). The screw guide (35) is provided with a smaller dimension in the elevation direction (±z direction) than the screw (34). Accordingly, when the screw (34) and the screw guide (35) are fastened, the screw (34) protrudes more than the screw guide (35) in the elevation direction (±z direction).

[0562] The screw (34) and the screw guide (35) are connected through screw threads. At this time, depending on the rotation of the screw (34), the screw guide (35) rises or falls in the t-direction (±z-direction) along the screw threads of the screw (34). When the screw (34) rotates forward, the screw guide (35) can rise by the rotation of the screw (34), and when the screw (34) rotates backward, the screw guide (35) can descend by the rotation of the screw (34).

[0563] The screw guide (35) supports the lifting member (36) through a step formed by the difference between the width direction (±x direction) dimension of the lower part and the width direction (±x direction) dimension of the upper part, and is connected to the lifting member (36) with the upper part inserted into the inside.

[0564] In other words, the screw guide (35) is coupled so that its upper part is inserted into the inside of the lifting member (36) and the lower part of the lifting member (36) comes into contact with the step portion (37) so that its lower part is positioned on the outside of the lifting member (36).

[0565] Accordingly, the lifting member (36) can rise or fall together with the raising and lowering of the screw guide (35).

[0566] A lifting guide portion (LG) is provided on the outer side of the lifting member (36) and the screw guide (35) to guide the raising and lowering of the lifting member (36) by being supported by the upper surface of the support member (SP). The lifting guide portion (LG) has a second insertion hole (LH) formed at the lower side thereof and communicating with a first insertion hole (SH) formed at the lower side of the screw guide (35). The lifting guide portion (LG) is provided on the outer side of the lifting member (36) and the screw guide (35) so that the second insertion hole (LH) corresponds to the first insertion hole (SH). A safety pin (SN) is provided to penetrate the first and second insertion holes (LH). The lifting drive portion (30) may not operate in a state in which the safety pin (SN) is engaged in the first and second insertion holes (LH). The lifting drive portion (30) may operate in a state in which the safety pin (SN) is disengaged.

[0567] Referring to FIGS. 38 and 40, one end of the link (50) is hingedly connected to the upper end of the lifting member (36) via the third hinge axis (HG3). Accordingly, the link (50) can rise or fall together with the rising or falling of the lifting member (36). The arrows shown on the lifting member (36) in FIG. 38 indicate the direction of elevation of the lifting member (36).

[0568] The link (50) rotates with a greater or lesser inclination relative to a vertical center line perpendicular to the elevation direction (±z direction) of the rotating part (12). This is achieved by the third hinge axis (HG3) and the second hinge axis (HG2). The arrows shown around the link (50) in Fig. 38 indicate the direction in which the link (50) rotates with a greater or lesser inclination relative to the vertical center line of the rotating part (12).

[0569] Specifically, the link (50) can rise in the lifting direction (±z direction) according to the rising of the lifting member (36). At this time, the link (50) rotates so that the inclination with respect to the vertical center line of the rotating part (12) becomes increasingly larger by the third hinge axis (HG3) and the second hinge axis (HG2). Accordingly, the horizontal support (20) rotates on the xz plane so that the angle between its horizontal center line in the width direction (±x direction) and the vertical center line of the vertical support (10) in the lifting direction (±z direction) forms a right angle. In this case, the horizontal support (20) is in a form that spreads away from the vertical support (10). Here, the right angle is 90°. The horizontal support (20) can rotate by hinge connection with the vertical support (10) through the first hinge axis (HG1).

[0570] In contrast, the link (50) can be lowered in the lifting direction (±z direction) according to the lowering of the lifting member (36). At this time, the link (50) rotates so that the inclination with respect to the vertical center line of the rotating part (12) becomes gradually smaller by the third hinge axis (HG3) and the second hinge axis (HG2). As a result, the horizontal support (20) rotates on the xz plane so that the angle between its horizontal center line and the vertical center line of the vertical support (10) forms an acute angle. In this case, the horizontal support (20) is folded so as to become closer to the vertical support (10) while gradually lowering its height based on the lifting direction (±z direction). Here, the acute angle is an angle range greater than 0° and less than 90°.

[0571] The link (50) is guided to move up and down by the lifting guide part (18) according to the up and down movement of the lifting member (36). The lifting guide part (18) may be provided with an opening cover means (not shown) that covers the opening portion constituting the lifting guide part (18). The opening cover means may be provided in the form of a Javara that can be folded and unfolded in a wrinkled shape, for example.

[0572] The opening cover means may preferably have an insertion hole through which a link (50) passes at least partly (preferably, the central part in the elevation direction (±z direction)). The opening cover means may pass the link (50) through the insertion hole and be combined with the link (50) to cover the opening portion of the elevation guide part (18) according to the elevation and descent of the link (50).

[0573] For example, the link (50) may penetrate the insertion hole of the opening cover means and be located on the outside of the rotating part (12). At this time, a wrinkled portion constituting the opening cover means may exist above and below the link (50) in the elevation direction (±z direction).

[0574] For example, when the link (50) rises, the opening cover means can spread the wrinkled portion existing at the lower portion of the link (50) in the rising direction of the link (50) as the link (50) rises. At this time, the wrinkled portion existing at the upper portion of the link (50) can fold in the rising direction of the link (50).

[0575] Conversely, when the link (50) is lowered, the opening cover means may unfold the wrinkled portion existing on the upper portion of the link (50) in the downward direction of the link (50) as the link (50) is lowered. At this time, the wrinkled portion existing on the lower portion of the link (50) may fold in the downward direction of the link (50).

[0576] The opening cover means can prevent foreign substances (e.g., rain, snow, or other contaminants) from penetrating into the inside of the rotating part (12) through the lifting guide part (18) formed by the opening portion of the rotating part (12).

[0577] Meanwhile, a drain hole (EX) may be separately provided in the rotating part (12). The drain hole (EX) may preferably be provided between the rotating part (12) and the rotating cover (44). The drain hole (EX) may be provided in the form of a hole, for example. The drain hole (EX) may serve to discharge foreign substances that have entered the interior of the rotating part (12) to the outside. Accordingly, the problem of the lifting drive part (30) provided inside the rotating part (12) and the rotating drive part (40) provided inside the fixed part (11) being contaminated or corroded by foreign substances can be prevented.

[0578] In this way, the horizontal support (20) is provided to be able to rotate and increase or decrease the incline with respect to the vertical support (10) through the lifting drive unit (30), and the vertical support (10) is provided to be able to rotate with respect to the fixed unit (11) through the rotation drive unit (40) and has a rotation unit (12) to which the horizontal support (20) is coupled.

[0579] The sensing unit (1100) installed on the horizontal support (20) can be easily installed and replaced.

[0580] Referring to Fig. 36, the sensing unit (1100) is preferably detachably installed on the head unit (21) that is coupled to one end of the horizontal support (20). The sensing unit (1100) is installed on the lower surface of the head unit (21) and is installed horizontally with the road surface.

[0581] The sensing unit (1100) is installed in the head unit (21) and can detect in real time any one of the road surface condition, road flooding, traffic volume, and weather conditions around the road.

[0582] The sensing unit (1100) may be configured to include a non-contact sound wave sensor, a traffic volume detection sensor, and a weather environment detection sensor.

[0583] A non-contact acoustic sensor transmits acoustic signals onto a road surface and receives reflected signals. The acoustic signals received by the acoustic sensor are preferably periodically transmitted to a control unit (1000). The acoustic signals transmitted to the control unit (1000) can be transmitted to an external server via a communication unit (1200).

[0584] An external server can convert the received signal into a digital signal and train an AI model by computing the converted signal. Accordingly, the platform of the present invention can estimate the road surface condition using an AI model trained on the scanning and reflection signal characteristics of these sound waves. The trained AI model can estimate whether the road surface condition is dry, wet, slippery due to black ice caused by snow or rain, or present with snow or rain.

[0585] The sensing unit (1100) can perform a function of checking whether the road is flooded. This can be preferably accomplished by checking whether the road surface is submerged in water using a non-contact sonic sensor. Alternatively, whether the road is flooded can be checked using another sensor capable of determining whether the road surface is submerged in water, in addition to the sonic sensor.

[0586] A traffic detection sensor may be, for example, a radar sensor. A radar sensor can detect an object by receiving a reflected signal reflected from an object, and measure the direction in which the object was detected, the distance to the detected object, the speed, the angle, etc. Such a radar sensor can be installed around a road and detect vehicles driving on the road, and transmit sensing information for analyzing and predicting traffic volume to a control unit (1000).

[0587] The meteorological environment detection sensor may be equipped with an atmospheric sensor that performs the function of measuring temperature, humidity, and atmospheric pressure in the atmosphere. Sensing information detected by the meteorological environment detection sensor may be transmitted to the control unit (1000).

[0588] The sensing unit (1100) can be positioned on the vehicle travel path side of the road to obtain sensing information by performing a sensing operation. The horizontal support (20) is raised by the elevation drive unit (30) while positioned on the vehicle travel path side through the rotation of the rotation unit (12) via the rotation drive unit (40), and is unfolded so that the angle between the horizontal center line and the vertical center line of the vertical support (10) forms a right angle.

[0589] Specifically, the rotary drive unit (40) can rotate the rotary unit (12) in one direction to position the horizontal support (20) on the guide side or rotate it in the other direction to position it on the vehicle travel path side. The rotary unit (12) rotates in one direction or the other direction by means of a rotary cover (44) coupled to a first worm wheel (42) that rotates according to the rotation of the first worm (41).

[0590] In order to obtain sensing information through the sensing unit (1100) in the platform of the present invention, the rotating unit (12) is rotated in one direction by the rotating driving unit (40). This allows the horizontal support (20) to be positioned on the vehicle travel path side. The horizontal support (20) is raised by the lifting driving unit (30) while positioned on the vehicle travel path side, and is unfolded so that the angle between its horizontal center line and the vertical center line of the vertical support (10) forms a right angle. The horizontal support (20) can be unfolded through an operation in which the second worm (31) is rotated in one direction, the screw (34) is rotated in the forward direction by the second worm wheel (32), and the screw guide (35) and the lifting member (36) are raised.

[0591] The various sensors (sound wave sensor, traffic volume detection sensor, weather environment detection sensor) included in the sensing unit (1100) are consumables and may need to be replaced.

[0592] The sensing unit (1100) of the present invention can be easily replaced by a structure that rotates the vertical support (10) and raises and lowers the horizontal support (20).

[0593] Specifically, the rotary part (12) can be rotated in one direction by the rotary drive part (40) so that the horizontal support (20) can be positioned toward the delivery side. Then, the horizontal support (20) is lowered by the lifting drive part (30). As a result, the horizontal support (20) is folded so that the angle between its horizontal center line and the vertical center line of the vertical support becomes an acute angle. Due to the lowering and folding motion of the horizontal support (20), the sensing part (1100) is positioned at the eye level of the worker without the worker having to climb up to the height of the sensing part (1100).

[0594] Traditionally, sensing units were installed on fixed horizontal and vertical supports. This required workers to remain in the vehicle's path when replacing the sensing units. Installing new sensing units also required workers to remain in the vehicle's path. This increased the risk of safety accidents during the work, and the requirement to control the road each time installation and replacement made the work cumbersome and inefficient.

[0595] However, the sensing unit (1100) of the present invention can be positioned on the delivery side during replacement and installation work by positioning the horizontal support (20) on the delivery side through the rotation of the vertical support (10). In addition, the sensing unit (1100) can be positioned at a height where the worker can work through the lowering and folding operation of the horizontal support (20).

[0596] In the platform of the present invention, the sensing unit (1100) can be moved to a position where the worker can work, so that installation and replacement work can be performed safely and easily on the sidewalk. In addition, when the sensing unit (1100) is newly installed, the installation work can be conveniently performed by rotating the vertical support (10) and lowering the horizontal support (20).

[0597] In this way, the sensing unit (1100) in the platform of the present invention can be easily installed around a road, thereby increasing the efficiency of installation and replacement work. Therefore, the platform of the present invention can significantly reduce the risk of safety accidents that occur due to the failure to prevent black ice due to the absence of the sensing unit (1100), which makes it difficult to install and replace the sensing unit (1100) on the road.

[0598] When performing a sensing operation, the sensing unit (1100) is located on the vehicle's driving path and performs the sensing operation in a horizontal state with the road surface by means of an unfolded horizontal support (20).

[0599] The sensing unit (1100) can obtain sensing information by measuring the state of the road surface, preferably through a sound wave sensor, while being horizontal to the road surface.

[0600] The head portion (21) on which the sensing portion (1100) is installed is provided to be rotatable with respect to the horizontal support (20).

[0601] Referring to Fig. 36, the head portion (21) may be preferably provided to be rotatable about the horizontal center axis of the horizontal support (20) as the axis of rotation when the horizontal support (20) is spread out and away from the vertical support (10). In the drawing of Fig. 36, the head portion (2 1) may be provided to be rotatable in the yz plane about the horizontal center axis of the horizontal support (20) as the axis of rotation. The arrow shown on the head portion (21) indicates the direction of rotation of the head portion (21).

[0602] The sensing unit (1100) is installed on a rotatable head unit (21) so that it can always be level with the road surface. The sensing unit (1100) is installed around the road and may not be level with the road surface due to external factors (wind, external force, etc.). The platform can check the level between the road surface and the sensing unit (1100) through a signal received from the sound wave sensor of the sensing unit (1100). (The level between the sensing unit (1100) and the road surface can also be checked through other means (camera or naked eye).)

[0603] When the sensing unit (1100) and the road surface are not level, the platform rotates the head unit (21) with the horizontal support (20) unfolded and the horizontal center axis of the horizontal support (20) as the rotation axis. The rotation of the head unit (21) can be implemented, for example, through a motor controlled by the control unit (1000). The sensing unit (1100) is installed on the rotatable head unit (21), so that its position can be easily adjusted from an unlevel state to a horizontal state with respect to the road surface. This can further improve the accuracy and efficiency of detection when detecting the road surface condition through the sensing unit (1100).

[0604] A distribution box (16) may be provided in the fixed part (11). A control unit (1000) and a communication unit (1200) are provided inside the distribution box (16).

[0605] Referring to FIG. 37, the control unit (1000) has a function of receiving sensing information from the sensing unit (1100) and transmitting the received sensing information to the communication unit (1200).

[0606] The communication unit (1200) performs a function of transmitting sensing information received from the control unit (1000) to the outside or receiving external information from an external server. The holding unit (TB) can communicate with an external server through the communication unit (1200) and communicate with other holding units (TB) located in the vicinity.

[0607] Specifically, the holding units (TB) can communicate with each other through the communication unit (1200). Any one of the plurality of holding units (TB) can transmit external information by communicating with an external server through the communication unit (1200). A holding unit (TB) that has transmitted external information from an external server can transmit the received external information to another holding unit (TB) in the vicinity. As a result, even a holding unit (TB) installed in a location where direct communication with an external server is difficult can transmit external information through another holding unit (TB).

[0608] In other words, each holding device (TB) can communicate with an external server through a communication unit (1200), and a holding device (TB) installed in a location where communication with an external server is difficult can receive external information by communicating with another holding device (TB) installed in a location where communication with an external server is possible.

[0609] A camera (1300) may be installed at the bottom of the distribution box (16). The camera (1300) may be installed for security purposes to prevent damage and destruction of a support beam (TB) installed on the road.

[0610] The platform of the present invention may further include a facility unit. The facility unit performs a function of performing necessary road response based on sensing information acquired through the sensing unit (1100). The facility unit may include a heating element and a salt spray means.

[0611] As an example, the control unit (1000) can determine the presence of black ice on the road surface based on sensing information obtained through the sonic sensor of the sensing unit (1100). The control unit (1000) can control the equipment unit to operate a heating wire linked to a support (TB) or spray salt water onto the road.

[0612] The sensing unit (1100) of the present invention, which acquires sensing information from a platform, can be installed and replaced safely on a platform by having a structure that rotates a rotating unit (12) through a rotating driving unit (40) and lowers a horizontal support (20) through an elevating driving unit (30). In addition, a worker can easily install or replace the sensing unit (1100) without having to climb up to the position of the sensing unit (1100) using a ladder.

[0613] In this way, the platform of the present invention facilitates the installation and replacement of the sensing unit (1100), thereby further increasing the installation rate of the sensing unit (1100) around roads. This further minimizes safety accidents that may arise due to the inability to prevent black ice on the road due to the absence of the sensing unit (1100).

[0614]

[0615] Meanwhile, below, a road surface information providing server and method for obtaining information on the condition of a road surface using sound wave signals and providing the information to a user, as well as an entire system for providing the same, will be described.

[0616] In one embodiment of the present disclosure, the entire system may be composed of a sensor that senses sound wave information about a road surface, a server that acquires data from the sensor and processes the task of providing the data, and a user that requests or acquires such data and utilizes the data.

[0617] FIG. 42 is a diagram illustrating a road information providing system according to various embodiments of the present disclosure.

[0618] Referring to Fig. 42, the entire system (3000) in which the road information provision server (hereinafter referred to as the server) operates can be configured by connecting the server (2100), the road classification device (2200), and the user terminal (2300) to a network (1), respectively. That is, the system (3000) can acquire and process data that is continuously and / or periodically updated within the server (2100).

[0619] According to one embodiment of the present disclosure, a server (2100) acquires sound wave data sensing the road surface from a sensor or the results of determining the road surface condition (hereinafter referred to as "road surface data"), and can additionally process not only the road surface data but also other types of data for use by the user. Furthermore, the server (2100) can provide data or perform an authentication step according to the user's conditions.

[0620] A road surface classification device (2200) according to one embodiment of the present disclosure is a device installed on road infrastructure or a moving object to determine the type or condition of a road surface. It may include a device that is composed solely of a sensor to obtain sound wave data sensing the road surface, or a device that includes a sensor to obtain data determining the condition of the road surface. Here, it can be understood that the road surface classification device (2200) can be applied to the above-described road surface type estimation device.

[0621] A user terminal (2300) according to one embodiment of the present disclosure may be used as a term referring to a terminal through which a user communicates with a server. Meanwhile, the term "user" in the present disclosure refers to anyone who uses data from the server, and may also refer to anyone who uses navigation that applies road conditions without directly requesting data, a business that does not directly use navigation but requests data from the server to apply information to navigation, and any other person who obtains data for the purpose of utilizing the data. The term will be appropriately expressed according to the description and should be interpreted accordingly.

[0622] The server (2100), the road surface classification device (2200), and the user terminal (2300) may each or all be connected to a network (1). Here, being connected to a network (1) can be understood to mean being electrically or communicationally connected.

[0623] Examples of the network (1) include, but are not limited to, a 3rd Generation Partnership Project (3GPP) network (such as a 3G network, a 4G or LTE (Long Term Evolution) network, a 5G or NR (New Radio) network, or a 6G network), a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), a Wireless Local Area Network (WLAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) network, and the like. According to one embodiment of the present disclosure, within the system, a server (2100) can obtain data for determining road information or derived road information through a road classification device (2200). In addition, the server (2100) can transmit data to a user terminal (2300) according to an appropriate standard or request.

[0624] According to one embodiment of the present disclosure, a server (2100) can process and store various types of data, and transmit and / or receive the processed data. More specifically, the server (2100) can be interconnected with a road surface classification device (2200) and a user terminal (2300) via a network (1). Accordingly, the server (2100) can transmit and / or receive data for linking data between the two via the network (1).

[0625] The server (2100) may be, for example, a platform, a host server providing a specific service, and / or a cloud server, but is not limited thereto. The server (2100) may be configured integrally with a database, for example, and may search for data corresponding to a request from a user terminal (2300), generate requested specific information based on the searched information, and transmit the generated specific information back to the user terminal (2300).

[0626] The server (2100) may exist as a single server or may be comprised of multiple servers. In one embodiment, the server (2100) may include multiple different servers depending on the entity operating the server.

[0627] Meanwhile, the road surface information in the present disclosure can indicate the degree of road slipperiness, and may include data that quantifies the degree of road slipperiness. Alternatively, the road surface information may include data indicating road surface conditions such as dryness, black ice, rain, or snow. Alternatively, the road surface information may provide both the road surface condition and data quantifying the degree of road slipperiness.

[0628] According to one embodiment of the present disclosure, the server (2100) can generate necessary data according to a user request. The data may include data related to traffic information, such as traffic volume, vehicle speed, and acquisition time at the location, as well as data regarding road surface conditions at each location.

[0629] According to one embodiment of the present disclosure, information related to data provision by a user terminal (2300) (e.g., history of using traffic data, amount of data transmitted from the server (2100) to the user terminal (2300), amount of data acquired from the road surface classification device (2200), etc.) is recorded by the server (2100), and this can be used to determine the data provision contribution of a sensor. That is, the server (2100) can store information related to data provision (e.g., history of using traffic data, amount of data transmitted from the server (2100) to the user terminal (2300), amount of data acquired from the road surface classification device (2200), etc.) and use the information related to data provision to determine the contribution. The server (2100) can provide compensation for data provision based on this data provision contribution.

[0630] A user terminal (2300) according to one embodiment of the present disclosure may include, but is not limited to, all types of input / output capable wired / wireless communication devices, such as, for example, a PCS (Personal Communication System), a GSM (Global System for Mobile communication), a PDC (Personal Digital Cellular), a PHS (Personal Handyphone System), a PDA (Personal Digital Assistant), an IMT (International Mobile Telecommunication)-2000, a CDMA (Code Division Multiple Access)-2000, a W-CDMA (WCode Division Multiple Access), a Wibro (Wireless Broadband Internet) terminal, a Smartphone, a SmartPad, a tablet PC, a laptop, a wearable device, a digital signage, etc.

[0631] The overall system configuration of the present disclosure is not limited to the above configuration and may include all other necessary configurations or omit some configurations. Furthermore, it may include configurations that can be interpreted identically to the above configuration.

[0632] FIG. 43 is a block diagram illustrating a configuration of a road information provision server according to one embodiment of the present disclosure.

[0633] The road information provision server (2100) may include a communication unit (2110), a memory (2120), and a processor (2130).

[0634] The communication unit (2110) can perform communication with the server (2100), the road surface classification device (2200), and the user terminal (2300), and the communication unit (2110) can include a data transmitter and a receiver. The communication unit (2110) can be implemented as a module of various standards that performs wired communication or wireless communication. For example, the communication unit (2110) can include a short-range communication unit (not shown) that performs according to a short-range communication standard such as BLE (Bluetooth Low Energy), Bluetooth, or Wi-Fi, or a long-range communication unit (not shown) that performs according to a long-range communication standard such as 3G, 4G, or 5G.

[0635] According to one embodiment of the present disclosure, the communication unit (2110) of the server (2100) can receive data such as road surface data, user authentication tokens, and location information from a road surface classification device, and can receive data such as user authentication tokens and request signals from a user terminal. In addition, when requesting data from a provider, location information and a request signal can be transmitted, or data can be transmitted in response to a user request.

[0636] Data acquired by the server (2100) can be stored in the memory (2120).

[0637] The memory (2120) may store processes and related data performed by the processor (2130).

[0638] According to one embodiment of the present disclosure, the memory (2120) may store sensing data and road surface data acquired from a road surface classification device. Alternatively, when the processor (2130) acquires sensing data and determines road surface conditions within the server, an AI algorithm related thereto may be stored.

[0639] According to one embodiment of the present disclosure, the memory (2120) may store a program for the processor (2130) to generate data to be transmitted to the UI / UX to provide road surface information. Additionally, the memory (2120) may store settings and processes for the processor (2130) to transmit data to a user terminal.

[0640] The memory (2120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory may store information temporarily, permanently, or semi-permanently, and may be provided as a built-in or removable type.

[0641] Additionally, the memory (2120) may be implemented as at least one independent sub-database. For example, each sub-database included in the database may be wirelessly connected via Bluetooth, Wi-Fi, etc., and may exchange data with each other via a network. Of course, the database is not limited to the above-described description, and may be implemented as an integrated database in which at least one independent sub-database is integrated into one. According to one embodiment, the memory (2120) may be implemented as an independently implemented database or may be implemented in a form in which data is exchanged through communication with an independently configured database.

[0642] According to one embodiment of the present disclosure, when the processor (2130) obtains sound wave data from the road surface classification device (2200), it can determine road surface information using an artificial intelligence model, and can generate additional data using location information and road surface information.

[0643] Additionally, the processor (2130) can determine conditions for providing information according to a user request signal, or determine data to be provided according to road conditions.

[0644] Additionally, the processor (2130) may further perform a task of determining a price to be requested from the user or a reward to be provided to the provider based on the transmitted and received data.

[0645] The processor (2130) may include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a state machine, an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), etc.

[0646] The server components, including a communication unit (2100), a memory (2200), and a processor (2130), are disclosed above, but in addition to the above configuration, all necessary configurations may be included, and all configurations that perform the same function as the above configuration and are judged to be substantially the same configuration may be included.

[0647] FIG. 44 is a diagram illustrating a type of road classification device that provides information to a road information provision server according to the present disclosure.

[0648] According to one embodiment of the present disclosure, the road surface classification device may refer to a first road surface classification device (2310) that is fixed and used on a streetlight, etc., and a second road surface classification device (2320) that is attached and used on a vehicle, etc. In this case, the first road surface classification device (2310) and the second road surface classification device (2320) may perform the same processing method for road surface classification. Since the first road surface classification device (2310) and the second road surface classification device (2320) can determine the condition of the road surface while being fixed or moving, respectively, there may be functional differences based on this.

[0649] According to one embodiment of the present disclosure, the first road surface classification device (2310) can determine the road surface condition at a fixed location, and thus, in addition to determining the road surface condition, can additionally determine data on traffic volume, average speed of vehicles passing through the road, etc.

[0650] According to one embodiment of the present disclosure, the second road surface classification device (2320) is attached to a vehicle or the like and thus may additionally include a configuration that performs a correction function by the moving object and additional sensing functions for autonomous driving. That is, the second road surface classification device (2320) can determine the road surface condition in real time for the vehicle to which it is attached and perform functions such as acceleration and deceleration control, and can provide an alarm to the driver regarding the road surface condition. Additionally, the second road surface classification device (2320) can share data through data sharing between them.

[0651] A second road surface classification device (2320) can be attached to each mobile device to collect data. The server can obtain data through the second road surface classification device (2320) or the provider terminal and provide compensation to the provider based on the collected data. In the present disclosure, the "provider" may refer to an entity that attaches a second road surface classification device (2320) to each mobile device to obtain road surface information and provide the information to the server (2100).

[0652] In a system according to an embodiment of the present disclosure, when data is collected by the first road surface classification device (2310), data can be continuously collected at a certain location. When data is collected by the second road surface device (2320), information about the location can be acquired as needed even in an area without a fixed sensor. Accordingly, the server can acquire road surface information in two ways. When road surface information is not acquired or when information needs to be updated, the server can transmit a signal to the second road surface classification device to additionally acquire information about the location where information is needed.

[0653] FIG. 45 is a diagram showing the overall flowchart of the operation of a road information provision system according to one embodiment of the present disclosure.

[0654] In S1401, the road surface classification device (2200) can transmit sound waves and sense and acquire sound waves reflected from the road surface. This allows for the acquisition of data regarding the road surface, which will be described later in FIG. 46. The data regarding the road surface at this time may include road surface condition classification data or numerical values ​​for the road surface's slipperiness.

[0655] In S1402, the server (2100) can obtain road surface data from the road surface classification device (2200). In addition to the acquired sonic data, the server can also obtain location information where the data was acquired. The sonic data can be transmitted in real time and stored in a manner matched with the location information.

[0656] The server (2100) can obtain authentication information of the provider transmitting data of the road surface classification device (2200). At this time, the authentication information of the provider can be stored as an authentication token.

[0657] In S1403, the server (2100) may process and generate data based on the acquired road surface information. Prior to processing and generating the data, the server (2100) may store the data acquired from the road surface classification device (2200). In this case, the server (2100) may match the acquisition location and acquisition time information of each piece of data and store them in memory.

[0658] The server can generate data to be provided to the user based on a request signal from the user terminal. The data to be provided to the user may be generated based on road surface data. Examples of the data to be provided to the user may include: notification criteria determined based on road surface conditions; map data indicating road surface conditions; traffic volume corresponding to a risk zone; average vehicle speed corresponding to a risk zone; time zones where a non-risk zone changed to a risk zone; and the total length of the risk zone.

[0659] The server (2100) may obtain additional data (such as traffic volume, vehicle speed, road surface condition and road type, and notification criteria according to terrain) for obtaining the above types of data from a road surface classification device or receive and use the data from the memory (2200) within the server.

[0660] In S1404, the server (2100) can transmit data to the user terminal in response to a stored transmission criterion or a user's signal.

[0661] For example, when the road surface status acquired from the road surface classification device is provided to the UI / UX of an application operated by a server, the server (2100) can set a transmission standard to continuously update at regular intervals.

[0662] For example, when the server (2100) provides road surface data acquired from the second road surface classification device to the UI / UX of an application operated by the server, the server (2100) may provide data in response to a condition in which a state indicating the road surface condition or the degree of slipperiness of the road surface changes.

[0663] For example, the server (2100) can continuously provide data to navigation-related users according to a set cycle.

[0664] For example, when a user requests data for a specific area, the server (2100) can identify the necessary data corresponding to the request signal. After identifying the type of data corresponding to the request signal, the server can transmit the corresponding data or generate data in response to the request signal. In this case, S1404 can generate data after S1405.

[0665] In S1405, the server (2100) may provide compensation to the provider terminal that provided sensor data or road surface data based on the data provided to the user terminal (2300). The compensation may be calculated based on the provided data. That is, if data acquired from a road surface classification device is provided to the server via a provider terminal, the extent to which each provider terminal contributed to the server data may be calculated and reasonable compensation may be provided based on this.

[0666] Through this method, the user terminal (2300) can use the data acquired in S1404 to generate additional data or utilize it for business purposes. This can increase the utilization of traffic data across all industries.

[0667] For example, the user terminal may be an electronic device within an insurance company that requests traffic data to determine insurance compliance. In this case, the server (2100) may use information such as road surface conditions and the average speed of surrounding vehicles to calculate insurance benefits.

[0668] For example, if a user terminal is an electronic device requesting government policy development, a dataset containing stored road condition information for the relevant section can be provided. This dataset can be used to develop regional policies.

[0669] For example, a user terminal may be an electronic device that collects or generates navigation data. In this case, the user terminal may obtain the road surface hazard level and corresponding hazard notification criteria and transmit this information to the driver terminal, thereby providing a hazard notification function to drivers passing through a hazardous area.

[0670] As previously explained, the order of business performed on the server (2100) may be configured by changing each order of the flowchart of FIG. 45, or some may be omitted, and additional configurations may be added and used.

[0671] FIG. 46 is a diagram illustrating a method for processing and generating data in a road information provision server according to various embodiments of the present disclosure.

[0672] According to various embodiments of the present disclosure, when a sensed acoustic wave reflection signal is acquired by a sensor, an artificial neural network can be used to perform a road surface classification method using the acoustic wave reflection signal.

[0673] According to one embodiment of the present disclosure, the server (2100) may generate first data or second data for transmission to a user terminal. The first data may be generated by the server (2100) or may be generated within a road surface classification device and transmitted to the server (2100).

[0674] The neural network model of the artificial neural network according to various embodiments of the present disclosure may include a plurality of layers or strata.

[0675] A neural network model can be implemented as a classifier that generates road surface classification information. The classifier can perform multiple classifications. For example, a neural network model can be a multi-classification model that classifies input data into multiple classes.

[0676] A neural network model according to one embodiment of the present disclosure may include a deep neural network (DNN) of a multi-layer perceptron algorithm including an input layer, multiple hidden layers, and an output layer.

[0677] According to another embodiment of the present disclosure, a neural network model may include a convolutional neural network (CNN). As the CNN structure, at least one of AlexNet, LENET, NIN, VGGNet, ResNet, WideResnet, GoogleNet, FractaNet, DenseNet, FitNet, RitResNet, HighwayNet, MobileNet, and DeeplySupervisedNet may be used. The neural network model may be implemented using multiple CNN structures.

[0678] For example, a neural network model can be implemented to include multiple VGGNet blocks. More specifically, the neural network model can be prepared by combining a first structure in which a CNN layer having 64 filters of 3x3 size, a BN (Batch Normalization) layer, and a ReLu layer are sequentially combined, and a second block in which a CNN layer having 128 filters of 3x3 size, a ReLu layer, and a BN layer are sequentially combined.

[0679] The neural network model may include a max pooling layer following each CNN block, and may include a global average pooling (GAP) layer, a fully connected (FC) layer, and an activation layer (e.g., sigmoid, softmax, etc.) at the end.

[0680] The artificial neural network according to various embodiments of the present disclosure refers to a neural network model for classifying a road surface by extracting characteristics from a frequency conversion signal of an acoustic signal, and is not limited to the examples described above.

[0681] A road surface classification artificial neural network according to various embodiments of the present disclosure can be trained using frequency domain data of a reflected signal as an input value, and the trained artificial neural network can classify a road surface on which a target signal is reflected using frequency domain data of a target signal as an input value.

[0682] The above frequency domain data may refer to data obtained by performing frequency domain conversion on a digital signal converted through ADC sampling of a reflected signal.

[0683] According to various embodiments of the present disclosure, a Short-Time Fourier Transform (STFT) transform, a Fast Fourier Transform (FFT) transform, a cepstrum transform, a wavelet transform, a cross-correlation method, a convolution transform, and the like may be used as frequency domain transformation methods. The above-described frequency domain transformation methods are exemplary and are not limited to the listed transformation methods, and various transformation or analysis methods may be used to analyze sound signals in the time domain in the frequency domain.

[0684] As an example of frequency domain data according to various embodiments of the present disclosure, spectrogram data obtained through STFT transformation may be included.

[0685] Another example of frequency domain data according to various embodiments of the present disclosure may include data acquired using a cross-correlation method. In this case, the cross-correlation synthesis of input data can correspond to the step of inputting data to a convolution layer, enabling CNN-based learning and classification.

[0686] Meanwhile, frequency domain data used for learning can be labeled with information necessary for road surface classification. The labeled information may include road surface type and / or atmospheric information.

[0687] According to one embodiment of the present disclosure, in order to train a road surface classification artificial neural network, a training data set may include a data set in which the type of road surface from which each data is acquired is labeled in frequency domain data.

[0688] The types (classes) of road surfaces classified by the road surface classification device according to one embodiment of the present disclosure may include classes such as asphalt, cement, dirt, ice, marble, paint, slush (a mixture of water and ice), snow, and water. The types of classes listed are exemplary, and the number of classes or groups to be classified may vary depending on the situation in various embodiments of the present disclosure. Meanwhile, instead of using such a direct labeling method or group names, each input data may be grouped in an arbitrary manner, such as a first class and a second class. Such an arbitrary grouping may be a classification result when using an unsupervised artificial neural network that does not include labels in the learning data, but is not limited thereto.

[0689] The first data determined by the road surface classification device according to one embodiment of the present disclosure may be expressed as a type of road surface or may correspond to numerical data of the slipperiness of the road surface.

[0690] According to one embodiment of the present disclosure, the server (2100) may generate second data. The second data may be generated based on the first data. In this case, the process of generating the second data from the first data may include generating data that visualizes road surface data, a dataset that can be used for UI / UX utilizing maps, a dataset that matches additional data to the location information of the first data, etc. The second data generated thereby may include map data that visually represents the road surface condition of each section of the road, map data that indicates dangerous sections, data obtained from traffic volume, speed, and acceleration sensors corresponding to each road surface condition, etc. In addition, the additional data may include data on traffic volume, average traffic speed, and data from other regions that have the same road surface data.

[0691] FIG. 47 is a drawing showing an example of a screen displayed on UI / UX when providing information according to one embodiment of the present disclosure.

[0692] According to one embodiment of the present disclosure, the first data and the second data acquired from the server may be provided to an application operated by the server or an application operated by an external server (not shown). Here, the application may refer to a mapping application, and the mapping application may refer to a navigation program, a navigation application, etc. that provides a driving service to a driver based on a map image. The external server (not shown) may be connected to the server (2100) via a network, etc. That is, the first data and the second data acquired from the server (2100) may be directly displayed in a mapping application that displays a map, etc., and the external server (not shown) may provide the data in the form of data to be displayed in the application.

[0693] According to one embodiment of the present disclosure, the server (2100) may provide the user terminal (2300) with information such as the location of a dangerous section ahead of the driver, the length of the dangerous section, and image data indicating the dangerous section. The dangerous section refers to a section with a high risk for driving or walking, such as a road covered in black ice.

[0694] For example, as illustrated, a warning message such as "10m black ice zone 100m ahead" can be provided, or black ice zones can be marked with different markings than other road sections. In this case, the criteria for displaying warning messages can be determined based on the risk zone. That is, the warning criteria for risk zones can also be provided as data from the server. The warning criteria for risk zones can be determined based on external conditions such as the length of the risk zone, the period during which the risk zone was sensed as a risk zone, the terrain on which the risk zone is located, and the length of the curved / straight section to the risk zone, as well as the conditions for the risk zone.

[0695] FIG. 48 is a flowchart illustrating a method by which a server according to one embodiment of the present disclosure provides a notification according to a road condition to a user terminal.

[0696] According to one embodiment of the present disclosure, even if data on dangerous sections is not continuously reflected and displayed on the mapping application, as illustrated in FIG. 47, a road hazard notification function can be provided through a user application, etc. The server can reduce the user's vehicle accident risk by consistently providing data even without a separate request signal from the user terminal. This method may not necessarily be provided for all sections, and may only be provided in certain areas, such as areas with a high accident probability, areas assessed as dangerous sections, or areas where road surface information is confirmed by a first road surface classification device, etc.

[0697] In S1701, the server can obtain location information of the user terminal and obtain road condition information based on the location of the user terminal.

[0698] After acquiring the user terminal's location, the server can check surrounding road condition information centered on the user terminal's location. Alternatively, if the user terminal displays a map application, the server can check the user terminal's driving path and identify dangerous sections along the path.

[0699] In S1702, if the road surface condition or slipperiness meets the established risk criteria, the server can check information other than road surface information, such as terrain conditions and road pavement conditions, corresponding to the user terminal's location information. This information, in addition to road surface information, can be used as a judgment criterion for sending notifications to the user terminal. In this case, geographical conditions such as terrain conditions and road pavement conditions can be obtained from an external server. This external server may correspond to a server used by a mapping application, such as a navigation company, or a data management server for a country or local government.

[0700] For example, the conditions the server acquires may correspond to weather conditions. For example, if the server determines road surface information indicates rain, it may acquire current precipitation information from an external server. In this case, the external server may refer to a server used to acquire terrain conditions and another server.

[0701] In S1703, the server can obtain the notification transmission criteria set according to the obtained information.

[0702] For example, the notification criteria may differ for mountainous areas where the road surface is determined to be black ice, and for national highways where the road surface is determined to be black ice. For tunnels where the road surface is determined to be black ice, the server may set the tunnel section entry as the notification criterion. If the gradient of a national highway where the road surface is determined to be black ice exceeds the set value, the server may further adjust the notification criteria upward.

[0703] In S1704, the server can send a notification to a user terminal that meets the notification criteria. By providing a warning to the user terminal, the server can reduce the probability of a vehicle accident.

[0704] Figure 49 is a flowchart illustrating a method for providing information in response to a user terminal request according to one embodiment of the present disclosure.

[0705] In S1801, the server (2100) can obtain a data request signal from a user terminal. In addition to navigation companies, user terminals requesting data may include insurance companies for post-traffic accident processing, and local governments for traffic policy formulation.

[0706] In S1802, the server (2100) may perform a user authentication procedure. At this time, the user authentication procedure performed by the server may be a token-based user authentication procedure.

[0707] According to one embodiment of the present disclosure, the server (2100) can obtain a user authentication token from a user terminal and verify the validity and expiration time of the user token. The user token can remain valid for a certain period of time. If the validity period of the user token has expired, the server can additionally transmit a token request signal.

[0708] In S1803, the server can determine which data to transmit based on user information and determine which data to additionally acquire based on this.

[0709] According to one embodiment of the present disclosure, the server can obtain necessary location information and necessary time information along with user information. The server can obtain road surface information at a location near the location information corresponding to the time information.

[0710] According to one embodiment of the present disclosure, the server may set data to be transmitted based on user terminal information. The server may obtain the necessary information along with the user's request signal, or obtain data to be transmitted in addition to the set information. For example, if the user terminal is an insurance company electronic device, the server may determine that data to be transmitted includes not only road surface information at the location requested by the user at the time, but also road surface information for other dates within the same time zone at the location, along with information such as respective traffic volumes and average vehicle speeds. Furthermore, the server may also provide average vehicle speeds in other areas with similar road surface information.

[0711] For example, if the user terminal is an electronic device belonging to a policy-making agency, such as a local government, the server may provide road surface information statistical data for a certain period of time in that area. Alternatively, the server may extract road surface information within a certain area and provide road surface information statistical data for that area.

[0712] In S1804, the server may obtain the second data determined in S1803, and transmit the first data and the additionally generated second data regarding road surface information to the user terminal in S1805.

[0713] Above, a method for utilizing the acquired road surface information as big data about the road surface was presented even if the server does not provide the road surface information in real time. However, in a business model using the road surface data stored by the server, the above order is not necessarily followed. For example, if the second data is not additionally acquired and is transmitted to the user only within the acquired data, steps S1803 and S1804 may be omitted, and various combinations, omissions, and additions of other steps may be implemented within the scope of the purpose of the present disclosure.

[0714] FIG. 50 is a flowchart of a method for improving data accuracy when providing information at the request of a user terminal according to one embodiment of the present disclosure.

[0715] According to one embodiment of the present disclosure, a road surface classification device is used in combination with a fixed first road surface classification device and a mobile second road surface classification device, so that road surface information can be acquired relatively consistently at a location sensed by the first road surface classification device. However, when road surface information is requested in an area where the first road surface classification device is not located, data accuracy can be improved through communication between users possessing the second road surface classification device.

[0716] In S1901, the server (2100) can obtain an information request signal from the user terminal (2300).

[0717] In S1902, when the server receives an information request signal from the user terminal, it can obtain location information of the area where the information was requested along with the user information. In this case, although not described in detail, steps such as determining the user's authentication method or data type, as in the previous Figure 49, may be added.

[0718] The server (2100) can obtain road surface information from the location information of the requested area. At this time, the server can obtain time information at which the road surface information was obtained. In step S1903, the server (2100) can identify, based on the time information, whether the time information at which the road surface information was obtained was obtained earlier than the reference time.

[0719] According to one embodiment of the present disclosure, the reference time may be a value stored on a server. This stored reference time may be set based on a period of time during which data accuracy is maintained to a certain degree, or may be set based on conditions such as the temperature not changing by a certain amount. The reference time may change based on external conditions such as season, time zone, and temperature.

[0720] In S1904, the server (2100) may transmit an information request message to the second road surface classification device if the time at which the first data was acquired was acquired before the reference time. The server transmitting the message to the second road surface classification device may mean that a signal is transmitted to the UI / UX of a vehicle equipped with the second road surface classification device, or to a terminal of an information provider to which the second road surface classification device is linked.

[0721] In S1905, the server (2100) may assign additional weight to the second road surface classification device when it obtains road surface information for the area from the second road surface classification device that transmitted the signal.

[0722] That is, if the second road surface classification device actively provides data requested by the server, it can be compensated with more weight compared to general data provision.

[0723] As described above, a method for improving data accuracy according to data signals has been described. However, the following steps may be added or omitted depending on the method of setting the reference time, etc.

[0724] Additionally, even if there is no information request signal from the user terminal, the server can determine whether data is updated and perform an information request to the second road surface classification device.

[0725] In S1905, the server may receive information in response to the request and provide the information to the user terminal, or, if data does not need to be updated in S1903, may provide stored data to the user terminal.

[0726] FIG. 51 is a flowchart of a method for compensating information providers according to one embodiment of the present disclosure.

[0727] In various embodiments of the present disclosure, the server provides users with data acquired through a road surface classification device. In the case of the first road surface classification device, data can be continuously acquired without any specific compensation after attachment. However, in the case of the second road surface classification device, compensation may be required for information providers using the second road surface classification device based on the data provided.

[0728] In S2001, the server performs security authentication of the provider terminal. Similar to the previous user terminal authentication, the provider terminal's security authentication verifies the validity and expiration date of the encrypted, transmitted authentication token. Once the provider terminal authentication process is complete, sensor data or road surface information obtained from the provider terminal is stored on the server.

[0729] In S2002, the server can check the total data provided per provider terminal at regular intervals. Based on this total data provided, various compensation criteria can be set.

[0730] For example, a server could use the amount of authenticated data uploads as a reward criterion.

[0731] For example, the server may use the total road condition detection distance calculated based on the location information of the amount of data uploaded as a compensation criterion.

[0732] For example, the server may determine a compensation standard based on first data obtained, such as whether the road surface's slipperiness is above a standard or whether the road surface is determined to be black ice.

[0733] That is, the server can base its reward criteria on the amount of uploaded authenticated data, but various methods can be devised for more precise reward criteria.

[0734] In S2003, the server can check the data information transmitted to the user terminal and use it as an additional compensation criterion or weight.

[0735] In S2004, for example, the server can assign data weights based on the length of a section detected as a risk section among the data sections acquired from the provider terminal.

[0736] For example, the server can determine the frequency of data obtained from the provider terminal and assign data weights to it.

[0737] For example, the server may assign additional weight to data if the provider terminal obtains road surface conditions and provides data thereon in response to a server request.

[0738] The data weights described above can be calculated differently for each case. For example, if data for a specific region is provided at the server's request, a higher weight may be assigned compared to other cases.

[0739] In S2005, the server calculates the compensation ratio by calculating the history of data and additional weights, and can calculate each compensation value according to the compensation ratio from the usage fee obtained from the user terminal.

[0740] In S2006, the server can provide compensation to the provider terminal. This compensation can be provided periodically and in the form of goods, transportation services, points, tokens, and other forms. Transportation services can refer to services provided by external service applications, such as navigation companies that obtain and provide data to the server, or applications directly operated by the server.

[0741] Although the operation of the road surface type estimation method and the road surface information providing system according to one embodiment of the present invention has been described above, it is also possible to implement a computer-readable recording medium storing a program for implementing the road surface type estimation method and the road surface information providing system, and a program stored in the computer-readable recording medium for implementing the road surface type estimation method.

[0742] That is, it will be readily understood by those skilled in the art that the above-described method for estimating road surface type using a vehicle-mounted sensor and the method for operating a road surface information providing system may be provided by including the program of commands for implementing the same in a computer-readable recording medium by being tangibly implemented. In other words, the program may be implemented in the form of program commands that can be executed through various computer means and may be recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., singly or in combination. The program commands recorded on the computer-readable recording medium may be those specially designed and configured for the present invention or may be those known to and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions such as ROMs, RAMs, flash memories, and USB memories. Examples of the program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0743] The present invention is not limited to the above-described embodiments, and the scope of application is diverse, and various modifications can be made without departing from the gist of the present invention as claimed in the claims.

Claims

1. In a road surface type estimation device using a vehicle-mounted sensor, A vehicle-mounted sensor (100) that transmits a sound wave signal and then receives the reflected signal; An analog-to-digital converter (200) for converting an analog signal into a digital signal for a preset range of signals received from a sensor installed in the vehicle; A signal filter unit (300) for filtering and separating only the signal reflected from the road surface from the above digital signal; An artificial neural network (400) that uses the filtered digital signal as an input signal and classifies the input signal based on a learned road surface classification model to estimate the type of the road surface; and A control unit (MCU) (500) that controls the operation of the sensor mounted on the vehicle, the analog-to-digital converter, the signal filter unit, and the artificial neural network A road surface type estimation device using a vehicle-mounted sensor including a 2. In paragraph 1, The above control unit (500). It is characterized in that the time of flight (ToF) is determined through height information (h) from the ground for the above vehicle's mounted sensor, The above signal filter unit (300) is A road surface type estimation device using a vehicle-mounted sensor, characterized in that it filters a signal received for a preset time after the time of flight (ToF) under the control of the above control unit (500).

3. In paragraph 1, The above control unit (500) A road surface type estimation device using a vehicle-mounted sensor, characterized in that the operation of the analog-to-digital converter, the signal filter unit, and the artificial neural network is controlled to estimate the road surface type only when the driving speed is a km / h or more or the acceleration is b ㎨ or more (a and b are preset threshold values).

4. In paragraph 1, The above artificial neural network (400) is A frequency converter (410) for performing frequency conversion on a preset region in the time domain of the filtered digital signal received from the signal filter unit to obtain a frequency domain signal; and A neural network (420) that uses the frequency domain signal as an input signal and extracts and classifies the characteristics of the input signal based on the learned road surface classification model to estimate the type of the road surface, The above neural network (420) is A road surface type estimation device using a vehicle-mounted sensor, characterized by being a deep neural network of a multi-layer perceptron algorithm including an input layer, multiple hidden layers, and an output layer.

5. In paragraph 1, The above artificial neural network (400) is A road surface type estimation device using a vehicle-mounted sensor, characterized in that the device is a deep convolution neural network (DCNN) of a multi-layer perceptron algorithm including a convolution performing unit (501) that receives the filtered digital signal transmitted from the signal filter unit and performs multiple convolution operations, a transmission layer (502), a plurality of hidden layers (503), and an output layer (504).

6. In paragraph 5, The above convolution performing unit (501) is It is characterized in that it performs multiple 1D convolution operations on the input signal, performs batch normalization, Relu function, and MaxPooling function for each convolution operation, and the output of the last convolution operation is flattened data. The above output layer is, It is characterized by outputting a probability value for each type of learned road surface, The above artificial neural network, A road surface type estimation device using a vehicle-mounted sensor, characterized in that it determines and outputs the type of road surface with the highest probability using softmax.

7. In a method for estimating road surface type using a vehicle-mounted sensor, A reflection signal receiving step (S120) in which a sound wave signal is transmitted from a vehicle's onboard sensor and then the reflected signal is received; A signal conversion step (S130) for converting an analog signal into a digital signal for a preset area of ​​the received signal according to the control of the control unit; A signal filtering step (S140) for filtering and separating only the signal reflected from the road surface from the digital signal according to the control of the above control unit; A frequency domain signal acquisition step (S150) for performing frequency conversion on the filtered digital signal under the control of the above control unit to obtain a frequency domain signal; and A method for estimating a road surface type using a vehicle-mounted sensor, including a first road surface type determination step (S160) of determining the type of the road surface by extracting and classifying the characteristics of the input signal based on a learned road surface classification model, according to the control of the control unit, using the frequency domain signal as an input signal of a neural network.

8. In a method for estimating road surface type using a vehicle-mounted sensor, A reflection signal receiving step (S120) in which a sound wave signal is transmitted from a vehicle's onboard sensor and then the reflected signal is received; A signal conversion step (S130) for converting an analog signal into a digital signal for a preset area of ​​the received signal according to the control of the control unit; A signal filtering step (S140) for filtering and separating only the signal reflected from the road surface from the digital signal according to the control of the above control unit; A convolution operation step (S170) that receives the filtered digital signal from the artificial neural network and performs multiple convolution operations according to the control of the above control unit; and A method for estimating a road surface type using a vehicle-mounted sensor, including a second road surface type determination step (S180) of extracting and classifying the characteristics of the convolution-operated signal based on a road surface classification model learned in the artificial neural network according to the control of the control unit, and determining the type of the road surface.

9. In paragraph 7 or 8, The control unit further includes a setup step (S115) for determining the time of flight (ToF) and a preset time (area of ​​interest) to be filtered through height information (h) from the ground for the vehicle's on-board sensor. The above signal filtering step (S140) is A method for estimating a road surface type using a vehicle-mounted sensor, characterized in that the signal received for the preset time after the above-mentioned time of flight (ToF) is filtered.

10. In paragraph 7 or 8, A method for estimating a road surface type using a vehicle-mounted sensor, characterized in that the road surface type is estimated only when the driving speed of the vehicle is a km / h or more or the acceleration is b ㎨ or more (a and b are preset threshold values) under the control of the control unit.

Images