Method And Apparatus for Controlling Terrain Mode Using Road Condition Judgment Model Based on Deep Learning
A deep learning-based system estimates road surface friction coefficients to autonomously control terrain modes, enhancing driver convenience and reducing manufacturing costs by eliminating the need for physical buttons, thereby facilitating advanced autonomous driving.
Patent Information
- Application Number
- KR1020200107699
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-08-26
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2040-08-26
AI Technical Summary
Conventional terrain mode control technology requires driver intervention for road surface assessment and manual manipulation, limiting the implementation of advanced autonomous driving technology and increasing manufacturing costs due to the need for physical buttons.
A pre-trained deep learning-based model uses a camera to estimate road surface friction coefficients, automatically determining the optimal terrain mode and controlling vehicle modules without driver intervention, eliminating the need for physical buttons and reducing manufacturing costs.
Improves driver convenience by allowing vehicles to determine road surface conditions autonomously, provides terrain mode information, and secures interior space by eliminating the need for physical buttons, thus enabling advanced autonomous driving.
Smart Images

Figure 112020089812734-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The embodiments of the present disclosure relate to a terrain mode control method and apparatus using a deep learning-based road surface judgment model. More specifically, the invention relates to a method for controlling a terrain mode and an apparatus for controlling a terrain mode using a pre-trained deep learning-based road surface judgment model and road surface data obtained from a camera. Background Technology
[0002] The content described in this section merely provides background information regarding the present invention and does not constitute prior art.
[0003] Vehicle driving performance varies in response to various road surface conditions, such as wet roads, off-road roads, roads containing sand and mud, and roads covered in snow. Automobile manufacturers have conducted research and development on electronic control units (ECUs) installed in vehicles to enable the vehicle to actively respond to diverse road conditions and improve driving performance.
[0004] As a result, terrain mode control technology is being commercialized, which allows the driver to control the vehicle's drive system, shift time, and braking system by selecting a specific terrain mode corresponding to the current road surface conditions.
[0005] Terrain mode control technology is executed through a process in which the driver judges road surface conditions using the naked eye, selects a specific terrain mode using a jog-shuttle dial, and changes the operation state of each module within the vehicle according to the selected terrain mode. A jog-shuttle dial refers to one of the various types of physical buttons that can be attached within a vehicle, and it may be changed to a different type of button during vehicle manufacturing.
[0006] However, conventional terrain mode control technology has limitations in that the driver must visually assess the road surface conditions, requires additional manipulation to change the terrain mode, and necessitates the separate provision of devices such as physical or soft buttons within the vehicle.
[0007] Currently, automotive technology is evolving toward the realization of autonomous vehicles that drive themselves; however, there is a problem in that it is difficult to implement advanced autonomous driving technology due to the need for human intervention at multiple stages when using conventional terrain mode control technology.
[0008] Therefore, as a foundational technology for implementing advanced autonomous driving technology at Level 4 or higher, it is necessary to introduce technology that enables vehicles to more actively perform terrain mode control and control autonomous driving assistance functions. The problem to be solved
[0009] One embodiment of the present disclosure has the main purpose of providing a technology that uses a pre-trained deep learning-based learning model to determine the road surface condition on its own without driver intervention and to change the driving state in response to various road surface conditions.
[0010] According to another aspect of the present embodiment, there is another primary purpose of providing information regarding the terrain mode to the driver after determining the road surface condition and the terrain mode optimized for the road surface, even in normal driving situations where the vehicle is not in a limit driving situation where the Electronic Stability Control (ESC) would intervene.
[0011] According to another aspect of the present embodiment, there is another primary objective to lower the manufacturing cost by eliminating the need to install separate physical buttons inside the vehicle during manufacturing, and to improve driver convenience by securing extra space inside the vehicle. means of solving the problem
[0012] According to one embodiment of the present disclosure, a method for automatically controlling the driving state of a vehicle in a terrain mode optimized for a road surface is provided, comprising: a process of obtaining road surface data captured from a camera mounted on the vehicle while the vehicle is driving on the road surface; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of determining a terrain mode optimized for the road surface while the vehicle is driving using the road friction coefficient and generating optimized terrain mode information; and a process of controlling a module within the vehicle based on the optimized terrain mode information.
[0013] According to another embodiment of the present disclosure, a method for assisting terrain mode control of a vehicle is provided, comprising: a process of obtaining road surface data captured from a camera mounted on the vehicle, which is a road surface on which the vehicle is driving; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of determining a terrain mode optimized for the road surface on which the vehicle is driving using the road friction coefficient; and a process of providing the optimized terrain mode information.
[0014] According to another embodiment of the present disclosure, a method for assisting autonomous driving control of a vehicle is provided, comprising: a process of obtaining road surface data captured from a camera mounted on the vehicle, which is a road surface on which the vehicle is driving; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; and a process of determining a driving limit of the vehicle using the road friction coefficient and controlling a module within the vehicle based on the driving limit.
[0015] According to another embodiment of the present disclosure, a terrain mode automatic control device for a vehicle is provided, characterized by comprising: a road surface condition estimation unit that obtains road surface data captured by a camera mounted on the vehicle and a road surface on which the vehicle is driving, and includes a pre-trained deep learning-based learning model, and inputs the road surface data into the deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a terrain mode determining unit that determines a terrain mode optimized for the road surface on which the vehicle is driving using the road surface friction coefficient; and a terrain mode control unit that controls a module within the vehicle using the terrain mode information.
[0016] According to another embodiment of the present disclosure, a computer program stored on a computer-readable recording medium is provided to execute each process included in a method for automatically controlling the driving state of a vehicle in a terrain mode optimized for such a road surface. Effects of the invention
[0017] As described above, according to the present disclosure, a camera mounted in a vehicle acquires road surface data, and an estimation unit for road surface condition inputs the road surface data into a pre-trained deep learning-based learning model to estimate road surface friction coefficient information corresponding to the road surface. This has the advantage of improving driver convenience by allowing the vehicle to determine the road surface condition itself and provide information regarding a terrain mode corresponding to the road surface condition to the driver, without the need for the driver to visually check the road surface condition directly.
[0018] In addition, the terrain mode determination unit receives road surface friction coefficient information from the road surface condition estimation unit, independently determines the terrain mode optimized for the road surface on which the vehicle is driving, and controls the driving state using terrain mode control units such as ESC, ECU (Engine Control Unit), and TCU (Transmission Control Unit) based on information regarding the optimized terrain mode, thereby having the advantage of enabling a change in terrain mode without additional operation by the driver.
[0019] Furthermore, since there is no need to install physical buttons to assist the driver, vehicle production costs can be lowered, and extra space can be secured inside the vehicle, thereby improving the freedom of interior design. Brief explanation of the drawing
[0020] FIG. 1(a) is a block diagram of a terrain mode automatic control device according to one embodiment of the present disclosure, and FIG. 1(b) is an illustrative diagram for explaining a related configuration for a method to assist in autonomous driving control of a vehicle according to one embodiment of the present disclosure. FIG. 2 is a block diagram of a deep learning-based learning unit according to one embodiment of the present disclosure. FIG. 3 is a flowchart for explaining a training method of a deep learning-based learning model according to one embodiment of the present disclosure. FIG. 4a is a flowchart illustrating a terrain mode automatic control method according to one embodiment of the present disclosure. FIG. 4b is a flowchart illustrating an autonomous driving control assistance method according to one embodiment of the present disclosure. Specific details for implementing the invention
[0021] Hereinafter, some embodiments of the present invention will be described in detail with reference to the exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the embodiments of the present invention, if it is determined that a detailed description of related known components or functions could obscure the essence of the present invention, such detailed description is omitted.
[0022] In addition, terms such as first, second, A, B, (a), (b), etc. may be used when describing the components of the present invention. These terms are intended merely to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by these terms. Throughout the specification, when a part is described as 'comprising' or 'equipped' with a certain component, unless specifically stated otherwise, this means that it does not exclude other components but may include additional components. Furthermore, terms such as '…part' or 'module' described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.
[0023] Recently, the ability to estimate physical information from image data based on deep learning techniques has been improved. In the case of the embodiment of the present disclosure, a new method is proposed in which a vehicle can determine the road surface condition and control the driving state based on the ability to estimate such information.
[0024] More specifically, we propose a device, a method, and a method of utilizing the same, wherein an estimation unit for road surface condition calculates road surface friction coefficient information based on a deep learning-based learning model and controls engine output, vehicle driving direction, braking speed, and braking timing without driver intervention.
[0025] Meanwhile, although the embodiments of the present disclosure describe a method for controlling a driving state in an autonomous driving or semi-autonomous driving vehicle, this is for convenience of explanation and can be applied to various means of transportation equipped with a camera and an electronic control unit.
[0026] The description of the invention disclosed below, together with the accompanying drawings, is intended to describe exemplary embodiments of the invention and is not intended to represent the only embodiment in which the invention can be practiced.
[0027] FIG. 1(a) is a block diagram of a terrain mode automatic control device according to one embodiment of the present disclosure.
[0028] The terrain mode automatic control apparatus (100) according to the present embodiment includes, in whole or in part, a data collection unit (DCU: Data Collection Unit, 106), an estimation unit for road friction coefficient (110), a learning unit (112), a terrain mode determining unit (114), and a terrain mode control unit (116).
[0029] The terrain mode automatic control device (100) illustrated in FIG. 1(a) is according to one embodiment of the present disclosure, and not all blocks illustrated in FIG. 1(a) are essential components, and some blocks included in the terrain mode automatic control device (100) in other embodiments may be added, changed, or deleted. For example, if a communication unit (not shown) capable of communicating with the driver is added between the terrain mode determination unit (114) and the terrain mode control unit (116), the terrain mode automatic control device (100) can operate as an apparatus for assisting control of terrain mode (not shown) of a vehicle that provides terrain mode information optimized for the current road surface condition being driven to the driver and controls each module within the vehicle according to the terrain mode responded to by the driver.
[0030] The terrain mode automatic control device (100) obtains road surface data, estimates the road surface friction coefficient, determines a terrain mode suitable for the road surface, and changes the driving state in response to various terrains. A terrain mode refers to a driving state in which an automobile manufacturer calculates an optimized reference value for each road surface and uses this value to pre-set each terrain mode control unit within the vehicle. When a driver or the system selects a specific terrain mode among various terrain modes, each terrain mode control unit within the vehicle can change the driving state to the pre-set reference value. Below, each component included in the terrain mode automatic control device (100) will be described.
[0031] The road surface condition estimation unit (102) transmits road friction coefficient information estimated based on road surface data obtained from the camera (104) and a learning model based on deep learning to the terrain mode determination unit. The road surface condition estimation unit (102) may be an electronic control unit that trains a learning model based on deep learning.
[0032] A camera (104) is mounted on one side of a vehicle and provides road surface data, such as images or videos, that capture the surrounding area of the vehicle. A data collection unit (106) may include a data storage unit (108) that receives road surface data from the camera (104) and stores it. In another embodiment, in addition to collecting road surface data from the camera (104), the data collection unit (106) may receive and store road surface data from a radar (RADAR, not shown) or lidar (LiDAR, not shown) mounted on one side of the vehicle that emits a signal to an object around the vehicle and then interprets the signal reflected back from the object.
[0033] The road surface friction coefficient estimation unit (110) estimates road surface friction coefficient information using road surface data corresponding to road condition features, temporarily stores the estimated road surface friction coefficient information in a memory device such as Random Access Memory (RAM), and then transmits the road surface friction coefficient to the terrain mode determination unit (114) using a CAN (Controller Area Network) bus.
[0034] The learning unit (112) obtains road surface data from a camera, radar, or lidar and generates labeling data by labeling the road surface friction coefficient on the road surface data. Subsequently, a deep learning-based learning model is trained using the labeling data. In one embodiment of the present disclosure, the learning unit (112) may be implemented as a separate, independent device linked with the road surface condition estimation unit (102).
[0035] The terrain mode determination unit (114) determines a terrain mode optimized for the road surface being driven based on the road surface friction coefficient received from the road surface condition estimation unit (102). For example, if the terrain mode determination unit (114) determines that the road surface being driven is a wet road as a result of comparing the road surface friction coefficient provided by the road surface friction coefficient estimation unit with the road surface friction coefficient pre-set for each road surface, it can transmit terrain mode information to the terrain mode control unit (116) to change the driving state to a pre-set low engine power.
[0036] The terrain mode control unit (116) controls each drive system and brake system using the terrain mode. For example, the engine control unit (ECU, not shown) can change the engine output to a preset engine mapping value according to the terrain mode selected by the terrain mode determination unit (114). The electronic stability control (ESC, not shown) operates a preset TCS controller according to the terrain mode selected by the terrain mode determination unit (114). The transmission control unit (TCU, not shown) can control the transmission to a preset shift time according to the terrain mode selected by the terrain mode determination unit (114).
[0037] FIG. 1(b) is an illustrative diagram for explaining a related configuration for a method to assist in autonomous driving control of a vehicle according to one embodiment of the present disclosure.
[0038] A vehicle (118) equipped with an autonomous driving control unit estimates the road surface friction coefficient using road surface data obtained from a camera (104) and controls each module within the vehicle using an autonomous driving control unit (120). The operation of the vehicle (118) equipped with an autonomous driving control unit is described later in FIG. 4b.
[0039] FIG. 2 is a block diagram of a deep learning-based learning unit according to one embodiment of the present disclosure.
[0040] Hereinafter, a learning unit (112) according to one embodiment of the present disclosure will be described with reference to FIG. 2.
[0041] A learning unit (112) according to one embodiment of the present disclosure includes, in whole or in part, a data acquisition unit (202), a labeling data generator (204), and a training unit (206).
[0042] The data acquisition unit (202) collects learning data for training a deep learning-based learning model.
[0043] The data acquisition unit (202) acquires the road surface friction coefficient collected for the road surface being driven from the road surface friction coefficient estimation unit (110). In another embodiment, the data acquisition unit (202) may acquire the road surface friction coefficient calculated based on the wheel speed and wheel torque values of the wheel mounted in the vehicle from the ESC.
[0044] The data acquisition unit (202) collects road surface data of the camera (104) corresponding to the road surface friction coefficient obtained from the road surface friction coefficient estimation unit (110) as part of the process of collecting training data for training a deep learning-based learning model.
[0045] Meanwhile, the road surface data from the camera (104) is an image of a road surface region located in front of the vehicle, and there is actually a certain distance difference between the road surface region in the image and the vehicle. Based on this, the data acquisition unit (202) according to one embodiment of the present disclosure collects accurate road surface data corresponding to the road surface friction coefficient by considering the distance difference between the road surface region in the image and the vehicle, such as the vehicle's tires, when collecting road surface data from the camera (104).
[0046] Meanwhile, in one embodiment of the present disclosure, the data acquisition unit (202) may selectively collect road surface data corresponding to the road surface friction coefficient only when event information is collected from the road surface friction coefficient estimation unit (110) along with the road surface friction coefficient. That is, when event information is detected from the data transmitted by the road surface friction coefficient estimation unit (110), the data acquisition unit (202) may determine that the reliability of the road surface friction coefficient is high, extract the road surface friction coefficient, and collect road surface data corresponding to the extracted road surface friction coefficient. At this time, the data acquisition unit (202) may specifically acquire a value between 20 ms and 1 s within the data as the road surface friction coefficient in order to accurately extract the road surface friction coefficient. In addition, the initial 1 s and the final 1 s data, which may contain event information, may be omitted.
[0047] Meanwhile, in one embodiment of the present disclosure, event information is generated by the road surface friction coefficient estimation unit (110) only when a preset event situation related to road surface friction coefficient learning occurs. At this time, the preset event situation may be determined according to the prediction accuracy of the road surface friction coefficient calculated by the road surface friction coefficient estimation unit (110) when the event situation occurs. For example, the road surface friction coefficient estimation unit (110) may set a situation as an event situation when the prediction accuracy of the calculated road surface friction coefficient is greater than or equal to a preset threshold.
[0048] The data acquisition unit (202) can perform data pre-processing on the collected road surface data. For example, the data acquisition unit (202) can detect a road surface region within the collected road surface data and extract and provide only the image data corresponding to the detected road surface region. This has the effect of simplifying the computation during the subsequent training process by reducing the pixel region that is the subject of the computation.
[0049] The labeling data generation unit (204) obtains road surface data and road surface friction coefficients from the data acquisition unit (202) and performs the function of generating training data for training a learning model based thereon.
[0050] In one embodiment of the present disclosure, the labeling data generation unit (204) generates labeling data based on road surface data and road surface friction coefficients obtained from the data acquisition unit (202), and provides the generated labeling data as training data.
[0051] For example, the labeling data generation unit (204) can generate labeling data by labeling the road surface friction coefficient on the collected road surface data.
[0052] The training unit (206) trains a deep learning-based learning model by utilizing the labeled data obtained from the labeling data generation unit (204) as training data. Meanwhile, the specific method by which the training unit (206) trains the learning model based on the training data is common in the relevant technical field, so a detailed explanation will be omitted.
[0053] According to the operation of the training unit (206) in accordance with the present embodiment, the learning model is trained to estimate road surface friction coefficient information using only the image of the camera (104). That is, the learning model learns and stores the road surface friction coefficient measured by the road surface friction coefficient estimation unit (110) for each road surface data having different feature information.
[0054] The road surface friction coefficient estimation unit (110) includes a pre-trained learning model according to the operation of the learning unit (112), and estimates the road surface friction coefficient corresponding to the road surface on which the vehicle is currently driving through a learning process using the same.
[0055] The road surface friction coefficient estimation unit (110) applies road surface data received from the data collection unit (106) to a learning model to estimate the road surface friction coefficient corresponding to the road surface.
[0056] The road surface friction coefficient estimation unit (110) applies road surface data to a learning model to extract matching learning data for road surface data that has feature information identical or similar to the road surface data, and estimates the road surface friction coefficient based on the extracted learning data. That is, the road surface friction coefficient estimation unit (110) can estimate the previous road surface friction coefficient information labeled for previous road surface data that has feature information identical or similar to the road surface data collected at the current time as the current road surface friction coefficient information.
[0057] FIG. 3 is a flowchart illustrating a learning method for estimating a road surface friction coefficient according to one embodiment of the present disclosure.
[0058] The data acquisition unit (202) obtains the road surface friction coefficient collected for the road surface on which the vehicle is driving from the road surface friction coefficient estimation unit (110) (S300).
[0059] The data acquisition unit (202) checks whether event information has been collected along with the road surface friction coefficient (S302). In step S302, the data acquisition unit (202) may receive event information calculated by the road surface friction coefficient estimation unit (110) only when a pre-set event situation related to road surface friction coefficient learning occurs. This event information indicates that the collected road surface friction coefficient is meaningful data for learning.
[0060] When the collection of event information is confirmed in step S302, the data acquisition unit (202) collects road surface data of the camera (104) corresponding to the road surface friction coefficient acquired in step S300 (S304). In step S304, the data acquisition unit (202) ensures that accurate road surface data corresponding to the actual road surface friction coefficient is collected by considering the actual distance difference between the road surface area in the image and the vehicle.
[0061] The data acquisition unit (202) can detect a road surface area from the collected road surface data and extract only the road surface data corresponding to the detected road surface area.
[0062] The labeling data generation unit (204) generates labeling data by labeling the road surface friction coefficient of step S300 onto the road surface data collected in step S304 (S306).
[0063] The training unit (206) trains a deep learning-based learning model using the labeled data generated in step S306 as training data (S308).
[0064] FIG. 4a is a flowchart for explaining a terrain mode automatic control method according to one embodiment of the present disclosure.
[0065] The data collection unit (106) collects road surface data captured by a camera mounted in the vehicle while the vehicle is driving on the road surface (S410).
[0066] The road surface friction coefficient estimation unit (110) estimates the road surface friction coefficient by inputting the road surface data collected by the data collection unit (106) into a pre-trained learning model (S412).
[0067] The road surface friction coefficient estimation unit (110) transmits the road surface friction coefficient estimated in step S412 to the terrain mode determination unit (114) using a CAN (controller area network) bus.
[0068] The terrain mode determination unit (114) compares the road surface friction coefficient with the road surface friction coefficient set for each road surface to determine the road surface condition and the terrain mode optimized for the road surface while the vehicle is driving (S414).
[0069] The terrain mode determination unit (114) transmits terrain mode information optimized for the road surface on which the vehicle is driving to the terrain mode control unit (116).
[0070] The terrain mode control unit (116) changes the driving state based on terrain mode information and a preset reference value for each terrain mode. (S416)
[0071] Here, the process of changing the driving state may be a process in which electronic control units such as ESC, ECU, and TCU control each module within the vehicle, such as the TCS controller, engine, and transmission.
[0072] FIG. 4b is a flowchart for explaining an autonomous driving control assistance method according to one embodiment of the present disclosure.
[0073] The data collection unit (106) obtains road surface data captured by a camera mounted in the vehicle while the vehicle is driving (S420).
[0074] The road surface friction coefficient estimation unit (110) estimates the road surface friction coefficient by inputting the road surface data obtained by the data collection unit (106) into a pre-trained learning model (S422).
[0075] The road surface friction coefficient estimation unit (110) transmits the road surface friction coefficient estimated in step S422 to the autonomous driving control unit using a CAN (controller area network) bus.
[0076] The autonomous driving control unit (120) compares road surface friction coefficient information with road surface friction coefficient information set for each road surface to determine the road surface condition in which the vehicle is driving and determines the driving threshold value of the vehicle (S424). Here, the driving threshold value refers to a threshold value, such as engine power, shift time, and braking time, required to increase fuel efficiency or ensure driving safety based on the road surface condition in which the vehicle is driving. For example, on a road surface where snow has accumulated at least 20 mm, the driving threshold value of the engine power may be 50 / 100 of the maximum speed that the vehicle can achieve.
[0077] The autonomous driving control unit (120) controls each module within the vehicle based on the driving limit value for the road surface on which the vehicle is driving (S426).
[0078] Meanwhile, the autonomous driving control unit (120) can improve autonomous driving performance by changing the driving state of the vehicle itself without driver intervention using the road surface friction coefficient. For example, the autonomous driving control unit (120) may be a shift pattern control apparatus including a Smart Cruise Control System or a brake control apparatus including a Forward Collision-Avoidance System.
[0079] A smart cruise control system refers to a control system that maintains a constant driving speed based on the speed set by the driver, and has the function of increasing or decreasing the driving speed in accordance with the traffic conditions of vehicles traveling on the road.
[0080] The shift pattern control unit changes the vehicle's driving speed in response to traffic volume and road surface conditions by changing the engine output and shift pattern using the engine control unit and the transmission control unit.
[0081] A forward collision avoidance system refers to a system that prevents a collision by notifying the driver or by automatically controlling the braking system when there is a risk of the vehicle colliding with the vehicle ahead—that is, when the distance between the vehicle and the vehicle ahead is measured to be below a preset threshold.
[0082] The brake control system reduces the vehicle's driving speed by controlling the braking initial speed and braking time to prevent a collision between the moving vehicle and the vehicle ahead.
[0083] Although FIGS. 3, 4a, and 4b describe the respective processes as being executed sequentially, this is merely an illustrative explanation of the technical concept of some embodiments of the present invention. In other words, a person skilled in the art to which some embodiments of the present invention belong may modify and adapt the processes described in FIGS. 3, 4a, and 4b in various ways, such as by changing and executing them or by executing one or more of the processes in parallel, without departing from the essential characteristics of some embodiments of the present invention; therefore, FIGS. 3, 4a, and 4b are not limited to a chronological order.
[0084] Meanwhile, the operation of the terrain mode automatic control device (100) according to the embodiment of the present invention described in FIG. 1 may be implemented as a program and recorded on a computer-readable recording medium. A computer-readable recording medium on which a program for implementing the operation of the terrain mode automatic control device (100) according to the embodiment of the present invention is recorded includes all types of recording devices in which data that can be read by a computer system is stored. Such a computer-readable recording medium may be a non-transitory medium such as ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device, and may also include a transitory medium such as a carrier wave (e.g., transmission via the Internet) and a data transmission medium. Additionally, the computer-readable recording medium may be distributed across networked computer systems, and computer-readable code may be stored and executed in a distributed manner.
[0085] Various embodiments of the systems and techniques described herein may be realized as digital electronic circuits, integrated circuits, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being implemented as one or more computer programs executable on a programmable system. A programmable system comprises a storage system, at least one input device, and at least one programmable processor (which may be a special-purpose processor or a general-purpose processor) coupled to receive data and instructions from at least one output device and to transmit data and instructions to them. Computer programs (which are also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable recording medium."
[0086] Computer-readable recording media include all types of recording devices in which data that can be read by a computer system is stored. Such computer-readable recording media may be non-volatile or non-transitory media such as ROM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, magneto-optical disk, and storage device, and may also include transitory media such as carrier waves (e.g., transmission over the Internet) and data transmission media. Additionally, computer-readable recording media may be distributed across networked computer systems, and computer-readable code may be stored and executed in a distributed manner.
[0087] Various embodiments of the systems and techniques described herein may be implemented by a programmable computer. Here, the computer includes a programmable processor, a data storage system (including volatile memory, non-volatile memory, or other types of storage systems, or a combination thereof), and at least one communication interface. For example, the programmable computer may be one of a server, a network device, a set-top box, an embedded device, a computer expansion module, a personal computer, a laptop, a PDA (Personal Data Assistant), a cloud computing system, or a mobile device.
[0088] The foregoing description is merely an illustrative explanation of the technical concept of the present invention. The embodiments of the present invention are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment. Explanation of the symbols
[0089] 100: Terrain Mode Automatic Control Device 102: Road Surface Condition Estimation Unit 104: Camera 106: Data acquisition unit 108: Data storage unit 110: Road surface friction coefficient estimation unit 112: Learning Unit 114: Terrain Mode Determination Unit 116: Terrain mode control unit 118: Vehicle equipped with autonomous driving control unit 120: Autonomous driving control unit 202: Data acquisition unit 204: Labeling Data Generation Unit 206: Training Unit
Claims
Claim 1 A method for automatically controlling the driving state of a vehicle in a terrain mode optimized for a road surface, comprising: a process of obtaining road surface data captured from a camera mounted on the vehicle while the vehicle is driving on the road surface; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of generating labeling data based on the road surface data obtained from the camera and the estimated road friction coefficient information when event information is collected along with the friction coefficient, and training the deep learning-based learning model by utilizing the labeling data as training data; and a process of determining a terrain mode optimized for the road surface while the vehicle is driving using the road friction coefficient and generating optimal terrain mode information. A method for automatically controlling the driving state of a vehicle, characterized by including a process of controlling a module within the vehicle based on the optimal terrain mode information, wherein the event information is generated when a pre-set event situation occurs, and the learning model is trained to estimate the road surface friction coefficient from the image by utilizing the road surface friction coefficient calculated based on the wheel speed and wheel torque values of the wheel mounted in the vehicle and the image of the road surface region located in front of the vehicle as learning data. Claim 2 A method for automatically controlling the driving state of a vehicle according to claim 1, characterized in that, in the process of obtaining the road surface data, additional road surface data measured by a radar and a lidar mounted on the vehicle is obtained. Claim 3 delete Claim 4 delete Claim 5 A method for automatically controlling the driving state of a vehicle according to claim 1, wherein, in the process of controlling the module, the module within the vehicle is an ESC, and the ESC operates a pre-set TCS (Traction Control System) controller for each terrain mode. Claim 6 A method for automatically controlling the driving state of a vehicle according to claim 1, wherein, in the process of controlling the module, the module within the vehicle is an ECU (Engine Control Unit), and the ECU changes a preset engine mapping value for each terrain mode. Claim 7 A method for automatically controlling the driving state of a vehicle according to claim 1, wherein, in the process of controlling the module, the module within the vehicle is a TCU (Transmission Control Unit), and the TCU changes the pre-set shift point for each terrain mode. Claim 8 A method for assisting in the control of a vehicle's terrain mode comprises: a process of obtaining road surface data captured by a camera mounted on the vehicle, which captures the road surface on which the vehicle is driving; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of generating labeling data based on the road surface data obtained from the camera and the estimated road friction coefficient information when event information is collected along with the friction coefficient, and training the deep learning-based learning model by utilizing the labeling data as training data; and a process of determining a terrain mode optimized for the road surface on which the vehicle is driving using the road friction coefficient. A method for assisting terrain mode control of a vehicle, characterized by including a process for providing optimal terrain mode information, wherein the event information is generated when a pre-set event situation occurs, and the learning model is trained to estimate the road surface friction coefficient from the image by utilizing the road surface friction coefficient calculated based on the wheel speed and wheel torque values of the wheel mounted in the vehicle and the image of the road surface region located in front of the vehicle as learning data. Claim 9 A method for assisting terrain mode control of a vehicle, characterized in that, in the process of obtaining the above-mentioned road surface data, additional road surface data measured by a radar and a lidar mounted on the vehicle is obtained. Claim 10 A method for assisting terrain mode control of a vehicle, characterized in that, in claim 8, the process of providing the optimized terrain mode information is performed by using either a display mounted on the vehicle to provide it visually or a speaker in the vehicle to provide it audibly. Claim 11 A method for assisting terrain mode control of a vehicle according to claim 8, further comprising, after the process of estimating the road surface friction coefficient, generating labeling data based on road surface data obtained from the camera and the estimated road surface friction coefficient information, and training a deep learning-based learning model using the labeling data as training data. Claim 12 delete Claim 13 A method for assisting autonomous driving control of a vehicle, comprising: a process of obtaining road surface data captured from a camera mounted on the vehicle, which is a road surface on which the vehicle is driving; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; and a process of determining a driving limit of the vehicle using the road friction coefficient and controlling a module within the vehicle based on the driving limit, wherein the driving limit is at least one limit value among engine output, gear shift timing, and braking timing determined based on the friction coefficient. Claim 14 A method for assisting autonomous driving control of a vehicle, characterized in that, in the process of controlling a module, an apparatus for controlling a shift pattern including a Smart Cruise Control (SCC) function changes the shift pattern based on the driving limit value determined using the road surface friction coefficient. Claim 15 A method for assisting autonomous driving control of a vehicle, characterized in that, in the process of controlling a module, an apparatus for controlling a brake system including a Forward Collision-Avoidance System changes the initial braking speed and the braking timing based on the driving limit value determined using the road surface friction coefficient. Claim 16 An estimation unit for road surface condition that includes a pre-trained deep learning-based learning model, obtains road surface data captured by a camera mounted on the vehicle on the road surface on which the vehicle is driving, inputs the road surface data into the deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface, and generates event information when a preset event occurs; and a terrain mode determining unit that determines a terrain mode optimized for the road surface on which the vehicle is driving using the road surface friction coefficient. A terrain mode automatic control device for a vehicle, characterized by including a terrain mode control unit that controls a module within the vehicle using the terrain mode information, wherein the learning model is trained by utilizing labeling data generated based on the road surface data and the road surface friction coefficient as training data when the event information is collected together with the friction coefficient, and is trained to estimate the road surface friction coefficient from the image by utilizing the road surface friction coefficient calculated based on the wheel speed and wheel torque values of the wheel mounted in the vehicle and an image of a road surface region located in front of the vehicle as training data. Claim 17 A computer program stored on a computer-readable recording medium to execute each process included in a method for automatically controlling a driving state according to any one of claims 1 to 2 and claims 5 to 7.
Citation Information
Patent Citations
Self-adaptive cruise method based on pavement adhesion coefficient recognition
CN110466520A
Road surface condition detection device of steering support system of motor vehicle
JP1994281539A
Vehicle control systems and control methods
JP2014520712A
Control system and method of controlling a driveline
WO2016079190A1
Systems and methods for surface monitoring
WO2017053415A1
