Terrain mode control method and device using deep learning-based road surface judgment model

Abstract

To provide a technique for allowing a vehicle to autonomously determine a road surface condition without driver intervention by using a deep learning-based learning model trained in advance and to change a driving state of the vehicle in response to various road surface conditions.SOLUTION: A vehicle autonomously estimates a road surface condition by using a deep learning-based learning model, determines a terrain mode optimized for the road surface being traveled by the vehicle, and controls each in-vehicle module thereby automatically controlling the terrain mode.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a method and apparatus for controlling a terrain mode using a deep learning-based road surface judgment model, and more particularly to a method and apparatus for controlling a terrain mode using a pre-trained deep learning-based road surface judgment model and road surface data obtained from a camera. [Background technology]

[0002] The material described in this section merely provides background information for the present invention and may not constitute prior art.

[0003] A vehicle's driving performance changes depending on various road surface conditions, such as wet roads, unpaved roads, roads containing sand and mud, and roads covered with snow. Automobile manufacturers have been researching and developing electronic control units to install in vehicles so that vehicles can actively adapt to various road conditions and improve driving performance.

[0004] As a result, terrain mode control technology has been commercialized, which allows drivers to select a specific terrain mode that corresponds to the road conditions they are currently driving on, thereby controlling the vehicle's drive system, transmission gear shift times, and brake system.

[0005] Terrain mode control technology involves a process in which the driver uses their naked eye to determine road conditions, a process in which a specific terrain mode is selected using a jog-shuttle dial, and a process in which the operation state of each module in the vehicle changes depending on the selected terrain mode. A jog-shuttle dial is one of various types of physical buttons that can be installed in a vehicle, and can be changed to other types of buttons when the vehicle is manufactured.

[0006] However, conventional terrain mode control technology has limitations in that the driver must directly judge road conditions using their naked eyes, additional manipulation by the driver is required to change the terrain mode, and a separate device such as a physical button or soft button must be installed in the vehicle.

[0007] Currently, automotive technology is evolving toward the realization of autonomous vehicles, which are vehicles that drive themselves. However, conventional terrain mode control technology requires human intervention at several steps, making it difficult to realize advanced autonomous driving technology.

[0008] Therefore, as a fundamental technology for realizing advanced autonomous driving technology at level four or above, it is necessary to introduce technology that enables vehicles to more actively control terrain mode and autonomous driving assistance functions. Summary of the Invention [Problem to be solved by the invention]

[0009] The primary purpose of one embodiment of the present disclosure is to provide a technology that uses a pre-trained deep learning-based learning model to enable a vehicle to determine road surface conditions by itself without driver intervention and change its driving state in response to various road surface conditions.

[0010] According to another aspect of this embodiment, another main purpose is to determine the road surface conditions and the terrain mode optimized for the road surface even in normal driving conditions, rather than in limit driving conditions where ESC (Electronic Stability Control) intervenes, and then provide information about the terrain mode to the driver.

[0011] Another main purpose of this embodiment is to reduce the manufacturing cost of the vehicle by eliminating the need to install a separate physical button inside the vehicle during vehicle manufacturing, and to improve driver convenience by creating more space inside the vehicle. [Means for solving the problem]

[0012] According to one embodiment of the present disclosure, there is provided a method for automatically controlling a driving state of a vehicle in a terrain mode optimized for a road surface on which the vehicle is driving, the method comprising: a process of acquiring road surface data photographed from a camera mounted on the vehicle; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of determining a terrain mode optimized for the road surface using the road friction coefficient and generating optimal terrain mode information; and a process of controlling a module in the vehicle based on the optimal terrain mode information.

[0013] According to another embodiment of the present disclosure, there is provided a method for assisting in terrain mode control of a vehicle, the method comprising: a process of acquiring road surface data from a camera mounted on the vehicle and configured to capture a road surface on which the vehicle is traveling; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of determining a terrain mode optimized for the road surface on which the vehicle is traveling using the road friction coefficient; and a process of providing optimal terrain mode information based on the optimized terrain mode.

[0014] According to yet another embodiment of the present disclosure, there is provided a method for assisting autonomous driving control of a vehicle, the method comprising: a process of obtaining road surface data of a road surface on which the vehicle is traveling from a camera mounted on the vehicle; a process of inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; and a process of determining a driving limit value of the vehicle based on the road friction coefficient and controlling a module in the vehicle based on the driving limit value.

[0015] According to yet another embodiment of the present disclosure, there is provided an automatic terrain mode control device for a vehicle, comprising: an estimation unit configured to obtain road surface data photographed of a road surface on which the vehicle is traveling from a camera mounted on the vehicle, input the road surface data into a pre-trained deep learning-based learning model, and estimate a road friction coefficient corresponding to the road surface; a terrain mode determination unit configured to determine a terrain mode optimized for the road surface based on the road surface friction coefficient; and a terrain mode control unit configured to control modules in the vehicle based on the determined terrain mode.

[0016] According to yet another embodiment of the present disclosure, there is provided a computer program stored on a recording medium that can be read by a computer to execute various processes including a method for automatically controlling a vehicle's driving state in a terrain mode optimized for such a road surface. [Effects of the Invention]

[0017] As described above, according to the present disclosure, a camera mounted in a vehicle acquires road surface data, and an estimation unit for road surface condition inputs the road surface data into a pre-trained deep learning-based learning model to estimate road surface friction coefficient information corresponding to the road surface. This eliminates the need for the driver to directly check the road surface condition with the naked eye, and the vehicle itself determines the road surface condition and provides the driver with information on the terrain mode corresponding to the road surface condition, thereby improving driver convenience.

[0018] In addition, the terrain mode determination unit receives the road surface friction coefficient information from the road surface condition estimation unit, determines the terrain mode optimized for the road surface on which the vehicle is traveling, and controls the driving state using information on the optimized terrain mode using terrain mode control units such as ESC, ECU (Engine Control Unit), and TCU (Transmission Control Unit), which has the advantage of making it possible to change the terrain mode without any additional operation by the driver.

[0019] In addition, by eliminating the need to equip the vehicle with physical buttons to assist the driver, the cost of vehicle production can be reduced, more space can be secured inside the vehicle, and the freedom of interior vehicle design can be increased. [Brief explanation of the drawings]

[0020] [Figure 1] 1A is a block diagram of a terrain mode automatic control device according to one embodiment of the present disclosure, and FIG. 1B is an illustrative diagram for explaining the related configuration for a method for assisting autonomous driving control of a vehicle according to one embodiment of the present disclosure. [Figure 2] FIG. 1 is a block diagram illustrating a deep learning-based learning unit according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart illustrating a method for training a deep learning-based learning model according to an embodiment of the present disclosure. [Figure 4a] 1 is a flowchart illustrating an automatic terrain mode control method according to an embodiment of the present disclosure. [Figure 4b] 1 is a flowchart illustrating an autonomous driving control assistance method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0021] Some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. When assigning reference numerals to components in each drawing, it should be noted that identical components are assigned the same numerals whenever possible, even if they are displayed in different drawings. Furthermore, when describing embodiments of the present invention, if it is determined that a detailed description of related well-known structures or functions would obscure the gist of the present invention, such a detailed description will be omitted.

[0022] Furthermore, when describing components of the present invention, terms such as "first," "second," "A," "B," "(a)," and "(b)" may be used. These terms are used to distinguish the component from other components and do not limit the nature, order, or sequence of the components. Throughout the specification, when a part "includes" or "comprises" a component, this does not mean that it excludes other components, but that it may further include other components, unless otherwise specified. Furthermore, terms such as "... unit," "module," and the like used in the specification refer to a unit that processes at least one function or operation, and this may be embodied in hardware, software, or a combination of hardware and software.

[0023] Recently, the ability to estimate physical information from image data based on deep learning techniques has improved. In the embodiment of the present disclosure, a new method is provided that enables a vehicle to determine road conditions and control its driving state based on the ability to estimate such information.

[0024] More specifically, the present invention proposes a device, method, and application thereof in which an estimation unit for road surface condition calculates road surface friction coefficient information based on a deep learning-based learning model, and controls engine output, vehicle driving direction, braking speed, and braking timing without driver intervention.

[0025] Meanwhile, although the embodiments of the present disclosure describe a driving state control method for an autonomous driving or semi-autonomous driving vehicle, this is for convenience of explanation and can be applied to various means of transportation equipped with a camera and an electronic control unit.

[0026] The description of the invention disclosed below in conjunction with the accompanying drawings sets forth exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced.

[0027] FIG. 1(a) is a block diagram of a terrain mode automatic control device according to an embodiment of the present disclosure.

[0028] The automatic terrain mode control apparatus 100 according to this embodiment includes all or part of a data collection unit (DCU) 106, a road friction coefficient estimation unit 110, a learning unit 112, a terrain mode determining unit 114, and a terrain mode control unit 116.

[0029] The automatic terrain mode control device 100 illustrated in Fig. 1(a) is an embodiment of the present disclosure, and not all of the blocks illustrated in Fig. 1(a) are required components. In other embodiments, some blocks included in the automatic terrain mode control device 100 may be added, modified, or deleted. For example, if a communication unit (not shown) capable of communicating with a driver is added between the terrain mode determination unit 114 and the terrain mode control unit 116, the automatic terrain mode control device 100 can provide the driver with information on a terrain mode optimized for the current road surface conditions and operate as an apparatus for assisting control of terrain mode (not shown) of the vehicle that controls each module in the vehicle according to the terrain mode selected by the driver.

[0030] The automatic terrain mode control device 100 acquires road surface data, estimates the road surface friction coefficient, determines the terrain mode appropriate for the road surface, and changes the driving state to accommodate various terrains. A terrain mode refers to a driving state that is preset in each terrain mode control unit in the vehicle using a reference value optimized for each road surface calculated by the automobile manufacturer. When the driver or the system selects a specific terrain mode from various terrain modes, each terrain mode control unit in the vehicle can change the driving state to the preset reference value. Each component included in the automatic terrain mode control device 100 will now be described.

[0031] The road surface condition estimation unit 102 transmits road surface data obtained from the camera 104 and road friction coefficient information estimated based on a deep learning-based learning model to the terrain mode determination unit. The road surface condition estimation unit 102 may be an electronic control unit that trains the deep learning-based learning model.

[0032] The camera 104 is mounted on one side of the vehicle and provides road surface data such as images or videos of the area around the vehicle. The data collection unit 106 includes a data storage unit 108 that acquires and stores road surface data from the camera 104. In another embodiment, the data collection unit 106, in addition to collecting road surface data from the camera 104, may acquire and store road surface data from a radar (RADAR, not shown) or lidar (LiDAR, not shown) that is mounted on one side of the vehicle and emits a signal to objects around the vehicle and analyzes the signal reflected from the object.

[0033] The road surface friction coefficient estimation unit 110 estimates road surface friction coefficient information using road surface data corresponding to road condition-related features, temporarily stores the estimated road surface friction coefficient information in a storage device such as a random access memory (RAM), and then transmits the road surface friction coefficient to the terrain mode determination unit 114 via a controller area network (CAN) bus.

[0034] The learning unit 112 acquires road surface data from a camera, radar, or LIDAR, and generates labeling data by labeling the road surface data with a road surface friction coefficient. The labeling data is then used to train a deep learning-based learning model. In one embodiment of the present disclosure, the learning unit 112 may be implemented as a separate autonomous device that operates in conjunction with the road surface condition estimation unit 102.

[0035] The terrain mode determination unit 114 determines a terrain mode optimized for the road surface on which the vehicle is traveling based on the road surface friction coefficient received from the road surface condition estimation unit 102. For example, if the terrain mode determination unit 114 determines that the road surface on which the vehicle is traveling is a wet road as a result of comparing the road surface friction coefficient provided by the road surface friction coefficient estimation unit with a road surface friction coefficient preset for each road surface, the terrain mode determination unit 114 transmits terrain mode information to the terrain mode control unit 116 to change the driving state to a preset low engine power.

[0036] The terrain mode control unit 116 controls the configuration of a drive system and a brake system, including an engine control unit (ECU, not shown), an electronic stability control (ESC, not shown), and a transmission control unit (TCU, not shown), using the terrain mode. For example, the engine control unit (ECU) changes engine output to a preset engine mapping value according to the terrain mode selected by the terrain mode determination unit 114. The ESC operates a preset TCS controller according to the terrain mode selected by the terrain mode determination unit 114. The transmission control unit (TCU) controls transmission gears at preset shift times according to the terrain mode selected by the terrain mode determination unit 114.

[0037] FIG. 1B is an exemplary diagram illustrating a related configuration for a method for assisting autonomous driving control of a vehicle according to an embodiment of the present disclosure.

[0038] The vehicle 118 equipped with the autonomous driving control unit estimates the road surface friction coefficient using road surface data obtained from the camera 104, and controls each module in the vehicle using an autonomous driving control unit 120. The operation of the vehicle 118 equipped with the autonomous driving control unit will be described later with reference to FIG. 4b.

[0039] FIG. 2 is a block diagram illustrating a deep learning-based learning unit according to an embodiment of the present disclosure.

[0040] Hereinafter, the learning unit 112 according to an embodiment of the present disclosure will be described with reference to FIG.

[0041] The learning unit 112 according to an embodiment of the present disclosure includes all or part of a data acquisition unit 202, a labeling data generator 204, and a training unit 206.

[0042] The data acquisition unit 202 collects learning data for training a deep learning-based learning model.

[0043] The data acquisition unit 202 acquires the road friction coefficient collected for the road surface on which the vehicle is traveling from the road friction coefficient estimation unit 110. In another embodiment, the data acquisition unit 202 acquires the road friction coefficient calculated based on the wheel speed and wheel torque values ​​of the wheels mounted in the vehicle from the ESC.

[0044] The data acquisition unit 202 collects road surface data from the camera 104 corresponding to the road surface friction coefficient acquired from the road surface friction coefficient estimation unit 110 as part of the learning data collection process for training the deep learning-based learning model.

[0045] Meanwhile, the road surface data from the camera 104 is an image capturing a road surface region located in front of the vehicle, and there is actually a certain distance difference between the road surface region in the image and the vehicle. For this reason, the data acquisition unit 202 according to an embodiment of the present disclosure considers the distance difference between the road surface region in the image and the vehicle, for example, the vehicle's tires, when the camera 104 collects road surface data, so that accurate road surface data corresponding to the road surface friction coefficient is collected.

[0046] Meanwhile, in one embodiment of the present disclosure, the data acquisition unit 202 can selectively collect road surface data corresponding to the road surface friction coefficient only when event information is collected from the road surface friction coefficient estimation unit 110 along with the road surface friction coefficient. That is, when event information is detected from the data transmitted by the road surface friction coefficient estimation unit 110, the data acquisition unit 202 determines that the reliability of the road surface friction coefficient is high, extracts the road surface friction coefficient, and collects road surface data corresponding to the extracted road surface friction coefficient. In this case, the data acquisition unit 202 identifies and acquires values ​​between 20 ms and 1 s in the data as the road surface friction coefficient to accurately extract the road surface friction coefficient. In addition, the first 1 s and the last 1 s of data containing event information are omitted.

[0047] Meanwhile, in one embodiment of the present disclosure, event information is generated by the road surface friction coefficient estimating unit 110 only when a predetermined event situation occurs in relation to learning of the road surface friction coefficient. At this time, the predetermined event situation is determined according to the prediction accuracy of the road surface friction coefficient calculated by the road surface friction coefficient estimating unit 110 when the event situation occurs. For example, the road surface friction coefficient estimating unit 110 may set the event situation to a situation in which the prediction accuracy of the calculated road surface friction coefficient is equal to or greater than a predetermined threshold.

[0048] The data acquisition unit 202 performs data pre-processing of the collected road surface data. For example, the data acquisition unit 202 detects road surface areas within the collected road surface data and extracts and provides only image data corresponding to the detected road surface areas. This has the effect of simplifying calculations in the training process that is executed later by reducing the pixel region that is the target of calculations.

[0049] The labeling data generation unit 204 acquires road surface data and road surface friction coefficients from the data acquisition unit 202, and performs the function of generating learning data for training the learning model based on the acquired data.

[0050] In one embodiment of the present disclosure, the labeling data generation unit 204 generates labeling data based on the road surface data and road surface friction coefficient acquired from the data acquisition unit 202, and provides the generated labeling data as learning data.

[0051] For example, the labeling data generating unit 204 generates labeling data by labeling the road surface friction coefficient onto the collected road surface data.

[0052] The training unit 206 trains a deep learning-based learning model by utilizing the labeling data acquired from the labeling data generation unit 204 as learning data. Meanwhile, a specific method by which the training unit 206 trains the learning model based on the learning data is common in the art, and a detailed description thereof will be omitted.

[0053] According to the operation of the training unit 206 according to this embodiment, the learning model is trained to be able to estimate road surface friction coefficient information using only the image of the camera 104. That is, the learning model learns and stores the road surface friction coefficient measured by the road surface friction coefficient estimation unit 110 for each piece of road surface data having different feature information.

[0054] In response to the operation of the learning unit 112, the road surface friction coefficient estimation unit 110 undergoes a learning process using a pre-trained learning model, and estimates the road surface friction coefficient corresponding to the road surface on which the vehicle is currently traveling.

[0055] The road surface friction coefficient estimation unit 110 applies the road surface data received from the data collection unit 106 to a learning model to estimate the road surface friction coefficient corresponding to the road surface.

[0056] The road surface friction coefficient estimation unit 110 applies road surface data to a learning model, extracts learning data matching with road surface data having feature information identical or similar to the road surface data, and estimates a road surface friction coefficient based on the extracted learning data. That is, the road surface friction coefficient estimation unit 110 estimates, as current road surface friction coefficient information, information on previous road surface friction coefficients labeled with previous road surface data having feature information identical or similar to the road surface data collected at the current time point.

[0057] FIG. 3 is a flowchart illustrating a learning method for estimating a road surface friction coefficient according to an embodiment of the present disclosure.

[0058] The data acquisition unit 202 acquires the road surface friction coefficient collected for the road surface on which the vehicle is traveling from the road surface friction coefficient estimation unit 110 (S300).

[0059] The data acquisition unit 202 checks whether event information has been collected along with the road friction coefficient (S302). In step S302, the data acquisition unit 202 receives calculated event information from the road friction coefficient estimation unit 110 only when a preset event situation occurs in relation to learning the road friction coefficient. Such event information indicates that the collected road friction coefficient is meaningful data for learning.

[0060] When the collection of event information is confirmed in step S302, the data acquisition unit 202 collects road surface data from the camera 104 corresponding to the road surface friction coefficient acquired in step S300 (S304). In step S304, the data acquisition unit 202 takes into account the actual distance difference between the road surface area in the image and the vehicle, so that accurate road surface data corresponding to the actual road surface friction coefficient can be collected.

[0061] The data acquisition unit 202 can detect a road surface area from the collected road surface data and extract only the road surface data corresponding to the detected road surface area.

[0062] The labeling data generating unit 204 generates labeling data by labeling the road surface friction coefficient of step S300 onto the road surface data collected in step S304 (S306).

[0063] The training unit 206 uses the labeling data generated in step S306 as learning data to train a deep learning-based learning model (S308).

[0064] FIG. 4a is a flowchart illustrating an automatic terrain mode control method according to an embodiment of the present disclosure.

[0065] The data collection unit 106 collects road surface data obtained by capturing images of the road surface on which the vehicle is traveling using a camera mounted inside the vehicle (S410).

[0066] The road surface friction coefficient estimation unit 110 inputs the road surface data collected by the data collection unit 106 into a pre-trained learning model to estimate the road surface friction coefficient (S412).

[0067] The road surface friction coefficient estimating unit 110 transmits the road surface friction coefficient estimated in step S412 to the terrain mode determining unit 114 using a CAN (controller area network) bus.

[0068] The terrain mode determination unit 114 compares the road surface friction coefficient with a road surface friction coefficient preset for each road surface, and determines the terrain mode optimized for the road surface condition and road surface on which the vehicle is traveling (S414).

[0069] The terrain mode determination unit 114 transmits terrain mode information optimized for the road surface on which the vehicle is traveling to the terrain mode control unit 116 .

[0070] The terrain mode control unit 116 changes the driving state to a reference value preset for each terrain mode based on the terrain mode information (S416).

[0071] Here, the process of changing the driving state may be a process in which an electronic control unit such as an ESC, ECU, or TCU controls each module in the vehicle, such as a TCS controller, engine, or transmission.

[0072] FIG. 4b is a flowchart illustrating an autonomous driving control assistance method according to an embodiment of the present disclosure.

[0073] The data collection unit 106 acquires road surface data obtained by photographing the road surface on which the vehicle is traveling from a camera mounted inside the vehicle (S420).

[0074] The road surface friction coefficient estimation unit 110 inputs the road surface data obtained by the data collection unit 106 into a pre-trained learning model to estimate the road surface friction coefficient (S422).

[0075] The road surface friction coefficient estimation unit 110 transmits the road surface friction coefficient estimated in step S422 to an autonomous driving control unit using a CAN (controller area network) bus.

[0076] The autonomous driving control unit 120 compares the road friction coefficient information with information on road friction coefficients preset for each road surface to determine the condition of the road surface on which the vehicle is traveling and determines the vehicle's driving threshold value (S424). Here, the driving threshold value refers to threshold values ​​for engine power, shift time, braking time, etc. required to improve fuel efficiency or ensure driving safety based on the road surface condition on which the vehicle is traveling. For example, on a road with 20 mm or more of snow accumulation, the driving threshold value for engine power is 50 / 100 of the maximum speed the vehicle can reach.

[0077] The autonomous driving control unit 120 controls each module in the vehicle based on the driving limit value of the road surface on which the vehicle is traveling (S426).

[0078] Meanwhile, the autonomous driving control unit 120 can improve autonomous driving performance by changing the driving state of the vehicle by itself without driver intervention using the road friction coefficient. For example, the autonomous driving control unit 120 is a shift pattern control apparatus including a smart cruise control system or a brake control apparatus including a forward collision-avoidance system.

[0079] A smart cruise control system is a control system that has the function of maintaining a constant driving speed according to the driver's set driving speed, and increasing or decreasing the driving speed according to the traffic conditions of the vehicle traveling on the road.

[0080] The gear shift pattern control device changes the engine output and gear shift pattern using an engine control unit and a transmission control unit, thereby changing the vehicle speed in response to traffic volume and road conditions.

[0081] A forward collision prevention system is a system that prevents a collision by notifying the driver or controlling the vehicle's own brake system when there is a risk of the vehicle colliding with a vehicle ahead, i.e., when the distance between the vehicle and the vehicle ahead is measured to be below a preset threshold.

[0082] A brake control device reduces the driving speed of a vehicle by controlling the initial braking speed and braking time to prevent a collision between a moving vehicle and a vehicle ahead.

[0083] 3, 4a, and 4b show the processes being performed sequentially, but this is merely an illustrative example of the technical concepts of some embodiments of the present invention. In other words, a person skilled in the art to which some embodiments of the present invention pertains can make various modifications and variations to the processes described in Figures 3, 4a, and 4b, or to perform one or more of the processes in parallel, without departing from the essential characteristics of some embodiments of the present invention. Therefore, Figures 3, 4a, and 4b are not limited to a chronological order.

[0084] Meanwhile, the operation of the automatic terrain mode control device 100 according to the embodiment of the present invention shown in FIG. 1 is embodied in a program and recorded on a computer-readable recording medium. The computer-readable recording medium on which the program for implementing the operation of the automatic terrain mode control device 100 according to the embodiment of the present invention is recorded includes all types of recording devices capable of storing data readable by a computer system. Such computer-readable recording media may be non-transitory media such as ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and may also include transitory media such as carrier waves (e.g., transmission via the Internet) and data transmission media. Furthermore, the computer-readable recording media may be distributed among computer systems connected to a network, so that the computer-readable code is stored and executed in a distributed manner.

[0085] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. Such various implementations include implementation in one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor (which may be a special-purpose processor or a general-purpose processor) coupled to receive data and commands from and transmit data and commands to a storage system, at least one input device, and at least one output device. The computer program (also known as a program, software, software application, or code) includes command words for the programmable processor and is stored on a "computer-readable storage medium."

[0086] Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Such computer-readable recording media may be non-volatile or non-transitory media such as ROMs, CD-ROMs, magnetic tapes, floppy disks, memory cards, hard disks, magneto-optical disks, and storage devices, and may also include transitory media such as carrier waves (e.g., transmissions over the Internet) and data transmission media. Furthermore, computer-readable recording media may be distributed across networked computer systems, so that computer-readable code is stored and executed in a distributed manner.

[0087] Various implementations of the systems and techniques described herein can be implemented by a programmable computer, which includes a programmable processor, a data storage system (including volatile memory, non-volatile memory, or other types of storage systems, or a combination thereof), and at least one communication interface. For example, the programmable computer can be a server, a network appliance, a set-top box, an embedded device, a computer expansion module, a personal computer, a laptop, a personal data assistant (PDA), a cloud computing system, or a mobile device.

[0088] The above description merely exemplifies the technical idea of ​​the present invention, and the examples of the present invention are intended to explain rather than limit the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these examples. The scope of protection of the present invention should be interpreted by the claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of rights of the present invention. [Explanation of symbols]

[0089] 100: Terrain mode automatic control device 102: Road surface condition estimation unit 104: Camera 106: Data collection unit 108: Data storage unit 110: Road surface friction coefficient estimation unit 112: Learning unit 114: Terrain mode determination unit 116: Terrain mode control unit 118: Vehicle equipped with an autonomous driving control unit 120: Autonomous driving control unit 202: Data acquisition unit 204: Labeling data generation unit 206: Training unit

Claims

1. A method for automatically controlling a vehicle's driving state in a terrain mode optimized for a road surface on which the vehicle is driving, a process of acquiring road surface data obtained by photographing the road surface from a camera mounted on the vehicle; inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of determining an optimized terrain mode for the road surface using the road surface friction coefficient and generating optimal terrain mode information; and a process for controlling a module within a vehicle based on the optimal terrain mode information; Including, The method for automatically controlling the driving state of a vehicle, wherein the learning model is trained to estimate the road surface friction coefficient from the image, using the road surface friction coefficient calculated based on the wheel speed and wheel torque values ​​of the wheels attached to the vehicle and an image of the road surface region located in front of the vehicle as learning data.

2. In the process of obtaining the road surface data, 2. The method for automatically controlling the driving state of a vehicle according to claim 1, further comprising obtaining additional road surface data using a radar and a lidar mounted on the vehicle.

3. a process for generating labeling data based on the road surface data and the estimated road surface friction coefficient; and The method for automatically controlling the driving state of a vehicle according to claim 1, further comprising a process of training the deep learning-based learning model by utilizing the labeling data as learning data.

4. The module in the vehicle includes an ESC (Electronic Stability Control), 2. The method for automatically controlling the vehicle driving state according to claim 1, wherein in the process of controlling the module, the ESC operates a TCS (Traction Control System) controller that is preset for each terrain mode.

5. The module in the vehicle includes an ECU (Engine Control Unit), 2. The method for automatically controlling a vehicle's driving state according to claim 1, wherein the process of controlling the module causes the ECU to change a preset engine mapping value for the terrain mode.

6. The module in the vehicle includes a TCU (Transmission Control Unit), 2. The method for automatically controlling a vehicle's driving state according to claim 1, wherein the process of controlling the module causes the TCU to change a preset shift point for shifting the transmission gear for the terrain mode.

7. 1. A method for assisting terrain mode control of a vehicle, comprising: obtaining road surface data from a camera mounted on the vehicle and configured to photograph the road surface on which the vehicle is traveling; inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process of determining an optimized terrain mode for the surface on which the vehicle is traveling using the road surface friction coefficient; and providing optimal terrain mode information based on the optimized terrain mode; Including, The learning model is trained to estimate the road surface friction coefficient from an image captured of a road surface region in front of the vehicle, using the road surface friction coefficient calculated based on wheel speed and wheel torque values ​​of wheels attached to the vehicle and an image captured of a road surface region in front of the vehicle as learning data.

8. In the process of obtaining the road surface data, 8. The method of claim 7, further comprising obtaining additional road surface data using a radar and a lidar mounted on the vehicle.

9. a process for generating labeling data based on the road surface data and the estimated road surface friction coefficient; and The method of claim 7, further comprising: training the deep learning-based learning model by utilizing the labeling data as learning data.

10. 1. A method for assisting autonomous driving control of a vehicle, comprising: a process of acquiring road surface data obtained by photographing the road surface on which the vehicle is traveling from a camera mounted on the vehicle; inputting the road surface data into a pre-trained deep learning-based learning model to estimate a road friction coefficient corresponding to the road surface; a process for determining a driving limit value of the vehicle based on the road friction coefficient and controlling a module in the vehicle based on the driving limit value; Including, The learning model is trained to estimate the road surface friction coefficient from a video image of a road surface region in front of the vehicle, using the road surface friction coefficient calculated based on the wheel speed and wheel torque values ​​of the wheels attached to the vehicle and a video image of the road surface region in front of the vehicle as learning data.

11. In the process that controls the module, 11. The method of claim 10, further comprising the step of: changing a shift pattern of a transmission gear of the vehicle based on the driving limit value.

12. In the process that controls the module, 11. The method for assisting autonomous driving control of a vehicle according to claim 10, further comprising a process for changing an initial braking speed and a braking time of a braking system of the vehicle based on the driving limit value.

13. In an automatic terrain mode control device for a vehicle, An estimation unit configured to obtain road surface data obtained by photographing the road surface on which the vehicle is traveling from a camera mounted on the vehicle, input the road surface data into a pre-trained deep learning-based learning model, and estimate a road friction coefficient corresponding to the road surface; a terrain mode determining unit for determining a terrain mode optimized for the road surface based on the road surface friction coefficient; and a terrain mode control unit that controls a module in the vehicle based on the determined terrain mode; Including, The learning model is trained to estimate the road surface friction coefficient from an image taken of a road surface region in front of the vehicle, using the road surface friction coefficient calculated based on the wheel speed and wheel torque values ​​of the wheels attached to the vehicle and an image taken of a road surface region in front of the vehicle as learning data.

14. A computer program stored on a computer-readable recording medium for executing each process included in the method for automatically controlling the running state of a vehicle according to any one of claims 1 to 6.

Images