CONTROL DEVICE OF AN AUTONOMOUS VEHICLE AND METHOD THEREFOR

Abstract

The present disclosure relates to an apparatus and method for avoiding an obstacle. The apparatus comprises a sensor (100) that detects an object external to a vehicle (VEH), a processor (200), and a memory (300) that stores instructions. The apparatus determines whether the object belongs to a preclassified type, determines a confidence value associated with information about the object based on the object not belonging to a preclassified type, and, based on the confidence value being greater than a confidence threshold, controls the control device to avoid the object.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an autonomous vehicle control device and method, and more particularly to a technology for controlling driving of a vehicle based on detection of an object outside the vehicle. BACKGROUND

[0002] An autonomous vehicle may refer to a vehicle capable of driving independently without intervention by a driver or passenger, and an automated vehicle and highway or expressway system may refer to a system that monitors and controls such an autonomous vehicle to drive independently. In addition, technologies have been developed to monitor the external environment (e.g., outside the vehicle) and control various driver-assistance functions based on the monitored external environment of the vehicle.

[0003] Furthermore, autonomous vehicles or vehicles equipped with driver assistance functions can detect an object by monitoring the vehicle's external environment and are controlled based on a scenario (e.g., a prediction) determined based on the detected object. In other words, autonomous driving may require an operation to determine the nature of an object outside the vehicle.

[0004] The objects outside the vehicle can be classified based on predetermined classes (e.g. classifications, categories, etc.).

[0005] However, the types of objects that can be classified in advance using classes are limited, and unexpected obstacles may appear on a road that do not fall under any known classification. If an autonomous vehicle encounters an object or obstacle that has not been classified in advance, it may attempt to avoid the object (e.g., by driving around it, swerving, decelerating, etc.). However, if autonomous vehicles are designed to avoid all unclassified objects, driving may be disrupted, thus affecting traffic flow even in situations where avoiding the object is not necessary. A sudden evasive maneuver may also compromise driving safety. SHORT DESCRIPTION

[0006] The present disclosure is intended to solve the above-mentioned problems of the known art while retaining the advantages of the known art.

[0007] One aspect of the present disclosure provides an obstacle avoidance apparatus and method that can efficiently perform autonomous driving when a non-pre-classified object is detected.

[0008] Another aspect of the present disclosure provides an obstacle avoidance apparatus and method that can reduce unnecessary vehicle controls caused by erroneous detection of an object by the sensors when the object that has not been classified in advance is detected.

[0009] The technical problems to be solved by the present disclosure are not limited to the above-mentioned problems, and any other technical problems not mentioned here will be clearly understood by those skilled in the art from the following description.

[0010] According to one or more embodiments of the present disclosure, a control device of an autonomous vehicle may include: a sensor configured to detect an object, a processor, and a memory. The memory may store instructions that, when executed by the processor, cause the device to: determine whether the object belongs to any pre-classified type; determine a confidence value associated with information about the object based on the object not belonging to any pre-classified type; and control the vehicle to avoid the object (e.g., by swerving, braking, and combinations thereof) based on the confidence value being greater than a confidence threshold.

[0011] For example, the sensor may include at least one of a camera, a lidar, and a radar. For example, when executed by the processor, the instructions may further cause the device to determine the reliability value based on at least two of the following: a first reliability value determined based on an image captured by the camera, a second reliability value determined based on data captured by the lidar, and a third reliability value determined based on a reflected radio wave received by the radar.

[0012] For example, the instructions, when executed by the processor, may further cause the device to determine the first reliability value based on a height of the object determined by the camera.

[0013] For example, the instructions, when executed by the processor, may further cause the device to determine the second reliability value based on whether the object is in contact with a road surface.

[0014] For example, the instructions, when executed by the processor, may further cause the device to determine the second reliability value further based on a height of the object detected by the lidar.

[0015] For example, the instructions, when executed by the processor, may further cause the device to determine the second reliability value further based on a width (e.g., in the vehicle transverse direction) of the object detected by the lidar.

[0016] For example, the instructions, when executed by the processor, may further cause the device to determine the third reliability value based on the strength of the reflected radio wave received by the radar.

[0017] For example, the instructions, when executed by the processor, may further cause the device to control the vehicle to ignore the object and maintain a current course or path of the vehicle based on the confidence value being less than a threshold.

[0018] For example, the instructions, when executed by the processor, may cause the device to control the vehicle to avoid the object in a transverse direction (e.g., of the vehicle) based on a determination that the object can be avoided in the transverse direction.

[0019] For example, the instructions, when executed by the processor, may cause the device to decelerate the vehicle based on the confidence value based on a determination that the object cannot be avoided in a lateral direction.

[0020] According to one or more exemplary embodiments of the present disclosure, a method for controlling an autonomous vehicle may include determining whether an object detected by a sensor of a vehicle belongs to any pre-classified type based on the object not belonging to any pre-classified type, determining a confidence value associated with information about the object, and controlling the vehicle based on the confidence value being greater than a confidence threshold to avoid the object (e.g., swerve, decelerate, and combinations thereof).

[0021] The method may, for example, further comprise: determining the reliability value based on at least one of the following values: a first reliability value determined based on an image captured by a camera of the sensor, a second reliability value determined based on data captured by a lidar of the sensor, and a third reliability value determined based on a reflected radio wave received by a radar of the sensor.

[0022] The method may, for example, further comprise: determining the first reliability value based on a height of the object determined by the camera (e.g. height above the road surface).

[0023] For example, the method may further comprise: determining the second reliability value based on whether the object is in contact with a road surface.

[0024] The method may, for example, further comprise: determining the second reliability value based on a height of the object detected by the lidar.

[0025] The method may, for example, further comprise: determining the second reliability value based on a width of the object detected by the lidar.

[0026] The method may, for example, further comprise: determining the third reliability value based on the strength of the reflected radio wave received by the radar.

[0027] The method may, for example, further comprise: determining whether a second (e.g., further or additional) object detected by the sensor belongs to a pre-classified type based on the second object not belonging to a pre-classified type, determining a second confidence value associated with information about the second object, and controlling the vehicle based on the second confidence value being less than a threshold value to ignore the second object and maintain a current course or path of the vehicle.

[0028] Controlling the vehicle may, for example, comprise: controlling the vehicle to avoid the object in a transverse direction (e.g., swerve or avoid it) based on a determination that the object can be avoided in the transverse direction.

[0029] The method may further comprise, for example: causing the vehicle to brake based on the reliability value. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings: Fig. 1 is a block diagram showing a configuration of a control device of an autonomous vehicle, Fig. 2 is a diagram showing a vehicle equipped with an autonomous vehicle control device, Fig. 3 is a schematic representation of an artificial neural network (ANN) for calculating reliability, Fig. 4 is a flowchart illustrating an obstacle avoidance method, Fig.5 to 7 are diagrams illustrating an example method for determining the second reliability, Fig. Fig. 8 is a diagram illustrating an example of determining a trip based on detecting a road surface marking on a road, Fig. 9 is a flowchart illustrating an exemplary method for obstacle avoidance, Fig. 10 and Fig. 11 are diagrams illustrating the determination of a collision point, Fig. 12 is a diagram showing the transverse avoidance within a lane, Fig. 13 is a diagram illustrating a method for determining whether transverse avoidance via a lane change is possible, and Fig. 14 shows a calculation system. DETAILED DESCRIPTION

[0031] Hereinafter, one or more embodiments of the present disclosure will be described in detail with reference to the example drawings. When adding reference numerals to the components of each drawing, it should be noted that the identical or equivalent component will be denoted by the identical reference numeral even if they are illustrated in other drawings. Furthermore, in describing the embodiments of the present disclosure, a detailed description of the related known configuration or function will be omitted if it is determined that it would interfere with the understanding of the embodiments of the present disclosure.

[0032] In describing the components of the one or more embodiments according to the present disclosure, terms such as "first...", "second...", "A", "B", "(a)", "(b)", and the like may be used. These terms are merely used to distinguish the components from other components, and the terms do not limit the type, order, or sequence of the components. Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.It is further understood that terms as defined in commonly used dictionaries are to be construed to have a meaning consistent with their meaning in the context of the relevant art and are not to be construed in an idealized or overly formal sense unless expressly so defined herein.

[0033] Hereinafter, the present disclosure will be described with reference to the Fig. 1 to 14 described in detail.

[0034] Fig. 1 is a block diagram illustrating a configuration of a control device of an autonomous vehicle, and Fig. 2 is a diagram showing a vehicle equipped with an autonomous vehicle control device.

[0035] As with reference to Fig. 1 and Fig.2, the control device of an autonomous vehicle may include a sensor 100, a processor 200, a memory 300, and a driving control device 400.

[0036] The sensor 100 may include at least one of a camera 110, a lidar (light imaging detection and ranging) 120, and a radar (radio detection and ranging) 130 for detecting an external object of a vehicle VEH (e.g., an object in the surroundings of the vehicle VEH).

[0037] The camera 110, which captures an external image of the vehicle VEH, can capture an image of a front area or a front side area of ​​the vehicle VEH, or an area in front of the vehicle and to the side of the vehicle. For example, the camera 110 can be arranged on a windshield to capture the image of the front area of ​​the vehicle VEH.

[0038] The lidar 120, which emits a laser and detects the object based on a reflected wave (e.g., radio wave, radar signal, etc.) from the laser reflected from the object, can be implemented according to a time-of-flight (TOF) or phase-shift principle. The lidar 120 can be mounted to be exposed to the exterior of the vehicle and can be arranged on a front bumper or a front grille.

[0039] The radar 130 may include electromagnetic wave transmission and reception modules. The radar 130 may be embodied as a pulse radar or a continuous wave radar based on the principle of radio wave radiation. The radar 130 may be implemented as a frequency modulated continuous wave (FMCW) radar or a frequency shift keying (FSK) radar based on a waveform among the continuous wave radar methods. The radar 130 may include a front radar 131 disposed in the center of the front surface of the vehicle VEH, front side radars 132 disposed at both ends of the front bumper, and a rear radar 133 disposed at a rear portion of the vehicle VEH.

[0040] The arrangement of the camera 110, the lidar 120 and the radar 130 is not necessarily limited to the Fig. 2 positions shown.

[0041] In addition to what is shown in the drawing, the sensor may include an ultrasonic sensor and / or an infrared sensor. The ultrasonic sensor may include modules for transmitting and receiving ultrasonic waves. The ultrasonic sensor may detect the object based on an ultrasonic wave and detect a location of the detected object, a distance to the detected object, and a relative speed. The ultrasonic sensor may be arranged at a suitable location on the outside of the vehicle to detect the object located in front of, behind, or to a side of the vehicle. The infrared sensor may include modules for transmitting and receiving infrared rays. The infrared sensor may detect the object based on the infrared ray and detect the location of the detected object, the distance to the detected object, and the relative speed.The infrared sensor can be located on the outside of the vehicle to detect the object located in front of, behind or on the side of the vehicle.

[0042] In addition, the sensor 100 may include a brake-pedal position sensor (BPS) and an accelerator position sensor (APS) that generate a speed control command to shift the vehicle.

[0043] The brake pedal position sensor may output a BPS signal based on the degree of application of a brake pedal located in the vehicle. For example, the BPS signal may output data from 0 to 100 based on the degree of brake pedal depression, where 0 may represent a case where the brake pedal is not depressed, and 100 may represent a case where the brake pedal is fully depressed.

[0044] The accelerator pedal position sensor may output an APS signal based on the degree of depression of an accelerator pedal in the vehicle. For example, the APS signal may output data from 0 to 100 based on the degree of depression of the accelerator pedal, where 0 may represent a case where the accelerator pedal is not depressed, and 100 may represent a case where the accelerator pedal is fully depressed.

[0045] The processor 200 can detect the object outside the vehicle and perform avoidance driving (e.g., bypassing and / or braking) based on the object.

[0046] For this purpose, the processor 200 can determine whether the object detected by the sensor 100 of the vehicle VEH is an obstacle of a preset or pre-classified type.

[0047] The preset type obstacle can be an object that can be observed on a road or an object that can be classified based on a class using deep learning. The preset type obstacle can include, for example, a vehicle (e.g., car, truck, bus, etc.), a two-wheeled vehicle (e.g., bicycle, motorcycle, etc.), a pedestrian, a traffic facility (e.g., sign gantry, traffic light, etc.), and the like.

[0048] Processor 200 may determine (e.g., calculate) the reliability (e.g., a reliability value) when the object is not the obstacle. Reliability may be a criterion for determining (e.g., using) information about the object.

[0049] The processor 200 may determine a final reliability based on at least one of the following: a first reliability determined by the camera 110 (e.g., based on its images), a second reliability determined by the lidar 120 (e.g., based on its detection data), and a third reliability determined by the radar 130 (e.g., based on its detection data).

[0050] The processor 200 may determine the object as the obstacle that is an avoidance target if the reliability is equal to or greater than a preset threshold. For example, the processor 200 may determine the object as the obstacle if the final reliability is equal to or greater than the threshold.

[0051] In addition, the processor 200 may ignore the object and maintain the route if the final reliability is less than the threshold.

[0052] The processor 200 can control the movement of the vehicle to avoid the obstacle (e.g., bypass) based on the detection of the obstacle.

[0053] The processor 200 may control a driving or steering control device to avoid (e.g., bypass) the obstacle if it is determined that the obstacle can be avoided in a lateral direction.

[0054] The processor 200 may decelerate the vehicle by controlling a braking control device when the obstacle cannot be avoided (e.g., bypassed) in the transverse direction. The processor 200 may decelerate the vehicle by setting a braking value based on (e.g., proportional to) a value of the final reliability.

[0055] Furthermore, based on the data provided by sensor 100, processor 200 may perform artificial intelligence learning to detect a target vehicle (e.g., a vehicle in an adjacent lane) and a dangerous vehicle (e.g., a vehicle in an adjacent lane that would block a lane change of the host vehicle). For this purpose, processor 200 may include an artificial intelligence processor (hereinafter referred to as an AI processor). The AI ​​processor may learn or train a neural network using a pre-stored program. The neural network for detecting the target vehicle and the dangerous vehicle may be designed to simulate the structure of the human brain on a computer and may include a plurality of network nodes with weights that simulate the neurons of a human neural network.The majority of network nodes can send and receive data based on connection relationships between them to simulate the synaptic activity of neurons that send and receive signals across synapses. The neural network can comprise a deep learning model developed from a neural network model. In the deep learning model, the majority of network nodes can transmit and receive data based on convolutional connection relationships while located in different layers. Examples of neural network models can comprise various deep learning techniques, such as a deep neural network (DNN), a convolutional deep neural network (CNN), a recurrent Boltzmann machine (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), and a deep Q-network.

[0056] Fig.Figure 3 is a schematic representation of an artificial neural network (ANN) for reliability calculation.

[0057] With reference to Fig. 3, the processor 200 may determine the final reliability (e.g., reliability value, reliability score, etc.) using the Fig. 3 shown artificial neural network (e.g. calculate).

[0058] The artificial neural network may be stored in the processor 200 or in memory. The artificial neural network mathematically models a connected form of human neurons and may have an input layer, a hidden layer, and an output layer.

[0059] The input layer can multiply input data by a weighting matrix and pass the result to a hidden layer. The input data can be variables used to determine (e.g., calculate) reliability. The input data can include, for example, the image captured by camera 110, the data captured by lidar 120, the reception strength of radar 130, and the like.

[0060] The hidden layer can process the input data based on an activation function. Furthermore, the hidden layer can multiply the processed input data by a new weight matrix and pass the result to the output layer.

[0061] The output layer can output a result by reflecting an activation function for the output. The output layer can output the final reliability as an output value.

[0062] In addition, the processor 200 may determine whether to avoid the obstacle based on the information about the object on the road surface provided by another vehicle via a communication device 500.

[0063] Memory 300 may store algorithms and the AI ​​processor for operating processor 200. Memory 300 may use a hard disk drive, flash memory, electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), ferroelectric random access memory (FRAM), phase-change random access memory (PRAM), magnetic random access memory (MRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double-date-rate SDRAM (DDR-SDRAM), and the like.

[0064] The driving control device 400, which serves to control the steering and the deceleration and acceleration of the vehicle in response to a control signal from the processor 200, may include (or communicate with) (e.g., at least one of the following) the steering control device, an engine control device, the brake control device, and a transmission control module.

[0065] The steering control device can be divided into a hydraulic power steering system (HPS), which controls the steering using hydraulic pressure generated by a hydraulic pump, and a motor-driven power steering system (MDPS), which controls the steering using the output torque of an electric motor.

[0066] The engine control device is an actuator that controls an internal combustion engine of the vehicle and controls the acceleration of the vehicle. The engine control device can be implemented as an engine management system (EMS). The engine control device controls the drive torque of the internal combustion engine based on the accelerator position information output by the accelerator position sensor. The engine control device controls the power of the internal combustion engine to follow a vehicle speed requested by the processor 200 during autonomous driving.

[0067] The brake control device is an actuator that controls the deceleration of the vehicle and may be implemented as an electronic stability control (ESC). The brake control device controls a brake pressure to follow a target speed requested by the processor 200. That is, the brake control device controls the deceleration of the vehicle.

[0068] The transmission control module is an actuator for controlling the vehicle's transmission and can be implemented as a shift-by-wire (SBW) system. The transmission control module controls the vehicle's shifting depending on a gear position and a gear state range.

[0069] Furthermore, the control device of an autonomous vehicle may include the communication device 500 and an alarm device 600.

[0070] The communication device 500 can communicate with a user terminal, another vehicle, and / or an external server.

[0071] The communication device 500 may support short-range communication using at least one of the following technologies: Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (Wireless USB).

[0072] The communication device 500 may include a GPS (Global Positioning System) module or a DGPS (Differential Global Positioning System) module for obtaining location information.

[0073] Furthermore, the communication device 500 may include a V2X communication module. The V2X communication module may include RF circuitry for a protocol for wireless communication with a server (vehicle to infrastructure, V2I), another vehicle (vehicle to vehicle, V2V), and / or a pedestrian (vehicle to pedestrian, V2P). The communication device 500 may receive sensor data acquired by a sensor of another vehicle and forward the data to the processor 200 via the V2X communication module.

[0074] The alarm device 600 can notify a vehicle occupant of the obstacle detected by the processor 200 and report an emergency braking or evasive situation caused by the obstacle. The alarm device 600 can include a display, a speaker, and the like.

[0075] Fig. 4 is a flowchart illustrating an obstacle avoidance method. Fig. 4 may be a procedure that differs from that described in Fig. 1 is controlled by the processor shown.

[0076] With reference to Fig. 1 to 4, the obstacle avoidance procedure is described as follows.

[0077] In S410, the processor 200 may determine (e.g., calculate) the reliability if the object detected by the sensor 100 is not of a pre-classified type (e.g., a known classification).

[0078] For example, processor 200 can learn the image captured by camera 110 using a semantic segmentation scheme and classify the objects contained in the image. Objects can be classified using predefined classes. If an object of a class other than the predefined classes is detected in the image, processor 200 can determine (e.g., calculate) the reliability.

[0079] An operation for determining (e.g., calculating) the reliability may include an operation for determining (e.g., calculating) the first reliability determined by the image captured by the camera 110. For example, the processor 200 may set or determine the first reliability as small if the height of an unclassified object among the objects captured in the image is small.

[0080] The process of determining (e.g., calculating) the reliability may include a process of determining (e.g., calculating) the second reliability determined by the data acquired by the lidar 120. For example, the processor 200 may determine a value of the second reliability of the object in contact with a road surface that is greater than the second reliability of the object not in contact with the road surface. Furthermore, the processor 200 may set the second reliability to be greater as a height of the object acquired by the lidar 120 is greater. Furthermore, the processor 200 may set the second reliability to be greater as the width of the object acquired by the lidar 120 is greater.

[0081] The process of determining (e.g., calculating) the reliability may include a process of determining (e.g., calculating) the third reliability determined by the strength of a reflected wave from radar 130. Processor 200 may receive the strength of the reflected wave received from radar 130 and determine the third reliability based on (e.g., proportional to) the strength of the reflected wave.

[0082] The processor 200 may determine (e.g., calculate) the final reliability using at least one of the first reliability, the second reliability, and the third reliability. Any of the first reliability, the second reliability, and the third reliability may be (e.g., alone) the final reliability. To more accurately determine information about the object, the processor 200 may also calculate the final reliability by selecting at least two of the first reliability, the second reliability, and the third reliability.

[0083] In S420, the processor 200 may determine the object as an obstacle that is the avoidance target based on the reliability equal to or greater than the threshold reliability.

[0084] The processor 200 may compare the final reliability to the threshold reliability, where the threshold reliability may be preset.

[0085] In S430, the processor 200 may control the movement of the vehicle to avoid the obstacle (e.g., swerve and / or brake).

[0086] For this purpose, the processor 200 can control the driving control device 400 to brake the vehicle or control the steering of the vehicle.

[0087] Each process is described below as follows.

[0088] Table 1 is a table showing an example of determining the initial reliability. <Tabelle 1> Size of the height of the object First Reliability CV1 Object height H_I > Hth1 60 Object height H_I ≤ Hth1 20

[0089] Referring to Table 1, the processor 200 may determine the first reliability CV1 based on an image-based object height H_I that exceeds a preset first threshold Hth1 as "60." The first threshold Hth1 may be determined as the height of a plane at which the vehicle VEH can ignore the object.

[0090] The image-based object height H_I may denote the height of the object calculated based on the image captured by the camera 110. The image captured by the camera 110 may be expressed in an image coordinate system. The image coordinate system may be a coordinate system in which an upper left pixel of the image is an origin (0,0), a horizontal position of a pixel in the image is expressed as a u-coordinate component, and a vertical position of the pixel is expressed as a v-coordinate component.

[0091] In the image coordinate system, the u-coordinate component may be an x-axis direction of a world or vehicle coordinate system, and the v-coordinate component may be a y-axis direction of the world or vehicle coordinate system. Assuming that the road surface is flat, an x-axis and a y-axis may refer to linear axes defining a plane parallel to the road surface, the y-axis direction may denote a forward direction of the vehicle, and the x-axis may denote the transverse direction of the vehicle perpendicular to the y-axis. In the world or vehicle coordinate system, a z-axis direction may denote a height with respect to the road surface.

[0092] Processor 200 may determine the actual height of the object based on image coordinates of the object captured from the image. Any known technology may be used as a method for processor 200 to determine the actual height of the object.

[0093] Furthermore, according to Table 1, the processor 200 may determine the first reliability CV1 as “20” when the image-based object height H_I is less than or equal to the first threshold Hth1.

[0094] Table 2 is a table showing an example of determining the second reliability. <Tabelle 2> Sizes of height / width and height offset of the object Second Reliability CV2 Height offset = 1 10 Height offset = 0, H_L ≤ Hth2 20 Height offset = 0, H_L > Hth2, Wd > Hth3 80

[0095] Referring to Table 2, the processor 200 may determine the second reliability CV2 based on the lidar-based object height, the lidar-based object width, a height offset, and the like.

[0096] The lidar acquisition data for determining the second reliability are described below.

[0097] Fig. 5 to 7 are diagrams showing an exemplary method for determining the second reliability.

[0098] With reference to Fig.5 to 7, the processor 200 can acquire x / y data where an object is located in an xy plane, and the height and height offset of the object can be determined based on a range where the x / y data is located.

[0099] The processor 200 can determine a distance from a road surface Rs to a top point of the object Ob as a height of the object Ob. For example, if the object Ob is located as in Fig. 5, is located near the vehicle VEH and is large, an upper area of ​​the object Ob may no longer be detected by the lidar 120. Therefore, the Fig. 5 shown height of the object Ob can be estimated at "H1". Alternatively, the height of the object in Fig. 6 shown object Whether can be determined as “H2” and can the height of the Fig. 7 shown object whether can be determined as “H3”.

[0100] The height offset can be a factor that indicates whether the object Ob is in contact with the road surface Rs. If the object Ob is in contact with the road surface Rs, the height offset can be set to "0", and if the object Ob is not in contact with the road surface Rs, the height offset can be set to "1". Therefore, all the data received from the Lidar 120 in Fig. 5 recorded height offsets of the object Ob are set to "0". Accordingly, the height offsets of the object in Fig. 5 shown object Ob and the one in Fig. 6 shown object Ob can be set to "0". In addition, a height offset of the object in Fig. 7 shown object Whether to be set to “1”.

[0101] The processor 200 may determine the second reliability CV2 as “10” when the height offset is “1”.

[0102] Furthermore, the processor 200 may determine the second reliability CV2 as “20” when the height offset is “0” and the height of the object is less than or equal to a preset second threshold Hth2.

[0103] In addition, the processor 200 may determine the second reliability CV2 as “80” when the height offset is “0”, the height of the object exceeds the preset second threshold Hth2, and the width of the object exceeds a preset third threshold Hth3.

[0104] Table 3 is a table showing an example of determining the third reliability. <Tabelle 3> Value of the strength of the reflected wave Third Reliability CV3 Strength > Pth1 40 Pth1 ≥ Strength > Pth2 20 Pth2 ≥ strength 0

[0105] Referring to Table 3, the processor 200 may determine the third reliability CV3 based on the strength of the reflected wave of the radar.

[0106] If the strength of the reflected radar wave exceeds the first critical strength Pth1, the processor 200 can determine the third reliability CV3 as “40”.

[0107] When the strength of the reflected radar wave is less than or equal to the first critical strength Pth1 and exceeds the second critical strength Pth2, the processor 200 may determine the third reliability CV3 as “20”.

[0108] When the strength of the reflected wave of the radar is less than or equal to the second critical strength Pth2, the processor 200 may determine the third reliability CV3 as “0 (zero)”.

[0109] The first critical strength Pth1 can be set larger than the second critical strength Pth2.

[0110] The processor 200 may determine the final reliability CV_f based on the first reliability CV1, the second reliability CV2, and the third reliability CV3. The processor 200 may determine the final reliability CV_f as shown in Equation 1 below. CV_f=al∗CV1+b1×CV2+c1×CV3

[0111] The final reliability CV_f can be determined by Equation 1, and a scheme for determining the final reliability CV_f can be implemented in several ways. In Equation 1, a1, b1, and c1 are variables used in the process of determining the final reliability CV_f and can be customized by the designer.

[0112] Table 4 shows a procedure to determine whether normal driving is possible and how much deceleration is based on the final reliability CV_f. <Tabelle 4> Final reliability CV_f Maximum brake tax amount Whether normal driving is possible CVth1 > CV_f not controlled mobile (possible) CVth2 > CV_f ≥ CVth1 - 4 m / s 2 not drivable (not possible) CVth3 > CV_f ≥ CVth2 - 6 m / s 2 not drivable (not possible) CV_f ≥ CVth3 - 9,8 m / s 2 not drivable (not possible)

[0113] Referring to Table 4, the processor 200 may determine that normal driving (e.g., continuing on the previous course / path at the previous speed) is impossible when the final reliability CV_f is equal to or greater than the first reliability threshold CVth1. Normal driving may refer to a state in which driving continues while maintaining the speed of the moving vehicle without changing lanes.

[0114] When the final reliability CV_f is equal to or greater than the first reliability threshold CVth1, the processor 200 may determine a maximum brake control amount for reducing the speed of the vehicle, and a value of the maximum brake control amount may be proportional to a value of the final reliability.

[0115] For example, if the final reliability CV_f is equal to or greater than the first reliability threshold CVth1 and less than the second reliability threshold CVth2, the processor 200 may set the maximum brake control amount as “-4 m / s 2 " determine.

[0116] Further, when the final reliability CV_f is equal to or greater than the second reliability threshold CVth2 and less than the third reliability threshold CVth3, the processor 200 may further set the maximum brake control amount as “-6 m / s 2 " determine.

[0117] Further, when the final reliability CV_f is equal to or greater than the third reliability threshold CVth3, the processor 200 may also set the maximum brake control value “-9.8 m / s 2 " determine.

[0118] The maximum brake control amount is shown in Table 4, and the value of the maximum brake control amount can be set to a different value.

[0119] Further, the processor 200 may ignore the object detected by the sensor 100 and maintain the driving state if the final reliability CV_f is less than the first reliability threshold CVth1.

[0120] The first reliability threshold CVth1 can be preset to “50”, the second reliability threshold CVth2 to “70”, and the third reliability threshold CVth3 to “80”, but the values ​​for the reliability thresholds CVth1, CVth2 and CVth3 are not limited to this.

[0121] An example of determining whether normal driving is possible based on the final reliability CV_f is described below.

[0122] As in Fig.7, an example of determining whether normal driving is possible for the object not in contact with the road surface is described as follows.

[0123] As in Fig. 7, when the object Ob is suspended in the air without being in contact with the road surface, the processor 200 may determine the second reliability CV2 as “10” based on Table 2.

[0124] If the object Ob is light, the strength of the reflected wave can be identified as weak, and the processor 200 can determine the third reliability CV3 as "0" based on Table 3. Therefore, if the object hovering above the road surface is a light object, the final reliability CV_f can be determined as "10." If the first reliability threshold CVth1 is "50", the processor 200 can maintain normal driving because the final reliability CV_f is determined as 10.

[0125] Further, if the object Ob is heavy, the strength of the reflected wave may be detected as large, and the processor 200 may determine the third reliability CV3 as "40" based on Table 3. Therefore, if the object hovering above the road surface is a heavy object, the final reliability CV_f may be determined to be "50." If the first reliability threshold CVth1 is "50," the processor 200 may control the driving control device 400 to decelerate the vehicle or perform the evasive maneuver when the final reliability CV_f is determined to be 50.

[0126] A reliability threshold is an example and can vary depending on the design.

[0127] With reference to Fig. 8, an example of determining whether to perform normal driving when a small structure is detected is described as follows.

[0128] Fig. 8 is a diagram showing an example of determining a trip based on detecting a road surface marking (e.g., road reflector) on a road.

[0129] As in Fig. As shown in Figure 8, a road surface marker RM in the form of a small dot protrudes not far from the road surface, so the road surface marker RM may not have a great impact on the vehicle's travel. However, due to the characteristics of the structure, the light reflection is large, so the road surface marker RM may be detected as a large object by the lidar 120 or the radar 130.

[0130] Since the small road surface marking RM detected by the camera 110 has a low height, the processor 200 may determine the first reliability CV1 as "20" based on Table 1. In addition, the processor 200 may determine the second reliability CV2 as "20" based on Table 2. As a result, the final reliability CV_f may be determined as "40", and the processor 200 may maintain the normal driving state based on Table 4.

[0131] Fig. 9 is a flowchart illustrating an exemplary method for obstacle avoidance. Fig. 9 may be a procedure that is different from the Fig. 1 is controlled by the control device shown.

[0132] With reference to Fig. 9, an exemplary method for obstacle avoidance is described as follows.

[0133] In S901, the processor 200 may detect the object in the area outside (e.g., surroundings) of the vehicle based on the detection data detected by the sensor 100.

[0134] In S902, the processor 200 may determine whether the detected object is of the preclassified type. The preclassified type object may refer to the object that can be classified based on the class via machine learning (e.g., deep learning).

[0135] In S903, if the detected object is of the pre-classified type, the processor 200 may determine (e.g., calculate) a predicted collision location of a host vehicle.

[0136] A method for determining (e.g. calculating) a collision location is described below with reference to Fig. 10 and Fig. 11 described.

[0137] Fig. 10 and Fig. 11 are illustrations showing the determination of a collision location.

[0138] As in Fig. 10 and Fig. 11, the processor 200 may calculate the collision location based on a degree of overlap between an obstacle Ob2 and the vehicle VEH.

[0139] The processor 200 may calculate a ratio of an overlap area OA between the vehicle VEH and the obstacle Ob2 with respect to a width of the vehicle VEH on the x-axis as the overlap degree. For example, as shown in Fig. 10, the overlap area OA between a vehicle width W1 and the obstacle Ob2 on the x-axis is equal to or greater than the vehicle width W1, the processor 200 may calculate the overlap as 100 (%). Alternatively, as shown in Fig. 11, the processor 200 can calculate the overlap to be 20 (%) when the overlap area OA in which the vehicle width W1 and the obstacle Ob2 overlap on the x-axis is 1 / 5 of the vehicle width W1.

[0140] In S904, the processor 200 may calculate the first reliability, the second reliability, and / or the third reliability if the detected object is not of the pre-classified type.

[0141] For example, processor 200 may calculate the first reliability using Table 1 based on the image captured by camera 110. Furthermore, processor 200 may calculate the second reliability using Table 2 based on the data acquired by lidar 120. Furthermore, processor 200 may calculate the third reliability using Table 3 based on the data acquired by radar 130.

[0142] In S905, the processor 200 may calculate the final reliability using at least one of the first reliability, the second reliability, and the third reliability. The processor 200 may determine (e.g., calculate) the final reliability using at least two of the first reliability, the second reliability, and the third reliability.

[0143] In S906, the processor 200 may calculate a brake control amount based on the final reliability.

[0144] The processor 200 may set a value of the brake control amount based on (e.g., proportional to) the value of the final reliability, as shown in Table 4.

[0145] Based on the expected collision location calculated in S903, the processor 200 may determine in S907 whether the obstacle can be avoided in the longitudinal direction.

[0146] If a collision with the obstacle can be avoided via deceleration without controlling the steering control device, the processor 200 may determine that the obstacle can be avoided in the longitudinal direction.

[0147] To determine whether the obstacle can be avoided in the longitudinal direction, the processor 200 may determine the distance to the obstacle, the traveling speed of the vehicle, and the braking control amount of the vehicle.

[0148] In S908, if it is determined that the longitudinal obstacle can be avoided, the processor 200 may control the brake control device of the vehicle.

[0149] In S909, if it is determined that longitudinal collision avoidance is not possible, the processor 200 may determine whether lateral avoidance within a lane is possible. The process S909 is described with reference to Fig. 12 as follows.

[0150] Fig.Figure 12 is a diagram showing the avoidance in the transverse direction within a lane.

[0151] The processor 200 can determine an avoidance route of the vehicle VEH to avoid the overlap area OA of the vehicle VEH and the obstacle Ob2. Even if the overlap area OA is avoided, if a clear space in which the vehicle VEH can travel is ensured within a lane L1, the processor 200 can determine that the obstacle Ob2 can be avoided in the transverse direction within the lane L1 without changing lanes.

[0152] In S910, if the transverse avoidance (e.g., bypassing) within the lane is not possible, the processor 200 may determine whether the transverse avoidance (e.g., bypassing) is possible by changing lanes. The process S910 is described with reference to Fig. 13 as follows.

[0153] Fig.13 is a diagram illustrating a method for determining whether transverse avoidance via a lane change is possible.

[0154] The processor 200 can determine the avoidance route of the vehicle VEH to avoid the overlap area OA of the vehicle VEH and the obstacle Ob2. If the overlap area OA cannot be avoided within the lane L1, the vehicle can plan the lane change to an adjacent lane L2. As shown in Fig. 13, the processor 200 may determine that the obstacle Ob2 cannot be avoided by the lane change if there is only one adjacent lane L2 and a collision with another vehicle VEH2 in the adjacent lane L2 is expected when the vehicle VEH, which is the host vehicle, changes lanes.

[0155] When it is determined in operations S909 and S910 that the obstacle Ob2 can be avoided in the lateral direction, the processor 200 can perform the avoidance in the lateral direction by controlling the control device in operation S911.

[0156] Further, if it is determined in S910 that transverse avoidance is not possible, the processor 200 may also perform longitudinal control by braking. In S910, the case where transverse avoidance (e.g., bypassing) is impossible may be a situation where the obstacle Ob2 cannot be avoided in both the longitudinal and transverse directions. If the obstacle Ob2 cannot be avoided and another vehicle is traveling in the adjacent lane, the processor 200 may decelerate the vehicle, selecting (e.g., possible) collision with the obstacle Ob2.

[0157] When the processor 200 enters operation S908 after operation S910, it may decelerate the vehicle with the maximum braking amount of the vehicle, regardless of the final reliability.

[0158] Fig. 14 shows a computer system.

[0159] Fig. 14 shows that a computer system 1000 may include at least a processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a memory 1600, and a network interface 1700 connected via a bus 1200.

[0160] Processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes operations with instructions stored in memory 1300 and / or storage 1600. Memory 1300 and storage 1600 may include various types of volatile or non-volatile storage media. For example, memory 1300 may include ROM (Read Only Memory) and RAM (Random Access Memory).

[0161] Thus, the operations of the method or algorithm described herein may be implemented directly in hardware or in a software module executed by processor 1100, or in a combination thereof. The software module may be located on a storage medium (i.e., memory 1300 and / or memory 1600), such as RAM, flash memory, ROM, EPROM, EEPROM, register, hard disk, removable disk, and CD-ROM.

[0162] The exemplary storage medium is coupled to processor 1100, which can read and write information to the storage medium. In another method, the storage medium may be integrated with processor 1100. The processor and storage medium may be packaged in an application-specific integrated circuit (ASIC). The ASIC may be packaged in the user terminal. In another method, the processor and storage medium may be packaged as separate components in the user terminal.

[0163] The above description merely shows the technical idea of ​​the present disclosure, and various modifications and changes can be made by those skilled in the art without departing from the essential features of the present disclosure.

[0164] Therefore, the embodiments disclosed in the present disclosure are not intended to limit the technical idea of ​​the present disclosure, but merely to demonstrate the present disclosure, and the scope of the technical idea of ​​the present disclosure is not limited by the embodiments. The scope of the present disclosure should be construed as being covered by the scope of the appended claims, and all technical ideas falling within the scope of the claims should be construed as being included in the scope of the present disclosure.

[0165] Even if the object that has not been classified in advance is detected, the information about the object can be identified more accurately based on the characteristics of the sensors to reduce unnecessary vehicle control.

[0166] Furthermore, if a certain sensor detects the object incorrectly, the reliability of the information detected by the corresponding sensor can be set low in order to detect the object more accurately and control the vehicle based on the reliability.

[0167] In addition, various effects identified directly or indirectly by this document may be provided.

[0168] Although the present disclosure has been described with reference to exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto, but may be variously modified and changed by those skilled in the art without departing from the spirit and scope of the present disclosure as claimed in the following claims. LIST OF REFERENCE SYMBOLS 100 sensors 110 Camera 120 Lidar 130 radars 200 processor 300 storage 400 control device 500 communication devices 600 alarm device 1100 processor 1300 storage 1400 User Interface Input Device 1500 User Interface Output Device 1600 memory 1700 network interface

Claims

[1] A control device of an autonomous vehicle (VEH), the device comprising: a sensor (100) configured to detect an object, a processor (200), and a memory (300) storing instructions which, when executed by the processor (200), cause the device to to determine whether the object belongs to any pre-classified type, to determine a reliability value associated with information about the object based on the object not belonging to a pre-classified type, and based on the reliability value being greater than a reliability threshold, to control the vehicle (VEH) to avoid the object. [2] The device according to claim 1, wherein the sensor (100) comprises at least one of a camera (110), a lidar (120) and a radar (130), wherein the instructions, when executed by the processor (200), further cause the device to determine the reliability value based on at least two of the following values: a first reliability value determined based on an image captured by the camera (110), a second reliability value determined based on data captured by the lidar (120), and a third reliability value determined based on a reflected radio wave received by the radar (130). [3] The apparatus of claim 2, wherein the instructions, when executed by the processor (200), further cause the apparatus to determine the first reliability value based on a height of the object determined by the camera (110). [4] The apparatus of claim 2 or 3, wherein the instructions, when executed by the processor (200), further cause the apparatus to determine the second reliability value based on whether the object is in contact with a road surface. [5] The apparatus of any of claims 2 to 4, wherein the instructions, when executed by the processor (200), further cause the apparatus to determine the second reliability value based on a height of the object detected by the lidar (120). [6] The apparatus of any one of claims 2 to 5, wherein the instructions, when executed by the processor (200), further cause the apparatus to determine the second reliability value based on a width of the object detected by the lidar (120). [7] The apparatus of any one of claims 2 to 6, wherein the instructions, when executed by the processor (200), further cause the apparatus to determine the third reliability value based on a strength of the reflected radio wave received by the radar (130). [8] The apparatus of any one of claims 2 to 7, wherein the instructions, when executed by the processor (200), further cause the apparatus to control the vehicle (VEH) to ignore the object and maintain a current course of the vehicle (VEH) based on the reliability value being less than a threshold value. [9] Apparatus according to any one of the preceding claims, wherein the instructions, when executed by the processor (200), cause the apparatus to control the vehicle (VEH) to avoid the object in a transverse direction based on determining that the object can be avoided in the transverse direction. [10] Apparatus according to any one of the preceding claims, wherein the instructions, when executed by the processor (200), cause the apparatus to decelerate the vehicle (VEH) based on the reliability value based on a determination that the object cannot be avoided in a lateral direction. [11] A method for controlling an autonomous vehicle (VEH), the method comprising: Determining (S902) whether an object detected by a sensor (100) of a vehicle (VEH) belongs to any pre-classified type, based on the fact that the object does not belong to a pre-classified type, determining (S904, S905, S906) a reliability value associated with information about the object, and Controlling (S908, S911) the vehicle (VEH) based on the reliability value being greater than a reliability threshold to avoid the object. [12] The method of claim 11, further comprising: Determining (S904, S905, S906) the reliability value based on at least one of a first reliability value determined based on an image captured by a camera (110) of the sensor (100), a second reliability value determined based on data captured by a lidar (120) of the sensor (100), or a third reliability value determined based on a reflected radio wave received by a radar (130) of the sensor (100). [13] The method of claim 12, further comprising: Determining (S904) the first reliability value further based on a height of the object determined by the camera (110). [14] The method according to claim 12 or 13, further comprising: Determining (S905) the second reliability value based on whether the object is in contact with a road surface. [15] A method according to any one of claims 12 to 14, further comprising: Determining (S905) the second reliability value further based on a height of the object detected by the lidar (120). [16] A method according to any one of claims 12 to 15, further comprising: Determining (S905) the second reliability value based on a width of the object detected by the lidar (120). [17] A method according to any one of claims 12 to 16, further comprising: Determining (S906) the third reliability value based on a strength of the reflected radio wave received by the radar (130). [18] A method according to any one of claims 11 to 17, further comprising: Determining whether a second object detected by the sensor (100) belongs to any pre-classified type, based on the second object not belonging to a pre-classified type, determining a second reliability value associated with information about the second object, and Controlling the vehicle (VEH) based on the second confidence value being less than a threshold to ignore the second object and maintain a current path of the vehicle (VEH). [19] A method according to any one of claims 11 to 18, further comprising: Controlling (S911) the vehicle (VEH) to avoid the object in a transverse direction based on determining that the object can be avoided in the transverse direction. [20] The method of any one of claims 11 to 19, further comprising causing the vehicle (VEH) to decelerate based on the reliability value.

Images