Apparatus for controlling autonomous vehicle and method thereof

Abstract

An apparatus for controlling an autonomous vehicle and a method thereof are presented. The invention relates to a device and a method for avoiding obstacles. The apparatus includes a sensor that detects an object external to the vehicle, a processor, and a memory that stores instructions. The apparatus determines whether the object belongs to any pre-classification type, determines a reliability value associated with information of the object based on the object not belonging to any pre-classification type, and controls the driving controller to avoid the object based on the reliability value being greater than a threshold reliability value.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to Korean Patent Application No. 10-2023-0077649 filed in the Korean Intellectual Property Office on June 16, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to an apparatus for controlling an autonomous vehicle and a method thereof, and more particularly to a technology for controlling the travel of a vehicle based on sensing an object outside the vehicle. Background Art

[0004] An autonomous vehicle may refer to a vehicle that is capable of operating on its own without the manipulation of a driver or passenger, and an automated vehicle and highway system may refer to a system that monitors and controls such an autonomous vehicle to enable it to operate on its own. In addition, technologies have been developed for monitoring the external environment of a vehicle (e.g., the exterior, surrounding environment) and operating various driving assistance features based on the monitored external environment of the vehicle.

[0005] In addition, one or more autonomous vehicles equipped with driving assistance features can detect objects by monitoring the vehicle's external environment and control based on a scenario (e.g., prediction) determined based on the detected objects. In other words, autonomous driving may require a process to determine the type of object external to the vehicle.

[0006] Objects external to the vehicle may be categorized based on predetermined categories (eg, classes, categories, etc.).

[0007] However, the types of objects that can be classified in advance based on categories are limited, and unexpected obstacles may appear on the road that do not fit any known classification. When encountering an object that has not been classified in advance, the autonomous vehicle may try to avoid the corresponding object. However, if the autonomous vehicle is designed to avoid all such unclassified objects, even when it is not necessary to avoid the object, the vehicle's travel may be disturbed and thus the traffic flow may be affected. Sudden avoidance maneuvers may also adversely affect driving safety. Summary of the invention

[0008] The present invention is made to solve the above-mentioned problems occurring in the prior art while completely maintaining the advantages achieved by the prior art.

[0009] An aspect of the present invention provides an apparatus and method for avoiding obstacles, which can effectively perform autonomous driving when an object that has not been classified in advance is sensed.

[0010] Another aspect of the present invention provides an apparatus and method for avoiding an obstacle, which can reduce unnecessary vehicle control caused by a sensor erroneously identifying an object when an object that has not been classified in advance is sensed.

[0011] The technical problems to be solved by the present invention are not limited to the above-mentioned problems, and those skilled in the art to which the present invention belongs will clearly understand any other technical problems not mentioned herein from the following description.

[0012] According to one or more exemplary embodiments of the present invention, an apparatus for controlling an autonomous vehicle may include: a sensor configured to detect an object; a processor; and a memory. The memory may store instructions that, when executed by the processor, cause the apparatus to: determine whether the object belongs to any pre-classified type; determine a reliability value associated with information about the object based on the object not belonging to any pre-classified type; and control the vehicle to avoid the object based on the reliability value being greater than a threshold reliability value.

[0013] The sensor may include at least one of a camera, a lidar, or a radar. When the instructions are executed by the processor, the device may further determine the reliability value based on at least two of the following: a first reliability value determined based on an image acquired by the camera, a second reliability value determined based on data acquired by the lidar, or a third reliability value determined based on reflected radio waves received by the radar.

[0014] The instructions, when executed by the processor, may further cause the device to determine the first reliability value further based on a height of the object determined by the camera.

[0015] The instructions, when executed by the processor, may further cause the device to determine the second reliability value further based on whether the object is in contact with a road surface.

[0016] The instructions, when executed by the processor, may further cause the device to determine the second reliability value further based on the height of the object detected by the lidar.

[0017] The instructions, when executed by the processor, may further cause the device to determine the second reliability value further based on a width of the object detected by the lidar.

[0018] The instructions, when executed by the processor, may further cause the device to determine the third reliability value further based on a power of the reflected radio waves received by the radar.

[0019] The instructions, when executed by the processor, may further cause the device to control the vehicle to ignore the object and maintain a current route of the vehicle based on the reliability value being less than a threshold.

[0020] The instructions, when executed by the processor, may cause the device to control the vehicle to avoid the object in the lateral direction based on determining that the object is avoidable in the lateral direction.

[0021] The instructions, when executed by the processor, may cause the apparatus to decelerate the vehicle based on the reliability value based on a determination that the object is not avoidable in a lateral direction.

[0022] According to one or more exemplary embodiments of the present invention, a method for controlling an autonomous vehicle may include: determining whether an object acquired by a sensor of the vehicle belongs to any preclassified type; based on the object not belonging to any preclassified type, determining a reliability value associated with information of the object; and based on the reliability value being greater than a threshold reliability value, controlling the vehicle to avoid the object.

[0023] The method may also include determining the reliability value based on at least one of: a first reliability value determined based on an image acquired by a camera of the sensor, a second reliability value determined based on data acquired by a lidar of the sensor, or a third reliability value determined based on reflected radio waves received by a radar of the sensor.

[0024] The method may further include determining the first reliability value further based on a height of the object determined by the camera.

[0025] The method may further include determining the second reliability value further based on whether the object is in contact with a road surface.

[0026] The method may further include determining the second reliability value further based on a height of the object detected by the laser radar.

[0027] The method may further include determining the second reliability value further based on a width of the object detected by the laser radar.

[0028] The method may further include determining the third reliability value further based on a power of the reflected radio wave received by the radar.

[0029] The method may also include: determining whether a second object acquired by the sensor belongs to any preclassified type; determining a second reliability value associated with information of the second object based on that the second object does not belong to any preclassified type; and controlling the vehicle to ignore the second object and maintain the current route of the vehicle based on that the second reliability value is less than a threshold.

[0030] Controlling the vehicle may include controlling the vehicle to avoid the object in the lateral direction based on determining that the object is avoidable in the lateral direction.

[0031] The method may further include decelerating the vehicle based on the reliability value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0033] Figure 1 is a block diagram showing a configuration of an apparatus for controlling an autonomous vehicle;

[0034] Figure 2 is a diagram showing a vehicle equipped with an apparatus for controlling an autonomous vehicle;

[0035] Figure 3 is a schematic diagram of an artificial neural network (ANN) used for reliability calculation;

[0036] Figure 4 is a flow chart for illustrating an obstacle avoidance method;

[0037] Figures 5 to 7 is a diagram for illustrating an exemplary method for determining a second reliability;

[0038] Figure 8 is a diagram for illustrating an example of determining whether to travel based on detection of a road marking provided on a road;

[0039] Fig. 9 is a flow chart for illustrating an exemplary obstacle avoidance method;

[0040] Fig.10 and Fig.11 is a diagram for illustrating determination of a collision position;

[0041] Fig.12 is a diagram showing lateral avoidance within a lane;

[0042] Fig.13 is a diagram for illustrating a method for determining whether lateral direction avoidance is possible via a lane change; and

[0043] Fig.14 A computing system is shown. DETAILED DESCRIPTION

[0044] Hereinafter, one or more exemplary embodiments of the present invention will be described in detail with reference to the exemplary drawings. When adding reference numerals to the components of each drawing, it should be noted that even when the same or equivalent components are shown on other drawings, the same or equivalent components are designated by the same reference numerals. In addition, when describing the exemplary embodiments of the present invention, when it is determined that the detailed description of the related known configurations or functions interferes with the understanding of the exemplary embodiments of the present invention, the detailed description of the related known configurations or functions will be omitted.

[0045] When describing components according to one or more embodiments of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are intended only to distinguish components from other components, and these terms do not limit the nature, order or sequence of the components. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by ordinary technicians in the field to which the present invention belongs. It should also be understood that terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant field and will not be interpreted in an idealized or overly formal meaning unless explicitly defined herein.

[0046] In the following, reference will be made to Figures 1 to 14 The present invention is described in detail.

[0047] Figure 1 is a block diagram showing a configuration of an apparatus for controlling an autonomous vehicle, and Figure 2 is a diagram showing a vehicle equipped with an apparatus for controlling an autonomous vehicle.

[0048] Reference Figure 1 and Figure 2 , an apparatus for controlling an autonomous vehicle may include a sensor 100 , a processor 200 , a memory 300 , and a driving controller 400 .

[0049] The sensor 100 may include at least one of a camera 110 , light imaging detection and ranging (lidar) 120 , or radio detection and ranging (radar) 130 for detecting objects external to the vehicle VEH.

[0050] The camera 110 for acquiring the external image of the vehicle VEH may acquire the image of the front area or the front side area of ​​the vehicle VEH. For example, the camera 110 may be arranged around the front windshield to acquire the image of the front area of ​​the vehicle VEH.

[0051] The laser radar 120 for emitting laser light and determining the object using a reflected wave (e.g., radio wave, radar signal, etc.) of the laser light reflected from the object can be implemented by a time of flight (TOF) scheme or a phase shift scheme. The laser radar 120 can be installed to be exposed to the outside of the vehicle and can be set around the front bumper or the front grille.

[0052] The radar 130 may include an electromagnetic wave transmitting and receiving module. In terms of the radio wave transmitting principle, the radar 130 may be implemented in a pulse radar scheme or a continuous wave radar scheme. The radar 130 may be implemented in a frequency modulated continuous wave (FMCW) scheme or a frequency shift keying (FSK) scheme based on a signal waveform in the continuous wave radar scheme. The radar 130 may include a front radar 131 located at the center of the front surface of the vehicle VEH, a front side radar 132 located at both ends of the front bumper, and a rear radar 133 located at the rear portion of the vehicle VEH.

[0053] The positions of the camera 110, the laser radar 120 and the radar 130 may not be limited to Figure 2 Position shown.

[0054] In addition to those shown in the figure, the sensor may also include an ultrasonic sensor and an infrared sensor. The ultrasonic sensor may include an ultrasonic transmitting and receiving module. The ultrasonic sensor may detect an object based on ultrasonic waves, and may detect the position of the detected object, the distance to the detected object, and the relative speed. The ultrasonic sensor may be set at an appropriate position outside the vehicle to sense an object located in front of, behind, or on the side of the vehicle. The infrared sensor may include an infrared transmitting and receiving module. The infrared sensor may detect an object based on infrared rays and detect the position of the detected object, the distance to the detected object, and the relative speed. The infrared sensor may be set outside the vehicle to detect an object located in front of, behind, or on the side of the vehicle.

[0055] In addition, the sensor 100 may further include a brake pedal position sensor (BPS) and an accelerator position sensor (APS) that generate a speed control command for shifting a vehicle.

[0056] The brake pedal position sensor may output a BPS signal based on the degree of depression of a brake pedal provided in the vehicle. As an example, the BPS signal may output data from 0 to 100 based on the degree of depression of the brake pedal, a value of 0 may be a case where the brake pedal is not depressed and a value of 100 may be a case where the brake pedal is depressed to the maximum.

[0057] The accelerator position sensor may output an APS signal based on the degree of depression of an accelerator pedal provided in the vehicle. As an example, the APS signal may output data from 0 to 100 based on the degree of depression of the accelerator pedal, a value of 0 may be a case where the accelerator pedal is not depressed and a value of 100 may be a case where the accelerator pedal is depressed to the maximum.

[0058] The processor 200 may determine objects external to the vehicle and perform avoidance driving based on the objects.

[0059] To this end, the processor 200 may determine whether the object acquired by the sensor 100 of the vehicle VEH is an obstacle of a preset type.

[0060] The preset type of obstacles may be objects observed on the road or objects that can be classified based on categories via deep learning. For example, the preset type of obstacles may include vehicles, two-wheeled vehicles, pedestrians, traffic facilities, etc.

[0061] When the object is not an obstacle, the processor 200 may determine (eg, calculate) reliability (eg, reliability value). The reliability may be used to provide a criterion for determining information about the object.

[0062] The processor 200 may obtain the final reliability based on at least one of the first reliability determined by the camera 110 , the second reliability determined by the lidar 120 , or the third reliability determined by the radar 130 .

[0063] The processor 200 may determine the object as an obstacle to be avoided based on the reliability being equal to or greater than a preset threshold. For example, the processor 200 may determine the object as an obstacle based on the final reliability being equal to or greater than a threshold.

[0064] Furthermore, the processor 200 may ignore the object and maintain the travel route based on the final reliability being less than a threshold value.

[0065] The processor 200 may control the travel of the vehicle to avoid the obstacle based on the detection of the obstacle.

[0066] The processor 200 may control the steering controller to avoid the obstacle based on determining that the obstacle can be avoided in the lateral direction.

[0067] When the obstacle cannot be avoided in the lateral direction, the processor 200 may decelerate the vehicle by controlling the brake controller. The processor 200 may decelerate the vehicle by setting the braking amount based on the value of the final reliability (eg, proportional to ).

[0068] In addition, the processor 200 may perform artificial intelligence learning on the data provided from the sensor 100 to detect the target vehicle and the dangerous vehicle. To this end, the processor 200 may include an artificial intelligence (hereinafter referred to as AI) processor. The AI ​​processor may use a pre-stored program to learn a neural network. The neural network for detecting target vehicles and dangerous vehicles may be designed to simulate the human brain structure on a computer, and may include a plurality of network nodes having weights of neurons simulating a human neural network. A plurality of network nodes may transmit and receive data based on the connection relationship therebetween to simulate the synaptic activity of neurons that transmit and receive signals via synaptic emission. The neural network may include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes may transmit and receive data based on a convolutional connection relationship when located at different layers. Examples of neural network models may include various deep learning techniques, such as a deep neural network (DNN), a convolutional deep neural network (CNN), a recursive Boltzmann machine (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), and a deep Q network.

[0069] Figure 3 is a schematic diagram of an artificial neural network (ANN) used for reliability calculation.

[0070] Reference Figure 3 , the processor 200 may use Figure 3 The artificial neural network shown determines (eg, calculates) a final reliability (eg, a reliability value, a reliability score, etc.).

[0071] The artificial neural network may be stored in the processor 200 or the memory. The artificial neural network mathematically models the connection pattern of human neurons and can include an input layer, a hidden layer, and an output layer.

[0072] The input layer may multiply the input data by the weight matrix and provide the result to the hidden layer. The input data may be a variable used to determine (e.g., calculate) reliability. For example, the input data may include an image acquired by the camera 110, data acquired by the laser radar 120, a received intensity of the radar 130, etc.

[0073] The hidden layer can process the input data based on the activation function. In addition, the hidden layer can multiply the processed input data by a new weight matrix and transmit the result to the output layer.

[0074] The output layer may output a result by reflecting an activation function on the output. The output layer may output a final reliability as an output value.

[0075] Furthermore, the processor 200 may determine whether to avoid an obstacle based on information about an object on a road surface provided from another vehicle via the communication device 500 .

[0076] The memory 300 may store algorithms and an AI processor for the operation of the processor 200. The memory 300 may use a hard disk drive, a flash memory, an electrically erasable programmable read-only memory (EEPROM), a static RAM (SRAM), a ferroelectric RAM (FRAM), a phase change RAM (PRAM), a magnetic RAM (MRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate SDRAM (DDR-SDRAM), etc.

[0077] The driving controller 400 for controlling the steering and deceleration and acceleration of the vehicle in response to a control signal from the processor 200 may include a steering controller, an engine controller, a brake controller, and a transmission control module.

[0078] The steering controller may be classified into a hydraulic power steering (HPS) system that controls steering using hydraulic pressure formed by a hydraulic pump and a motor driven power steering (hereinafter, “MDPS”) system that controls steering using output torque of an electric motor.

[0079] The engine controller is an actuator that controls the engine of the vehicle and controls the acceleration of the vehicle. The engine controller can be implemented as an engine management system (EMS). The engine controller controls the driving torque of the engine based on the accelerator pedal position information output from the accelerator position sensor. During autonomous driving, the engine controller controls the output of the engine to follow the travel speed of the vehicle requested by the processor 200.

[0080] The brake controller is an actuator that controls the deceleration of the vehicle and can be implemented as an electronic stability control (ESC). The brake controller controls the brake pressure to follow the target speed requested by the processor 200. That is, the brake controller controls the deceleration of the vehicle.

[0081] The transmission control module is an actuator for controlling a transmission of a vehicle and can be implemented as a shift-by-wire (SBW). The transmission control module controls the shifting of the vehicle based on a gear position and a gear state range.

[0082] In addition, the apparatus for controlling an autonomous vehicle may further include a communication device 500 and an alarm device 600 .

[0083] The communication device 500 may communicate with a user terminal, another vehicle, or an external server.

[0084] The communication device 500 may support short-range communication using at least one of: Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, near field communication (NFC), wireless fidelity (Wi-Fi), Wi-Fi Direct, and wireless universal serial bus (wireless USB) technology.

[0085] The communication device 500 may include a global positioning system (GPS) module or a differential global positioning system (DGPS) module for obtaining location information.

[0086] In addition, the communication device 500 may include a V2X communication module. The V2X communication module may include an RF circuit for a protocol for wireless communication with a server (vehicle to infrastructure; V2I), another vehicle (vehicle to vehicle; V2V), or a pedestrian (vehicle to pedestrian; V2P). The communication device 500 may receive sensing data acquired by a sensor of another vehicle and provide the data to the processor 200 via the V2X communication module.

[0087] The warning device 600 may notify the occupants of the vehicle of obstacles determined by the processor 200 and may notify emergency braking or avoidance situations caused by the obstacles. The warning device 600 may include a display, a speaker, and the like.

[0088] Figure 4 is a flowchart for illustrating an obstacle avoidance method. Figure 4 Can be by Figure 1 The processor controlled procedures shown.

[0089] refer to Figures 1 to 4 ,The obstacle avoidance method will be described as follows.

[0090] In S410 , when the object acquired by the sensor 100 does not have a pre-classification type (eg, a known classification), the processor 200 may determine (eg, calculate) reliability.

[0091] For example, the processor 200 may use a semantic segmentation scheme to learn the image acquired by the camera 110 and classify the objects contained in the image. Object classification may be performed via a preset category. When an object of a category other than the preset category is detected in the image, the processor 200 may determine (e.g., calculate) reliability.

[0092] The process of determining (e.g., calculating) the reliability may include a process of determining (e.g., calculating) a first reliability determined by the image acquired by the camera 110. For example, when the height of the unclassified object among the objects detected in the image is small, the processor 200 may set the first reliability to be small.

[0093] The process of determining (e.g., calculating) the reliability may include a process of determining (e.g., calculating) a second reliability determined by data acquired by the laser radar 120. For example, the processor 200 may determine that the value of the second reliability of the object in contact with the road surface is greater than the value of the second reliability of the object not in contact with the road surface. In addition, when the height of the object detected by the laser radar 120 is large, the processor 200 may set the second reliability to be large. In addition, when the width of the object detected by the laser radar 120 is large, the processor 200 may set the second reliability to be large.

[0094] The process of determining (e.g., calculating) the reliability may include a process of determining (e.g., calculating) a third reliability determined by the power of the reflected wave of the radar 130. The processor 200 may receive the power of the reflected wave received by the radar 130 and may set the third reliability based on the power of the reflected wave (e.g., in proportion to the power of the reflected wave).

[0095] The processor 200 may determine (e.g., calculate) the final reliability using at least one of the first reliability, the second reliability, or the third reliability. Each of the first reliability, the second reliability, and the third reliability may be the final reliability. In addition, in order to more accurately determine the information about the object, the processor 200 may calculate the final reliability by selecting at least two of the first reliability, the second reliability, and the third reliability.

[0096] In S420, the processor 200 may determine the object as an obstacle to be avoided based on the reliability being equal to or greater than a threshold reliability.

[0097] The processor 200 may compare the final reliability with a threshold reliability, and the threshold reliability may be preset.

[0098] In S430 , the processor 200 may control the travel of the vehicle to avoid obstacles.

[0099] To this end, the processor 200 may control the driving controller 400 to decelerate the vehicle or control the steering of the vehicle.

[0100] In the following, each procedure will be described as follows.

[0101] Table 1 is a table showing an example of determining the first reliability.

[0102]

[0103] The value of the object's height First reliability CV1 Object height H_I>Hth1 60 Object height H_I≤Hth1 20

[0104] Referring to Table 1, the processor 200 may determine the first reliability CV1 as “60” based on the image-based object height H_I exceeding a preset first threshold Hth1. The first threshold Hth1 may be determined as a level at which the vehicle VEH may travel while ignoring the object.

[0105] The image-based object height H_I may represent the height of the object calculated based on the image acquired by the camera 110. The image acquired by the camera 110 may be expressed in an image coordinate system. The image coordinate system may be a coordinate system in which the upper left corner pixel of the image is the origin (0,0), the horizontal position of the pixel in the image is represented as the "u" coordinate component, and the vertical position of the pixel is represented as the "v" coordinate component.

[0106] In the image coordinate system, the "u" coordinate component may be the x-axis direction of the world coordinate system, and the "v" coordinate component may be the y-axis direction of the world coordinate system. Assuming that the road surface is flat, the x-axis and the y-axis may refer to linear axes that determine a plane parallel to the road surface, the y-axis direction may refer to the forward direction of the vehicle, and the x-axis direction may refer to the lateral direction of the vehicle perpendicular to the y-axis. In the world coordinate system, the z-axis direction may represent the height relative to the road surface.

[0107] The processor 200 may acquire the actual height of the object based on the image coordinates of the object detected from the image. Any known technology may be used as a method for the processor 200 to acquire the actual height of the object.

[0108] In addition, referring to Table 1, the processor 200 may determine that the first reliability CV1 is "20" based on the image-based object height H_I being less than or equal to the first threshold value Hth1.

[0109] Table 2 is a table showing an example of determining the second reliability.

[0110]

[0111] Height / width values ​​and object height offset The second reliability CV2 Height_offset = 1 10 Height_offset=0,H_L≤Hth2 20 Height_offset=0,H_L>Hth2,Wd>Hth3 80

[0112] Referring to Table 2, the processor 200 may determine the second reliability CV2 based on the laser radar-based object height, the laser radar-based object width, the height offset, and the like.

[0113] The sensing data of the laser radar used to determine the second reliability will be described as follows.

[0114] Figures 5 to 7 is a diagram for illustrating an exemplary method for determining the second reliability.

[0115] Reference Figures 5 to 7, the processor 200 may acquire x / y data when the object Ob is located on the xy plane, and may determine the height and height offset of the object based on the area where the x / y data is located.

[0116] The processor 200 may determine the distance from the road surface Rs to the highest point of the object Ob as the height of the object Ob. Figure 5 As shown, when the object Ob is located close to the vehicle VEH and is large, the laser radar 120 may not detect the upper area of ​​the object Ob. Figure 5 The height of the object Ob shown may be estimated as "H1". Alternatively, Figure 6 The height of the object Ob shown may be determined as "H2", and Figure 7 The height of the illustrated object Ob may be determined as "H3".

[0117] The height offset may refer to a factor indicating whether the object Ob is in contact with the road surface Rs. When the object Ob is in contact with the road surface Rs, the height offset may be set to "0", and when the object Ob is not in contact with the road surface Rs, the height offset may be set to "1". Therefore, Figure 5 All height offsets of the object Ob detected by the laser radar 120 in can be set to "0". Therefore, Figure 5 The objects Ob and Figure 6 The height offset of the object Ob shown can be set to "0". Figure 7 The height offset of the object Ob shown may be set to "1".

[0118] When the height offset is “1”, the processor 200 may determine the second reliability CV2 to be “10”.

[0119] In addition, when the height offset is “0” and the height of the object is less than or equal to a preset second threshold value Hth2, the processor 200 may determine that the second reliability CV2 is “20”.

[0120] In addition, when the height offset is “0”, the height of the object exceeds the preset second threshold value Hth2, and the width of the object exceeds the preset third threshold value Hth3, the processor 200 may determine that the second reliability CV2 is “80”.

[0121] Table 3 is a table showing an example of determining the third reliability.

[0122]

[0123] The power value of the reflected wave The third reliability CV3 Power>Pth1 40 Pth1≥Power>Pth2 20 Pth2≥Power 0

[0124] Referring to Table 3, the processor 200 may determine the third reliability CV3 based on the power of the reflected wave of the radar.

[0125] When the power of the reflected wave of the radar exceeds the first critical power Pth1, the processor 200 may determine that the third reliability CV3 is "40".

[0126] When the power of the reflected wave of the radar is less than or equal to the first critical power Pth1 and exceeds the second critical power Pth2, the processor 200 may determine that the third reliability CV3 is "20".

[0127] When the power of the reflected wave of the radar is less than or equal to the second critical power Pth2, the processor 200 may determine that the third reliability CV3 is "0 (zero)".

[0128] The first critical power Pth1 may be set to be greater than the second critical power Pth2 .

[0129] The processor 200 may determine the final reliability CV_f based on the first reliability CV1, the second reliability CV2, and the third reliability CV3. The processor 200 may determine the final reliability CV_f as shown in Equation 1 below.

[0130] <Equation 1>

[0131] CV_f=a1×CV1+b1×CV2+c1×CV3

[0132] The final reliability CV_f may be determined by Equation 1, and the scheme of determining the final reliability CV_f may be performed in various ways. In Equation 1, a1, b1, and c1 are variables used in the process of determining the final reliability CV_f and may be adjusted by a designer.

[0133] Table 4 is a table showing a method for determining whether normal driving is possible and the amount of deceleration based on the final reliability CV_f.

[0134]

[0135] Final reliability CV_f Maximum brake control amount Is it possible to drive normally? CVth1>CV_f Out of control Driving CVth2>CV_f≥CVth1 <![CDATA[-4m / s 2 ]]> Not drivable CVth3>CV_f≥CVth2 <![CDATA[-6m / s 2 ]]> Not drivable CV_f ≥ CVth3 <![CDATA[-9.8m / s 2 ]]> Not drivable

[0136] Referring to Table 4, when the final reliability CV_f is equal to or greater than the first threshold reliability CVth1, the processor 200 may determine that normal driving is impossible. Normal driving may refer to a state of maintaining travel while maintaining the speed of a moving vehicle without lane change.

[0137] When the final reliability CV_f is equal to or greater than the first threshold reliability CVth1, the processor 200 may determine a maximum braking control amount for reducing the vehicle speed, and the value of the maximum braking control amount may be proportional to the value of the final reliability.

[0138] For example, when the final reliability CV_f is equal to or greater than the first threshold reliability CVth1 and less than the second threshold reliability CVth2, the processor 200 may determine that the maximum braking control amount is "-4 m / s 2 ”.

[0139] In addition, when the final reliability CV_f is equal to or greater than the second threshold reliability CVth2 and less than the third threshold reliability CVth3, the processor 200 may determine that the maximum braking control amount is "-6m / s 2 ”.

[0140] In addition, when the final reliability CV_f is equal to or greater than the third threshold reliability CVth3, the processor 200 may determine the maximum braking control amount "-9.8 m / s 2 ”.

[0141] The maximum braking control amount and the value of the maximum braking control amount shown in Table 4 can be set to different values.

[0142] In addition, when the final reliability CV_f is less than the first threshold reliability CVth1, the processor 200 may ignore the object acquired by the sensor 100 and maintain the traveling state.

[0143] The first threshold reliability CVth1 may be preset to “50”, the second threshold reliability CVth2 may be preset to “70”, and the third threshold reliability CVth3 may be preset to “80”, but the values ​​of the threshold reliabilities CVth1, CVth2, and CVth3 may not be limited thereto.

[0144] An example of determining whether normal driving is possible based on the final reliability CV_f will be described as follows.

[0145] like Figure 7 As shown, an example of determining whether normal driving is possible for an object that is not in contact with the road surface will be described as follows.

[0146] like Figure 7 As shown, when the object Ob is floating in the air without contacting the road surface, the processor 200 may determine the second reliability CV2 to be "10" based on Table 2.

[0147] When the object Ob is light, the power of the reflected wave may be identified as weak, and the processor 200 may determine the third reliability CV3 as "0" based on Table 3. Therefore, when the object floating above the road surface is a light object, the final reliability CV_f may be determined as "10". When the first threshold reliability CVth1 is "50", the processor 200 may maintain normal driving based on the final reliability CV_f determined as 10.

[0148] Alternatively, when the object Ob is heavy, the power of the reflected wave may be identified as strong, and the processor 200 may determine the third reliability CV3 as "40" based on Table 3. Therefore, when the object floating above the road surface is a heavy object, the final reliability CV_f may be determined as "50". When the first threshold reliability CVth1 is "50", the processor 200 may control the driving controller 400 to decelerate the vehicle or perform avoidance driving based on the final reliability CV_f determined as 50.

[0149] The value of the threshold reliability may be an example and may vary based on the design.

[0150] refer to Figure 8 , an example of determining whether to perform normal driving when a small structure is detected will be described as follows.

[0151] Figure 8 is a diagram for illustrating an example of determining whether to travel based on detection of a road marking provided on a road.

[0152] like Figure 8 As shown, since the road marker RM in the form of a Botts point does not protrude high from the road surface, the road marker RM does not have a great impact on the travel of the vehicle. However, due to the characteristics of the structure, the light reflectivity is large, so that the road marker RM may be recognized as a large object by the laser radar 120 or the radar 130.

[0153] Because the small road mark RM determined by the camera 110 has a small height, the processor 200 may determine the first reliability CV1 to be "20" based on Table 1. In addition, the processor 200 may determine the second reliability CV2 to be "20" based on Table 2. Therefore, the final reliability CV_f may be determined to be "40", and the processor 200 may maintain the normal driving state based on Table 4.

[0154] Fig. 9 is a flow chart illustrating an exemplary obstacle avoidance method. Fig. 9 Can be by Figure 1 The processor controlled procedures shown.

[0155] refer to Fig. 9 , an exemplary obstacle avoidance method will be described as follows.

[0156] In S901 , the processor 200 may detect an object in an area outside the vehicle based on sensing data acquired by the sensor 100 .

[0157] In S902, the processor 200 may determine whether the detected object has a pre-classified type. The pre-classified type of object may refer to an object that can be classified based on a category via machine learning (eg, deep learning).

[0158] In S903 , when the detected object has a pre-classified type, the processor 200 may determine (eg, calculate) a predicted collision position of the host vehicle.

[0159] The following will refer to Fig.10 and Fig.11 Methods for determining (eg, calculating) a collision location are described.

[0160] Fig.10 and Fig.11 It is a diagram for illustrating determination of a collision position.

[0161] like Fig.10 and Fig.11 As shown, the processor 200 may calculate the collision position based on the degree of overlap between the obstacle Ob2 and the vehicle VEH.

[0162] The processor 200 may calculate the ratio of the overlapping area OA between the vehicle VEH and the obstacle Ob2 relative to the width of the vehicle VEH on the x-axis as the overlapping degree. Fig.10 As shown, when the overlapping area OA of the vehicle width W1 on the x-axis and the obstacle Ob2 is equal to or greater than the vehicle width W1, the processor 200 may calculate the overlap as 100(%). Fig.11 As shown, when the overlapping area OA in which the vehicle width W1 and the obstacle Ob2 overlap on the x-axis corresponds to 1 / 5 of the vehicle width W1, the processor 200 may calculate the overlap as 20(%).

[0163] In S904, when the detected object does not have the pre-classified type, the processor 200 may calculate the first reliability, the second reliability and / or the third reliability.

[0164] For example, the processor 200 may calculate the first reliability using Table 1 based on the image acquired by the camera 110. In addition, the processor 200 may calculate the second reliability using Table 2 based on the data acquired by the laser radar 120. In addition, the processor 200 may calculate the third reliability using Table 3 based on the data acquired by the radar 130.

[0165] In S905, the processor 200 may calculate the final reliability using at least one of the first reliability, the second reliability, and the third reliability. The processor 200 may determine (eg, calculate) the final reliability using at least two of the first reliability, the second reliability, and the third reliability.

[0166] In S906 , the processor 200 may calculate the braking control amount based on the final reliability.

[0167] The processor 200 may set the value of the braking control amount based on (eg, proportional to) the value of the final reliability, as shown in Table 4.

[0168] Based on the predicted collision position calculated in S903, the processor 200 may determine in S907 whether the obstacle can be avoided in the longitudinal direction.

[0169] When a collision with an obstacle can be avoided via deceleration without controlling the steering controller, the processor 200 may determine that the obstacle can be avoided in the longitudinal direction.

[0170] In order to determine whether the obstacle can be avoided in the longitudinal direction, the processor 200 may identify the distance to the obstacle, the traveling speed of the vehicle, and the braking control amount of the vehicle.

[0171] In S908, when it is determined that the obstacle can be avoided in the longitudinal direction, the processor 200 may control the brake controller to decelerate the vehicle.

[0172] In S909, when it is determined that longitudinal collision avoidance is not possible, the processor 200 may determine whether lateral collision avoidance can be performed within the lane. Fig.12 Operation S909 is described as follows.

[0173] Fig.12 It is a diagram showing the lateral avoidance within the lane.

[0174] The processor 200 may determine an avoidance route of the vehicle VEH to avoid the overlapping area OA of the vehicle VEH and the obstacle Ob2. Even when avoiding the overlapping area OA, when a free space in which the vehicle VEH can travel is ensured within the lane L1, the processor 200 may determine that the obstacle Ob2 can be avoided in the lateral direction within the lane L1 without changing lanes.

[0175] In S910, when lateral avoidance is not possible within the lane, the processor 200 may determine whether lateral avoidance can be performed via lane change. Fig.13 Operation S910 is described as follows.

[0176] Fig.13 1 is a diagram for illustrating a method for determining whether lateral avoidance is possible via a lane change.

[0177] The processor 200 may determine an avoidance route for the vehicle VEH to avoid the overlap area OA of the vehicle VEH and the obstacle Ob2. When the overlap area OA cannot be avoided in the lane L1, the vehicle may plan a lane change to the adjacent lane L2. Fig.13As shown, when there is only one adjacent lane L2 and a collision with another vehicle VEH2 in the adjacent lane L2 is expected when the vehicle VEH as the host vehicle changes lanes, the processor 200 may determine that the obstacle Ob2 cannot be avoided via a lane change.

[0178] When it is determined in operations S909 and S910 that the obstacle Ob2 can be avoided in the lateral direction, the processor 200 may perform lateral direction avoidance driving by controlling the steering controller in operation S911.

[0179] In addition, when it is determined in S910 that lateral direction avoidance is impossible, the processor 200 may perform longitudinal direction control via braking. In S910, the case where lateral direction avoidance is impossible may be a case where the obstacle Ob2 cannot be avoided in both the longitudinal direction and the lateral direction. When avoidance of the obstacle Ob2 is impossible and another vehicle is traveling in an adjacent lane, the processor 200 may decelerate the vehicle while selecting a collision with the obstacle Ob2.

[0180] When entering operation S908 after operation S910, the processor 200 may brake the vehicle using the maximum braking amount of the vehicle regardless of the final reliability.

[0181] Fig.14 A computing system is shown.

[0182] Reference Fig.14 , the computing system 1000 may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage device 1600 , and a network interface 1700 connected via a bus 1200 .

[0183] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that performs processing on commands stored in the memory 1300 and / or the storage device 1600. The memory 1300 and the storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) and a RAM (Random Access Memory).

[0184] Therefore, the operations of the methods or algorithms described herein may be embodied directly in hardware or in a software module executed by the processor 1100, or in a combination thereof. The software module may reside on a storage medium (i.e., the memory 1300 and / or the storage device 1600) such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, and CD-ROM.

[0185] An exemplary storage medium is coupled to the processor 1100, which can read information from the storage medium and write information to the storage medium. In another method, the storage medium can be integrated with the processor 1100. The processor and the storage medium can reside in an application specific integrated circuit (ASIC). The ASIC can reside in a user terminal. In another method, the processor and the storage medium can reside in a user terminal as separate components.

[0186] The above description is merely an illustration of the technical idea of ​​the present invention, and those skilled in the art may make various modifications and changes without departing from the essential characteristics of the present invention.

[0187] Therefore, one or more exemplary embodiments disclosed in the present invention are not intended to limit the technical concept of the present invention, but to illustrate the present invention, and the scope of the technical concept of the present invention is not limited by the exemplary embodiments. The scope of the present invention should be interpreted as being covered by the scope of the attached claims, and all technical concepts falling within the scope of the claims should be interpreted as being included in the scope of the present invention.

[0188] Even when a sensed object is not classified in advance, information about the object may be more accurately recognized based on the characteristics of the sensor to reduce unnecessary vehicle control.

[0189] Furthermore, when a specific sensor erroneously recognizes an object depending on the object, the reliability of information recognized by the corresponding sensor may be set low to more accurately determine the object and control the vehicle based on the reliability.

[0190] Furthermore, various effects directly or indirectly identified via this document may be provided.

[0191] Although the present invention has been described above with reference to the exemplary embodiments and the accompanying drawings, the present invention is not limited thereto but may be variously modified and changed by those skilled in the art without departing from the spirit and scope of the invention as claimed in the appended claims.

Claims

1. A device for controlling an autonomous vehicle, the device include: a sensor configured to detect an object; processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to: determining whether the object belongs to any of the pre-classified categories; determining a reliability value associated with the information of the object based on the object not belonging to any of the pre-classified categories; as well as Based on the reliability value being greater than a threshold reliability value, the vehicle is controlled to avoid the object.

2. The device of claim 1, wherein the sensor comprises at least one of a camera, a lidar, or a radar, The instructions, when executed by the processor, also cause the device to determine the reliability value based on at least two of the following: a first reliability value determined based on an image acquired by the camera, a second reliability value determined based on data acquired by the lidar, or a third reliability value determined based on reflected radio waves received by the radar. 3 . The apparatus of claim 2 , wherein the instructions, when executed by the processor, further cause the apparatus to determine the first reliability value based on a height of the object determined by the camera. 4 . The apparatus of claim 2 , wherein the instructions, when executed by the processor, further cause the apparatus to determine the second reliability value based on whether the object is in contact with a road surface.

5. A device according to claim 2, wherein the instructions, when executed by the processor, also cause the device to determine the second reliability value based on the height of the object detected by the lidar.

6. A device according to claim 2, wherein the instructions, when executed by the processor, also cause the device to determine the second reliability value based on the width of the object detected by the lidar. 7 . The apparatus of claim 2 , wherein the instructions, when executed by the processor, further cause the apparatus to determine the third reliability value based on a power of the reflected radio waves received by the radar. 8 . The device of claim 2 , wherein the instructions, when executed by the processor, further cause the device to control the vehicle to ignore the object and maintain a current route of the vehicle based on the reliability value being less than a threshold. 9 . The device of claim 1 , wherein the instructions, when executed by the processor, cause the device to control the vehicle to avoid the object in a lateral direction based on determining that the object is avoidable in the lateral direction.

10. The apparatus of claim 1, wherein the instructions, when executed by the processor, cause the apparatus to decelerate the vehicle based on the reliability value based on a determination that the object is not avoidable in a lateral direction.

11. A method for controlling an autonomous vehicle, the method include: determining whether an object acquired by a sensor of the vehicle belongs to any pre-classified type; determining a reliability value associated with the information of the object based on the object not belonging to any of the pre-classified categories; as well as Based on the reliability value being greater than a threshold reliability value, the vehicle is controlled to avoid the object.

12. The method according to claim 11, further comprising: include: The reliability value is determined based on at least one of: a first reliability value determined based on an image acquired by a camera of the sensor, a second reliability value determined based on data acquired by a lidar of the sensor, or a third reliability value determined based on reflected radio waves received by a radar of the sensor.

13. The method according to claim 12, further comprising: include: The first reliability value is also determined based on a height of the object determined by the camera.

14. The method according to claim 12, further comprising: include: The second reliability value is also determined based on whether the object is in contact with a road surface.

15. The method according to claim 12, further comprising: include: The second reliability value is also determined based on the height of the object detected by the laser radar.

16. The method according to claim 12, further comprising: include: The second reliability value is also determined based on a width of the object detected by the laser radar.

17. The method according to claim 12, further comprising: include: The third reliability value is also determined based on the power of the reflected radio waves received by the radar.

18. The method according to claim 11, further comprising: include: determining whether a second object acquired by the sensor belongs to any pre-classified type; determining a second reliability value associated with the information of the second object based on the second object not belonging to any pre-classified type; as well as Based on the second reliability value being less than a threshold, the vehicle is controlled to ignore the second object and maintain a current route of the vehicle.

19. The method of claim 11, wherein the control of the vehicle include: The vehicle is controlled to avoid the object in the lateral direction based on determining that the object is avoidable in the lateral direction.

20. The method according to claim 11, further comprising: include: The vehicle is decelerated based on the reliability value.

Images