Vehicle and Its Control Method

Abstract

The present disclosure relates to a vehicle and a control method thereof, which can effectively perform automatic parking driving by forming an occupancy map and a probability map using a camera and an ultrasonic sensor. The vehicle includes: a camera disposed on the vehicle, having a plurality of channels, and configured to obtain an image around the vehicle; a sensing device including an ultrasonic sensor and configured to obtain distance information between an object and the vehicle; and a controller configured to match the distance information with the image around the vehicle, divide the image around the vehicle into a plurality of regions, determine the risk level of each of the plurality of regions by matching the objects included in the plurality of regions to a predetermined category, and form a probability map and an occupancy map corresponding to the image around the vehicle based on the risk level.

Description

[0001] Cross - reference to related applications

[0002] This application claims priority to Korean Patent Application No. 10 - 2019 - 0166112, filed with the Korean Intellectual Property Office on December 12, 2019, the disclosure of which is incorporated herein by reference in its entirety. Technical field

[0003] The present disclosure relates to a vehicle for recognizing an image around a vehicle and a control method thereof. Background art

[0004] The autonomous driving technology of a vehicle is a technology that enables a vehicle to grasp road conditions and drive autonomously even when a driver does not control a brake, a steering wheel, or an accelerator pedal.

[0005] Autonomous driving technology is a core technology for realizing intelligent vehicles. Autonomous driving technology may include highway driving assistance (HDA, a technology for automatically maintaining a vehicle distance), blind spot detection (BSD, a technology for detecting surrounding vehicles during reverse driving and issuing an alarm), automatic emergency braking (AEB, a technology for activating a braking system when a vehicle cannot recognize a preceding vehicle), lane departure warning system (LDWS), lane - keeping assist system (LKAS, a technology for compensating for lane departure without a turn signal), advanced intelligent cruise control (ASCC, a technology for maintaining a constant vehicle distance at a set speed and driving at a constant speed), traffic jam assist (TJA), parking collision avoidance assist (PCA), and remote intelligent parking assist (RSPA).

[0006] In particular, the RSPA system uses only ultrasonic sensors to recognize a parking space, and thus can perform automatic parking only when the vehicle is in the vicinity by generating a control trajectory.

[0007] In order to improve the integrity of a parking space or a parking layout without a vehicle, an identification system that recognizes a lane type outside a vehicle and transmits the lane type to a control system is required. Summary of the invention

[0008] One aspect of the present disclosure provides a vehicle and a control method thereof, which can effectively perform automatic parking driving by forming an occupancy map and a probability map using a camera and an ultrasonic sensor. As used herein, parking driving may refer to driving a vehicle to park or attempt to park the vehicle in a parking lot or a parking area.

[0009] Other aspects of the present disclosure are partially set forth in the following description and partially will be apparent from the description or can be learned by practicing the present disclosure.

[0010] According to one aspect of the present disclosure, a vehicle includes: a camera disposed on the vehicle, having a plurality of channels, and configured to obtain an image around the vehicle; a sensing device including an ultrasonic sensor and configured to obtain distance information between an object and the vehicle; and a controller configured to match the distance information with the image around the vehicle, divide the image around the vehicle into a plurality of regions, determine the risk of each of the plurality of regions by matching the objects included in the plurality of regions to a predetermined category, and form a probability map and an occupancy map corresponding to the image around the vehicle based on the risk.

[0011] The controller may be configured to assign weights to the objects included in the plurality of regions and determine the risk based on the weight values.

[0012] The controller may be configured to determine a risk probability corresponding to each region based on the relative risks between the plurality of regions and form a probability map based on the risk probability.

[0013] The controller may be configured to determine the risk probability based on the signal obtained by the ultrasonic sensor when the distance between the vehicle and the object is less than a predetermined distance.

[0014] The controller may be configured to determine an update period of the ultrasonic sensor signal based on the position information of the object.

[0015] The controller may be configured to match the risk probability to the image around the vehicle by matching the risk probability with a predetermined scale.

[0016] The controller may be configured to guide a moving path of the vehicle based on the probability map.

[0017] The controller may be configured to form a top view image using the occupancy map and the probability map, the occupancy map and the probability map being formed corresponding to the image around the vehicle obtained from each of the plurality of channels of the camera.

[0018] According to another aspect of the present disclosure, a method for controlling a vehicle includes: a camera having a plurality of channels obtains an image around the vehicle; a sensing device including an ultrasonic sensor obtains distance information between an object and the vehicle; a controller matches the distance information with the image around the vehicle; the controller divides the image around the vehicle into a plurality of regions; the controller determines the risk of each of the plurality of regions by matching the objects included in the plurality of regions to a predetermined category; and the controller forms a probability map and an occupancy map corresponding to the image around the vehicle based on the risk.

[0019] Determining the risk of each of the plurality of regions may include: assigning weights to the objects included in the plurality of regions; and determining the risk based on the weight values.

[0020] Forming a probability map may include: determining a risk probability corresponding to each region based on a relative risk between a plurality of regions; and forming a probability map based on the risk probability.

[0021] Determining the risk probability may include: when a distance between a vehicle and an object is less than a predetermined distance, determining the risk probability based on a signal obtained by an ultrasonic sensor.

[0022] The method may further include: a controller determining an update period of the ultrasonic sensor signal based on position information of the object.

[0023] The method may further include: the controller matching the risk probability to an image around the vehicle by matching the risk probability with a predetermined scale.

[0024] The method may further include: the controller guiding a movement path of the vehicle based on the probability map.

[0025] The method may further include: the controller forming a top view image using an occupancy map and a probability map, the occupancy map and the probability map being formed corresponding to an image around the vehicle obtained from each of a plurality of channels of a camera. Description of the Drawings

[0026] These and / or other aspects of the present disclosure will become apparent and be more readily understood from the following description of embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 is a control block diagram according to an embodiment.

[0028] Figure 2 is a diagram showing a relationship between an image around a vehicle, an occupancy map, and a probability map obtained from each channel of a camera according to an embodiment.

[0029] Figure 3 is a diagram showing an image around a vehicle and a plurality of regions according to an embodiment.

[0030] Figure 4 is a diagram for describing a relationship between a predetermined category and an image around a vehicle according to an embodiment.

[0031] Figure 5 is a diagram for describing an operation of determining an update period of an ultrasonic sensor signal based on position information of an object according to an embodiment.

[0032] Figure 6 is a diagram for describing a scale representing a risk probability according to an embodiment.

[0033] Figure 7It is a diagram showing the formation of an occupancy map according to an embodiment.

[0034] Figure 8 It is a diagram showing the formation of a top - view image from an occupancy map obtained from multiple channels of a camera according to an embodiment.

[0035] Figure 9 It is a flowchart according to an embodiment. Detailed Description

[0036] Throughout the specification, the same reference numerals denote the same or equivalent elements. Not all elements of the embodiments of the present disclosure are described, but the description of elements known in the art or elements overlapping in the embodiments is omitted. Terms such as "~ component", "~ module", "~ member", "~ block" used throughout the specification can be implemented in software and / or hardware, and multiple "~ components", "~ modules", "~ members" or "~ blocks" can be implemented in a single element, or a single "~ component", "~ module", "~ member" or "~ block" can include multiple elements. When a component, module, member, block, component, device, element, etc. of the present disclosure is described as having a purpose or performing an operation, function, etc., the component, module, member, block, component, device or element should be regarded as being "configured to" achieve that purpose or perform that operation, function in this document. Further, the controller described in this document may include a processor programmed to execute the operations, functions, calculations, etc.

[0037] It should be further understood that the term "connected" and its derivatives refer to both direct connection and indirect connection, and the indirect connection includes connection through a wireless communication network. The terms "comprising (or comprising of)" and "including (or including of)" are inclusive or open - ended and do not exclude additional, unlisted elements or method steps unless otherwise stated. It should be further understood that the term "member" and its derivatives refer to both the case where a member contacts another member and the case where there is another member between these two members. It should be understood that although terms such as "first", "second", "third" etc. may be used in this document to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections are not limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section.

[0038] It should be understood that the singular forms "a", "an", "the" include plural references unless the context clearly dictates otherwise. The reference numerals used for method steps are only for convenience of explanation and do not limit the order of the steps. Therefore, unless the context clearly dictates otherwise, they can be implemented in other writing orders.

[0039] Hereinafter, the operating principle and embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0040] Figure 1 is a control block diagram according to an embodiment.

[0041] Referring to Figure 1 , the vehicle 1 according to an embodiment may include a camera 300, a sensing device 100, a display 400, and a controller 200.

[0042] The camera 300 has a plurality of channels and can obtain an image around the vehicle 1, for example, an image of the surrounding environment of the vehicle.

[0043] The camera 300 installed in the vehicle 1 may include a charge-coupled device (CCD) camera or a CMOS color image sensor. Here, both CCD and CMOS refer to sensors that convert the light received through the lens of the camera into an electrical signal and store the electrical signal.

[0044] The sensing device 100 may include an ultrasonic sensor.

[0045] The ultrasonic sensor may adopt a method of emitting ultrasonic waves and using the ultrasonic waves reflected from an obstacle to detect the distance to the obstacle.

[0046] The sensing device 100 can obtain the distance information between the vehicle 1 and the obstacles provided around the vehicle 1.

[0047] The display 400 may be set as an instrument panel provided in the vehicle 1 or a display device provided in the center instrument panel.

[0048] The display 400 may include a cathode ray tube (CRT), a digital light processing (DLP) panel, a plasma display panel (PDP), a liquid crystal display (LCD) panel, an electroluminescent (EL) panel, an electrophoretic display (EPD) panel, an electrochromic display (ECD) panel, a light emitting diode (LED) panel, or an organic light emitting diode (OLED) panel, but is not limited thereto.

[0049] The controller 200 can match the distance information with the image around the vehicle 1 and can divide the image around the vehicle 1 into a plurality of regions.

[0050] The plurality of regions may cover the regions where obstacles exist in the image around the vehicle 1.

[0051] The controller 200 can determine the risk level of each of the plurality of regions by matching the objects included in the plurality of regions to a predetermined category.

[0052] The predetermined category may refer to the type of the object included in each region. This will be described in detail in the corresponding drawings.

[0053] The controller 200 may form a probability map and an occupancy map corresponding to an image around the vehicle 1 based on the degree of danger.

[0054] The controller 200 may assign weights to objects included in multiple regions and determine the degree of danger based on the weight values.

[0055] Weights may be assigned according to the above categories.

[0056] The controller 200 may determine the degree of danger corresponding to each object by summing the weight values corresponding to each object.

[0057] The controller 200 may determine the danger probability corresponding to each region based on the relative degree of danger between multiple regions.

[0058] The degree of danger is a value determined based on the weights corresponding to each region, while the danger probability is a relative concept of the degree of danger of each region.

[0059] For example, a danger probability of 0% may be assigned to a free space where there is no object in the image of the vehicle 1. This will be described in detail later.

[0060] When the distance between the vehicle 1 and an object is less than a predetermined distance, the controller 200 may determine the danger probability based on a signal obtained by an ultrasonic sensor.

[0061] The distance at which the ultrasonic signal can be obtained is limited. According to an embodiment, when the object is within 3 m of the vehicle, the controller 200 may simultaneously utilize the image around the vehicle 1 and the ultrasonic sensor to obtain information.

[0062] The controller 200 may determine the update period of the ultrasonic sensor signal based on the position information of the object.

[0063] In particular, when it is determined that the object is located in the corresponding part, the controller 200 may determine that the update period of the information around the object is shorter than the update period of the information farther from the object.

[0064] The controller 200 may match the danger probability to the image around the vehicle 1 by matching the danger probability with a predetermined scale. As described below, the scale may be determined using a predetermined gray scale.

[0065] The controller 200 may guide the movement path of the vehicle 1 based on the probability map. Since the probability map includes information on objects that may collide with the vehicle 1, the controller 200 may guide the vehicle 1 based on the information on the objects to prevent a collision with the objects.

[0066] The controller 200 can form a top - view image by using an occupancy map and a probability map, which are formed corresponding to the images around the vehicle 1 obtained from each of the multiple channels of the camera 300.

[0067] In addition, the controller 200 can output the top - view image to the above - mentioned display 400.

[0068] The controller 200 can be implemented by using a memory and a processor. The memory stores algorithms for controlling the operations of components in the vehicle 1 or data related to programs for reproducing the algorithms, and the processor performs the above - mentioned operations by using the data stored in the memory. The memory and the processor can be implemented on separate chips. Optionally, the memory and the processor can be implemented on a single chip.

[0069] It can be added or removed at least one component corresponding to Figure 1 the performance of the components of the vehicle 1 shown. It should be easily understood by those skilled in the art that the relative positions of the components can be changed corresponding to the performance or structure of the vehicle 1.

[0070] Meanwhile, Figure 1 each of the components shown can be regarded as a hardware component such as a field - programmable gate array (FPGA) and an application - specific integrated circuit (ASIC) and / or software.

[0071] Figure 2 is a diagram showing the relationship between the image V2 around the vehicle 1, the occupancy map O2, and the probability map P2 obtained from each channel of the camera 300 according to an embodiment.

[0072] Referring to Figure 2 , the image obtained by the camera 300 can be classified into objects and roads and displayed.

[0073] When identifying object information, the controller 200 can execute a pre - learned semantic segmentation algorithm by receiving a 4 - channel image input from the camera 300.

[0074] The controller 200 can identify the free space and obstacle information around the vehicle 1.

[0075] In addition, based on the obtained information, the controller 200 can judge whether an object occupies space and can form an occupancy map O2 based on this.

[0076] A probability map P2 can also be formed based on the relative risk probability of the object.

[0077] When forming the occupancy map O2 and the probability map P2, the images formed by each channel of the camera 300 can be synthesized. Hereinafter, the operations of forming the occupancy map O2 and the probability map P2 will be described in detail step by step.

[0078] Figure 3 is a diagram showing an image around a vehicle and multiple regions according to an embodiment.

[0079] The controller 200 may divide the image obtained by the camera 300 into predetermined regions.

[0080] According to an embodiment, if there are obstacles in the image around the vehicle 1 up to the L3 region, the predetermined regions may be assigned to the images up to the corresponding regions. According to an embodiment, the controller 200 may assign regions for the image recognition results based on the actual distance.

[0081] Regions may be assigned by setting one region to 20 cm × 20 cm.

[0082] Figure 4 is a diagram for describing the relationship between a predetermined category according to an embodiment and the image around the vehicle 1.

[0083] Referring to Figure 4 , shows Figure 3 the image around the vehicle 1 described above matching the predetermined regions.

[0084] On the other hand, the predetermined category according to an embodiment may be determined as an empty space E41, a space E42 such as a stop line through which the vehicle 1 can pass, a space E43 such as a blocking member and a curb through which the vehicle 1 can pass when necessary, and a space E44 such as a pillar, an obstacle, another vehicle, or other objects through which the vehicle 1 cannot pass.

[0085] The controller 200 may assign a weight value of 0 to the category corresponding to the space through which the vehicle 1 can pass.

[0086] When necessary, the controller 200 may assign a low weight to the space through which the vehicle 1 can pass.

[0087] On the other hand, a high weight may be assigned to the space through which the vehicle 1 cannot pass.

[0088] Meanwhile, the controller 200 may determine the degree of danger by summing the weights of each category.

[0089] For example, in the case of E44, the object occupying the corresponding region may be determined as another vehicle. Since another vehicle corresponds to the space through which the vehicle 1 cannot pass, the controller 200 may determine the degree of danger by assigning a high weight and summing all the weights of the corresponding region.

[0090] The controller 200 may determine that the degree of danger of E44 is high. In addition, when determining the degree of danger, the larger the region occupied by the category with a large weight, the higher the degree of danger that can be determined.

[0091] On the other hand, in the case of E41, since the corresponding area is an empty space corresponding to the area where the vehicle 1 can move, the controller 200 can assign a weight of 0 and determine that the risk level is low.

[0092] At the same time, the controller 200 can determine the risk probability corresponding to each area based on the relative risk levels between multiple areas. In particular, since the risk level of E44 is higher than that of E41, the controller 200 can determine that the risk probability is high.

[0093] The controller 200 can determine a probability map based on the risk probability determined through such operations.

[0094] Figure 5 It is a diagram for describing the operation of determining the update period of the ultrasonic sensor signal based on the position information of an object according to an embodiment.

[0095] Referring to Figure 5 , P5 can refer to the position where an object exists obtained by the vehicle 1.

[0096] The controller 200 can determine the update period of the ultrasonic sensor signal based on the position information of the object.

[0097] Specifically, since the surrounding area P5-1 where the object exists has a high collision probability, the update period of the ultrasonic sensor signal can be determined to be short.

[0098] On the other hand, since the nearby area P5-2 where the object exists has a low collision probability, the update period of the ultrasonic sensor signal can be determined to be long.

[0099] In other words, the controller 200 can obtain a large amount of information by determining that the update period of the area close to the object is short, and can obtain a small amount of information by determining that the update period of the area far from the object is long. Through this operation, the controller 200 can perform effective information management.

[0100] Figure 6 It is a diagram for describing a scale representing the risk probability according to an embodiment.

[0101] The scale can be set using a predetermined gray scale GS6.

[0102] When the risk probability of the corresponding area is high, the controller 200 can form a probability map by displaying the corresponding surrounding image with a scale close to black.

[0103] On the other hand, when the risk probability of the corresponding area is low, the controller 200 can form a probability map by displaying the corresponding surrounding image with a scale close to white.

[0104] Meanwhile, Figure 6 The scale mentioned above is only an embodiment of the present disclosure, and the display form of the scale is not limited.

[0105] Figure 7 is a diagram showing the formation of an occupancy map according to an embodiment, Figure 8 is a diagram showing the formation of a top view image from the occupancy maps obtained from multiple channels of the camera 300 according to an embodiment.

[0106] The controller 200 can form an occupancy map by using the risk level of each area. When the risk level of the corresponding area is 0, the controller 200 can determine that the corresponding area is not occupied and display the corresponding area as 0. In Figure 7 this case, since the Z71 area corresponds to an empty space, the controller 200 can assign 0 to the corresponding area.

[0107] On the other hand, when the risk level of the corresponding area is not 0, the controller 200 can determine that the corresponding area is an occupied area Z72 and display the corresponding area as 1.

[0108] Based on this, the controller 200 can generate an occupancy map by using the unoccupied areas as movable areas.

[0109] Figure 8 Shows that according to an embodiment, a top view image T8 can be formed from the occupancy map O8 obtained from multiple channels of the camera 300.

[0110] Based on the above operations, the controller 200 can form an occupancy map O8 based on the images obtained by the camera 300.

[0111] According to an embodiment, since the camera 300 can include four channels, an occupancy map O8 can be formed according to each image obtained from each channel.

[0112] The controller 200 can synthesize each occupancy map formed in this way to form a top view image T8.

[0113] On the other hand, in the top view image formed by the controller 200, it can be determined that there are no obstacles in the Z81 area, so the controller 200 can avoid collisions by guiding the vehicle 1 to drive to the corresponding area.

[0114] In addition, the controller 200 can output the top view image formed by such operations to the display 400.

[0115] Figure 9 is a flowchart according to an embodiment.

[0116] Referring to Figure 9, the controller 200 can obtain the image and distance information around the vehicle 1 (1001).

[0117] In addition, the controller 200 can divide the corresponding image into regions of a predetermined size (1002).

[0118] The controller 200 can determine the degree of danger and the probability of danger of the corresponding region based on the information obtained by the ultrasonic sensor and the image around the vehicle 1 obtained by the camera 300 (1003).

[0119] In addition, an occupancy map can be formed based on the degree of danger, and a probability map can be formed based on the probability of danger (1004).

[0120] According to an embodiment of the present disclosure, a vehicle and its control method can use a camera and an ultrasonic sensor to form an occupancy map and a probability map, and thus can effectively perform automatic parking driving.

[0121] The disclosed embodiment can be implemented in the form of a recording medium storing computer-executable instructions executable by a processor. The instructions can be stored in the form of program code, and when executed by the processor, the instructions can generate program modules to perform the operations of the disclosed embodiment. The recording medium can be implemented as a non-transitory computer-readable recording medium.

[0122] The non-transitory computer-readable recording medium can include all kinds of recording media storing commands that can be interpreted by a computer. For example, the non-transitory computer-readable recording medium can be, for example, ROM, RAM, magnetic tape, magnetic disk, flash memory, optical data storage device, etc.

[0123] So far, the embodiments of the present disclosure have been described with reference to the accompanying drawings. It should be apparent to those skilled in the art that the present disclosure can be implemented in other forms than the above embodiments without changing the technical idea or essential features of the present disclosure. The above embodiments are only examples and should not be construed in a limiting sense.

Claims

1. A vehicle, comprising: a camera disposed on the vehicle, having a plurality of channels, and obtaining an image around the vehicle; a sensing device including an ultrasonic sensor and obtaining distance information between an object and the vehicle; and a controller that matches the distance information with the image around the vehicle, divides the image around the vehicle into a plurality of regions, determines the risk level of each of the plurality of regions by matching the object included in the plurality of regions to a predetermined category, and forms a probability map and an occupancy map corresponding to the image around the vehicle based on the risk level.

2. The vehicle according to claim 1, wherein the controller assigns weights to the objects included in the plurality of regions and determines the risk level based on the weight values.

3. The vehicle according to claim 2, wherein the controller determines a risk probability corresponding to each region based on the relative risk levels between the plurality of regions and forms the probability map based on the risk probability.

4. The vehicle according to claim 3, wherein when the distance between the vehicle and the object is less than a predetermined distance, the controller determines the risk probability based on the signal obtained by the ultrasonic sensor.

5. The vehicle according to claim 4, wherein the controller determines an update period of the ultrasonic sensor signal based on the position information of the object.

6. The vehicle according to claim 3, wherein the controller matches the risk probability to the image around the vehicle by matching the risk probability with a predetermined scale.

7. The vehicle according to claim 3, wherein the controller guides a moving path of the vehicle based on the probability map.

8. The vehicle according to claim 1, wherein the controller forms a top view image using the occupancy map and the probability map, and the occupancy map and the probability map are formed corresponding to the image around the vehicle obtained from each of the plurality of channels of the camera.

9. A control method for a vehicle, comprising: a camera having a plurality of channels obtaining an image around the vehicle; a sensing device including an ultrasonic sensor obtaining distance information between an object and the vehicle; a controller matching the distance information with the image around the vehicle; the controller dividing the image around the vehicle into a plurality of regions; the controller determining the risk level of each of the plurality of regions by matching the object included in the plurality of regions to a predetermined category; and the controller forming a probability map and an occupancy map corresponding to the image around the vehicle based on the risk level.

10. The method according to claim 9, wherein determining the risk level of each of the plurality of regions includes: assigning weights to the objects included in the plurality of regions; and determining the risk level based on the weight values.

11. The method according to claim 10, wherein forming the probability map includes: determining a risk probability corresponding to each region based on the relative risk levels between the plurality of regions; and Form the probability map based on the risk probability.

12. The method according to claim 11, wherein, determining the risk probability includes: when the distance between the vehicle and the object is less than a predetermined distance, determining the risk probability based on the signal obtained by the ultrasonic sensor.

13. The method according to claim 12, further including: The controller determines an update period of the ultrasonic sensor signal based on the position information of the object.

14. The method according to claim 11, further including: The controller matches the risk probability to an image around the vehicle by matching the risk probability with a predetermined scale.

15. The method according to claim 11, further including: The controller guides a moving path of the vehicle based on the probability map.

16. The method according to claim 9, further including: The controller forms a top view image using the occupancy map and the probability map, and the occupancy map and the probability map are formed corresponding to the images around the vehicle obtained from each of the multiple channels of the camera.

Images