A real-time AVM system for vehicles that reflects vehicle operation information and method thereof

Abstract

The vehicle AVM (Around View Monitoring) system includes: an AVM operation information storage unit that stores AVM operation information including pre-set video display information for at least one predicted event expected to occur while the vehicle is in operation or stopped; a video collection unit that collects real-time video of the area around a first vehicle while it is in operation or stopped; a video processing unit that generates real-time AVM video from the collected video in response to an actual event occurring at the first vehicle; and a display unit that displays the real-time AVM video. The video processing unit detects video display information corresponding to the actual event from the AVM operation information storage unit and generates the real-time AVM video based on that video display information, thereby selectively providing information necessary for safe operation that reflects the vehicle's operation information, and thus appropriately providing the driver with the information they need, which has the advantage of improving the driver's cognitive ability and effectively preventing accidents.

Description

Technical Field

[0001] The present invention relates to a vehicle AVM (Around View Monitoring) system.

Background Art

[0002] The AVM system is a technology for monitoring the surrounding situation during the running of an automobile, and combines the images collected through a predetermined number of cameras to provide an image of the surrounding of the vehicle. For this purpose, the AVM system usually includes four cameras installed in the front, rear, left, and right of the vehicle, and a display installed inside the vehicle, combines the images collected through the cameras to generate a 360-degree omnidirectional image, and then displays it on the display.

[0003] [[ID=十六]]Thus, the AVM system can greatly assist in preventing accidents that may occur during parking and driving by enabling the driver to accurately grasp the surrounding situation of the vehicle. Not only that, recently, there is a trend that even autonomous driving vehicles actively utilize such an AVM system.

[0004] In relation to this, Patent Document 1: Korean Patent No. 10-1366112 discloses an AVM (Around View Monitoring) system for a vehicle, including a first camera network constructed by some of the cameras constituting the AVM system, a second camera network constructed by some other cameras constituting the AVM system, and a processor that generates a first image from the images generated by the cameras constituting the first camera network and generates a second image from the images generated by the cameras constituting the second camera network. The cameras constituting the first camera network and the cameras constituting the second camera network automatically vary according to the traveling direction of the vehicle, and the number of cameras in the first camera network and the number of cameras in the second camera network maintain the original number. An AVM system is disclosed.

[0005] According to the aforementioned patent document, even when installing more cameras on special vehicles such as buses and trucks than in the AVM system of general vehicles to eliminate blind spots due to vehicle characteristics, smooth networking is possible and video delay does not occur, which has the advantage of preventing accidents caused by blind spots and video delay in advance.

[0006] However, conventional technologies do not provide AVM video services that take vehicle operation information into account. Therefore, they cannot adequately provide necessary information to drivers of vehicles where the vehicle's operating status (e.g., stopping, departing, etc.) changes constantly, and where the area the driver should focus on varies depending on the vehicle's operating status, such as buses with a large number of passengers boarding and alighting. Consequently, accident prevention has been limited.

[0007] In other words, conventionally, the entire image synthesized using a predetermined number of cameras is displayed around the vehicle, or the image information collected by each of the cameras is displayed collectively using multiple divided regions. This forces the driver to selectively check the area of ​​the overall image that should be focused on depending on the driving situation. However, in the former case, there are technically blind spots, and in the latter case, providing multiple divided regions simultaneously impairs cognitive ability and can actually cause accidents. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Korean Patent Publication No. 10-1366112 [Overview of the project] [Problems that the invention aims to solve]

[0009] Therefore, in order to solve the aforementioned problems, the present invention aims to provide a real-time AVM system and method for vehicles that can appropriately provide necessary information to the driver by reflecting vehicle operation information and selectively providing information necessary for safe operation, thereby improving the driver's cognitive ability and effectively preventing accidents.

[0010] Furthermore, the present invention aims to provide a real-time AVM system and method for vehicles that can automatically select and display information about areas that the driver should focus on, by pre-setting video display information for each predicted event that may occur while the vehicle is in operation or stopped, and then automatically switching the display screen of the real-time AVM video each time an actual event occurs based on the video display information, thereby effectively preventing accidents caused by driver error.

[0011] Furthermore, the present invention aims to provide a real-time AVM system and method for vehicles that includes at least one additional camera that collects additional images under separate operating conditions, and that enhances monitoring of areas with a high risk of accidents by preferentially displaying the additional images collected from the additional camera, or composite images with the additional images, thereby enabling concentrated prevention of accidents under specific conditions with a high risk of accidents. [Means for solving the problem]

[0012] To achieve the above objective, the present invention provides a real-time AVM system for vehicles, which is an AVM (Around View Monitoring) system for vehicles, comprising: an AVM operation information storage unit that stores AVM operation information including pre-set video display information for at least one predicted event expected to occur while the vehicle is in operation or stopped; a video collection unit that collects real-time video of the area around a first vehicle while it is in operation or stopped; a video processing unit that generates real-time AVM video from the collected video in response to an actual event occurring in the first vehicle; and a display unit that displays the real-time AVM video, wherein the video processing unit detects video display information corresponding to the actual event from the AVM operation information storage unit and generates the real-time AVM video based on the video display information.

[0013] Preferably, the video display information may include screen division information of the display unit, which is set to be different for each predicted event in order to display the real-time AVM video, and video matching information, which matches the video to be displayed on each of the divided screens divided based on the division information.

[0014] Preferably, the AVM operation information further includes camera drive information matched with information for at least one AVM camera to be driven for each predicted event, and the video acquisition unit can detect the camera drive information corresponding to the actual event from the AVM operation information storage unit and drive the AVM camera based on the camera drive information to collect real-time video of the area around the first vehicle.

[0015] Preferably, the vehicle AVM system further includes a first additional camera that selectively operates only when a preset first operating condition is met and collects a first additional image, which is a real-time additional image of the right front of the first vehicle, and the display unit can switch the display screen to display the first additional image according to preset first additional image display information in response to the operation of the first additional camera.

[0016] Preferably, the vehicle AVM system further includes an object detection unit that senses the presence or absence of an object located to the right front of the first vehicle and the direction of movement of the object, and when the object detection unit detects an object approaching toward the right front of the first vehicle, it can generate a first additional event to activate the first additional camera.

[0017] Preferably, the vehicle AVM system further includes a second additional camera that selectively operates only when a preset second operating condition is met and collects a second additional image, which is a real-time additional image of the interior of the first vehicle, and the display unit can switch the display screen to display the second additional image according to preset second additional image display information in response to the operation of the second additional camera.

[0018] On the other hand, in order to achieve the above objective, the present invention provides an AVM (Around View Monitoring) method using a vehicle AVM (Around View Monitoring) system, comprising: an AVM operation information storage step in which the vehicle AVM (Around View Monitoring) system stores AVM operation information including pre-set video display information for at least one predicted event that is expected to occur while the vehicle is in operation or stopped; a video collection step in which real-time video of the area around a first vehicle is collected while the vehicle is in operation or stopped; a video processing step in which real-time AVM video is generated from the collected video in response to an actual event that occurred in the first vehicle; and a video display step in which the real-time AVM video is displayed, wherein the video processing step is characterized in that, after detecting video display information corresponding to the actual event from the AVM operation information, the real-time AVM video is generated based on the video display information.

[0019] Preferably, the video display information may include screen division information of the display unit, which is set to be different for each predicted event in order to display the real-time AVM video, and video matching information, which matches the video to be displayed on each of the divided screens divided based on the division information.

[0020] Preferably, the AVM operation information further includes camera drive information matched with information about at least one AVM camera to be driven for each predicted event, and the video acquisition step can detect the camera drive information corresponding to the actual event from the AVM operation information, and then drive the AVM camera based on that camera drive information to acquire real-time video of the area around the first vehicle.

[0021] Preferably, the video acquisition step further includes a first additional video acquisition step, which acquires a first additional video, which is a real-time additional video of the right front of the first vehicle, when a preset first operating condition is met, and the video display step can automatically switch the display screen of the display unit to display the first additional video according to preset first additional video display information in response to the acquisition of the first additional video.

[0022] Preferably, the first additional video acquisition step further includes a monitoring step of centrally monitoring the right front of the first vehicle, and if the first additional video acquisition step detects an object approaching the right front of the first vehicle as a result of the monitoring, the first additional video can be acquired.

[0023] Preferably, the video acquisition step further includes a second additional video acquisition step in which, when a preset second operating condition is met, a second additional video is acquired, which is a real-time additional video of the interior of the first vehicle, and the video display step can automatically switch the display screen of the display unit to display the second additional video according to preset second additional video display information in response to the acquisition of the second additional video. [Effects of the Invention]

[0024] The real-time AVM system and method for vehicles of the present invention as described above can reflect the driving information of the vehicle and selectively provide the information necessary for safe driving, thereby appropriately providing the information necessary for the driver, and thereby having the advantage of improving the driver's cognitive ability and effectively preventing accidents.

[0025] Further, the present invention presetts video display information for each predicted event that may occur during or while the vehicle is stopped, and then automatically switches the display screen of the real-time AVM video every time an actual event occurs based on the video display information, so that the information about the area that the driver should focus on can be automatically selected and presented, and thereby having the advantage of effectively preventing accidents caused by driver errors.

[0026] Further, the present invention includes at least one additional camera that collects additional videos under separate operating conditions, and by preferentially presenting the additional videos collected from the additional camera or the composite video with the additional videos, the monitoring of areas with high accident risks can be strengthened, and thereby having the advantage of intensively preventing accidents caused by specific conditions with high accident risks.

Brief Description of the Drawings

[0027] [Figure 1] It is a schematic block diagram of a real-time AVM system for vehicles according to an embodiment of the present invention. [Figure 2a] It is a diagram for explaining the installation position and operation information of an AVM camera according to an embodiment of the present invention. [Figure 2b] It is a diagram for explaining the installation position and operation information of an AVM camera according to an embodiment of the present invention. [Figure 3] It is a diagram for explaining the attachment position of a sensor unit for sensing things approaching a vehicle equipped with a real-time AVM system for vehicles according to an embodiment of the present invention. [Figure 4a]This figure illustrates the principle of detecting the approach of an object using a sensor attached to a real-time AVM system for vehicles according to one embodiment of the present invention. [Figure 4b] This figure illustrates the principle of detecting the approach of an object using a sensor attached to a real-time AVM system for vehicles according to one embodiment of the present invention. [Figure 5] This figure illustrates AVM operation information pre-configured for each predicted event according to one embodiment of the present invention. [Figure 6a] This figure illustrates the display screen of a real-time AVM system for vehicles according to one embodiment of the present invention. [Figure 6b] This figure illustrates the display screen of a real-time AVM system for vehicles according to one embodiment of the present invention. [Figure 6c] This figure illustrates the display screen of a real-time AVM system for vehicles according to one embodiment of the present invention. [Figure 6d] This figure illustrates the display screen of a real-time AVM system for vehicles according to one embodiment of the present invention. [Figure 7] This is a flowchart illustrating the general processing of a real-time AVM method according to one embodiment of the present invention. [Figure 8] This is a flowchart illustrating the general processing of a real-time AVM method according to one embodiment of the present invention. [Modes for carrying out the invention]

[0028] In the following description of embodiments of the present invention with reference to the accompanying drawings, the descriptions will be detailed in such a way that a person with ordinary skill in the art to which the present invention pertains can easily implement the present invention. However, the present invention can be embodied in a variety of different forms and is not limited to the embodiments described herein. On the other hand, in order to clearly illustrate the present invention, parts that are not relevant to the description have been omitted from the drawings, and similar parts throughout the specification have been given similar reference numerals. Furthermore, descriptions of parts that can be easily understood by a person skilled in the art have been omitted.

[0029] Throughout the specification and claims, when a part includes a component, this means, unless otherwise stated, that it does not exclude other components, but rather that it may further include other components.

[0030] Figure 1 is a schematic block diagram of a real-time AVM system for vehicles according to one embodiment of the present invention. Referring to Figure 1, the real-time AVM system 100 for vehicles according to one embodiment of the present invention includes an AVM camera 110, an additional camera 120, an image acquisition unit 130, an object detection unit 140, an event acquisition unit 150, an AVM operation information storage unit 160, an image processing unit 170, a display unit 180, and a control unit 190.

[0031] The AVM camera 110 is installed inside / outside the vehicle and captures real-time video of the area around the vehicle. To this end, the AVM camera 110 includes a front camera 111 that captures the area in front of the vehicle, a rear camera 112 that captures the area behind the vehicle, a right-side camera 113 that captures the area to the right of the vehicle, and a left-side camera 114 that captures the area to the left of the vehicle.

[0032] Each of the cameras 111 to 114 is driven when the vehicle is in motion or stopped, or in response to an operation command transmitted from an external source, and can capture real-time video of the area around the vehicle depending on its installation position.

[0033] Here, each of the cameras 111 to 114 can be driven simultaneously and collectively to generate real-time 360° video of the area around the vehicle, or can be driven selectively according to pre-set operation information (i.e., camera drive information) for each type of event predicted to occur on the vehicle (i.e., predicted event). To this end, the control unit 190 can either drive all cameras 111 to 114 simultaneously when the operation of the vehicle real-time AVM system 100 starts, or detect camera drive information corresponding to an actual event by matching an event that actually occurred while the vehicle was running or stopped (i.e., an actual event) with the predicted event, and then selectively drive a camera based on the result.

[0034] Here, the predicted events may include the operation of the left / right turn signal lights, the opening and closing of the vehicle's entrance / exit gates, the operation of at least one switch installed in the driver's seat, and the operation of the exit signal bell.

[0035] For example, when the vehicle is traveling at a constant speed, the control unit 190 operates all cameras 111 to 114 collectively and provides the driver with a top-view image generated using the information collected at that time. When the vehicle's right turn signal light illuminates, i.e., when a right turn signal light event occurs, the control unit 190 can selectively operate cameras 111 to 114 according to pre-set operation information. If the camera drive information corresponding to the right turn signal light event includes only the right-side camera 113, the control unit 190 can operate only the right-side camera 113, thereby controlling cameras 111 to 114 and peripheral devices (e.g., the display unit 180) to intensively collect and provide only real-time images of the right side of the vehicle.

[0036] The additional camera 120 operates under separate operating conditions that differ from the operating information of the AVM camera 110, and captures additional images. In other words, the additional camera 120 is a camera installed to intensively prevent accidents under specific conditions where the risk of accidents is high. Normally it operates in standby mode, but when a specific event occurs in the vehicle (for example, activation of the right turn signal or opening of an entrance / exit gate) or when a pre-set separate operating condition is met (for example, a request from the driver or detection of an object or person approaching the vehicle), it activates and captures the corresponding area.

[0037] Here, the specific conditions under which the risk of accident is high may include when the vehicle is turning right, when a person or object is detected around the vehicle, when a person or object is detected moving in the direction of the vehicle, when passengers are boarding / alighting a route bus, or when a dangerous situation occurs inside the vehicle.

[0038] The additional camera 120 may include a right-front camera 121 that photographs the right front of the vehicle and an interior camera 122 that photographs the interior of the vehicle. The additional images collected by the additional camera 120 may be provided to the driver independently or combined with images collected by the AVM camera 110.

[0039] The right-front camera 121 is a camera designed to focus on capturing images of the right front of the vehicle and can operate in special situations where enhanced monitoring of the right front is required. For example, if the control unit 190 detects an event that activates the vehicle's right turn signal, if the vehicle's steering wheel rotates to the right by a predetermined angle or more, or if it detects an object or person approaching the right front of the vehicle, it can activate the right-front camera 121 to enhance monitoring of the right front.

[0040] This is to reduce the risk of accidents caused by the fact that, due to the characteristics of a vehicle where the driver's seat is located on the left front of the vehicle, it is difficult for the driver to secure a field of view to the right front, making it difficult to perceive the surrounding situation to the right front of the vehicle, and also because there is a blind spot in the right front corner of the vehicle that is difficult to capture with only the front camera 111 and the right-side camera 113. Therefore, when a rightward rotation is detected by a specific event that occurs in the vehicle (for example, the illumination of the right turn signal, the angle of rotation of the steering wheel to the right, etc.), or when a person or object is detected in the right direction of travel of the vehicle or around the right front of the vehicle by a sensing device installed outside the vehicle, the control unit 190 can activate the right-side front camera 121, so that the driver can concentrate on monitoring the right front of the vehicle, thereby eliminating the blind spot to the right front of the vehicle and reducing the risk of accidents.

[0041] In particular, in traffic systems where drivers of vehicles traveling to the right turn right at an intersection without receiving a separate signal, this system can prevent safety accidents that occur when drivers of vehicles turning right are unable to recognize pedestrians.

[0042] Thus, the present invention includes a right-front camera 121 that operates according to separate operating conditions, and by operating the right-front camera 121 when those conditions are met, it is possible to prevent accidents that occur when the driver has difficulty recognizing people or objects located in the right direction of travel of the vehicle or in the right front area of ​​the vehicle.

[0043] On the other hand, in the case of vehicles where the driver's seat is located on the right front, by installing an additional camera on the left front (not the right front), monitoring of the left front can be enhanced in special situations where enhanced monitoring of the left front is necessary (for example, when turning left).

[0044] The interior camera 122 is a camera designed to intensively photograph the interior of the vehicle and can operate in special situations where enhanced monitoring of the vehicle's interior (i.e., interior) is required. For example, if the control unit 190 detects an event that causes the vehicle's entrance / exit gate to open or a request signal from the driver, it can activate the interior camera 122 to enhance monitoring of the vehicle's interior. For this purpose, at least one interior camera 122 may be installed inside the vehicle and distributed throughout the vehicle to photograph the entire interior. Furthermore, operation keys for operating such interior cameras 122 may be included around the driver's seat.

[0045] As a result, when the aforementioned entrance / exit opening event occurs, or when a driver who determines that it is necessary to monitor the interior of the vehicle operates the operation key, the interior camera 122 will activate to photograph the interior of the vehicle and provide that information to the driver in real time.

[0046] This is to prevent safety accidents that may occur when a driver is unable to accurately grasp the boarding / alighting status of passengers or to recognize people or objects located around the vehicle's stopping position, in the case of vehicles that repeatedly stop and depart to assist an unspecified number of passengers boarding / alighting, such as route buses that operate periodically on predetermined routes.

[0047] In other words, the operation of the interior camera 122 is determined in accordance with the opening and closing signal of the entrance gate, and it can operate when the entrance gate is open to collect surrounding images. Alternatively, when an entrance gate opening signal is generated, the interior camera 122 operates together with the right-side camera 113, and by combining the images collected by each, the surrounding conditions inside and outside the vehicle entrance gate can be accurately conveyed to the driver.

[0048] On the other hand, the right-side front camera 121 and the interior camera 122 can be operated by the control unit 190, or by the control of the video acquisition unit 130 which is operated by the control unit 190.

[0049] Figures 2a and 2b are diagrams illustrating the installation position and operation information of an AVM camera according to one embodiment of the present invention. Figure 2a illustrates the installation position and shooting area of ​​the AVM camera 110 and the additional camera 120, respectively. Figure 2b illustrates a shooting area that reflects vehicle operation information, and Figure 2b illustrates a monitoring area determined by vehicle operation information.

[0050] Referring to Figure 2a, it can be seen that the vehicle real-time AVM system 100 of the present invention includes a front camera 111 (shooting area 111A) installed on the front of the vehicle 10, a rear camera 112 (shooting area omitted) installed on the rear of the vehicle 10, a right-side camera 113 (shooting area 113A) installed on the right side of the vehicle 10, a left-side camera 114 (shooting area 114A) installed on the left side of the vehicle 10, a right-front camera 121 (shooting area 121A) installed on the right front of the vehicle 10 (i.e., at the right front vertex position), and an interior camera 122 (shooting area 122A) installed on the left interior side of the vehicle 10.

[0051] The vehicle real-time AVM system 100 of the present invention can determine a monitoring area that reflects the operation information of the vehicle 10 by operating each of the cameras 111 to 114, 121, and 122 installed as illustrated in Figure 2a, either collectively or selectively. Referring to Figure 2b, when turning right, the system can control the system to concentrate on monitoring the first area S1 by driving the front camera 111, the right-side camera 113, and the front right-side camera 121; when stopped, the system can control the system to concentrate on monitoring the second area S2 by driving the right-side camera 113 and the interior camera 122; and when turning left, the system can control the system to concentrate on monitoring the third area S3 by driving the front camera 111 and the left-side camera 114.

[0052] Thus, the vehicle real-time AVM system 100 of the present invention can capture images of corresponding areas 111A to 114A, 121A, and 122A collectively or selectively, depending on the operating state of the vehicle 10, by operating cameras 111 to 114, 121, and 122 collectively or selectively, and by selectively generating and providing real-time AVM images of the area around the vehicle 10 using this, it is possible to improve the driver's cognitive ability and effectively prevent accidents.

[0053] The video acquisition unit 130 collects real-time video of the area around the vehicle 10 from at least one of the AVM camera 110 and the additional camera 120. To this end, the video acquisition unit 130 can be controlled by the control unit 190 and collects real-time video of a corresponding area from at least one of the front camera 111, rear camera 112, right-side camera 113, left-side camera 114, right-front camera 121, and interior camera 122 installed on the vehicle 10 while it is in motion or stationary. In particular, the video acquisition unit 130 can collect real-time video from the right-front camera 121 and the interior camera 122 only when the pre-set operating conditions for each of the right-front camera 121 and the interior camera 122 are met. For example, the video acquisition unit 130 can detect camera drive information corresponding to an actual event from the AVM operation information storage unit 160 (described later), and then drive the AVM cameras 111 to 114 and additional cameras 121 and 122 based on that camera drive information to collect real-time video of the area around the corresponding vehicle 10.

[0054] The object detection unit 140 senses the presence or absence of objects or people located around the vehicle and their direction of movement, and generates additional events based on the results. For example, the object detection unit 140 can sense objects or people located to the right front of the vehicle, where the driver's field of view is difficult to secure, and their direction of movement, and based on the results, it can generate an additional event to activate the right front camera 121 and transmit it to the control unit 190. For this purpose, the object detection unit 140 can be implemented by multiple ultrasonic sensors or multiple LiDAR sensors, and is preferably installed on the right front of the vehicle.

[0055] Figure 3 is a diagram illustrating the attachment position of a sensor unit for detecting objects approaching a vehicle equipped with a real-time AVM system for vehicles according to one embodiment of the present invention, showing an example in which an object detection unit 140, composed of multiple sensors 141, is installed adjacent to the right front of the vehicle 10 (i.e., a bus). Figures 4a and 4b are diagrams illustrating the principle of detecting the approach of objects using a sensor unit attached to a real-time AVM system for vehicles according to one embodiment of the present invention, with Figure 4a illustrating a detection area 141A formed in front of a vehicle rotating to the right with a sensor 141 attached to the right front, and Figure 4b illustrating the principle of determining whether or not an object is approaching by detecting a pedestrian and the direction of travel of the pedestrian in the detection area 141A.

[0056] Referring to Figures 3, 4a, and 4b, each of the multiple sensors 141 installed adjacent to each other on the right front of the vehicle outputs sensing signals L1 and L2 (for example, an ultrasonic signal and a laser pulse) in a linear direction to detect objects or people approaching the vehicle 10, and outputs the sensing signal in a direction in which the angle widens as the distance from the vehicle 10 increases. This is to secure a sensing area for detecting the object or person, and if the angle is narrow, the sensing distance is long but the sensing area around the vehicle becomes narrow, and conversely, if the angle is narrow, the sensing distance is short but the sensing area around the vehicle becomes wide. Therefore, it is preferable to determine the installation interval of the multiple sensors 141 considering the height of the vehicle, the width of the vehicle, etc.

[0057] Referring to Figure 4b, each of the sensing areas 141A is formed by multiple sensing signals L1 and L2 in the manner illustrated on the left side of the figure, and as shown on the right side of the figure, the direction of travel of a pedestrian can be determined by the order in which the pedestrian comes into contact with the multiple sensing signals L1 and L2. For example, if a pedestrian passes through the first sensing signal L1 and then comes into contact with the second sensing signal L2, the pedestrian's direction of travel is from the right side to the left side of the figure, and if it is the other way around, the pedestrian's direction of travel is also reversed. If a pedestrian comes into contact with only one of the multiple sensing signals L1 and L2, it can be determined that the pedestrian is not approaching the vehicle 10. On the other hand, whether or not the pedestrian has come into contact with the multiple sensing signals L1 and L2 can be determined by whether or not the phenomenon of each of the multiple sensing signals L1 and L2 being interrupted occurs.

[0058] Through such operation, the object detection unit 140, which senses the presence and direction of movement of objects or people located around the vehicle, generates the additional event and may further include means for generating a warning sound to alert pedestrians of danger (e.g., a buzzer). Alternatively, the vehicle real-time AVM system 100 of the present invention may further include sound output means (e.g., a speaker) (not shown), and the control unit 190 may output a warning sound by controlling the sound output means in response to the additional event.

[0059] The event collection unit 150 collects actual events occurring in the vehicle 10 in real time and transmits them to the control unit 190. To this end, the event collection unit 150 can collect information from the electronic control unit that controls the operation of the vehicle 10, or it can collect the actual events in real time using signals sensed via a separate sensor unit.

[0060] The AVM operation information storage unit 160 stores pre-configured operation information for the vehicle's real-time AVM system 100. Specifically, the AVM operation information storage unit 160 can store AVM operation information, including pre-configured video display information and camera drive information, for each of at least one predicted event that is expected to occur while the vehicle is running or stopped.

[0061] Here, the video display information may include screen division information of the display unit, which is set to be different for each predicted event in order to display real-time AVM video, and video matching information, which matches the video to be displayed on each of the divided screens divided based on the division information. On the other hand, the camera drive information may include information about at least one AVM camera to be driven for each predicted event.

[0062] Figure 5 is a diagram illustrating AVM operation information pre-set for each predicted event according to one embodiment of the present invention, and illustrates AVM operation information stored in the AVM operation information storage unit 160.

[0063] Referring to Figure 5, the AVM operation information may include fields for event type 161, operating camera 162, and display image 163. The event type 161 field stores the predicted events that may occur, the operating camera 162 field stores information about the camera that should selectively operate in response to the predicted event, and the display image 163 field may store information about the area that should be intensively monitored in response to the predicted event in order to improve the driver's cognitive ability. Here, the types of predicted events and the corresponding operating camera and display image information are illustrated in Figure 5. On the other hand, Figure 5 is merely an example of the AVM operation information, and the types of predicted events and the corresponding AVM operation information are not limited to those illustrated in Figure 5.

[0064] The video processing unit 170 generates real-time AVM video from video collected via the video collection unit 130 in response to actual events occurring in the vehicle 10. To this end, the video processing unit 170 can detect video display information corresponding to the actual event from the AVM operation information storage unit 160 and generate the real-time AVM video based on that video display information. For example, if the actual event collected via the event collection unit 150 is a disembarkation signal bell, the video processing unit 170 can detect AVM operation information (i.e., camera operation information and video display information) corresponding to the disembarkation signal bell from the AVM operation information storage unit 160, as illustrated in Figure 5, and then, based on the results, generate real-time AVM video that displays the interior of the vehicle by collecting video captured by cameras installed inside the vehicle.

[0065] Here, the video processing unit 170, in generating real-time AVM video using a pre-set video processing algorithm, can process the video collected from at least one camera that operates selectively according to the AVM operation information to be divided and displayed, or it can combine them.

[0066] The display unit 180 displays real-time AVM video generated by the video processing unit 170. To this end, the display unit 180 operates under the control of the control unit 190, and its display screen can be configured differently depending on the type of actual event. That is, the display unit 180 can configure and display the screen in a way that enhances the driver's cognitive ability depending on the type of actual event that may occur while the vehicle is moving or stopped. To this end, the display unit 180 displays the real-time AVM video based on video display information set so that the number of divided screens, the size and position of each divided screen differ depending on the type of actual event, and operates under the control of the control unit 190, which controls the operation of the display unit 180 based on video display information stored in the AVM operation information storage unit 160.

[0067] On the other hand, the video display information may be pre-configured to differ according to the type of predicted event and stored in the AVM operation information storage unit 160.

[0068] Furthermore, the display unit 180 can automatically switch the display screen to show additional video footage collected via the additional camera 120, in response to the operation of the additional camera 120. For this purpose, the video display information can further store additional video display information for automatically switching the screen depending on the type of additional video footage collected via the additional camera 120.

[0069] For example, when the display unit 180 automatically switches the display screen to display the right front image of the vehicle in response to the operation of the right front camera 121, if a pedestrian approaches the right front of the vehicle, the display unit 180 can automatically switch the screen so that the entire screen displays the right front image of the vehicle, switch the display state to display the right front image in a pop-up form while maintaining the current screen, or switch the display state to replace a part of the current screen with the right front image.

[0070] On the other hand, when the display unit 180 automatically switches the display screen to display the interior image of the vehicle in response to the operation of the interior camera 122, in particular, when the vehicle's entrance / exit gate closes or the vehicle enters a departure waiting state, the display unit 180 can automatically switch the screen so that the entire screen displays the images from the interior camera 122 and the right-side camera 113, or a composite image thereof, in order to inform the passengers whether there are passengers boarding or alighting, or it can switch the display state to display a real-time image including the interior image around the entrance / exit gate as a pop-up while maintaining the current screen, or it can switch the display state to replace a part of the current screen with the interior image.

[0071] For this purpose, video display information for displaying the right front image and video display information for displaying real-time video including the indoor image around the entrance / exit must be pre-registered in the AVM operation information storage unit 160, and the control unit 190 can control the operation of the display unit 180 based on the information stored in the AVM operation information storage unit 160.

[0072] Figures 6a to 6d illustrate the display screens of a real-time AVM system for vehicles according to one embodiment of the present invention. Figures 6a and 6b illustrate screens A and B, which are displayed while the vehicle is in motion. Figure 6c illustrates screen C, which displays images of the surroundings of a stationary vehicle. Figure 6d illustrates screen D, which displays both a top-view image and an interior image of a moving vehicle.

[0073] In the example shown in Figure 6a, the display unit 180 divides the display screen A into four split screens A1, A2, A3, and A4, displaying a 360° top view image on the first split screen A1, a forward view image on the second split screen A2, a left-side view image on the third split screen A3, and a right-side view image on the fourth split screen A4.

[0074] In the example shown in Figure 6b, the display unit 180 divides the display screen B into three split screens B1, B2, and B3, with the first split screen B1 displaying the front view, the second split screen B2 displaying the left view, and the third split screen B3 displaying the right view.

[0075] In the example shown in Figure 6c, the display unit 180 divides the display screen C into five split screens C1, C2, C3, C4, and C5. The first split screen C1 displays a 360° top view of the stationary vehicle, the second split screen C2 displays the left front view of the stationary vehicle, the third split screen C3 displays the right front view of the stationary vehicle, the fourth split screen C4 displays the left rear view of the stationary vehicle, and the fifth split screen C5 displays the right rear view of the stationary vehicle.

[0076] In the example shown in Figure 6d, the display unit 180 divides the display screen D into two split screens D1 and D2, with the first split screen D1 displaying a 360° top-view image of the moving vehicle and the second split screen D2 displaying an interior image.

[0077] Thus, the present invention divides the display screen of the display unit 180 into various forms based on pre-set video display information, and by using information that is set to differ depending on the type of actual event occurring in a moving or stationary vehicle, the display screen can be operated in a way that enhances the driver's cognitive ability.

[0078] Furthermore, the present invention further includes additional video display information for displaying video footage captured in accordance with specific conditions where the risk of accidents is high, and by automatically switching the screen of the display unit 180 accordingly, the driver's awareness of dangerous situations can be further improved.

[0079] The control unit 190 controls the overall operation of the vehicle real-time AVM system 100 based on a preset AVM system control algorithm. Specifically, the control unit 190 controls the operation of the AVM camera 110, additional camera 120, image acquisition unit 130, object detection unit 140, event acquisition unit 150, AVM operation information storage unit 160, image processing unit 170, and display unit 180 based on the AVM system control algorithm.

[0080] Figures 7 and 8 are flowcharts illustrating the general processing for a real-time AVM method according to one embodiment of the present invention. Referring to Figures 1 to 8, the real-time AVM method according to one embodiment of the present invention will be described as follows.

[0081] First, in step S105, the vehicle real-time AVM system 100 stores pre-configured AVM operation information for each of at least one predicted event that is expected to occur while the vehicle is running or stopped. To this end, the vehicle real-time AVM system 100 can receive the AVM operation information via a user interface (not shown), via a communication network, or store it offline via a separate storage medium on which the AVM operation information has already been stored.

[0082] Here, the AVM operation information may include the predicted event-specific camera drive information and the video display information. The camera drive information includes information about at least one AVM camera to be driven for each predicted event (e.g., camera identification information or camera installation location information), and the video display information may include video processing information for generating the real-time AVM video (e.g., video synthesis / splitting algorithm), screen splitting information for the display unit 180 which is set differently for each predicted event in order to display the real-time AVM video, and video matching information which matches the video to be displayed on each of the split screens divided based on the splitting information.

[0083] In stage S110, the video acquisition unit 130 collects real-time video of the area around the first vehicle while it is in motion or stopped. Since stage S110 is the stage before an event (i.e., an actual event) occurs on the first vehicle, real-time video can be collected from each of the cameras 111, 114, 121, and 122 included in the vehicle real-time AVM system 100, based on the system's initial setup information.

[0084] In step S115, the control unit 190 checks whether an event (i.e., an actual event) has occurred in the first vehicle. This is done in order to generate AVM video while taking into account the vehicle's operational information.

[0085] In other words, if the confirmation in step S115 indicates that the actual event did not occur, the control unit 190 generates AVM video based on the real-time video collected in step S110, controls the video processing unit 170 to generate initial AVM video in step S120, and controls the display unit 180 to display the initial AVM video in step S125. Here, initial AVM video refers to AVM video generated without considering vehicle operation information (i.e., the occurrence of the actual event), and may mean top-view video provided by conventional AVM systems.

[0086] On the other hand, if the confirmation in step S115 indicates that the actual event has occurred, the control unit 190 can detect AVM operation information corresponding to the actual event and control the video acquisition unit 130, the video processing unit 170, and the display unit 180 based on that information. That is, the control unit 190 can collect video corresponding to the actual event and then control the video acquisition unit 130, the video processing unit 170, and the display unit 180 to generate and display real-time AVM video from the collected video.

[0087] To this end, in step S130, the control unit 190 first checks whether camera drive information corresponding to the actual event is stored in the AVM operation information. If camera drive information corresponding to the actual event is stored, in steps S135 and S140, the control unit 190 controls the video acquisition unit 130 to select and drive the AVM camera corresponding to the actual event and then collect the selected video. That is, in step S140, the video acquisition unit 130 can collect real-time video of the region from at least one camera 111 to 114, 121, and 122 driven by the control unit 190, i.e., real-time video of the area around the first vehicle corresponding to each of the driven cameras.

[0088] If, as a result of the check in step S130, camera drive information corresponding to the actual event is not saved, steps S135 and S140 may be omitted.

[0089] In step S145, the control unit 190 detects video display information corresponding to the actual event from the event-specific video display information included in the AVM operation information. In steps S150 and S155, the control unit 190 controls the video processing unit 170 and the display unit 180 to generate and display AVM video (i.e., event-specific AVM video) that takes the actual event into account based on the detected video display information.

[0090] In step S160, the control unit 190 collects and processes additional video according to pre-set separate operating conditions, and in step S170, it controls the video collection unit 130 and the display unit 180 to automatically switch the screen in order to display the collected additional video.

[0091] To this end, the control unit 190 can store in advance the operating conditions for each of the additional cameras 120, and when a situation that satisfies those operating conditions occurs, it can operate the corresponding additional camera 120 to collect the relevant video footage. For example, the control unit 190 can store in advance specific conditions that indicate a high risk of accidents, and when a driving situation that satisfies those specific conditions occurs, it can operate the additional camera 120 to collect video footage of the corresponding area.

[0092] Here, the specific conditions under which the risk of accident is high may include when the vehicle is turning right, when a person or object is detected around the vehicle, when a person or object is detected moving in the direction of the vehicle, when passengers are boarding / alighting a route bus, or when a dangerous situation occurs inside the vehicle.

[0093] In other words, the control unit 190 can store in advance a first operating condition for operating the right front camera 121, and in steps S162 and S164, if a situation occurs that satisfies the first operating condition, it can control the right front camera 121 and the image acquisition unit 130 to collect additional images of the right front. Here, the first operating condition may include the detection of an event that causes the right turn signal light to be activated on the vehicle, the vehicle's steering wheel being rotated to the right by a predetermined angle or more, and the detection of an object or person approaching the right front of the vehicle. Therefore, if it is determined in step S162 that the first operating condition is met, the control unit 190 can activate the right front camera 121 in step S164 to enhance monitoring of the right front. To this end, the control unit 190 can detect if there is an object or person approaching the right front of the first vehicle by controlling the object detection unit 140 to centrally monitor the right front of the first vehicle.

[0094] Furthermore, the control unit 190 can store in advance a second operating condition for operating the interior camera 122, and in steps S166 and S168, if a situation arises that satisfies the second operating condition, it can control the interior camera 122 and the video acquisition unit 130 to collect additional video of the vehicle's interior. Here, the second operating condition may include the detection of an event that opens the vehicle's entrance / exit gate, or the detection of a request signal from the driver. Therefore, if it is determined in step S166 that the second operating condition is met, the control unit 190 can activate the interior camera 122 in step S168 to enhance the monitoring of the vehicle's interior.

[0095] In step S170, the control unit 190 can automatically switch the display screen of the display unit 180 to display the additional video, and can automatically switch the display screen based on information set to differ depending on the type of actual event.

[0096] For example, the control unit 190 can store in advance the degree of danger or urgency depending on the type of actual event, and when automatically switching the display screen of the display unit 180 based on that degree of danger or urgency, it can automatically switch the screen so that the entire screen of the display unit 180 displays the additional video, switch the display state so that the additional video is displayed in a pop-up form while maintaining the current screen, or switch the display state so that a part of the current screen is replaced with the additional video.

[0097] Thus, the real-time AVM system and method for vehicles of the present invention reflect vehicle operation information and selectively provide information necessary for safe operation, thereby appropriately providing the driver with the necessary information, and thereby improving the driver's cognitive abilities and effectively preventing accidents.

[0098] Furthermore, the present invention has the advantage of automatically selecting and displaying information about areas that the driver should focus on, by pre-setting video display information for each predicted event that may occur while the vehicle is in operation or stopped, and then automatically switching the display screen of the real-time AVM video each time an actual event occurs based on the video display information. This effectively prevents accidents caused by driver error.

[0099] Furthermore, the present invention includes at least one additional camera that collects additional images under separate operating conditions, and by preferentially displaying the additional images collected by the additional camera, or images combined with the additional images, it is possible to enhance monitoring of areas with a high risk of accidents, thereby enabling concentrated prevention of accidents under specific conditions that pose a high risk of accidents.

[0100] Although embodiments of the present invention have been described above, the scope of the present invention is not limited thereto and includes all changes and modifications that are readily apparent to a person with ordinary skill in the art to which the present invention pertains and are deemed equivalent thereto. [Explanation of Symbols]

[0101] 100 Vehicle AVM System 110 AVM Camera 111 Front Camera 112 Rear camera 113 Right-side camera 114 Left-side camera 120 additional cameras 121 Right front camera 122 Indoor Camera 130 Video Collection Department 140 Object sensing part 150 Event Collection Department 160 AVM operation information storage section 170 Video Processing Section 180 Display section 190 Control Unit

Claims

1. A vehicle AVM (Around View Monitoring) system, AVM operation information storage unit that stores AVM operation information including pre-configured video display information for each of at least one predicted events that are expected to occur while the vehicle is in operation or stopped, A video collection unit that collects real-time video footage of the area around the first vehicle while it is in operation or stopped, A video processing unit that generates real-time AVM video from the collected video in response to an actual event occurring in the first vehicle, It includes a display unit that displays the real-time AVM video, The video processing unit detects video display information corresponding to the actual event from the AVM operation information storage unit and generates the real-time AVM video based on that video display information. A real-time AVM system for vehicles, characterized by the following features.

2. The aforementioned video display information is, In order to display the real-time AVM video, the screen division information of the display unit is set differently for each predicted event, Includes video matching information which matches the video to be displayed on each of the split screens divided based on the aforementioned division information. The vehicle real-time AVM system according to claim 1.

3. The AVM operation information further includes camera drive information obtained by matching information for at least one AVM camera to be driven according to the predicted event, The video acquisition unit detects the camera drive information corresponding to the actual event from the AVM operation information storage unit, and drives the AVM camera based on the camera drive information to acquire real-time video of the area around the first vehicle. The vehicle real-time AVM system according to claim 2.

4. The system further includes a first additional camera that selectively operates only when a pre-set first operating condition is met, and collects a first additional image which is a real-time additional image of the right front of the first vehicle. The display unit switches the display screen to display the first additional video according to the preset first additional video display information in response to the operation of the first additional camera. The vehicle real-time AVM system according to claim 1.

5. The vehicle further includes an object sensing unit that senses the presence or absence of an object located to the right front of the first vehicle and the direction of movement of the object, When the object detection unit detects an object approaching the right front of the first vehicle, it generates a first additional event to activate the first additional camera. The vehicle real-time AVM system according to claim 4.

6. The system further includes a second additional camera that selectively operates only when a pre-set second operating condition is met, and collects a second additional image, which is a real-time additional image of the interior of the first vehicle. The display unit switches the display screen to display the second additional video according to the second additional video display information that has been set in advance, in response to the operation of the second additional camera. The vehicle real-time AVM system according to claim 4.

7. An AVM (Around View Monitoring) method using a vehicle AVM (Around View Monitoring) system, AVM (Around View Monitoring) system for vehicles stores AVM operation information storage step, which stores AVM operation information including pre-configured video display information for each of at least one predicted event that is expected to occur while the vehicle is in operation or stopped, The video collection stage involves collecting real-time video footage of the area around the first vehicle while it is in operation or stopped, A video processing step in which, in response to an actual event occurring in the first vehicle, real-time AVM video is generated from the collected video, The process includes a video display step in which the real-time AVM video is displayed, The aforementioned video processing step involves detecting video display information corresponding to the actual event from the AVM operation information, and then generating the real-time AVM video based on that video display information. A real-time AVM method characterized by the following:

8. The aforementioned video display information is, In order to display the real-time AVM video, the screen division information of the display unit is set differently for each predicted event, Includes video matching information which matches the video to be displayed on each of the split screens divided based on the aforementioned division information. The real-time AVM method according to claim 7.

9. The AVM operation information further includes camera drive information obtained by matching information for at least one AVM camera to be driven according to the predicted event, The aforementioned video acquisition step involves detecting the camera drive information corresponding to the actual event from the AVM operation information, and then driving the AVM camera based on that camera drive information to acquire real-time video of the area around the first vehicle. The real-time AVM method described in item 8.

10. The aforementioned video acquisition stage further includes a first additional video acquisition stage, which, when a predetermined first operating condition is met, acquires a first additional video, which is a real-time additional video of the right front of the first vehicle. In the aforementioned video display stage, the display screen of the display unit is automatically switched to display the first additional video according to the first additional video display information set in advance, in response to the collection of the first additional video. The real-time AVM method according to claim 7.

11. The first additional video acquisition stage further includes a monitoring stage in which the right front of the first vehicle is centrally monitored. The first additional video collection stage involves collecting the first additional video if, as a result of the monitoring, an object approaching the right front of the first vehicle is detected. The real-time AVM method according to claim 10.

12. The aforementioned video acquisition stage further includes a second additional video acquisition stage, which, if a pre-set second operating condition is met, acquires a second additional video, which is a real-time additional video of the interior of the first vehicle. In the aforementioned video display stage, the display screen of the display unit is automatically switched to display the second additional video according to the pre-set second additional video display information, in response to the collection of the second additional video. The real-time AVM method according to claim 10.

Images