Vehicle real-time AVM system reflecting driving information of vehicle, and method therefor

Abstract

The present invention relates to a vehicle around view monitoring (AVM) system comprising: an AVM operation information storage unit for storing AVM operation information including image display information preset for each of one or more prediction events predicted to occur when a vehicle is driven or stopped; an image collection unit for collecting real-time images of the surroundings of a first vehicle which is being driven or stopped; an image processing unit for generating real-time AVM images from the collected images in response to an actual event occurring in the first vehicle; and a display unit for displaying the real-time AVM images, wherein the image processing unit detects image display information corresponding to the actual event from the AVM operation information storage unit, and generates the real-time AVM images on the basis of the image display information so as to selectively provide information necessary for safe driving by reflecting driving information of the vehicle, and thus can appropriately provide the necessary information to a driver such that the awareness of the driver is improved to effectively prevent an accident.

Description

Real-time AVM system for vehicles reflecting vehicle operation information and method thereof

[0001] The present invention relates to an AVM (Around View Monitoring) system for a vehicle.

[0002] AVM is a technology for monitoring the surroundings of a vehicle while it is in motion. It collects images from a set number of cameras and collates them to provide a video of the vehicle's surroundings. To this end, an AVM typically includes four cameras installed on the front, rear, left, and right sides of the vehicle, as well as a display installed inside the vehicle. The images collected by these cameras are collated to create a 360-degree, omnidirectional image, which is then displayed on the display.

[0003] Therefore, AVMs can significantly help prevent accidents that may occur during parking and driving by allowing drivers to accurately perceive the surroundings of the vehicle. Furthermore, autonomous vehicles are actively utilizing AVMs.

[0004] In relation to this, Korean Patent No. 10-1366112 discloses an AVM (Around View Monitoring) system for a vehicle, comprising: a first camera network constructed with some of the cameras constituting the AVM system; a second camera network constructed with other some of the cameras constituting the AVM system; and a processor that generates a first image from images generated by the cameras constituting the first camera network and generates a second image from images generated by the cameras constituting the second camera network; wherein the cameras constituting the first camera network and the cameras constituting the second camera network are automatically varied according to the direction of travel of the vehicle, and the number of cameras in the first camera network and the number of cameras in the second camera network are maintained at their original numbers.

[0005] According to the above patent, even when more cameras are installed in special vehicles such as buses and trucks than in the AVM system of general vehicles to eliminate blind spots according to the characteristics of the vehicle, smooth networking is possible, so that video delay does not occur, and thus there is an advantage in that accidents caused by blind spots and video delay can be prevented in advance.

[0006] However, these conventional technologies do not provide AVM video service that takes into account the vehicle's operating information, and thus, they cannot provide the information necessary for drivers of vehicles in which the vehicle's operating conditions (e.g., stops and starts, etc.) change frequently and the area that the driver must focus on changes depending on the vehicle's operating conditions, such as buses in which an unspecified number of passengers get on and off, and as a result, accident prevention is limited.

[0007] That is, in the past, the entire synthesized image was displayed around the vehicle using a certain number of cameras, or the image information collected by each of the cameras was displayed in a single, divided area, so that the driver had to selectively check the area to focus on among the entire image depending on the driving situation. However, in the former case, there was a problem that a blind spot existed technically, and in the latter case, by providing multiple divided areas at the same time, cognitive ability was reduced, which rather caused an accident.

[0008]

[0009] [Prior Art Literature]

[0010] [Patent Document]

[0011] (Patent Document 1) Korean Patent No. 10-1366112

[0012] Therefore, in order to solve the above problem, the present invention provides a real-time AVM system for a vehicle and a method thereof, which can appropriately provide the driver with the information necessary for safe driving by reflecting the vehicle's driving information and selectively providing the information necessary for safe driving, thereby improving the driver's cognitive ability and effectively preventing accidents.

[0013] In addition, the present invention provides a real-time AVM system for a vehicle and a method thereof, which can automatically select and display information on an area that a driver should focus on by automatically switching the display screen of a real-time AVM image whenever an actual event occurs based on the image display information set in advance for each predicted event that may occur when a vehicle is in operation or stopped, and thereby effectively prevent accidents caused by driver error.

[0014] In addition, the present invention provides a real-time AVM system for a vehicle and a method thereof, which can strengthen surveillance of an area with a high risk of accidents by providing at least one additional camera that collects additional images according to separate operating conditions and preferentially displaying an additional image collected from the additional camera or a composite image with the additional image, thereby enabling intensive prevention of accidents due to specific conditions with a high risk of accidents.

[0015] In order to achieve the above object, the real-time AVM system for a vehicle provided by the present invention is a vehicle AVM (Around View Monitoring) system, comprising: an AVM operation information storage unit for storing AVM operation information including preset image display information for at least one predicted event predicted to occur when a vehicle is in operation or stopped; an image collection unit for collecting real-time images around a first vehicle while the vehicle is in operation or stopped; an image processing unit for generating a real-time AVM image from the collected images in response to an actual event occurring in the first vehicle; and a display unit for displaying the real-time AVM image, wherein the image processing unit detects image display information corresponding to the actual event from the AVM operation information storage unit and generates the real-time AVM image based on the image display information.

[0016] Preferably, the image display information may include: split information of the display screen set differently for each predicted event to display the real-time AVM image; and image matching information that matches images to be displayed on each of the split screens split based on the split information.

[0017] Preferably, the AVM operation information further includes camera operation information that matches information about at least one AVM camera to be operated for each predicted event, and the image collection unit detects the camera operation information corresponding to the actual event from the AVM operation information storage unit, and operates the AVM cameras based on the camera operation information to collect real-time images around the first vehicle.

[0018] Preferably, the vehicle AVM system further includes a first additional camera that selectively operates only when a preset first operating condition is satisfied, and collects a first additional image that is a real-time additional image for the right front of the first vehicle, and the display unit can switch a display screen to display the first additional image by preset first additional image display information in response to the operation of the first additional camera.

[0019] Preferably, the vehicle AVM system further includes an object detection unit that detects the presence of an object located at the right front of the first vehicle and the direction of movement of the object, and the object detection unit can generate a first additional event for operating the first additional camera when an object approaching the right front of the first vehicle is detected.

[0020] Preferably, the vehicle AVM system further includes a second additional camera that selectively operates only when a preset second operating condition is satisfied, and collects a second additional image, which is a real-time additional image of the interior of the first vehicle, and the display unit can switch the display screen to display the second additional image by preset second additional image display information in response to the operation of the second additional camera.

[0021] Meanwhile, in order to achieve the above object, the AVM method provided in the present invention is an AVM (Around View Monitoring) method using a vehicle AVM (Around View Monitoring) system, comprising: an AVM operation information storage step for storing AVM operation information including preset image display information for at least one predicted event predicted to occur when the vehicle is in operation or stopped; an image collection step for collecting real-time images around a first vehicle while the vehicle is in operation or stopped; an image processing step for generating a real-time AVM image from the collected images in response to an actual event occurring in the first vehicle; and an image display step for displaying the real-time AVM image, wherein the image processing step is characterized in that the real-time AVM image is generated based on the image display information after detecting image display information corresponding to the actual event from the AVM operation information.

[0022] Preferably, the image display information may include split information of the display screen that is set differently for each prediction event to display the real-time AVM image, and image matching information that matches an image to be displayed on each of the split screens split based on the split information.

[0023] Preferably, the AVM operation information further includes camera operation information that matches information about at least one AVM camera to be operated for each predicted event, and the image collection step detects the camera operation information corresponding to the actual event from the AVM operation information, and then operates the AVM camera based on the camera operation information to collect real-time images around the first vehicle.

[0024] Preferably, the image collection step further includes a first additional image collection step of collecting a first additional image, which is a real-time additional image for the right front of the first vehicle, when a preset first operating condition is satisfied, and the image display step can automatically switch the display screen of the display unit to display the first additional image by preset first additional image display information in response to the collection of the first additional image.

[0025] Preferably, the first additional image collection step further includes a monitoring step for intensively monitoring the right front of the first vehicle, and the first additional image collection step can collect the first additional image when, as a result of the monitoring, an object approaching the right front of the first vehicle is detected.

[0026] Preferably, the image collection step further includes a second additional image collection step of collecting a second additional image, which is a real-time additional image of the interior of the first vehicle, when a preset second operating condition is satisfied, and the image display step can automatically switch the display screen of the display unit to display the second additional image by preset second additional image display information in response to the collection of the second additional image.

[0027] The real-time AVM system for a vehicle of the present invention and its method as described above can provide the driver with the information he needs appropriately by reflecting the vehicle's driving information and selectively providing the information necessary for safe driving, thereby improving the driver's cognitive ability and effectively preventing accidents.

[0028] In addition, the present invention sets image display information in advance for each predicted event that may occur when a vehicle is in operation or stopped, and then automatically switches the display screen of a real-time AVM image each time an actual event occurs based on the image display information, thereby automatically selecting and displaying information on an area that the driver should focus on, thereby having the advantage of effectively preventing accidents caused by driver error.

[0029] In addition, the present invention has at least one additional camera that collects additional images according to separate operating conditions, and by preferentially displaying additional images collected from the additional cameras or composite images with the additional images, surveillance can be strengthened for areas with a high risk of accidents, and thus, there is an advantage in that accidents due to specific conditions with a high risk of accidents can be intensively prevented.

[0030] FIG. 1 is a schematic block diagram of a real-time AVM system for a vehicle according to one embodiment of the present invention.

[0031] FIG. 2a and FIG. 2b are drawings for explaining the installation location and operation information of an AVM camera according to one embodiment of the present invention.

[0032] FIG. 3 is a drawing for explaining the attachment location of a sensor unit for detecting an object approaching a vehicle in which a real-time AVM system for a vehicle according to one embodiment of the present invention is installed.

[0033] FIGS. 4A and 4B are drawings for explaining the principle of detecting the approach of an object using a sensor unit attached to a real-time AVM system for a vehicle according to one embodiment of the present invention.

[0034] FIG. 5 is a diagram illustrating AVM operation information preset for each prediction event according to one embodiment of the present invention.

[0035] FIGS. 6A to 6D are drawings illustrating display screens of a real-time AVM system for a vehicle according to one embodiment of the present invention.

[0036] FIGS. 7 and 8 are schematic processing flowcharts for a real-time AVM method according to one embodiment of the present invention.

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice the present invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. Meanwhile, in the drawings, parts that are not related to the description are omitted in order to clearly explain the present invention, and similar parts are designated with similar reference numerals throughout the specification. In addition, the description of parts that can be easily understood by those skilled in the art even if a detailed description is omitted is omitted.

[0038] Throughout the specification and claims, whenever a part is said to include a certain component, this does not mean that it excludes other components, but rather that it may include other components, unless specifically stated otherwise.

[0039] FIG. 1 is a schematic block diagram of a real-time AVM system for a vehicle according to an embodiment of the present invention. Referring to FIG. 1, a real-time AVM system (100) for a vehicle according to an embodiment of the present invention includes an AVM camera (110), an additional camera (120), an image collection unit (130), an object detection unit (140), an event collection unit (150), an AVM operation information storage unit (160), an image processing unit (170), a display unit (180), and a control unit (190).

[0040]

[0041] The AVM camera (110) is installed inside / outside the vehicle and captures real-time images of the surroundings of the vehicle. To this end, the AVM camera (110) includes a front camera (111) that captures the front of the vehicle, a rear camera (112) that captures the rear of the vehicle, a right-side camera (113) that captures the right-side of the vehicle, and a left-side camera (114) that captures the left-side of the vehicle.

[0042] Each camera (111 to 114) can be driven by an operation command transmitted from the outside or while the vehicle is in operation or stopped, and can capture real-time images of the surroundings of the vehicle depending on its installation location.

[0043] At this time, each of the cameras (111 to 114) may be operated simultaneously and in batches to generate a real-time 360° image around the vehicle, or may be selectively operated by preset operation information (i.e., camera operation information) according to the type of event predicted to occur in the vehicle (i.e., according to the predicted event). To this end, the control unit (190) may operate all cameras (111 to 114) simultaneously when the operation of the vehicle real-time AVM system (100) is initiated, or may detect camera operation information corresponding to an event that actually occurred while the vehicle was driving or stopped (i.e., an actual event) by matching the predicted event, and then selectively operate the cameras according to the result.

[0044] At this time, the above predicted event may include operation of the left / right turn signal, operation of opening / closing the vehicle door, operation of at least one switch installed on the driver's seat, and operation of the exit signal bell.

[0045] For example, when the vehicle is driving at a constant speed, the control unit (190) operates all of the cameras (111 to 114) simultaneously and provides the driver with a top-view image generated using the information collected at that time. However, when the right turn signal of the vehicle is turned on, that is, when a right turn signal event occurs, the cameras (111 to 114) can be selectively operated in response to this based on preset operation information. When only the right-side camera (113) is included in the camera operation information corresponding to the right turn signal event, the control unit (190) can operate only the right-side camera (113) to control the cameras (111 to 114) and peripheral devices (e.g., the display unit (180)) to intensively collect and provide only real-time images for the right side of the vehicle.

[0046]

[0047] The additional camera (120) operates under separate operating conditions different from the operating information of the AVM camera (110) to capture additional images. That is, the additional camera (120) is a camera additionally installed to intensively prevent accidents caused by specific conditions with a high risk of accidents, and normally operates in standby mode, but operates to capture the corresponding area when a specific event occurring in the vehicle (e.g., operation of the right turn signal, or opening of a door, etc.) or a separate, preset operating condition (e.g., driver's request, detection of an object or person approaching the vehicle, etc.) is satisfied.

[0048] At this time, the specific conditions with a high risk of accident may include when the vehicle makes a right turn, when a person or object is detected around the vehicle, when a person or object is detected moving in the direction of the vehicle, when passengers on a route bus get on / off, or when a dangerous situation occurs inside the vehicle.

[0049] The additional camera (120) may include a right front camera (121) that photographs the right front of the vehicle, and an interior camera (122) that photographs the interior of the vehicle, and additional images collected from the additional camera (120) may be provided to the driver independently or in combination with images collected from the AVM camera (110).

[0050] The right front camera (121) is a camera that focuses on the right front of the vehicle and can be activated in special situations where right front surveillance must be strengthened. For example, if an event is detected to activate the right turn signal in the vehicle, the steering wheel of the vehicle is turned to the right by a predetermined angle or more, or an object or person approaching the right front of the vehicle is detected, the control unit (190) can activate the right front camera (121) to strengthen right front surveillance.

[0051] This is to reduce the risk of an accident caused by the fact that it is difficult for the driver of a vehicle with the driver's seat located on the left front of the vehicle to secure a view of the right front, making it difficult to recognize the surroundings on the right front of the vehicle, and that there is a blind spot on the right front corner of the vehicle that is difficult to capture with only the front camera (111) and the right-side camera (113). Therefore, when a right turn is detected by a specific event that occurred in the vehicle (e.g., the right turn signal is turned on, the steering wheel is turned to the right, etc.), or a person or object is detected in the right direction of travel of the vehicle or around the right front of the vehicle by a detection device installed outside the vehicle, the control unit (190) operates the right front camera (121) so that the driver can focus on monitoring the right front of the vehicle, thereby eliminating the blind spot on the right front of the vehicle and reducing the risk of an accident.

[0052] In particular, in a traffic system where drivers of vehicles driving on the right turn right at intersections without receiving a separate signal, it is possible to prevent safety accidents that occur when drivers of vehicles turning right fail to recognize pedestrians.

[0053] In this way, the present invention has a right front camera (121) that operates according to separate operating conditions, and by operating the right front camera (121) when the conditions are satisfied, accidents occurring due to the driver having difficulty recognizing a person or object located in the right direction of travel of the vehicle or around the right front of the vehicle can be prevented.

[0054] Meanwhile, in the case of vehicles with the driver's seat located on the right front, by installing an additional camera on the left front instead of the right front, left front surveillance can be strengthened in special situations (e.g., left turns) where left front surveillance must be strengthened.

[0055] The interior camera (122) is a camera designed to intensively photograph the interior of a vehicle and can be operated in special situations where surveillance of the interior (i.e., the interior) of the vehicle must be strengthened. For example, when an event to open a door in the vehicle is detected or a driver's request signal is detected, the control unit (190) can operate the interior camera (122) to strengthen surveillance of the interior of the vehicle. To this end, at least one interior camera (122) can be installed inside the vehicle, and the cameras can be distributed throughout the vehicle so as to capture the entire interior of the vehicle. In addition, an operation key for operating the interior camera (122) can be further included around the driver's seat.

[0056] Accordingly, when the above door opening event occurs or the driver, who determines that it is necessary to monitor the situation inside the vehicle, operates the above operation key, the interior camera (122) operates to photograph the inside of the vehicle and provides the information to the driver in real time.

[0057] This is to prevent safety accidents that may occur due to the driver not being able to accurately determine the passenger boarding / disembarking situation or not being able to recognize people or objects located around the vehicle's stopping location, in the case of vehicles that repeatedly stop and start to help an unspecified number of passengers board / disembark, such as route buses that run on a predetermined route periodically.

[0058] That is, the indoor camera (122) determines its operation in response to the door opening / closing signal, and can operate to collect surrounding images when the door is opened. Alternatively, when a door opening signal is generated, the indoor camera (122) can operate together with the right-side camera (113) to synthesize the images collected from each, thereby accurately conveying the surrounding situation inside / outside the vehicle door to the driver.

[0059] Meanwhile, the right front camera (121) and the indoor camera (122) can operate under the control of the control unit (190) or the image collection unit (130) that operates under the control of the control unit (190).

[0060] FIG. 2A and FIG. 2B are drawings for explaining the installation location and operation information of an AVM camera according to one embodiment of the present invention. FIG. 2A illustrates the installation location and respective shooting areas of an AVM camera (110) and an additional camera (120), FIG. 2B illustrates a shooting area reflecting the driving information of a vehicle, and FIG. 2B illustrates a surveillance area determined by reflecting the driving information of a vehicle.

[0061] Referring to FIG. 2A, it can be seen that the real-time AVM system (100) for a vehicle of the present invention includes a front camera (111) (recording area (111A)) installed on the front of the vehicle (10), a rear camera (112) (recording area omitted) installed on the rear of the vehicle (10), a right-side camera (113) (recording area (113A)) installed on the right side of the vehicle (10), a left-side camera (114) (recording area (114A)) installed on the left side of the vehicle (10), a right-side camera (121) (recording area (121A)) installed on the right front (i.e., the right front vertex position) of the vehicle (10), and an indoor camera (122) (recording area (122A)) installed on the left side inside the vehicle (10).

[0062] The real-time AVM system (100) for a vehicle of the present invention can determine a surveillance area reflecting the driving information of the vehicle (10) by operating each of the installed cameras (111 to 114, 121, and 122) as illustrated in FIG. 2a in batches or selectively. Referring to FIG. 2b, when turning right, the front camera (111), the right-side camera (113), and the front-right camera (121) are controlled to be intensively monitored, and when stopping, the right-side camera (113) and the interior camera (122) are controlled to be intensively monitored, and when turning left, the front camera (111) and the left-side camera (114) are controlled to be intensively monitored, and the third area (S3) is controlled to be intensively monitored.

[0063] In this way, the real-time AVM system (100) for a vehicle of the present invention can collectively or selectively capture images of areas (111A to 114A, 121A, and 122A) corresponding to the driving status of the vehicle (10) by operating each of the cameras (111 to 114, 121, and 122) in batches or selectively by reflecting the driving information of the vehicle (10), and can selectively generate and provide real-time AVM images around the vehicle (10) using this, thereby improving the driver's cognitive ability and effectively preventing accidents.

[0064] The image collection unit (130) collects real-time images of the surroundings of the vehicle (10) from at least one of the AVM camera (110) and the additional cameras (120). To this end, the image collection unit (130) can be controlled by the control unit (190), and collects images captured in real time for a corresponding area from at least one of the front camera (111), the rear camera (112), the right-side camera (113), the left-side camera (114), the right-side front camera (121), and the indoor camera (122) installed in the vehicle (10) while it is in operation or stopped. In particular, the image collection unit (130) can collect images captured in real time from the right-side front camera (121) and the indoor camera (122) only when the preset operating conditions for the right-side front camera (121) and the indoor camera (122) are satisfied, respectively. For example, the image collection unit (130) can detect camera operation information corresponding to an actual event from the AVM operation information storage unit (160) described later, and then operate the AVM cameras (111 to 114) and additional cameras (121 and 122) based on the camera operation information to collect real-time images around the corresponding vehicle (10).

[0065] The object detection unit (140) detects the presence and movement direction of objects or people located around the vehicle, and generates additional events based on the results. For example, the object detection unit (140) may detect objects or people located on the right front of the vehicle, where it is difficult for the driver to secure a field of vision, and their movement direction, and generate an additional event to operate the right front camera (121) based on the results, and transmit the event to the control unit (190). To this end, the object detection unit (140) may be implemented with a plurality of ultrasonic sensors or a plurality of LiDAR sensors, and is preferably installed on the right front of the vehicle.

[0066] FIG. 3 is a drawing for explaining the attachment position of a sensor unit for detecting an object approaching a vehicle having a real-time AVM system for vehicles installed according to an embodiment of the present invention, and illustrates an example in which an object detection unit (140) composed of a plurality of sensors (141) is installed adjacent to the right front side of a vehicle (10) (i.e., a bus), and FIGS. 4A and 4B are drawings for explaining the principle of detecting the approach of an object using a sensor unit attached to a real-time AVM system for vehicles according to an embodiment of the present invention, and FIG. 4A illustrates a detection area (141A) formed in front of vehicles turning right by attaching a sensor (141) to the right front side, and FIG. 4B illustrates the principle of detecting a pedestrian and the direction of movement of the pedestrian in the detection area (141A) to determine whether to approach.

[0067] Referring to FIGS. 3, 4a, and 4b, each of the plurality of sensors (141) installed adjacent to the right front side of the vehicle outputs detection signals (L1, L2) (e.g., ultrasonic signals, laser pulses, etc.) in a straight direction to detect an object or person approaching the vehicle (10). The detection signals are output in a direction in which the angle between the sensors increases as the distance from the vehicle (10) increases. This is to secure a detection area for detecting the object or person, and if the angle between the sensors is narrow, the detection distance is long, but the detection area around the vehicle becomes narrower, and in the opposite case, the detection distance is short, but the detection area around the vehicle becomes wide. Therefore, it is preferable to determine the installation interval of the plurality of sensors (141) in consideration of the height of the vehicle, the width of the vehicle, etc.

[0068] Referring to FIG. 4B, the detection areas (141A) can be formed in a shape as illustrated on the left side of the drawing by a plurality of detection signals (L1, L2), and the direction in which the pedestrian is moving can be determined by the order in which the pedestrian comes into contact with the plurality of detection signals (L1, L2), as shown on the right side of the drawing. For example, if the pedestrian comes into contact with the second detection signal (L2) after passing through the first detection signal (L1), the direction in which the pedestrian is moving is from the right side of the drawing to the left, and in the opposite case, the direction in which the pedestrian is moving is also opposite. If the pedestrian comes into contact with only one of the plurality of detection signals (L1, L2), it can be determined that the pedestrian is not approaching the vehicle (10). Meanwhile, whether or not the plurality of detection signals (L1, L2) and the pedestrian come into contact can be determined by whether or not a phenomenon in which each of the plurality of detection signals (L1, L2) is disconnected occurs.

[0069] By this operation, the object detection unit (140) that detects the presence and movement direction of an object or person located around the vehicle may further include a means (e.g., a buzzer, etc.) for generating the additional event and generating a warning sound to alert pedestrians of danger. Alternatively, the vehicle real-time AVM system (100) of the present invention may further include an audio output means (e.g., a speaker, etc.) (not shown), and the control unit (190) may control the audio output means in response to the additional event to thereby output a warning sound.

[0070]

[0071] The event collection unit (150) collects real-time events occurring in the vehicle (10) and transmits them to the control unit (190). To this end, the event collection unit (150) may collect information from an electronic control unit that controls the operation of the vehicle (10), or may collect the real-time events by using signals detected through a separate sensor unit.

[0072] The AVM operation information storage unit (160) stores operation information of a preset real-time vehicle AVM system (100). That is, the AVM operation information storage unit (160) can store AVM operation information including preset image display information and camera operation information for at least one predicted event predicted to occur when the vehicle is in operation or stopped.

[0073] At this time, the image display information may include split information of the display screen, which is set differently for each predicted event to display real-time AVM images; and image matching information that matches images to be displayed on each of the split screens divided based on the split information. Meanwhile, the camera operation information may include information on at least one AVM camera to be operated for each predicted event.

[0074] FIG. 5 is a drawing illustrating AVM operation information preset for each prediction event according to one embodiment of the present invention, and illustrates AVM operation information stored in an AVM operation information storage unit (160).

[0075] Referring to FIG. 5, the AVM operation information may include an event type (161), an operation camera (162), and a display image (163) field. The event type (161) field stores the predicted events that may occur, the operation camera (162) field stores camera information that should be selectively operated in response to the predicted event, and the display image (163) field may store information on an area that should be intensively monitored for the predicted event in order to improve the driver's cognitive ability. At this time, the type of the predicted event and the corresponding operation camera and display image information are as illustrated in FIG. 5. Meanwhile, FIG. 5 only illustrates an example of the AVM operation information, and examples of the type of the predicted event and the corresponding AVM operation information are not limited to the contents illustrated in FIG. 5.

[0076] The image processing unit (170) generates a real-time AVM image from the image collected through the image collection unit (130) in response to an actual event occurring in the vehicle (10). To this end, the image processing unit (170) can detect image display information corresponding to the actual event from the AVM operation information storage unit (160) and generate the real-time AVM image based on the image display information. For example, when the actual event collected through the event collection unit (150) is an exit signal bell, the image processing unit (170) can detect AVM operation information (i.e., camera operation information and image display information) corresponding to the exit signal bell from the AVM operation information storage unit (160), as illustrated in FIG. 5, and then collect images captured by a camera installed inside the vehicle based on the results to generate a real-time AVM image that displays an image of the inside of the vehicle.

[0077] At this time, the image processing unit (170) generates a real-time AVM image using a preset image processing algorithm, and may process and display images collected from at least one camera that is selectively operated by the AVM operation information by dividing the images or synthesizing them.

[0078] The display unit (180) displays real-time AVM images generated by the image processing unit (170). To this end, the display unit (180) operates under the control of the control unit (190), and its display screen can be configured differently depending on the type of the actual event. That is, the display unit (180) can configure and display the screen in a way that increases the driver's recognition ability depending on the type of actual events that may occur while the vehicle is driving or stopped. To this end, the display unit (180) displays the real-time AVM images based on image display information in which the number of split screens and the size and position of each split screen are set differently depending on the type of the actual event, and can operate under the control of the control unit (190) that controls the operation of the display unit (180) based on image display information stored in the AVM operation information storage unit (160).

[0079] Meanwhile, the above image display information may be preset differently for each type of prediction event and stored in the AVM operation information storage unit (160).

[0080] In addition, the display unit (180) can automatically switch the display screen to display additional images collected through the additional camera (120) in response to the operation of the additional camera (120). To this end, the image display information can further store additional image display information for automatically switching the screen according to the type of additional images collected through the additional camera (120).

[0081] For example, the display unit (180) automatically switches the display screen to display the right front image of the vehicle in response to the operation of the right front camera (121), but when a pedestrian approaches the right front of the vehicle, the entire screen of the display unit (180) may automatically switch to display the right front image of the vehicle, or the display state may be switched to display the right front image in a pop-up form while maintaining the current screen, or the display state may be switched to replace a portion of the current screen with the right front image.

[0082] Meanwhile, the display unit (180) automatically switches the display screen to display the interior image of the vehicle in response to the operation of the interior camera (122). In particular, when the vehicle door is closed or the vehicle is in a waiting state for departure, the display unit (180) may automatically switch the screen to display the images of the interior camera (122) and the right-side camera (113) or a composite image thereof in the entire screen to indicate whether there are any customers getting on or off the vehicle, or may switch the display state to display a real-time image including the interior image around the door in a pop-up form while maintaining the current screen, or may switch the display state to replace a portion of the current screen with the interior image.

[0083] To this end, image display information for displaying the right front image and image display information for displaying real-time images including indoor images around the entrance door must be registered in advance in the AVM operation information storage unit (160), and the control unit (190) can control the operation of the display unit (180) based on the information stored in the AVM operation information storage unit (160).

[0084] FIGS. 6A to 6D are drawings illustrating display screens of a real-time AVM system for a vehicle according to an embodiment of the present invention. FIGS. 6A and 6B illustrate screens (A, B) displayed while a vehicle is in motion, FIG. 6C illustrates a screen (C) displaying a surrounding image of a stopped vehicle, and FIG. 6D illustrates a screen (D) displaying a top-view image of a vehicle in motion and an interior image together.

[0085] In the example of FIG. 6a, the display unit (180) divides the display screen (A) into four split screens (A1, A2, A3, A4), and displays a 360° top view image on the first split screen (A1), a front image on the second split screen (A2), a left-side image on the third split screen (A3), and a right-side image on the fourth split screen (A4).

[0086] In the example of FIG. 6b, the display unit (180) divides the display screen (B) into three split screens (B1, B2, B3), and displays the front image on the first split screen (B1), the left-side image on the second split screen (B2), and the right-side image on the third split screen (B3).

[0087] In the example of FIG. 6c, the display unit (180) divides the display screen (C) into five split screens (C1, C2, C3, C4, C5), and displays a 360° top view image of a stopped vehicle on the first split screen (C1), a left front image of the stopped vehicle on the second split screen (C2), a right front image of the stopped vehicle on the third split screen (C3), a left rear image of the stopped vehicle on the fourth split screen (C4), and a right rear image of the stopped vehicle on the fifth split screen (C5).

[0088] In the example of FIG. 6d, the display unit (180) divides the display screen (D) into two split screens (D1, D2), and displays a 360° top view image of a moving vehicle on the first split screen (D1), and an indoor image on the second split screen (D2).

[0089] In this way, the present invention divides the display screen of the display unit (180) into various forms based on preset image display information, and by using information set differently depending on the type of actual event occurring in a vehicle that is in motion or stopped, the display screen can be operated in a way that enhances the driver's cognitive ability.

[0090] In addition, the present invention further includes additional image display information for displaying an image captured in response to a specific condition with a high risk of accident, and automatically switches the screen of the display unit (180) in response thereto, thereby further improving the driver's awareness of a dangerous situation.

[0091]

[0092] The control unit (190) controls the overall operation of the vehicle real-time AVM system (100) based on a preset AVM system control algorithm. That is, the control unit (190) controls the operation of each of the AVM camera (110), additional camera (120), image collection unit (130), object detection unit (140), event collection unit (150), AVM operation information storage unit (160), image processing unit (170), and display unit (180) based on the AVM system control algorithm.

[0093]

[0094] Figures 7 and 8 are schematic flowcharts of a real-time AVM method according to an embodiment of the present invention. Referring to Figures 1 to 8, the real-time AVM method according to an embodiment of the present invention is described as follows.

[0095] First, in step S105, the vehicle real-time AVM system (100) stores preset AVM operation information for at least one predicted event predicted to occur when the vehicle is in operation or stopped. To this end, the vehicle real-time AVM system (100) may receive the AVM operation information through a user interface (not shown), transmit it through a communication network, or store it offline through a separate storage medium in which the AVM operation information is previously stored.

[0096] At this time, the AVM operation information may include camera operation information and image display information for each predicted event. The camera operation information may include information on at least one AVM camera to be operated for each predicted event (e.g., camera identification information, camera installation location information, etc.), and the image display information may include image processing information for generating the real-time AVM image (e.g., image synthesis / segmentation algorithm, etc.), split information of the display unit (180) screen set differently for each predicted event to display the real-time AVM image, and image matching information that matches images to be displayed on each of the split screens split based on the split information.

[0097] In step S110, the image collection unit (130) collects real-time images around the first vehicle while it is in operation or stopped. Since step S110 is a step before an event (i.e., an actual event) occurs in the first vehicle, real-time images can be collected from each of all cameras (111 to 114, 121, and 122) included in the vehicle real-time AVM system (100) based on the initial setting information of the system.

[0098] In step S115, the control unit (190) determines whether an event (i.e., an actual event) occurs in the first vehicle. This is to generate an AVM image by taking into account the vehicle's driving information.

[0099] That is, if the actual event has not occurred as a result of the confirmation in step S115, the control unit (190) generates an AVM image based on the real-time image collected in step S110, and in step S120, controls the image processing unit (170) to generate an initial AVM image, and in step S125, controls the display unit (180) to display the initial AVM image. At this time, the initial AVM image refers to an AVM image generated without considering the vehicle's driving information (i.e., the occurrence of the actual event), and may mean a top-view image provided by a conventional AVM system.

[0100] Meanwhile, if the actual event has occurred as a result of the confirmation in step S115, the control unit (190) can detect AVM operation information corresponding to the actual event and control the image collection unit (130), the image processing unit (170), and the display unit (180) based on the information. That is, the control unit (190) can control the image collection unit (130), the image processing unit (170), and the display unit (180) to collect images corresponding to the actual event and then generate and display real-time AVM images from the collected images.

[0101] To this end, first, in step S130, the control unit (190) checks whether camera operation information corresponding to the actual event is stored in the AVM operation information, and if the camera operation information corresponding to the actual event is stored, in steps S135 and S140, the control unit (130) controls the image collection unit (130) to selectively operate the AVM camera corresponding to the actual event and then collect the selected image. That is, in step S140, the image collection unit (130) can collect real-time images of the corresponding area, i.e., real-time images of the surroundings of the first vehicle corresponding to each of the operated cameras, from at least one of the cameras (111 to 114, 121, and 122) operated under the control of the control unit (190).

[0102] As a result of the verification in step S130, if the camera operation information corresponding to the actual event is not stored, steps S135 and S140 may be omitted.

[0103] In step S145, the control unit (190) detects image display information corresponding to the actual event from the event-specific image display information included in the AVM operation information, and in steps S150 and S155, the control unit (190) controls the image processing unit (170) and the display unit (180) to generate and display an AVM image (i.e., an event-specific AVM image) that takes the actual event into consideration based on the detected image display information.

[0104] In step S160, the control unit (190) collects and processes additional images according to separate operating conditions set in advance, and in step S170, the image collection unit (130) and the display unit (180) are controlled to automatically switch the screen to display the collected additional images.

[0105] To this end, the control unit (190) may store in advance the operating conditions for operating each additional camera (120), and when a situation satisfying the operating conditions occurs, the corresponding additional camera (120) may be operated to collect the corresponding image. For example, the control unit (190) may store in advance specific conditions with a high risk of accidents, and when a driving situation satisfying the specific conditions occurs, the additional camera (120) may be operated to collect the image for the corresponding area.

[0106] At this time, the specific conditions with a high risk of accident may include when the vehicle makes a right turn, when a person or object is detected around the vehicle, when a person or object is detected moving in the direction of the vehicle, when passengers on a route bus get on / off, or when a dangerous situation occurs inside the vehicle.

[0107] That is, the control unit (190) may store in advance the first operating condition for operating the right front camera (121), and in steps S162 and S164, when a situation satisfying the first operating condition occurs, control the right front camera (121) and the image collection unit (130) to collect additional images of the right front. At this time, the first operating condition may include a case where an event for operating the right turn signal in the vehicle is detected, a case where the steering wheel of the vehicle is turned to the right by a predetermined angle or more, and a case where an object or person approaching the right front of the vehicle is detected. Therefore, when it is determined in step S162 that the first operating condition is satisfied, the control unit (190) may operate the right front camera (121) to strengthen right front surveillance in step S164. To this end, the control unit (190) controls the object detection unit (140) to intensively monitor the right front of the first vehicle, thereby detecting the presence of an object or person approaching the right front of the first vehicle.

[0108] In addition, the control unit (190) may store in advance the second operating condition for operating the indoor camera (122), and in steps S166 and S168, if a situation that satisfies the second operating condition occurs, the control unit (190) may control the indoor camera (122) and the image collection unit (130) to collect additional images of the interior of the vehicle. At this time, the second operating condition may include a case where an event for opening a door in the vehicle is detected, or a case where a driver's request signal is detected. Therefore, if it is determined in step S166 that the second operating condition is satisfied, the control unit (190) may operate the indoor camera (122) in step S168 to strengthen the internal surveillance of the vehicle.

[0109] In step S170, the control unit (190) automatically switches the display screen of the display unit (180) to display the additional image, and may automatically switch the display screen based on information set differently depending on the type of the actual event.

[0110] For example, the control unit (190) may store the risk or urgency in advance according to the type of the actual event, and automatically switch the display screen of the display unit (180) based on the risk or urgency, automatically switching the screen so that the entire screen of the display unit (180) displays the additional image, switching the display state so that the additional image is displayed in a pop-up form while maintaining the current screen, or switching the display state so that a portion of the current screen is replaced with the additional image.

[0111]

[0112] By doing so, the real-time AVM system for a vehicle of the present invention and its method can provide the driver with the necessary information appropriately by reflecting the vehicle's driving information and selectively providing the information necessary for safe driving, thereby improving the driver's cognitive ability and effectively preventing accidents.

[0113] In addition, the present invention has the feature of automatically selecting and displaying information on an area that the driver should focus on by automatically switching the display screen of a real-time AVM image each time an actual event occurs based on the image display information set in advance for each predicted event that may occur when the vehicle is in operation or stopped, thereby effectively preventing accidents caused by driver error.

[0114] In addition, the present invention has the feature that surveillance can be strengthened for areas with a high risk of accidents by providing at least one additional camera that collects additional images according to separate operating conditions and preferentially displaying additional images collected from the additional cameras or composite images with the additional images, thereby enabling intensive prevention of accidents due to specific conditions with a high risk of accidents.

[0115] Although the embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and the present invention includes all changes and modifications that can be easily modified by a person having ordinary skill in the art to which the present invention pertains from the embodiments and are recognized as equivalent.

[0116] [Explanation of symbols]

[0117] 100: AVM system for vehicle 110: AVM camera

[0118] 111: Front camera 112: Rear camera

[0119] 113: Right camera 114: Left camera

[0120] 120: Additional camera 121: Right front camera

[0121] 122: Indoor camera 130: Video collection unit

[0122] 140: Object detection unit 150: Event collection unit

[0123] 160: AVM operation information storage unit 170: Image processing unit

[0124] 180: Display section 190: Control section

Claims

1. In the vehicle AVM (Around View Monitoring) system, An AVM operation information storage unit storing AVM operation information including preset image display information for at least one predicted event predicted to occur when a vehicle is in operation or stopped; A video collection unit that collects real-time video around the first vehicle while it is in operation or stopped; An image processing unit that generates real-time AVM images from the collected images in response to an actual event occurring in the first vehicle; and Including a display unit that displays the above real-time AVM image, The above image processing unit A real-time AVM system for a vehicle, characterized in that it detects image display information corresponding to the actual event from the AVM operation information storage unit and generates the real-time AVM image based on the image display information.

2. In paragraph 1, the image display information is Split information of the display screen set differently for each predicted event to display the real-time AVM image; and A real-time AVM system for a vehicle, characterized in that it includes image matching information that matches images to be displayed on each of the split screens divided based on the above-mentioned split information.

3. In the second paragraph, the AVM operation information Further including camera operation information matching information on at least one AVM camera to be operated for each of the above prediction events, The above image collection unit A vehicle real-time AVM system characterized in that it detects the camera operation information corresponding to the actual event from the AVM operation information storage unit, and operates the AVM cameras based on the camera operation information to collect real-time images around the first vehicle.

4. In paragraph 1, It further includes a first additional camera that selectively operates only when a preset first operating condition is satisfied, and collects a first additional image, which is a real-time additional image for the right front of the first vehicle. The above display part A real-time AVM system for a vehicle, characterized in that, in response to the operation of the first additional camera, the display screen is switched to display the first additional image using preset first additional image display information.

5. In paragraph 4, It further includes an object detection unit that detects the presence of an object located on the right front side of the first vehicle and the direction of movement of the object, The above object detection unit A real-time AVM system for a vehicle, characterized in that a first additional event is generated to operate the first additional camera when an object approaching the right front of the first vehicle is detected.

6. In paragraph 4, It further includes a second additional camera that collects a second additional image, which is a real-time additional image of the interior of the first vehicle, and operates selectively only when a preset second operating condition is satisfied. The above display part A real-time AVM system for a vehicle, characterized in that, in response to the operation of the second additional camera, the display screen is switched to display the second additional image using preset second additional image display information.

7. In the AVM (Around View Monitoring) method using the vehicle AVM (Around View Monitoring) system, An AVM (Around View Monitoring) system for a vehicle stores AVM operation information including preset image display information for at least one predicted event predicted to occur when the vehicle is in operation or stopped; A video collection step for collecting real-time video around the first vehicle while it is in operation or stopped; An image processing step for generating a real-time AVM image from the collected image in response to an actual event occurring in the first vehicle; and Including an image display step for displaying the above real-time AVM image, The above image processing step A real-time AVM method characterized in that the real-time AVM image is generated based on the image display information after detecting image display information corresponding to the actual event from the AVM operation information.

8. In paragraph 7, the image display information is Split information of the display screen set differently for each predicted event to display the real-time AVM image, and A real-time AVM method characterized by including image matching information that matches images to be displayed on each of the split screens divided based on the above split information.

9. In paragraph 8, the AVM operation information is Further including camera operation information matching information on at least one AVM camera to be operated for each of the above prediction events, The above image collection step A real-time AVM method characterized in that the camera operation information corresponding to the actual event is detected from the AVM operation information, and then the AVM camera is operated based on the camera operation information to collect real-time images around the first vehicle.

10. In paragraph 7, the image collection step It further includes a first additional image collection step of collecting a first additional image, which is a real-time additional image for the right front of the first vehicle, when a preset first operation condition is satisfied. The above video display step A real-time AVM method characterized in that, in response to the collection of the first additional image, the display screen of the display unit is automatically switched to display the first additional image by preset first additional image display information.

11. In the 10th paragraph, the first additional image collection step Further comprising a monitoring step for intensively monitoring the right front of the first vehicle, The above first additional image collection step A real-time AVM method characterized in that, as a result of the above monitoring, when an object approaching the right front of the first vehicle is detected, the first additional image is collected.

12. In paragraph 10, the image collection step It further includes a second additional image collection step of collecting a second additional image, which is a real-time additional image of the interior of the first vehicle, when a preset second operating condition is satisfied. The above video display step A real-time AVM method characterized in that, in response to the collection of the second additional image, the display screen of the display unit is automatically switched to display the second additional image using preset second additional image display information.

Images