Perimeter monitoring system with object detection function

Abstract

To provide a perimeter monitoring system with an object detection function. [Solution] The perimeter monitoring system includes a vehicle (10), a first camera (11), a second camera (12), a third camera (13), a fourth camera (14), an image processing device (15), a display device (16), a storage device (17), and a warning device (18). The first camera is directed toward the front of the vehicle and captures a first image (111), the second camera is directed toward the rear of the vehicle and captures a second image (121), the third camera is directed toward the left side of the vehicle and captures a third image (131), and the fourth camera is directed toward the right side of the vehicle and captures a fourth image (141). An image stitching module (154) of the image processing device acquires a perimeter image, an object detection module (155) acquires an object image, and an event trigger module (156) triggers a recording event based on the distance between the object and the vehicle, causing the warning device to issue a warning message (181).

Description

[Technical Field]

[0001] The present invention relates to a perimeter monitoring system with an object detection function, and more particularly to a perimeter monitoring system with an object detection function that is mounted on a vehicle to monitor the surrounding environment and issue an early warning. [Background technology]

[0002] With the rapid development of automotive technology, the demand for driver assistance systems is also increasing. Among these, 360-degree surround monitoring systems, which provide comprehensive visual assistance to drivers, are an important tool for improving driving safety and convenience. Existing surround monitoring systems use cameras to capture images of the vehicle's surroundings, then process and fuse them to generate a complete surround image, which is then displayed on an in-vehicle display. This allows drivers to have a comprehensive understanding of their surroundings, making it easier to drive, park, or navigate in complex traffic situations.

[0003] While existing peripheral assistance systems can display images of the surrounding environment, drivers often need to pay attention to the traffic ahead, making it difficult for them to simultaneously focus on the peripheral image. Even if people, vehicles, or obstacles appear in the peripheral image, they are often out of the driver's sight, resulting in ineffective warnings and attention. Furthermore, peripheral images are often captured with wide-angle lenses, which can result in distortion. This makes it difficult for drivers to determine the actual location and distance of objects in the image, potentially leading to incorrect judgments and compromising driving safety.

[0004] In light of this situation, existing dashcam systems still have room for improvement, and they have difficulty effectively detecting changes in the surrounding environment and issuing immediate warnings. This may lead to accidents while the vehicle is running or moving. Therefore, the inventors of this invention have studied and designed a perimeter monitoring system with object detection function, aiming to eliminate the shortcomings of existing technology and promote its use in the industry. Summary of the Invention [Problem to be solved by the invention]

[0005] In view of the above problems of the prior art, the object of the present invention is to provide a surroundings monitoring system with an object detection function to solve the problem that it is difficult for conventional drive recorders to detect objects in the surrounding environment and to issue a warning notification in a timely manner to avoid a vehicle accident. [Means for solving the problem]

[0006] One object of the present invention is to provide a perimeter monitoring system with an object detection function. The system includes a vehicle, a first camera, a second camera, a third camera, and a fourth camera, an image processing device, a display device, a storage device, and a warning device. The first camera is located at the front of the vehicle and faces forward to capture a first image. The second camera is located at the rear of the vehicle and faces backward to capture a second image. The third camera is located on the left side of the vehicle and faces leftward to capture a third image. The fourth camera is located on the right side of the vehicle and faces rightward to capture a fourth image. An image processing device is connected to the first, second, third, and fourth camera devices, respectively, and receives the first, second, third, and fourth images. A processor in the image processing device accesses a memory and executes an image temporary storage module, an image stitching module, an object detection module, and an event trigger module. The image temporary storage module is configured to store the first, second, third, and fourth images in a temporary storage area of ​​the memory. The image stitching module is connected to the image temporary storage module and synthesizes the first image, the second image, the third image, and the fourth image to generate a surrounding image. The object detection module is connected to the image temporary storage module and detects an object in the first image, the second image, the third image, or the fourth image to obtain an object image and calculate a distance between the object and the vehicle. The event trigger module is connected to the object detection module and triggers a recording event to record the object image when the distance is equal to or less than a predetermined distance. The display device is connected to the image processing device and simultaneously displays the surrounding image and the object image. The storage device is connected to the image processing device and stores the object image when a recording event is triggered. The warning device is connected to the image processing device and issues a warning message when a recording event is triggered.

[0007] Preferably, the first, second, third and fourth photographing devices are wide-angle photographing devices, and the detection ranges of the first and second photographing devices are wider than the width of the vehicle, and the detection ranges of the third and fourth photographing devices are wider than the length of the vehicle.

[0008] Preferably, the first, second, third and fourth image capturing devices can be arranged at an angle of 30° to 60° with respect to the horizontal plane.

[0009] Preferably, the object detection module can identify an object in the first image, the second image, the third image, or the fourth image through an artificial intelligence deep learning program.

[0010] Preferably, the object includes a pedestrian, a car, a motorcycle, or a traffic cone.

[0011] Preferably, the object detection module can determine the gap distance based on the camera parameters, the camera installation position, and the image stitching correction information.

[0012] Preferably, the predetermined distance includes a first predetermined distance and a second predetermined distance, and the first predetermined distance is greater than the second predetermined distance.

[0013] Preferably, the warning device comprises an indicator light and a buzzer, and when the distance is equal to or less than a first predetermined distance, the object image displayed on the display device emits a light warning by the indicator light, and when the distance is equal to or less than a second predetermined distance, the buzzer emits a sound warning.

[0014] Preferably, the first predetermined distance is 1 to 3 meters, and the second predetermined distance is within 1 meter. [Effects of the Invention]

[0015] As described above, the perimeter monitoring system with object detection function according to the present invention has one or more of the following advantages.

[0016] (1) The perimeter monitoring system with object detection function of the present invention acquires peripheral images through an image stitching module and provides a 360° bird's-eye view of the surrounding environment, helping the driver understand the situation around the vehicle and determine whether there are driving risks around the vehicle based on the actual image screen, effectively reducing the occurrence of vehicle accidents and improving driving safety.

[0017] (2) The perimeter monitoring system with object detection function of this invention uses the object detection module to detect objects in the image and calculate the distance between the object and the vehicle, thereby accurately determining the object's position and relative distance and reducing the risk of accidents. Furthermore, the system's automatic object detection function prevents the driver from paying excessive attention to the auxiliary peripheral image, which affects the normal driving vision, further improving driving safety.

[0018] (3) The perimeter monitoring system with object detection function of the present invention uses an event trigger module to monitor changes in the distance between an object and a vehicle, and issues a warning message when an object enters the warning range, allowing the driver to control the vehicle in advance to avoid traffic accidents, thereby improving safety and reducing economic losses. [Brief explanation of the drawings]

[0019] In order to make the technical features, contents, advantages and effects of the present invention clearer, the present invention will be described in detail in the form of an embodiment accompanied by the accompanying drawings. [Figure 1] 1 is a block diagram of a perimeter monitoring system with an object detection function according to an embodiment of the present invention; [Figure 2] 1 is a schematic diagram showing the installation position of the imaging device according to an embodiment of the present invention; [Figure 3] 3A and 3B are schematic diagrams showing the installation angles of the imaging device according to one embodiment of the present invention. [Figure 4] 1 is a schematic diagram of a display device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0020] In order to facilitate understanding of the technical features, contents, advantages, and achievable effects of the present invention, the present invention will be described in detail below in the form of an embodiment with reference to the accompanying drawings. The drawings used herein are for explanatory purposes only and may not reflect the actual proportions or precise configurations of the present invention after implementation. Therefore, it should be noted in advance that the proportions and configurations of the accompanying drawings should not be interpreted as limiting the scope of the actual implementation of the present invention.

[0021] 1 is a block diagram of a perimeter monitoring system with an object detection function according to one embodiment of the present invention. As shown, the perimeter monitoring system 100 with an object detection function includes a vehicle 10, a first image capturing device 11, a second image capturing device 12, a third image capturing device 13, a fourth image capturing device 14, an image processing device 15, a display device 16, a storage device 17, and an alarm device 18. The perimeter monitoring system 100 with an object detection function is mounted on a vehicle 10 that travels on a road, such as a gasoline-powered vehicle or an electric vehicle. While the vehicle 10 is traveling on a road or moving, the system monitors the surrounding environment.

[0022] The first camera 11 is located at the front of the vehicle 10 and captures a first image 111 facing forward. The second camera 12 is located at the rear of the vehicle 10 and captures a second image 121 facing backward. The third camera 13 is located on the left side of the vehicle 10 and captures a third image 131 facing leftward. The fourth camera 14 is located on the right side of the vehicle 10 and captures a fourth image 141 facing rightward. The detection ranges of the first camera 11 and the second camera 12 are wider than the width of the vehicle 10, and the detection ranges of the third camera 13 and the fourth camera 14 are wider than the length of the vehicle 10. By partially overlapping these detection ranges, a 360° panoramic view of the vehicle is captured. To cover the detection range, the first camera 11, the second camera 12, the third camera 13, and the fourth camera 14 may be wide-angle camera devices, or each camera may include multiple lenses so that the captured image covers the required range.

[0023] The image processing device 15 is connected to the first, second, third, and fourth image capturing devices 11, 12, 13, and 14, respectively, and receives the first, second, third, and fourth images 111, 121, 131, and 141. The image processing device 15 may be a system on chip (SoC) for an in-vehicle computer or a driver's mobile terminal. The image processing device 15 receives images from the image capturing devices via a wired or wireless connection. The image processing device 15 includes a processor 151 and a memory 152. The processor 151 accesses programs in the memory 152 to execute an image temporary storage module 153, an image stitching module 154, an object detection module 155, and an event trigger module 156.

[0024] The image temporary storage module 153 is configured to store the first image 111, the second image 121, the third image 131, and the fourth image 141 in a temporary storage area of ​​the memory 152. Since the image capturing devices are arranged in four directions around the vehicle 10, the captured images are oriented in different directions. These images are first saved in the temporary storage area for later use by the analysis module.

[0025] The image stitching module 154 is connected to the image temporary storage module 153 and accesses the first image 111, the second image 121, the third image 131, and the fourth image 141 in the temporary storage area. The image stitching module 154 stitches these individual images together to generate the surrounding image 161. Although the first image 111, the second image 121, the third image 131, and the fourth image 141 were captured in different directions, their detection ranges partially overlap. Based on this overlap, the image stitching module 154 uses an image stitching algorithm to synthesize these images into a seamless 360° surrounding image 161. This allows the user of the vehicle 10 to simultaneously observe the surrounding environment, improving visibility and driving safety. To prevent fisheye-like distortion caused by the wide-angle lens of the capture device, the image stitching module 154 uses an algorithm to correct this distortion, adapting to the user's visual inertia and preventing misjudgments.

[0026] The object detection module 155 is connected to the image storage module 153 and detects objects in the first image 111, the second image 121, the third image 131, or the fourth image 141 to obtain an object image 162 and calculate a distance between the object and the vehicle 10. The object detection module 155 uses an artificial intelligence deep learning algorithm to perform operations on the first image 111, the second image 121, the third image 131, or the fourth image 141 to identify the presence of specific objects, such as pedestrians, cars, motorcycles, traffic cones, etc., and mark the objects as subsequent tracking targets. The distance between the object and the vehicle 10 is determined by combining the position in the image with the camera parameters, the installation position of the camera, and image stitching correction information.

[0027] The event trigger module 156 is connected to the object detection module 155 and triggers a recording event to record an object image 162 when the distance between the object and the vehicle 10 falls below a predetermined distance. Because the camera continuously captures images of the surroundings, the distance between the object and the vehicle 10 changes over time when an object is detected. The event trigger module 156 can preset a predetermined distance and compare it with the predetermined distance as an event trigger criterion. For example, if the distance between the object and the vehicle 10 is less than one meter, the event trigger module 156 determines that there is a risk of contact or collision between the object and the vehicle 10, generates a warning message, and records the object image 162. In another embodiment, the predetermined distance includes a first predetermined distance and a second predetermined distance, and the first predetermined distance is greater than the second predetermined distance. The first predetermined distance may be between one and three meters, and the second predetermined distance may be within one meter. Presetting different distances can issue a warning when the distance is too far, or provide a clear warning or directly apply the brakes when the vehicle approaches a dangerous area. For example, when the distance between the detected object and the vehicle 10 is within 3 meters, only the image of the object may be displayed on a particular lens for the user to see, and the direction of the object may be displayed through different colored lights in the peripheral image 161. However, when the distance between the detected object and the vehicle 10 is within 1 meter, in addition to the visual and light warnings described above, a buzzer may sound to notify the user that there is a risk of contact or collision and that immediate action is required.

[0028] The display device 16 is connected to the image processing device 15 and simultaneously displays a peripheral image 161 and an object image 162. The display device 16 can be a vehicle monitor or the display screen of a user's mobile device. The peripheral image 161 generated by the image stitching module 154 and the object image 162 detected by the object detection module 155 can be simultaneously displayed on the display device 16. The user can observe the surrounding environment of the vehicle 10 through the peripheral image 161, achieving a 360-degree bird's-eye view of the surroundings. At the same time, the display device 16 displays an image from a single imaging device. For example, normally, an image of the front of the vehicle 10 captured by the first imaging device 11 at the front of the vehicle is displayed. When an object is detected, the display device 16 switches to displaying the object image 162 captured in the direction of the object. The distance between the object and the vehicle 10 can be determined by referring to auxiliary lines on the screen, and the detection result is visually displayed.

[0029] The storage device 17 is connected to the image processing device 15. When the event trigger module 156 determines that the distance to the object is equal to or less than a predetermined distance, a recording event is triggered, and an object image 162 is stored in the storage device. The storage device 17 can be a memory card, a hard drive, or various storage media. When an event is triggered, the object image 162 of the detected object is stored and can be accessed and reviewed later. During recording, a red dot or a display icon such as REC can be displayed on the screen of the display device 16 to notify the user that the system is recording. In other embodiments, the storage device 17 can continuously store the detection results, such as continuously storing the surrounding image 161 or the object image 162 after powering on, but this is an inherent function of the dashcam and is not limited to this.

[0030] The warning device 18 is connected to the image processing device 15. The event trigger module 156 triggers a recording event and issues a warning message 181 when it determines that the distance to the object is equal to or less than a predetermined distance. As described above, when an object is detected and the distance is equal to or less than a predetermined distance, it is necessary to notify the user using a warning message. In this embodiment, the warning device 18 may include an indicator light and a buzzer. When the distance is equal to or less than a first predetermined distance, the object image 162 displayed on the display device 16 turns on a warning light by changing the color of the detection area or by flashing the area where the object appears. When the distance is equal to or less than a second predetermined distance, the buzzer issues an audible warning to directly notify the user that an object is approaching.

[0031] FIG. 2 is a schematic diagram showing the installation locations of camera devices according to one embodiment of the present invention. As shown, the vehicle 20 may be a gasoline-powered vehicle, an electric vehicle, or the like. The vehicle 20 monitors its surroundings while traveling on a road. The first camera 21 is mounted on the front of the vehicle 20, e.g., on the hood between the headlights or on the front grille, and captures a first image directed toward the front of the vehicle 20. The second camera 22 is mounted on the rear of the vehicle 20, e.g., on the trunk lid between the taillights or on the license plate, and captures a second image directed toward the rear of the vehicle 20. The third camera 23 is attached to the left side of the vehicle 20, e.g., on the left rearview mirror, and captures a third image directed toward the left side of the vehicle 20. The fourth camera 24 is attached to the right side of the vehicle 20, e.g., on the right rearview mirror, and captures a fourth image directed toward the right side of the vehicle 20.

[0032] This four-way configuration allows each imaging device to capture detection ranges in all four directions using a wide-angle lens. The first detection range 211 of the first imaging device 21 and the second detection range 221 of the second imaging device 22 are wider than the width of the vehicle 20. The third detection range 231 of the third imaging device 23 and the fourth detection range 241 of the fourth imaging device 24 are wider than the length of the vehicle 20. As shown in the figure, the first detection range 211 partially overlaps the third detection range 231 and the fourth detection range 241 on the left and right. The second detection range 221 also partially overlaps the third detection range 231 and the fourth detection range 241 on the left and right. This allows the vehicle 20 to capture 360-degree panoramic images, and a surrounding image can be generated by image stitching.

[0033] 3A and 3B are schematic diagrams of the installation angles of the image capturing devices of this embodiment. Also, referring to FIG. 2, FIG. 3A is a side view of the installation angle of the image capturing device of this embodiment, and FIG. 3B is a rear view of the installation angle of the image capturing device of this embodiment. As shown in FIG. 3A, the first image capturing device 21 is installed at the front of the vehicle, and the second image capturing device 22 is installed at the rear. The second image capturing device 22 is installed at a higher height than the first image capturing device 21, but this is not limited to this, and the installation height can be adjusted depending on the appearance and front and rear shapes of the vehicle. Regarding the installation angle, the first image capturing device 21 and the second image capturing device 22 can be installed at an angle of 30° to 60° from the horizontal plane to capture the surroundings of the vehicle, particularly the surrounding road conditions. In this embodiment, both the first image capturing device 21 and the second image capturing device 22 are installed at an angle of 30° downward from the horizontal plane, which can be adjusted depending on the installation height of the image capturing device.

[0034] As shown in Figure 3B, the third camera 23 is attached to the left rearview mirror, and the fourth camera 24 is attached to the right rearview mirror. The third camera 23 is installed at the same height as the fourth camera 24. To capture images of the vehicle's surroundings, particularly the road, the third camera 23 and the fourth camera 24 can be installed at an angle of 30 to 60 degrees downward from the horizontal. In this embodiment, the angle of both camera devices is set to 30 degrees downward from the horizontal, but this can be adjusted depending on the installation height of the camera devices.

[0035] 4 is a schematic diagram of a display device according to one embodiment of the present invention. As shown, a first camera 31 is disposed at the front of the vehicle 30 and captures a first image facing forward. A second camera 32 is disposed at the rear of the vehicle 30 and captures a second image facing backward. A third camera 33 is disposed on the left side of the vehicle 30 and captures a third image facing left. A fourth camera 34 is disposed on the right side of the vehicle 30 and captures a fourth image facing right. After accessing the first, second, third, and fourth images, the image processing device stitches these individual images together to form a surrounding image 361. The surrounding image 361 is a 360-degree bird's-eye view image divided into 12 detection areas for detecting objects around the vehicle 30.

[0036] As shown on the right side of FIG. 4 , when a pedestrian 50 approaches the right rear of the vehicle 30, the object detection module uses an artificial intelligence deep learning algorithm to identify the pedestrian 50 as an object in the fourth image and further mark the pedestrian 50 as object 51 in the fourth image, forming object image 362. The display device 36 simultaneously displays the surrounding image 361 and the object image 362, allowing the user to simultaneously grasp the surrounding situation and clearly see the actual appearance of the object. This allows the user to more clearly grasp the situation around the vehicle 30 and prevent a collision with the pedestrian 50 when maneuvering the vehicle, such as backing up or parking.

[0037] When the distance between the pedestrian 50 and the vehicle 30 on the display screen falls below a predetermined distance, the event trigger module triggers a recording event and records an object image 362. This image is saved in a storage device, and at the same time, the color of the detection area in the surrounding image 361 changes to indicate the direction of the detected object and issue a warning to the user. If the distance continues to decrease during continuous detection and reaches a preset danger distance, a buzzer will sound to issue an audible warning, preventing a collision between the vehicle 30 and the pedestrian and improving driving safety.

[0038] The above description is for illustrative purposes only and is not intended to be limiting. Any equivalent modifications or variations that do not depart from the spirit and scope of the present invention are intended to be encompassed by the appended claims. [Explanation of symbols]

[0039] 10, 20, 30 vehicles 11, 21, 31 First imaging device 12, 22, 32 Second imaging device 13, 23, 33 Third imaging device 14,24,34 Fourth imaging device 15 Image processing device 16,36 Display device 17 Storage device 18 Warning device 50 Pedestrians 51 Object Mark 100 Perimeter monitoring system with object detection function 111 Image 1 121 2nd image 131 Third image 141 4th image 151 processors 152 memory 153 Image Temporary Storage Module 154 Image Stitching Module 155 Object Detection Module 156 Event Trigger Module 161,361 surrounding images 162,362 object images 181 Warning Messages 211 First detection range 221 Second detection range 231 Third detection range 241 4th detection range

Claims

1. A perimeter monitoring system with an object detection function, Vehicles and a first image capturing device disposed at a front portion of the vehicle and directed forward of the vehicle to capture a first image; a second image capturing device disposed at a rear portion of the vehicle and directed toward the rear of the vehicle to capture a second image; a third image capturing device disposed on the left side of the vehicle and directed toward the left side of the vehicle to capture a third image; a fourth image capturing device disposed on a right side of the vehicle and directed toward the right side of the vehicle to capture a fourth image; an image processing device connected to the first image capturing device, the second image capturing device, the third image capturing device, and the fourth image capturing device, respectively, and configured to receive the first image, the second image, the third image, and the fourth image; The processor of the image processing device accesses the memory and an image temporary storage module configured to store the first image, the second image, the third image, and the fourth image in a temporary storage area of ​​the memory; an image stitching module connected to the image temporary storage module, the image stitching module stitching the first image, the second image, the third image, and the fourth image to generate a peripheral image; an object detection module connected to the image temporary storage module, the object detection module detecting an object in the first image, the second image, the third image, or the fourth image to obtain an object image, and calculating a distance between the object and the vehicle; an event trigger module connected to the object detection module, the event trigger module triggering a recording event for recording the object image when the gap distance is equal to or less than a predetermined distance; The perimeter monitoring system having the object detection function further comprises: a display device connected to the image processing device and configured to simultaneously display the peripheral image and the object image; a storage device connected to the image processing device, the storage device storing the object image when the recording event is triggered; a warning device connected to the image processing device, the warning device issuing a warning message when the recording event is triggered; A perimeter monitoring system with an object detection function, comprising:

2. 2. The perimeter monitoring system with object detection function described in claim 1, characterized in that the first, second, third, and fourth photographing devices are wide-angle photographing devices, the detection ranges of the first and second photographing devices are wider than the width of the vehicle, and the detection ranges of the third and fourth photographing devices are wider than the length of the vehicle.

3. The perimeter monitoring system with object detection function described in claim 1, characterized in that the first imaging device, the second imaging device, the third imaging device, and the fourth imaging device are arranged at an angle of 30° to 60° downward from the horizontal plane.

4. The perimeter monitoring system with object detection function described in claim 1, characterized in that the object detection module identifies the object in the first image, the second image, the third image, or the fourth image using an artificial intelligence deep learning program.

5. The perimeter monitoring system with object detection function according to claim 4 , wherein the object includes a pedestrian, a car, a motorcycle, or a traffic cone.

6. The perimeter monitoring system with object detection function according to claim 4 , wherein the object detection module determines the gap distance based on the parameters of the camera, the installation position of the camera, and image stitch correction information.

7. 2. The perimeter monitoring system with object detection function according to claim 1, wherein the predetermined distance includes a first predetermined distance and a second predetermined distance, and the first predetermined distance is greater than the second predetermined distance.

8. 8. The perimeter monitoring system with object detection function described in claim 7, wherein the warning device is equipped with an indicator light and a buzzer, and when the distance is equal to or less than a first predetermined distance, the object image displayed on the display device emits a light warning by the indicator light, and when the distance is equal to or less than a second predetermined distance, the buzzer emits a sound warning.

9. 8. The perimeter monitoring system with object detection function according to claim 7, wherein the first predetermined distance is 1 to 3 meters, and the second predetermined distance is within 1 meter.

Images