Rear monitoring system for vehicle and rear monitoring method for vehicle

Abstract

In the rear monitoring system for a vehicle and the rear monitoring method for a vehicle of the present application, the rear monitoring system for a vehicle includes: a lower camera that captures a road surface behind the truck; an upper camera that is disposed at a position higher than the lower camera and captures the road surface behind the truck; a conversion unit that converts a lower camera image (G1) captured by the lower camera into a lower bird's-eye view (T1) and converts an upper camera image (G2) captured by the upper camera into an upper bird's-eye view (T2); an object detection unit that compares the upper bird's-eye view and the lower bird's-eye view and detects an object on the road surface based on a difference in a correction portion of the object on the road surface between the upper camera image and the lower camera image; and a collision possibility determination unit that notifies a driver of the truck of a possibility of a collision when the object detected by the object detection unit has a possibility of a collision with the truck.

Description

Rear monitoring systems and methods for vehicles Technical Field

[0001] This disclosure relates to a rear-view monitoring system for vehicles and a method for monitoring the rear of vehicles. Background Technology

[0002] Vehicles exist that have a detection unit installed to detect objects around the vehicle for the purpose of preventing collisions. As a detection unit, an ultrasonic sensor, also known as a sonar sensor or gap sonar, is known (Patent Document 1). Additionally, a technique is known that uses a camera installed as a detection unit to extract objects from images captured by the camera through machine learning (Patent Document 2).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent document 1: Japanese Patent Application Publication No. 2022-23870.

[0006] Patent Document 2: Japanese Patent Application Publication No. 2022-16027. Summary of the Invention

[0007] The problem the invention aims to solve

[0008] In this case, when detecting objects behind a freight vehicle, the detection unit needs to be mounted on a vehicle-mounted device that serves as a shelf. However, the shape and size of the vehicle-mounted device vary depending on the intended use. Therefore, when installing a gap sonar like that in Patent Document 1, the optimal location differs depending on the type of vehicle-mounted device. Furthermore, since various protrusions and movable parts exist in the vehicle-mounted device, there is a possibility that the ultrasonic waves irradiated by the gap sonar may encounter these parts and misdetect the device as objects around the vehicle. Therefore, sometimes it is difficult to install a gap sonar on a freight vehicle due to issues with installation location and potential for misdetection.

[0009] Furthermore, the method for extracting objects using machine learning, as described in Patent Document 2, suffers from the following problems: the need for massive amounts of data for machine learning, and the time and cost of learning from that data. Additionally, the camera mounting heights on freight vehicles vary widely, depending on the onboard equipment, and are not fixed. Therefore, the observed appearance of objects differs depending on the mounting height. Consequently, there is a problem with the inability to perform machine learning with high precision.

[0010] This disclosure was made in view of the above-mentioned problems, and its purpose is to provide a rear-view monitoring system for a vehicle that can easily determine the possibility of a collision between an object behind the vehicle and the vehicle based on images captured by a monocular camera.

[0011] Solution to the problem

[0012] One aspect of this disclosure for achieving the above-mentioned objective is a vehicle rear-view monitoring system that monitors objects located behind the vehicle. It is characterized by comprising: a lower camera, mounted on the vehicle, which is a monocular camera for capturing images of the road surface behind the vehicle; an upper camera, positioned above the lower camera on the vehicle, which is also a monocular camera for capturing images of the road surface behind the vehicle; and a conversion unit that converts the image of the road surface captured by the lower camera (i.e., the lower camera image) into a top-down view of the road surface from above (i.e., a lower bird's-eye view), and converts the image captured by the upper camera... The image of the road surface, i.e., the image from the upper camera, is converted into a top-down view of the road surface, i.e., an upper bird's-eye view; an object detection unit compares the upper bird's-eye view and the lower bird's-eye view, and detects the object located on the road surface based on the difference between the portions of the object hidden on the road surface in the upper camera image and the lower camera image; and a collision probability determination unit determines the probability of the object detected by the object detection unit colliding with the vehicle, and if it is determined that there is a possibility of a collision, informs the driver of the vehicle that there is a possibility of a collision.

[0013] Another aspect of this disclosure is a method for monitoring the rear of a vehicle, which monitors objects located behind the vehicle. It is characterized by comprising: a shooting step, in which a lower camera mounted on the vehicle and an upper camera positioned above the lower camera capture images of the road surface behind the vehicle; both the lower and upper cameras are monocular cameras; and a conversion step, in which the image of the road surface captured by the lower camera (i.e., the lower camera image) is converted into a top-down view of the road surface from above (i.e., a lower bird's-eye view), and the image of the road surface captured by the upper camera is converted into... The process involves converting the upper camera image into a top-down view of the road surface, i.e., an upper bird's-eye view; an object detection process comparing the upper and lower bird's-eye views, and detecting the object on the road surface based on the difference between the lower and upper camera images, where the object is hidden and not reflected; and a collision probability determination process determining the probability of a collision between the object detected by the object detection process and the vehicle, and in the case of a collision probability determination, informing the driver of the vehicle of the possibility of a collision.

[0014] Invention Effects

[0015] According to this disclosure, a rear-view monitoring system for a vehicle can be provided that can easily determine the likelihood of a collision between the vehicle and an object behind it based on images captured by a monocular camera. Attached Figure Description

[0016] Figure 1 is a side view of a freight vehicle equipped with a rear monitoring system according to an embodiment of the present disclosure.

[0017] Figure 2 is a functional block diagram of the rear monitoring system.

[0018] Figure 3 is a diagram illustrating the steps of detecting objects behind a freight vehicle using a rear monitoring system. In Figure 3, (a) is a side view and (b) is a top view of (a).

[0019] In Figure 4, (a) is a schematic diagram of the image behind the vehicle captured by the lower camera, i.e., the lower camera image, and (b) is a schematic diagram of the image behind the vehicle captured by the upper camera, i.e., the upper camera image.

[0020] In Figure 5, (a) is a schematic diagram of converting the lower camera image in Figure 4(a) into a lower bird's-eye view, and (b) is a schematic diagram of converting the upper camera image in Figure 4(b) into an upper bird's-eye view.

[0021] Figure 6 is a flowchart illustrating the steps of the rear-end monitoring method using the rear-end monitoring system of this embodiment.

[0022] Explanation of reference numerals in the attached figures

[0023] 1: Rear monitoring system

[0024] 3: Lower camera

[0025] 5: Upper camera

[0026] 13: Monitoring and Control Department

[0027] 15: Conversion Section

[0028] 17: Object Detection Department

[0029] 17a: Difference-related information

[0030] 19: Collision Probability Determination Department

[0031] 23: Display Section

[0032] 25: Speaker

[0033] 27: Rudder Angle Sensor

[0034] 29: Speed ​​sensor

[0035] 31: Pillar

[0036] 33, 35: White lines

[0037] 41, 43, 45: Parking areas

[0038] 61: Frontend

[0039] 64: Difference

[0040] 65: Correction Department

[0041] 100: Truck

[0042] 103: Chassis

[0043] 105: Control Room

[0044] 107: Shelves

[0045] 109: Road surface Detailed Implementation

[0046] Hereinafter, suitable embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. As a rear monitoring system 1, an example is shown that is, a system for detecting objects on the road surface 109 behind the rack 107 of a truck 100, the truck 100 being a vehicle with a van-type rack 107. Furthermore, in the following figures, the X direction is defined as the forward / backward direction of the truck 100, the Y direction as the width direction of the truck 100, and the Z direction as the vertical direction.

[0047] First, the general structure of the truck 100 will be described with reference to FIG1. ​​The truck 100 shown in FIG1 includes: a chassis 103, which supports the equipment constituting the truck 100; a driver's cab 105, which is provided at the front end of the chassis 103; a rack 107 provided at the rear of the driver's cab 105 on the chassis 103; and a rear monitoring system 1.

[0048] Next, the detailed structure of the rear monitoring system 1 will be described with reference to Figures 1 to 5. The rear monitoring system 1 detects objects on the road surface 109 behind the shelf 107 and informs the driver of the truck 100 of the possibility of a collision if the truck 100 collides with a detected object. As shown in Figures 1 and 2, the rear monitoring system 1 includes a lower camera 3, an upper camera 5, and a monitoring control unit 13.

[0049] The lower camera 3 is a monocular camera installed on the truck 100 to capture images of the road surface 109 behind the truck's cargo rack 107. Regarding the specific camera structure, any known monocular camera can be used. Furthermore, the light source for the lower camera 3 to capture images of the road surface 109 can be any type of light located at the rear of the truck 100, such as a taillight (not shown), but a camera that does not require a light source, such as an infrared camera, can also be used.

[0050] To facilitate filming the rear of the truck 100, the lower camera 3 is preferably positioned at the rear of the truck 100 in the longitudinal direction. If the lower camera 3 is positioned at the center of the vehicle width direction, it can film an area of ​​equal length to the left and right of the vehicle width, which is preferred. The lower camera 3 is positioned vertically to film objects behind the vehicle, and preferably at a lower position. However, if the position is too low, there is a possibility that the lower camera 3 will come into contact with the road surface 109; therefore, it is positioned within a range that does not contact the road surface 109. Specifically, as shown in Figure 1, it is preferably positioned at the rear of the chassis 103.

[0051] As shown in Figures 3(a) and 3(b), the shooting range R1 of the lower camera 3 includes the road surface 109 behind the truck 100. The viewing angle and optical axis A2 of the lower camera 3 are set to be able to capture the shooting range R1. Furthermore, since the lower camera 3 is positioned near the road surface 109, the optical axis A2 can be horizontal. However, it is preferable that the rear end of the shelf 107 enters the shooting range R1 of the lower camera 3. This is because it is easier to determine the distance between the rear end of the shelf 107 and the detected object in the captured image.

[0052] For example, as shown in Figure 3(b), the truck 100 reverses in the direction of X1 to park in parking area 43 of the parking areas 41, 43, 45 enclosed by white lines 33 and 35. Furthermore, the parking area 43 is assumed to have pillars 31 of height H1, such as traffic cones, making it an area where direct parking is not possible. In this case, if we schematically represent the image captured by the lower camera 3, i.e., the lower camera image G1, it is as shown in Figure 4(a). As shown in Figure 4(a), in the lower camera image G1 captured by the lower camera 3, at least the rear end of the shelf 107, the white lines 33 and 35, and the pillar 31 are visible. However, in the lower camera image G1, the portion behind the pillar 31 is hidden behind the pillar 31 and is not visible.

[0053] The upper camera 5 shown in Figures 1 and 2 is a monocular camera positioned above the lower camera 3 on the truck 100, used to capture images of the road surface 109 behind the truck 100. Specifically, the structure of the upper camera 5 is the same as that of the lower camera 3.

[0054] Similar to the lower camera 3, to facilitate capturing images of the rear of the truck 100, the upper camera 5 is preferably positioned at the rear of the truck 100 in the longitudinal direction. The upper camera 5 is positioned the same as the lower camera 3 in the width direction. The upper camera 5 is positioned in the height direction to capture objects behind the vehicle, and preferably at the top. This is because, in this embodiment, the images captured by the lower camera 3 and the upper camera 5 are converted into a bird's-eye view. Objects are detected based on the difference in height between the bird's-eye view and the lower camera 3. The greater the height difference between the lower camera 3 and the upper camera 5, the greater the difference in the bird's-eye view, and the easier it is to detect objects at a lower height. Specifically, it is preferably positioned at a height sufficient to detect a child as a pedestrian, and more preferably at the top and rear of the shelf 107.

[0055] As shown in Figures 3(a) and 3(b), the shooting range R1 of the upper camera 5 includes the road surface 109 behind the truck 100. The viewing angle and optical axis A1 of the upper camera 5 are set so that it can capture images within the shooting range R1. However, it is preferable that the rear end of the shelf 107 enters the shooting range R1 of the upper camera 5. This is because it is easier to determine the distance between the rear end of the shelf 107 and the detected object in the captured image.

[0056] For example, as shown in Figures 3(a) and 3(b), when truck 100 is reversing in the direction of X1 to park in parking area 43, the image captured by the upper camera 5, i.e., upper camera image G2, is schematically represented as shown in Figure 4(b). As shown in Figure 4(b), in the upper camera image G2 captured by the upper camera 5, at least the rear end of the shelf 107, white lines 33 and 35, and support column 31 are visible. In the upper camera image G2, the part behind the support column 31 is also hidden behind the support column 31 and is not visible. However, the upper camera image G2 is taken from a position higher than the lower camera 3, therefore, the part hidden behind the support column 31 is smaller than the part hidden behind the support column 31 in the lower camera image G1 captured by the lower camera 3 shown in Figure 4(a).

[0057] The monitoring and control unit 13 shown in Figure 2 is a computer that detects objects based on the lower camera image G1 and the upper camera image G2 captured by the lower camera 3 and the upper camera 5, respectively, of the area behind the truck 100. The monitoring and control unit 13 is also a computer that informs the driver of the possibility of a collision if a detected object is detected and there is a possibility of a collision with the truck 100. The monitoring and control unit 13 is, for example, located in the driver's cab 105. As shown in Figure 2, the monitoring and control unit 13 includes a conversion unit 15, an object detection unit 17, and a collision probability determination unit 19.

[0058] The conversion unit 15 converts the lower camera image G1 of the road surface 109 captured by the lower camera 3 into a top-down view of the road surface 109, i.e., a lower bird's-eye view T1. The conversion unit 15 also converts the upper camera image G2 of the road surface 109 captured by the upper camera 5 into a top-down view of the road surface 109, i.e., an upper bird's-eye view T2. The virtual viewpoint for generating the lower bird's-eye view T1 and the upper bird's-eye view T2 can be set above the center of the image within the shooting range R1. Furthermore, known image processing techniques can be used to generate the bird's-eye view. In Figure 2, the conversion unit 15 is included in the structure of the monitoring and control unit 13, but it is also possible that the lower camera 3 and the upper camera 5 each have their own conversion unit 15. Moreover, if the truck 100 is equipped with a device that displays bird's-eye views, such as a panoramic monitor, the device for generating bird's-eye views displayed on the panoramic monitor can be replaced by the conversion unit 15.

[0059] Figure 5(a) shows a schematic diagram of converting the lower camera image G1, captured by the lower camera 3 in Figure 4(a), which shows the view behind the truck 100, into a lower bird's-eye view T1. Figure 5(b) shows a schematic diagram of converting the upper camera image G2, captured by the upper camera 5 in Figure 4(b), which shows the view behind the truck 100, into an upper bird's-eye view T2.

[0060] As shown in Figures 5(a) and 5(b), objects without height, such as white lines 33 and 35, have shapes in the lower bird's-eye view T1 and upper bird's-eye view T2 that are identical to their actual shapes. Furthermore, the shapes and sizes of white lines 33 and 35 shown in the lower bird's-eye view T1 and upper bird's-eye view T2 are identical. On the other hand, objects with height, such as pillar 31, have shapes in the lower bird's-eye view T1 and upper bird's-eye view T2 that are significantly different from their actual shapes. Moreover, comparing the lower bird's-eye view T1 and upper bird's-eye view T2 reveals significant differences in shape and size. Specifically, in the lower camera image G1 and upper camera image G2, image correction was performed by setting the area behind the portion hidden behind pillar 31 that is not shown to be the same as pillar 31. Since the pillar 31 is located near the center of the truck 100 in the width direction, image correction is performed such that a trapezoidal correction portion 65 is set behind the portion of the pillar 31 that is not shown in the lower camera image G1 and the upper camera image G2. Regarding the size of the correction portion 65, it is smaller in the upper bird's-eye view T2 shown in FIG. 5(b) than in the lower bird's-eye view T1 shown in FIG. 5(a). This is because the upper bird's-eye view T2 is a bird's-eye view converted from the upper camera image G2 of the road surface 109 captured by the upper camera 5; therefore, the portion behind the pillar 31 that is obscured is smaller compared to the lower camera image G1.

[0061] The object detection unit 17 shown in Figure 2 compares the upper bird's-eye view T2 and the lower bird's-eye view T1, and detects the portion of the support pillar 31 located on the road surface 109 based on the difference 64 of the correction unit 65. The correction unit 65 is the portion of the support pillar 31 located on the road surface 109 that is hidden in the lower camera image G1 and the upper camera image G2 and not reflected. Specifically, based on the difference 64, the position, height, and width in the vehicle width direction of the support pillar 31, which is an object, are detected. The specific method for detecting these can be exemplified below.

[0062] First, the upper bird's-eye view T2 and the lower bird's-eye view T1 are compared, and the parts with different appearances are extracted. Specifically, the difference 64 of the correction part 65 between the upper bird's-eye view T2 and the lower bird's-eye view T1 shown in Figure 5(a) and Figure 5(b) is extracted. This part is the rear of the part where the object exists.

[0063] Next, regarding the object's position on the plane, the upper bird's-eye view T2 and the lower bird's-eye view T1 can be compared, and the object's position on the plane can be detected based on the position of the difference 64. Specifically, since the difference 64 shown in Figure 5(a) is located behind the support column 31, there is a correlation between the position of the difference 64 and the actual position of the support column 31. Therefore, it can also be configured such that, as shown in Figure 2, the object detection unit 17 has a correlation that has been experimentally determined in advance as the aforementioned correlation, and the position of the difference 64 is applied to the difference-related information 17a to determine the position of the support column 31. Alternatively, the correlation between the position of the difference 64 and the actual position of the support column 31 can be determined by calculation rather than by experiment. In addition, the difference 64 is part of the correction unit 65, which is a part that is identical to the support column 31 and has undergone image correction. Therefore, the front end 61 of the correction unit 65 can also be detected as the center position of the support column 31. Once the position of the support column 31 is detected, the distance D between the support column 31 and the rear end of the cab 105 of the truck 100 can be detected based on the upper bird's-eye view T2 and the lower bird's-eye view T1.

[0064] Next, as a method for detecting the height of the support column 31, the following two methods can be exemplified. First, the following method can be exemplified: comparing the upper bird's-eye view T2 and the lower bird's-eye view T1, and detecting the height of the object based on the length ΔL in the front-back direction of the difference 64. This is because the higher the support column 31, the greater the length ΔL; therefore, there is a correlation between the length ΔL of the difference 64 and the actual height of the object. Regarding this correlation between the length ΔL of the difference 64 and the actual height of the object, it can also be as shown in Figure 2, where the object detection unit 17 has a correlation calculated beforehand through experiments as a difference-related information 17a. In this case, the object detection unit 17 calculates the length ΔL of the difference 64 and applies the calculated value to the difference-related information 17a to detect the height of the object. Alternatively, the correlation between the length ΔL and the height of the support column 31 can also be calculated rather than an experimental value.

[0065] As a method for detecting the height of the support column 31, the following method can also be exemplified: compare the upper bird's-eye view T2 and the lower bird's-eye view T1, and detect the height of the object based on the area ΔS of the difference 64 of the correction unit 65. This is because the higher the support column 31 is, the larger the area ΔS is; therefore, there is a correlation between the area ΔS and the height of the support column 31. Regarding this correlation, the object detection unit 17 may have a correlation obtained experimentally as a difference-related information 17a, or the correlation may be obtained through calculation.

[0066] Regarding whether to use the length ΔL or the area ΔS of the difference 64, the appropriate choice can be made by considering the advantages of each. For example, compared to the method of detecting the height of an object based on the area ΔS of the difference 64, the method of detecting the height of an object based on the length ΔL of the difference 64 is advantageous in terms of detection accuracy because it has a stronger correlation with the height of the object. On the other hand, compared to the method of calculating the length ΔL, the method of detecting the height of an object based on the area ΔS of the difference 64 is advantageous because it does not require setting a reference point for calculating the length, and therefore the processing of calculating the area ΔS is easier.

[0067] Next, as a method for detecting the width of the support column 31, the following two methods can be exemplified. First, the following method can be exemplified: compare the upper bird's-eye view T2 and the lower bird's-eye view T1, and detect the width of the object based on the width in the vehicle width direction of the difference 64. For example, detect the shortest width ΔW in the Y direction of the difference 64 of the correction part 65 of the lower bird's-eye view T1 shown in Figure 5(a). This is because the longer the width of the support column 31, the longer the shortest width ΔW, therefore, there is a correlation between the shortest width ΔW and the width of the support column 31. Regarding the correlation between the shortest width ΔW of the difference 64 and the actual width of the object, the object detection unit 17 can have a correlation obtained in advance through experiments as the difference-related information 17a shown in Figure 2, or the correlation can be obtained through calculation.

[0068] As a method for detecting the width of the support column 31, the following method can also be exemplified: the width of the object is detected based on the area ΔS of the difference 64 of the correction section 65. For example, the width of the support column 31 is detected based on the area ΔS of the difference 64 of the correction section 65 in the lower bird's-eye view T1 shown in FIG. 5(a). This is because the longer the width of the support column 31, the larger the area ΔS is; therefore, there is a correlation between the area ΔS and the width of the support column 31. Regarding this correlation, the object detection unit 17 may have a correlation obtained experimentally as a difference-related information 17a, or the correlation may be obtained through calculation.

[0069] The collision probability determination unit 19 shown in Figure 2 is a device that determines the probability of a collision between an object detected by the object detection unit 17 and the truck 100, and informs the driver of the truck 100 of the possibility of a collision if it is determined that a collision is possible. As a method to inform the driver of the truck 100 of the possibility of a collision, a speaker 25 can be installed in the driver's cab 105 as shown in Figure 2 to convey the possibility of a collision through sound. Alternatively, a display unit 23 for displaying text, graphics, etc., can be installed in the driver's cab 105, and the possibility of a collision can be conveyed through text on the display unit 23. In particular, when a panoramic monitor is provided in the driver's cab 105 of the truck 100, the panoramic monitor can also be used as the display unit 23, and the possibility of a collision can be conveyed by emphasizing the detected object displayed in the image on the panoramic monitor.

[0070] In this way, the rear monitoring system 1 compares the bird's-eye view generated from the camera images captured by the lower camera 3 and the upper camera 5, and detects objects based on the difference 64 between the parts of the camera images that are hidden from view and not reflected, thus determining the likelihood of a collision with the vehicle. Therefore, the rear monitoring system 1 can easily determine the likelihood of an object behind the truck 100 colliding with the truck 100 based on the image captured by the monocular camera. In particular, if the truck 100 has a device that displays a bird's-eye view like a panoramic monitor, the device that generates the bird's-eye view displayed on the panoramic monitor can be replaced by the conversion unit 15. Therefore, without adding image processing devices or software, the likelihood of an object behind the truck 100 colliding with the truck 100 can be easily determined.

[0071] Specifically, as a criterion for the collision probability determination unit 19 to determine that a collision is possible, an example can be given when the distance D between the rear end of the truck 100 and the object detected by the object detection unit 17—the pillar 31 in Figure 3—is less than or equal to a predetermined distance. The predetermined distance is a value obtained by multiplying the braking distance by a predetermined safety factor. This braking distance is the distance at which the truck 100 will stop before colliding with the pillar 31 if the driver of the truck 100 applies the brakes.

[0072] In this way, if the distance D between the pillar 31 and the rear end of the truck 100 is less than a predetermined distance, it is determined that there is a possibility of collision. Therefore, as long as the driver applies the brakes when he knows that there is a possibility of collision, he can avoid a collision with the pillar 31.

[0073] Furthermore, the predetermined distance used by the collision probability determination unit 19 to determine the probability of a collision may vary depending on the relative speed and direction of movement of the truck 100 and the object. For example, if the object is a moving object such as a pedestrian rather than a support pillar 31, the predetermined distance is shorter when the pedestrian is walking towards the vehicle compared to when the object is a stationary support pillar 31. Additionally, the faster the truck 100 or the object moves relative to each other, the shorter the predetermined distance. Therefore, the collision probability determination unit 19 obtains the direction and speed of the truck 100 from the steering angle sensor 27 and speed sensor 29 installed on the truck 100 to set the predetermined distance. Furthermore, the lower camera 3 and the upper camera 5 capture images of the road surface 109 at predetermined time intervals to generate a bird's-eye view. Objects are detected using the differential 64, and changes in the position and size of the differential 64 are obtained. The speed and direction of the object's movement are then detected to set the predetermined distance. More specifically, the closer the truck 100 and the object are to each other and the faster their relative speeds are, the shorter the specified distance is set.

[0074] In this way, the collision probability determination unit 19 sets a predetermined distance based on the relative speed between the truck 100 and the detected object and the direction of movement, thereby enabling it to determine the probability of a collision between the truck 100 and the detected object with higher accuracy.

[0075] Furthermore, the collision probability determination unit 19 only treats objects with a height above a predetermined height among the detected objects as objects with a probability of collision. The predetermined height can be exemplified, for example, by the height of the lowest portion of the truck 100's height above the road surface 109, i.e., the lowest ground height H2, as shown in FIG3(a). This is because objects, such as small stones on the road surface 109, with a height lower than the lowest ground height H2, have a low probability of colliding with the truck 100's rack 107.

[0076] Furthermore, the conversion unit 15, object detection unit 17, and collision probability determination unit 19 shown in Figure 2 can each be configured as hardware. Alternatively, they can be configured as a general-purpose computer that enables a computer to perform these functions and uses software-based instructions to perform these functions. The above is a detailed description of the structure of the rear monitoring system 1.

[0077] Next, the steps of the rear monitoring method using the rear monitoring system 1 will be briefly described with reference to FIG6. First, the monitoring control unit 13 and the conversion unit 15 of the monitoring control unit 13 shown in FIG2 issue instructions to the lower camera 3 and the upper camera 5 to capture images of the shooting range R1 of the road surface 109 behind the truck 100 at predetermined time intervals. Upon receiving the instructions, the lower camera 3 and the upper camera 5 capture images of the shooting range R1 and send the captured images G1 and G2 from the lower camera to the conversion unit 15 (S1, shooting process in FIG6).

[0078] Next, the conversion unit 15 shown in FIG2 converts the acquired lower camera image G1 and upper camera image G2 into a lower bird's-eye view T1 and an upper bird's-eye view T2, and sends them to the object detection unit 17 (S2 of FIG6, conversion process). The object detection unit 17 compares the lower bird's-eye view T1 and the upper bird's-eye view T2, and extracts the difference 64 of the correction unit 65, which is the part of the lower camera image G1 and the upper camera image G2 that is hidden in the objects located on the road surface 109 and is not reflected. Furthermore, the object detection unit 17 detects the objects located on the road surface 109 based on the difference 64 of the correction unit 65, calculates the position, height, width, moving speed and direction of the detected objects, and sends them to the collision probability determination unit 19 (S3 of FIG6, object detection process).

[0079] Next, the collision probability determination unit 19 shown in FIG2 determines whether the object detected by the object detection unit 17 has a probability of colliding with the truck 100. If it is determined that there is a probability of collision, it proceeds to S5; if it is determined that there is no probability of collision, it returns (S4 in FIG6). When it is determined in S4 that there is a probability of collision between the object and the truck 100, the collision probability determination unit 19 uses the display unit 23 shown in FIG2 or the speaker 25 to inform the driver of the truck 100 of the probability of collision (S5 in FIG6). Furthermore, S4 and S5 are also referred to as the collision probability determination process. The above is an explanation of the steps of the rear monitoring method using the rear monitoring system 1.

[0080] Thus, according to this embodiment, the rear monitoring system 1 includes: a lower camera 3, an upper camera 5, a conversion unit 15, an object detection unit 17, and a collision probability determination unit 19. In this structure, a bird's-eye view generated from camera images of the lower camera 3 and the upper camera 5 is compared, and objects are detected based on the difference 64 between the parts of the camera images that are hidden from view and not reflected, thereby determining the probability of an object colliding with the truck 100. Therefore, the probability of an object behind the truck 100 colliding with the truck 100 can be determined simply based on the image captured by the monocular camera.

[0081] The present disclosure has been described above based on the embodiments; however, the present disclosure is not limited to the embodiments. Those skilled in the art will naturally conceive of various modifications and improvements within the scope of the technical concept of the present disclosure, and these are also included in the present disclosure.

Claims

1. A rear-view monitoring system for vehicles, used to monitor objects on the road surface behind a vehicle with a rack, characterized in that, The system includes: a lower camera, a monocular camera positioned at the rear end of the vehicle's rack, for capturing images of the road surface behind the vehicle; an upper camera, positioned above the lower camera at the rear end of the vehicle's rack, also a monocular camera for capturing images of the road surface behind the vehicle; a conversion unit that converts the image of the road surface captured by the lower camera (lower camera image) into a top-down view of the road surface from above (lower bird's-eye view), and converts the image of the road surface captured by the upper camera (upper camera image) into a top-down view of the road surface from above (upper bird's-eye view); and an object detection unit that compares the upper bird's-eye view and the lower bird's-eye view, and detects objects located on the road surface based on the difference between the upper camera image and the lower camera image that are hidden and not reflected in the image. The system includes a collision probability determination unit, which determines the probability of a collision between the object detected by the object detection unit and the vehicle. If a collision is determined to be possible, the system informs the driver of the vehicle of the possibility of a collision. The object detection unit detects the position of the object based on the position of the difference and detects the height of the object based on the length of the difference in the vehicle's longitudinal direction or the area of ​​the difference. The collision probability determination unit only determines objects detected by the object detection unit that are at least a predetermined distance from the vehicle and have a predetermined height or more as objects with a probability of a collision.

2. The vehicle rear monitoring system as described in claim 1, wherein, The collision probability determination unit determines that there is a possibility of a collision if the distance between the vehicle and the detected object is within a predetermined distance, and informs the driver of the vehicle of the possibility of a collision.

3. The vehicle rear monitoring system as described in claim 2, wherein, The collision probability determination unit sets the predetermined distance based on the relative speed between the vehicle and the object and the direction of movement.

4. A method for monitoring the rear of a vehicle, wherein objects on the road surface located behind a vehicle with a cargo rack are monitored, characterized in that, include: The shooting process involves a lower camera positioned at the rear of the vehicle's rack and an upper camera positioned above the lower camera at the rear of the vehicle's rack, both of which are monocular cameras. The conversion process involves converting the image of the road surface captured by the lower camera (the lower camera image) into a top-down view of the road surface (a lower bird's-eye view), and converting the image of the road surface captured by the upper camera (the upper camera image) into a top-down view of the road surface (an upper bird's-eye view). The object detection process involves comparing the upper bird's-eye view and the lower bird's-eye view, and detecting the object located on the road surface based on the difference between the lower camera image and the upper camera image, specifically the portion of the object hidden on the road surface that is not reflected. The system includes a collision probability determination process, which determines the probability of a collision between the object detected by the object detection process and the vehicle. If a collision is determined to be possible, the driver of the vehicle is informed of the possibility of a collision. In the object detection process, the position of the object is detected based on the position of the difference, and the height of the object is detected based on the length of the difference in the front-rear direction of the vehicle or the area of ​​the difference. In the collision probability determination process, only objects detected in the object detection process that are within a specified distance from the vehicle and have a height above a specified height are determined to be objects with a probability of collision.

Images