Rear view image display system

The rear image display system addresses blind spots and unnatural image discontinuities by using multiple trailer-mounted cameras and hitch angle detection to generate a seamless and natural image of the towing vehicle's surroundings.

JP2024167770APending Publication Date: 2024-12-04ALPS ALPINE CO LTD

Patent Information

Application Number
JP2023084078
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-12-04

AI Technical Summary

Technical Problem

Existing rear image display systems for towing vehicles with trailers suffer from blind spots and unnatural image discontinuities due to varying hitch angles, which obscure camera views and create areas that cannot be captured or displayed naturally.

Method used

A rear image display system using multiple cameras positioned at the rear, right side, and left side of the trailer, along with a hitch angle detection mechanism, to generate a seamless and natural image by replacing obscured camera views with corresponding views from other cameras, ensuring a continuous and comprehensive view behind the towing vehicle.

Benefits of technology

The system effectively reduces blind spots and provides a natural, continuous image of the area behind the towing vehicle, enhancing visibility and reducing discontinuities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167770000001_ABST
    Figure 2024167770000001_ABST
Patent Text Reader

Abstract

To provide a "rear view image display system" to display a tractor's rear view image with less blind corners naturally.SOLUTION: A rear view image display system is configured to: replace regions of a region hidden by a trailer 1 in a rear view image b captured by a tractor rear camera 215 located at a rear side of a tractor 2, the regions including a scene viewed by the tractor rear camera 215, the scene being captured by a side camera of the trailer 1 without being hidden by the trailer 1, by video c1, c3 obtained by converting viewpoint of video captured by the side camera into a viewpoint of the tractor rear camera 215; and replace remaining regions including a scene viewed by the tractor rear camera 215 and captured by a rear camera of the trailer 1 without being hidden by the trailer 1, by video c2 obtained by converting viewpoint of video of the rear camera of the trailer 1 into a viewpoint of the tractor rear camera 215, to generate rear video d to be displayed. Each region is calculated based on a detected hitch angle of the trailer 1.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for displaying a rear image to a user of a towing vehicle towing a trailer. [Background technology]

[0002] One known technology for displaying rear-view images to a user of a towing vehicle towing a trailer (towed vehicle) is one that combines and displays images captured by a camera mounted at the rear of the trailer and images captured of the rear by a camera mounted on the towing vehicle to present a single scene (for example, Patent Document 1).

[0003] Also, a known technology for displaying rearward images to a user of a towing vehicle towing a trailer is to synthesize images captured by three cameras installed on the left and right sides and the rear of the trailer to generate a panoramic view image that broadly shows the surroundings of the trailer, and then cut out a fixed-size range from the panoramic view image and display it superimposed on the area of ​​the image captured behind the trailer by the camera at the rear of the towing vehicle (for example, Patent Document 2). Here, with this technology, the range cut out and displayed from the panoramic view image is changed according to the hitch angle, which is the relative angle between the front-to-rear direction of the trailer and the front-to-rear direction of the towing vehicle.

[0004] Other known technologies for detecting the hitch angle of a trailer (towed vehicle) include a technology that detects the hitch angle by recognizing the center axis of the trailer drawbar from images captured by a camera (e.g., Patent Document 3), a technology that calculates the hitch angle from the fore-and-aft distance from the towing vehicle to both left and right ends of the trailer measured with a laser rangefinder (e.g., Patent Document 4), and a technology that detects the trailer hitch angle from the inclination of a pattern of a marker affixed to the top surface of the trailer drawbar, which is captured by a camera (e.g., Patent Document 5). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-170826 A [Patent Document 2] Special Publication No. 2022-547068 [Patent Document 3] JP 2020-1631 A [Patent Document 4] Patent Publication No. 2021-177145 [Patent Document 5] JP 2019-199150 A Summary of the Invention [Problem to be solved by the invention]

[0006] According to a technology that combines and displays images captured by a camera mounted at the rear of the trailer with images captured of the rear from a camera mounted on the towing vehicle, when the trailer hitch angle becomes large, an area is created behind the towing vehicle that is obscured by the side of the trailer from the camera mounted on the towing vehicle, resulting in an area that cannot be captured by either camera and therefore cannot be displayed, which is a blind spot for the user of the towing vehicle.

[0007] In addition, in the technology described above of cutting out and displaying a portion of the panoramic view image in accordance with the hitch angle of the trailer, depending on the hitch angle, the area behind the towing vehicle that is hidden by the side of the trailer from the camera installed on the towing vehicle and that is not cut out from the panoramic view image is a blind spot for the user of the towing vehicle.

[0008] In addition, the image captured by a camera mounted on the towing vehicle and the image cut out from the panoramic view image have different viewpoints and viewing directions, and the size of the side of the trailer that appears in the image captured behind the vehicle by a camera at the rear of the towing vehicle, and therefore the size of the area in which the trailer appears in the image, changes depending on the trailer's hitch angle, whereas the size of the portion of the panoramic view image that is displayed and superimposed on the area in which the trailer appears is fixed, resulting in a discontinuous and unnatural image.

[0009] Therefore, an object of the present invention is to display an image showing the situation behind the towing vehicle in a more natural manner with fewer blind spots. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, the present invention provides a rear image display system that displays a rear image showing the situation behind a towing vehicle towing a trailer, the rear image display system comprising a towing vehicle rear camera that is a camera that takes images of the rear from the rear of the towing vehicle, a trailer rear camera that is a camera that takes images of the rear from the rear of the trailer, a trailer right camera that is a camera that takes images of the surroundings of the trailer from the right side of the trailer, a trailer left camera that is a camera that takes images of the surroundings of the trailer from the left side of the trailer, a hitch angle detection means that detects the hitch angle of the trailer, and a rear image generation unit that generates the rear image.Here, the rear image generating unit generates, based on the hitch angle detected by the hitch angle detection means, a blocked direction range which is a direction range seen from the towing vehicle rear camera in which the rear side is blocked by the trailer with respect to the towing vehicle rear camera, a right replacement direction range which is a direction range seen from the towing vehicle rear camera within the blocked direction range and which is not blocked at all by the trailer with respect to the trailer right camera, a left replacement direction range which is a direction range seen from the towing vehicle rear camera within the blocked direction range and which is not blocked at all by the trailer with respect to the trailer left camera, and a rear replacement direction range which is a direction range excluding the right replacement direction range and the left direction range and which is not blocked by the trailer with respect to the trailer rear camera. and replacement means for replacing a portion of the towing vehicle rear image, which is image captured by the towing vehicle rear camera, corresponding to the right replacement direction range with an image corresponding to the right replacement direction range of an image obtained by replacing the image captured by the trailer right camera with an image observed from the same viewpoint as the towing vehicle rear image, replacing a portion of the towing vehicle rear image corresponding to the left replacement direction range with an image corresponding to the left replacement direction range of an image obtained by replacing the image captured by the trailer left camera with an image observed from the same viewpoint as the towing vehicle rear image, and replacing a portion of the towing vehicle rear image corresponding to the rear replacement direction range with an image corresponding to the rear replacement direction range of an image obtained by replacing the image captured by the trailer rear camera with an image observed from the same viewpoint as the towing vehicle rear image, to generate the rear image.

[0011] Here, such a rear image display system may be configured in such a way that the replacement means replaces a portion of the towing vehicle rear image corresponding to a direction range within the blocked direction range excluding the right replacement direction range, the left replacement direction range, and the rear replacement direction range, with a predetermined pattern to generate the rear image.

[0012] In addition, in order to achieve the above-mentioned objective, the present invention provides a rear image display system that displays a rear image, which is an image showing the rear of a towing vehicle towing a trailer, comprising a towing vehicle rear camera that is a camera that takes images of the rear from the rear of the towing vehicle, a trailer rear camera that is a camera that takes images of the rear from the rear of the trailer, a trailer right camera that is a camera that takes images of the surroundings of the trailer from the right side of the trailer, a trailer left camera that is a camera that takes images of the surroundings of the trailer from the left side of the trailer, a hitch angle detection means that detects the hitch angle of the trailer, and a rear image generation unit that generates the rear image. Here, the rear image generating unit includes a range calculation means for calculating a right replacement direction range, which is a directional range overlapping the right side of the trailer as seen from the towing vehicle rear camera, a left replacement direction range, which is a directional range overlapping the left side of the trailer as seen from the towing vehicle rear camera, and a rear replacement direction range, which is a directional range overlapping the rear surface of the trailer as seen from the towing vehicle rear camera, based on the hitch angle detected by the hitch angle detection means, and a range calculation means for calculating a right replacement direction range, which is a directional range overlapping the left side of the trailer as seen from the towing vehicle rear camera, based on the hitch angle detected by the hitch angle detection means, and a rear replacement direction range, which is a directional range overlapping the rear surface of the trailer as seen from the towing vehicle rear camera, a replacement means for replacing the portion of the towing vehicle rear image corresponding to the right replacement direction range of an image obtained by replacing the image captured by the trailer left camera with an image observed from the same viewpoint as the towing vehicle rear image, and replacing the portion of the towing vehicle rear image corresponding to the rear replacement direction range with an image corresponding to the rear replacement direction range of an image obtained by replacing the image captured by the trailer rear camera with an image observed from the same viewpoint as the towing vehicle rear image, thereby generating the rear image.

[0013] Here, such a rear image display system may be configured such that the range calculation means calculates a mask direction range, which is a directional range that overlaps with the bottom surface of the trailer as seen from the towing vehicle rear camera, based on the hitch angle detected by the hitch angle detection means, and the replacement means replaces a portion of the towing vehicle rear image that corresponds to the mask direction range with a predetermined pattern to generate the rear image.

[0014] As described above, the rear image display system replaces the part of the image captured by the camera at the rear of the towing vehicle that is obscured by the trailer with an image captured in the same directional range as that part by three cameras positioned at the rear, right side, and left side of the trailer, and generates a rear image that shows the situation behind the towing vehicle.

[0015] Here, by using three cameras arranged at the rear, right side, and left side of the trailer, it is possible to capture images of the rear, right, and left sides of the trailer without omission. Additionally, any portion of the image captured by the camera at the rear of the towing vehicle that is obscured by the trailer is replaced with an image from the same perspective as that image. Therefore, with this rear image display system, as long as the trailer is generally transparent, it is possible to generate and display a rear image that shows a wide area behind the towing vehicle as observed from the viewpoint of the camera at the rear of the towing vehicle, seamlessly and without any unnaturalness, with few blind spots. Effect of the Invention

[0016] As described above, according to the present invention, it is possible to display an image showing the situation behind the towing vehicle in a more natural form with fewer blind spots. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a configuration of a driving assistance system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing a towing vehicle and a trailer according to an embodiment of the present invention. [Diagram 3] FIG. 4 is a diagram showing a hitch angle detected in an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing a range of use of a video according to an embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram showing a synthesis area of ​​an image according to an embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing an example of a rear view image according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described. FIG. 1 shows the configuration of a driving assistance system according to this embodiment. The driving assistance system assists the driving of a towing vehicle 2 that tows a trailer 1 such as a camper trailer shown in Figs. 2a and 2b. Returning to FIG. 1, the driving assistance system comprises a trailer side system 11 mounted on the trailer 1 and a towing vehicle side system 21 mounted on the towing vehicle 2. The trailer side system 11 is equipped with three cameras: a trailer right camera 111, a trailer rear camera 112, and a trailer left camera 113, and a transmitter 114 that wirelessly transmits images captured by each of the above cameras to the towing vehicle side system 21. The towing vehicle side system 21 also includes an input device 211, a status sensor 212 that detects various states of the towing vehicle 2, such as the forward / reverse driving state of the towing vehicle 2 and the steering angle of the towing vehicle 2, a display device 213, a receiving device 214 that receives images wirelessly transmitted by the transmitting device 114 of the trailer side system 11, a towing vehicle rear camera 215, a hitch angle sensor 216 that detects the hitch angle of the trailer 1, a rear image synthesis unit 217 that synthesizes the images of the trailer right camera 111, the trailer rear camera 112, and the trailer left camera 113 received by the receiving device 214 with the image taken by the towing vehicle rear camera 215 to generate a rear image and displays it on the display device 213, and a control unit 218 that controls each of the above units.

[0019] However, the images from the trailer right camera 111, trailer rear camera 112, and trailer left camera 113 may be transmitted from the trailer side system 11 to the towing vehicle side system 21 by wired transmission. In this case, the transmitter 114 and receiver 214 are not provided, and the images from the trailer right camera 111, trailer rear camera 112, and trailer left camera 113 are transmitted directly to the rear image synthesis unit 217. Next, as shown in Figures 2a and 2b, the trailer right camera 111 of the trailer side system 11 takes pictures of the rear from a position near the front end of the right side of the trailer 1, the trailer rear camera 112 takes pictures of the rear from a central position in the left-right direction of the rear of the trailer 1, and the trailer left camera 113 takes pictures of the rear from a position near the front end of the left side of the trailer 1.

[0020] In addition, the towing vehicle rear camera 215 of the towing vehicle side system 21 photographs the rear from a central position in the left-right direction of the rear of the towing vehicle 2. A drawbar 12 that connects to the chassis extends forward from the lower front end of the trailer 1, and the trailer 1 is connected to the towing vehicle 2 by engaging a coupler provided at the front end of the drawbar 12 with a hitch ball provided at the rear end of a hitch member 22 that extends from the lower rear end of the towing vehicle 2 to the rear of the towing vehicle 2 so that the coupler and hitch ball can swing around the vertical central axis of the coupler and hitch ball.

[0021] Therefore, as shown in Figures 3a and 3b, the trailer 1 can swing relative to the towing vehicle 2 around the vertical central axis of the coupler and hitch ball, and the angle θ of this swing is the hitch angle of the trailer 1 detected by the hitch angle sensor 216. The generation of the rear image performed by the rear image synthesizer 217 will now be described. Here, the rear image may be generated and displayed at all times, or may be generated only while the status sensor 212 detects that the towing vehicle 2 is moving backward, or may be generated and displayed in response to a user operation. In the following description, the image output by trailer right camera 111 is referred to as the trailer right image, the image output by trailer left camera 113 as the trailer left image, the image output by trailer rear camera 112 as the trailer rear image, and the image output by towing vehicle rear camera 215 as the towing vehicle rear image. However, it is assumed that each camera outputs an image with the left and right reversed (mirror image).

[0022] First, the positional relationship between each of the trailer right camera 111, trailer left camera 113 and trailer rear camera 112 and the towing vehicle rear camera 215 is determined from the hitch angle θ of the trailer 1 detected by the hitch angle sensor 216, and according to the determined positional relationships, a viewpoint conversion function is set that converts the trailer right image into an image taken from the position of the towing vehicle rear camera 215, a viewpoint conversion function that converts the trailer left image into an image taken from the position of the towing vehicle rear camera 215, and a viewpoint conversion function that converts the trailer rear image into an image taken from the position of the towing vehicle rear camera 215.

[0023] However, a viewpoint conversion function for each image corresponding to each hitch angle may be calculated and registered in advance, and when generating a rear image, the viewpoint conversion function registered corresponding to the hitch angle θ detected by the hitch angle sensor 216 may be used as the viewpoint conversion function to be set. In addition, the rear image synthesis unit 217 sets an extraction area, which is an area for extracting image portions to be synthesized from the trailer right image, the trailer left image, and the trailer rear image into the towing vehicle rear image, based on the hitch angle θ of the trailer 1 detected by the hitch angle sensor 216. The extraction region is set as follows. That is, for the left-right angular range in which the elevation / depression angle (up-down angle) as seen from the towing vehicle rear camera 215 shown in Figure 4a is γA, the angular range Sφm that is blocked by the trailer 1 with respect to the towing vehicle rear camera 215 is calculated, as shown in Figures 4b1 and 4b2, within the horizontal shooting angle of view φmR+Sφm+φmL of the towing vehicle rear camera 215, and is determined according to the hitch angle θ.

[0024] Then, within the angle range Sφm, when the trailer right image is viewpoint-converted to an image taken from the position of the towing vehicle rear camera 215 using the set viewpoint conversion function, the area of ​​the trailer right image corresponding to the angle range φR in which an image in the direction of elevation / depression angle γA behind the towing vehicle 2 appears in the viewpoint-converted trailer right image without being obstructed at all by the trailer 1 is determined according to the hitch angle θ.

[0025] Similarly, for other elevation / depression angles within the angle of view of the towing vehicle rear camera 215, the area of ​​the trailer right image corresponding to the angle range φR is determined according to the hitch angle θ, and the combined area determined for each elevation / depression angle is set as the extraction area for extracting the image portion from the trailer right image.

[0026] In addition, within the angle range Sφm, when the trailer left image is viewpoint-converted to an image taken from the position of the towing vehicle rear camera 215 using the set viewpoint conversion function, the area of ​​the trailer left image corresponding to the angle range φL in which the image in the direction of the elevation / depression angle γA behind the towing vehicle 2 appears in the viewpoint-converted trailer left image without being obstructed at all by the trailer 1 is determined according to the hitch angle θ.

[0027] Similarly, for other elevation / depression angles within the angle of view of the towing vehicle rear camera 215, the area of ​​the trailer left image corresponding to the angle range φL is determined according to the hitch angle θ, and the combined area determined for each elevation / depression angle is set as the extraction area for extracting the image portion from the trailer left image.

[0028] In addition, when the trailer rear image is viewpoint-converted to an image captured from the position of the towing vehicle rear camera 215 using the set viewpoint conversion function within the angle range SφRL between the angle ranges φR and φL, the area of ​​the trailer rear image corresponding to the angle range φB in which an image in the direction of elevation / depression angle γA behind the towing vehicle 2 appears in the viewpoint-converted trailer rear image is determined according to the hitch angle θ.

[0029] Similarly, for other elevation / depression angles within the angle of view of the towing vehicle rear camera 215, the area of ​​the trailer rear image corresponding to the angle range φB is determined according to the hitch angle θ, and the combined area determined for each elevation / depression angle is set as the extraction area for extracting the image portion from the trailer rear image.

[0030] Here, the angle ranges φR, φL, and φL for extracting the images are determined according to the hitch angle θ, and change according to the hitch angle θ, as shown in Figs. 4b1 and 4b2. Furthermore, the angle ranges φR, φL, and φB from which images are extracted differ for each range of elevation / depression angles as seen from the towing vehicle rear camera 215. For example, for the left-right range of elevation / depression angle γB in which the portion between the trailers 1 of the towing vehicle 2 is photographed by the towing vehicle rear camera 215 in Fig. 4a, there is no angle range Sφm in which the trailer 1 is shielded from the towing vehicle rear camera 215 (ignoring shielding by the drawbar 12). Also, for the left-right range of elevation / depression angle γC that is directly below the trailer 1 as seen from the towing vehicle rear camera 215, this is not photographed by the trailer right camera 111, the trailer left camera 113, or the trailer rear camera 112, so the angle ranges φR, φL, and φB are not set.

[0031] The extraction area may be set by determining and registering an extraction area for each image corresponding to each hitch angle in advance, and using the extraction area registered corresponding to the hitch angle θ detected by the hitch angle sensor 216 as the extraction area to be set when generating the rear image. Next, image portions within the extraction regions of the trailer right image, trailer left image, and trailer rear image set according to the hitch angle θ are extracted, and the extracted image portions are viewpoint-converted to an image captured from the position of the towing vehicle rear camera 215 using the set viewpoint conversion function. However, the order of extracting the image portions and converting the viewpoint may be reversed, and each image portion may be extracted after converting the viewpoint of the trailer right image, trailer left image, and trailer rear image to images taken from the position of the towing vehicle rear camera 215. Then, a rear image is generated by replacing an area corresponding to the trailer right image part of the towing vehicle rear image with a trailer right image part which is an image obtained by viewpoint conversion of an image part extracted from the trailer right image, replacing an area corresponding to the trailer left image part of the towing vehicle rear image with a trailer left image part which is an image obtained by viewpoint conversion of an image part extracted from the trailer left image, and replacing an area corresponding to the trailer rear image part of the towing vehicle rear image with the trailer rear image part which is an image obtained by viewpoint conversion of an image part extracted from the trailer rear image.

[0032] 5a1, b1, and c1 show the areas of the towing vehicle rear image that are replaced by the trailer right image portion, trailer left image portion, and trailer rear image portion in the above manner, with R being the area replaced by the trailer right image portion, L being the area replaced by the trailer left image portion, and B being the area replaced by the trailer rear image portion. Also, M indicates a blind spot area where the situation behind the towing vehicle 2 is not captured by any of the cameras.

[0033] Also, Figure 5a1 shows the areas of the rear image of the towing vehicle that are replaced when the hitch angle θ is as shown in Figure 5a2, Figure 5b1 shows the areas when the hitch angle θ is as shown in Figure 5b2, and Figure 5c1 shows the areas when the hitch angle θ is as shown in Figure 5c2, and as shown, these areas change depending on the hitch angle θ.

[0034] As shown in Figures 4 and 5, when the shape of the trailer 1 is approximately box-shaped (rectangular prism), the trailer right image portion and area R of the towing vehicle rear image replaced by the trailer right image portion will approximately represent the range in the direction overlapping with the right side of the trailer 1 as seen from the towing vehicle rear camera 215, the trailer left image portion and area L of the towing vehicle rear image replaced by the trailer left image portion will approximately represent the range in the direction overlapping with the left side of the trailer 1 as seen from the towing vehicle rear camera 215, the trailer rear image portion and area B of the towing vehicle rear image replaced by the trailer rear image portion will approximately represent the range in the direction overlapping with the rear of the trailer 1 as seen from the towing vehicle rear camera 215, and blind spot area M will approximately represent the range in the direction overlapping with the bottom of the trailer 1 as seen from the towing vehicle rear camera 215.

[0035] Therefore, by replacing areas R, L, and B of the towing vehicle rear image with the trailer right image portion, trailer left image portion, and trailer rear image portion, respectively, an image is generated which observes the situation behind the towing vehicle 2 by looking through each part of the trailer 1 except for the part that overlaps with the bottom surface of the trailer 1 from the position of the towing vehicle rear camera 215.

[0036] FIG. 6 shows an example of a rear view image generated in this manner. Now, in a situation where there is another vehicle 3 behind the trailer 1, a person 4 to the right of the trailer 1, and the trailer 1 blocks the other vehicle 3 and person 4 from the towing vehicle rear camera 215, as shown in Figure 6a, the towing vehicle rear camera 215 captures an image of the rear of the towing vehicle as shown in Figure 6b, the image obtained by viewpoint conversion of the trailer left image captured by the trailer left camera 113 is as shown in Figure 6c1, the image obtained by viewpoint conversion of the trailer rear image captured by the trailer rear camera 112 includes an image of the other vehicle 3 as shown in Figure 6c2, and the image obtained by viewpoint conversion of the trailer right image captured by the trailer right camera 111 includes an image of the person 4 as shown in Figure 6c3.

[0037] In this case, the rear view is generated and displayed as shown in FIG. 6d. As shown in the figure, the rear image is an image of the area in the towing vehicle rear image that is blocked from the towing vehicle rear camera 215 by various parts of the trailer 1 except for the part that overlaps with the bottom of the trailer, replaced with images of the areas indicated by dashed lines in the left trailer image after viewpoint conversion in Figure 6c1, the trailer rear image after viewpoint conversion in Figure 6c2, and the right trailer image after viewpoint conversion in Figure 6c3, and the rear image includes images of other vehicles 3 and people 4 that cannot be captured by the towing vehicle rear camera 215.

[0038] Therefore, the rear image seamlessly shows a wide area behind the towing vehicle 2, including images of other vehicles 3 and people 4, with few blind spots, as would be observed from the viewpoint of the towing vehicle rear camera 215 if the trailer 1 were transparent. In this example, for the blind spot area M shown in Fig. 5, where the view behind the towing vehicle 2 is not captured by any of the cameras, if the rear image of the towing vehicle were to be displayed as is, it would be discontinuous with the surroundings and would look unnatural, so the rear image is generated by filling it in with a single color. Note that this filling may be performed using other patterns. The embodiment of the present invention has been described above.

[0039] Here, as described above, when the shape of the trailer 1 is approximately box-shaped (rectangular parallelepiped), the trailer right image portion and area R of the towing vehicle rear image replaced with the trailer right image portion are approximately areas that represent a range in a direction overlapping with the right side of the trailer 1 as viewed from the towing vehicle rear camera 215, the trailer left image portion and area L of the towing vehicle rear image replaced with the trailer left image portion are approximately areas that represent a range in a direction overlapping with the left side of the trailer 1 as viewed from the towing vehicle rear camera 215, and the trailer rear image portion and area B of the towing vehicle rear image replaced with the trailer rear image portion are approximately areas that represent a range in a direction overlapping with the rear of the trailer 1 as viewed from the towing vehicle rear camera 215. Therefore, when applied to a trailer 1 that is approximately box-shaped (rectangular parallelepiped), in the above embodiment, the trailer right image portion, trailer left image portion, and trailer rear image portion may be generated as follows based on the hitch angle θ.

[0040] That is, the trailer right image portion may be obtained by extracting an area from the trailer right image that will represent the range of the direction in which the trailer right image overlaps with the right side of the trailer 1 when the viewpoint is converted, and converting the viewpoint, or by extracting an area of ​​the viewpoint-converted trailer right image that represents the range of the direction in which the trailer left image overlaps with the right side of the trailer 1 when the viewpoint is converted. Similarly, the trailer left image portion may be obtained by extracting an area from the trailer left image that will represent the range of the direction in which the trailer left image overlaps with the left side of the trailer 1 when the viewpoint is converted, and converting the viewpoint, or by extracting an area of ​​the viewpoint-converted trailer left image that represents the range of the direction in which the trailer left image overlaps with the left side of the trailer 1 when the viewpoint is converted. Also, the trailer rear image portion may be obtained by extracting an area from the trailer left image that will represent the range of the direction in which the trailer rear image overlaps with the rear of the trailer 1 when the viewpoint is converted, and converting the viewpoint, or by extracting an area of ​​the viewpoint-converted trailer rear image that represents the range of the direction in which the trailer rear image overlaps with the rear of the trailer 1 when the viewpoint is converted. [Explanation of symbols]

[0041] 1...trailer, 2...towing vehicle, 3...other vehicle, 4...person, 11...trailer side system, 12...drawbar, 21...towing vehicle side system, 22...hitch member, 111...trailer right camera, 112...trailer rear camera, 113...trailer left camera, 114...transmitter, 211...input device, 212...status sensor, 213...display device, 214...receiving device, 215...towing vehicle rear camera, 216...hitch angle sensor, 217...rear image synthesis unit, 218...control unit.

Claims

1. A rear image display system that displays a rear image that shows the rear of a towing vehicle towing a trailer, A towing vehicle rear camera that captures images from the rear of the towing vehicle; A trailer rear camera that captures images from the rear of the trailer; A trailer right camera that photographs the periphery of the trailer from the right side of the trailer; A trailer left camera that photographs the surroundings of the trailer from the left side of the trailer; a hitch angle detection means for detecting a hitch angle of the trailer; a rear image generating unit that generates the rear image, The rear image generating unit is a range calculation means for calculating, based on the hitch angle detected by the hitch angle detection means, a blocked direction range which is a direction range seen from the towing vehicle rear camera in which the rear side is blocked by the trailer with respect to the towing vehicle rear camera, a right replacement direction range which is a direction range within the blocked direction range seen from the towing vehicle rear camera which is not blocked at all by the trailer with respect to the trailer right camera, a left replacement direction range which is a direction range within the blocked direction range seen from the towing vehicle rear camera which is not blocked at all by the trailer with respect to the trailer left camera, and a rear replacement direction range which is a direction range excluding the right replacement direction range and the left direction range and which is not blocked by the trailer with respect to the trailer rear camera and a replacement means for replacing a portion of the towing vehicle rear image, which is image captured by the towing vehicle rear camera, corresponding to the right replacement direction range with an image obtained by replacing the image captured by the trailer right camera with an image observed from the same viewpoint as the towing vehicle rear image, replacing a portion of the towing vehicle rear image which corresponds to the right replacement direction range with an image obtained by replacing the image captured by the trailer left camera with an image observed from the same viewpoint as the towing vehicle rear image, and replacing a portion of the towing vehicle rear image which corresponds to the left replacement direction range with an image obtained by replacing the image captured by the trailer left camera with an image observed from the same viewpoint as the towing vehicle rear image, thereby generating the rear image.

2. A rear image display system that displays a rear image that shows the rear of a towing vehicle towing a trailer, A towing vehicle rear camera that captures images from the rear of the towing vehicle; A trailer rear camera that captures images from the rear of the trailer; A trailer right camera that photographs the periphery of the trailer from the right side of the trailer; A trailer left camera that photographs the surroundings of the trailer from the left side of the trailer; a hitch angle detection means for detecting a hitch angle of the trailer; a rear image generating unit that generates the rear image, The rear image generating unit is a range calculation means for calculating a right replacement direction range, which is a directional range overlapping with the right side of the trailer as viewed from the towing vehicle rear camera, a left replacement direction range, which is a directional range overlapping with the left side of the trailer as viewed from the towing vehicle rear camera, and a rear replacement direction range, which is a directional range overlapping with the rear surface of the trailer as viewed from the towing vehicle rear camera, based on the hitch angle detected by the hitch angle detection means; and a replacement means for replacing a portion of the towing vehicle rear image, which is image captured by the towing vehicle rear camera, corresponding to the right replacement direction range with an image obtained by replacing the image captured by the trailer right camera with an image observed from the same viewpoint as the towing vehicle rear image, replacing a portion of the towing vehicle rear image, which corresponds to the right replacement direction range, with an image corresponding to the right replacement direction range of an image obtained by replacing the image captured by the trailer left camera with an image observed from the same viewpoint as the towing vehicle rear image, and replacing a portion of the towing vehicle rear image, which corresponds to the left replacement direction range, with an image corresponding to the left replacement direction range of an image obtained by replacing the image captured by the trailer left camera with an image observed from the same viewpoint as the towing vehicle rear image, and replacing a portion of the towing vehicle rear image, which corresponds to the rear replacement direction range, with an image corresponding to the rear replacement direction range of an image obtained by replacing the image captured by the trailer rear camera with an image observed from the same viewpoint as the towing vehicle rear image, to generate the rear image.

3. 2. The rear image display system according to claim 1, The rear image display system is characterized in that the replacement means replaces a portion of the towing vehicle rear image corresponding to a direction range excluding the right replacement direction range, the left replacement direction range, and the rear replacement direction range in the blocked direction range with a predetermined pattern to generate the rear image.

4. 3. The rear image display system according to claim 2, the range calculation means calculates a mask direction range, which is a direction range overlapping with a bottom surface of the trailer as viewed from the towing vehicle rear camera, based on the hitch angle detected by the hitch angle detection means; The rear image display system according to claim 1, wherein the replacement means replaces a portion of the rear image of the towing vehicle that corresponds to the mask direction range with a predetermined pattern to generate the rear image.

Citation Information

Patent Citations

  • Towing support device

    JP2019199150A

  • Towing support device

    JP2020001631A

  • Method and apparatus for displaying peripheral scene of combination of vehicle and tracked vehicle

    JP2021170826A

  • Controller, control system and derivation method

    JP2021177145A

  • Trailer connection perspective view

    JP2022547068A

Cited By

  • Control apparatus, control method, and storage medium

    JP2026044539A