Video display control system and video display control method

Abstract

To provide an image display control system which enables a rider to obtain an image of the rear side of the own vehicle when necessary while facing the front side of a motor cycle.SOLUTION: An image display control system according to the present invention comprises: an image control device which has a composite image data generation unit (42) that generates composite image data by combining image data captured by a plurality of cameras (11, 12, 13) that image the rear side of a saddle-riding type vehicle (100); and an image display device (50) which displays the composite image on the front visual field of a rider of the saddle-riding type vehicle by performing image processing of the composite image data. The image control device transmits the composite image data to the image display device (50) when detecting a signal from a direction indicator (70) of the saddle-riding type vehicle by the input from the rider.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image display control system and method for controlling an image displayed in the forward field of view of a rider while the rider is traveling in a vehicle, particularly a straddle-type vehicle. [Background technology]

[0002] A conventional image display system for motorcycles is known that includes an imaging device, a vehicle direction sensor, a control device, and a wireless communication device, and image data of the area behind the motorcycle captured by the imaging device is transferred by the wireless communication device to a display device installed in the helmet, allowing the rider to view the area behind the motorcycle without having to check the monitor display installed in the side mirror or instrument panel (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2017-30530 Summary of the Invention [Problem to be solved by the invention]

[0004] Since the image displayed on the display device can obstruct the rider's forward visibility, it is desirable to display the necessary image on the display device when needed. According to the invention of Patent Document 1, when the rider's head angle is within a predetermined reference angle range including 0 degrees, the image captured by the imaging device is not displayed on the display screen. This prevents the image from being displayed on the display screen when the rider is looking ahead, ensuring the rider's forward visibility. However, when a rider wants to check ahead while simultaneously checking for the presence of vehicles behind, for example, when changing lanes into the passing lane on a highway to overtake a vehicle ahead, the invention of Patent Document 1 makes it impossible for the rider to check for the presence of vehicles behind while facing forward.

[0005] The present invention has been made in light of the above-mentioned problems, and aims to provide an image display control system and method that enables a rider to obtain an image of the area behind the motorcycle when necessary while still facing forward. [Means for solving the problem]

[0006] The image display control system of the present invention comprises an image control device having a composite image data generation unit that generates composite image data by synthesizing image data captured by multiple cameras that capture images of the area behind a saddle-riding vehicle, and an image display device that displays the composite image in the forward field of view of a rider of the saddle-riding vehicle by image processing the composite image data, and when the image control device detects a signal from a turn signal of the saddle-riding vehicle input by the rider, it transmits the composite image data to the image display device.

[0007] The control method according to the present invention includes the steps of acquiring image data by capturing images of the area behind a saddle-riding vehicle using a plurality of cameras, synthesizing the image data to generate composite image data, detecting signals from turn signals of the saddle-riding vehicle input by a rider, evaluating the signals and transmitting the composite image data to an image display device, and displaying the composite image in the rider's forward field of view by image processing the composite image data. [Effects of the Invention]

[0008] In the image display control system according to the present invention, when the turn signal of the saddle-riding type vehicle is turned on by the rider's input, the generated composite image is displayed in the rider's field of view in front of the rider, allowing the rider to view the composite image covering a wide area behind the saddle-riding type vehicle while facing forward, and since the composite image is not displayed when the turn signal is not turned on, it is possible to provide a system that does not excessively obstruct the rider's field of view. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an explanatory diagram illustrating a video display control system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing a shooting range of the video display control system according to the embodiment of the present invention. [Figure 3] 1 is a block diagram of a video display control system according to an embodiment of the present invention. [Figure 4] 5 is a flowchart showing the operation of a device provided in a vehicle in the video display control system according to the first embodiment of the present invention. [Figure 5] 1 is a diagram illustrating a scene in which a video display control system according to a first embodiment of the present invention is applied. [Figure 6] 10 is a flowchart showing the operation of a device provided in a vehicle in a video display control system according to a second embodiment of the present invention. [Figure 7] FIG. 10 is a diagram illustrating a scene in which a video display control system according to a second embodiment of the present invention is applied. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a video display control system and a video display control method according to the present invention will be described with reference to the drawings.

[0011] The configuration, operation, etc. described below are merely examples, and the video display control system according to the present invention is not limited to such configuration, operation, etc.

[0012] For example, although the following description will be given of a case in which the image display control system according to the present invention is used in a motorcycle, the image display control system according to the present invention may also be used in saddle-ride vehicles other than motorcycles. A saddle-ride vehicle refers to any vehicle on which a rider straddles. Examples of saddle-ride vehicles include motorcycles (motorcycles and motor tricycles), buggies, bicycles, etc. A motorcycle includes a motor cycle or motor tricycle that uses an engine as a propulsion source, and a motor cycle or motor tricycle that uses an electric motor as a propulsion source, and includes, for example, a motorcycle, a scooter, an electric scooter, etc. Furthermore, a bicycle refers to any vehicle that can be propelled down a road by the rider's pedaling force applied to the pedals.

[0013] In the following description, structural details are appropriately simplified or omitted from the drawings, and overlapping or similar descriptions are appropriately simplified or omitted.

[0014] (First embodiment) Fig. 1 shows an image display control system according to an embodiment of the present invention. As shown in Fig. 1, this embodiment is an application of the image display control system of the present invention to a system that displays an image behind a saddle-riding type vehicle 100. The image display control system 1 in Fig. 1 includes a right rear camera 11 that takes images of the right rear of the saddle-riding type vehicle 100, a left rear camera 12 that takes images of the left rear of the saddle-riding type vehicle 100, a central rear camera 13 that takes images directly behind the saddle-riding type vehicle, an image control device 40, an image display device 50, a turn signal 70, and a rider assistance device 80, which are provided in either the saddle-riding type vehicle 100 as a vehicle or a helmet 3 as a head-mounted device for a rider.

[0015] 1, the image display device 50 is provided on the helmet 3. The image display device 50 includes a projector 51, a processing device 52, a storage unit 53, and a wireless communication device 62.

[0016] 1, the projector 51 is provided at a predetermined upper position inside the helmet main body 4. Also, inside the helmet main body 4, light emitted from the projector 51 is projected onto a predetermined position on the transparent shield 5 provided in front of the helmet main body 4, and an image based on the light emitted from the projector 51 is projected onto the predetermined position on the transparent shield 5. The predetermined portion of the transparent shield 5 is semi-transparent, and transmits external light, and also reflects the image from the projector 51 toward the eyes of the rider wearing the helmet 3. Under the control of the processing device 52, the wireless communication device 62 performs wireless communication with the video control device 40 provided in the saddle-ride type vehicle in accordance with a predetermined short-range communication standard (for example, Bluetooth (registered trademark)). The processing device 52 processes the composite image data received from the image control device 40 via the wireless communication device 62 and causes the projector 51 to project the composite image data onto the predetermined portion of the transparent shield 5 of the helmet 3 . The storage unit 53 stores programs that can be executed by the processing device 52, various information required for processing by the processing device 52, information generated by processing by the processing device 52, and the like.

[0017] The position of the display screen in the forward field of view of the rider wearing the helmet 3 may be any position that does not interfere with the user's view of the front, for example, any position other than the center of the forward field of view F, and may be the upper right, upper left, lower right, or lower left part of the forward field of view F, or may be the upper, lower, left, or right part of the forward field of view F. The mounting position and angle of the projector 51 relative to the helmet 3 are determined depending on the position of the display screen in the forward field of view F.

[0018] Furthermore, although the helmet 3 shown in FIG. 1 is a full-face type helmet, the helmet 3 is not limited to a full-face type helmet. Furthermore, the position where the image display device 50 is attached to the helmet 3 is not limited, and the image display device 50 may be attached to another position as long as the display screen is within the user's forward field of view. For example, in this embodiment, the projector 51 is provided separately from the processing device 52, etc., but the projector 51, processing device 52, memory unit 53, and wireless communication device 62 may be integrated on a single board. Furthermore, in this embodiment, the light emitted from the projector 51 is directly irradiated onto the transparent shield 5 provided in front of the helmet main body 4. However, an optical system (not shown) may be provided that guides the light emitted from the projector 51 to a predetermined portion of the transparent shield 5 provided in front of the helmet main body 4, so that an image based on the light emitted from the projector 51 is projected onto the predetermined portion of the transparent shield 5.

[0019] Furthermore, the image display device 50 may be attached to eyewear 3' shown in Fig. 1 instead of the helmet 3. The eyewear may be, for example, smart glasses, which can project composite image data transferred by wireless communication from the image control device 40 via a projector 51' onto transparent glass 5'. A laser module and a mirror deflection unit are incorporated into the projector 51', and in this embodiment, an electronic circuit for driving the projector, a wireless communication device 62 that receives composite image data transferred from the image control device 40, a processing device that processes the composite image data, a storage device, etc. are provided in a module 54' separate from the projector 51'.

[0020] Fig. 1 shows a view of a saddle-riding type vehicle 100 as seen from above. The right rear camera 11, left rear camera 12, and central rear camera 13 provided on the saddle-riding type vehicle 100 shown in Fig. 1c are, for example, CCD cameras. As shown in Fig. 1, the right rear camera 11 is provided at the position of the right side mirror of the saddle-riding type vehicle 100, the left rear camera 12 is provided at the position of the left side mirror of the saddle-riding type vehicle 100, and the central rear camera 13 is provided near the tail lamp at the rearmost part of the saddle-riding type vehicle 100. Note that the cameras 11, 12, and 13 may be provided at other positions on the saddle-riding type vehicle 100 as long as they can capture images of predetermined areas around the saddle-riding type vehicle 100 shown in Fig. 2.

[0021] FIG. 2 shows the shooting directions and shooting ranges of the right rear camera 11, the left rear camera 12, and the center rear camera 13. The right rear camera 11 captures images in a capturing range 11a in Fig. 2. Therefore, the right rear camera 11 can include in its capturing range vehicles traveling in the right lane adjacent to the lane in which the saddle riding vehicle 100 is traveling. In addition, because the right rear camera 11 covers a wide angle, it can also include in its capturing range an area 11b to the side of the right side mirror, which can be a blind spot with a fixed side mirror.

[0022] Similarly, the imaging direction of the left rear camera 12 is to the left rear of the saddle riding type vehicle 100, and the left rear camera 12 captures an area in imaging range 12a in Fig. 2. Therefore, the left rear camera 12 can include in its imaging range vehicles traveling in the left lane adjacent to the lane in which the saddle riding type vehicle 100 is traveling. In addition, the imaging range 12a can also include an area 12b that would be a blind spot in a fixed side mirror.

[0023] The shooting direction of the central rear camera 13 is directly behind the saddle riding type vehicle 100, and the central rear camera 13 shoots an area in a shooting range 13a in Fig. 2. The central rear camera 13 can include in its shooting range vehicles traveling behind the driving lane of the saddle riding type vehicle 100, which cannot be photographed by the right rear camera 11 or the left rear camera 12.

[0024] 1 , the image control device 40 provided in the saddle-riding vehicle 100 is a device that combines image data captured by the right rear camera 11, the left rear camera 12, and the center rear camera 13, and controls the transfer of the combined image data to the image display device 50 in accordance with the driving state of the saddle-riding vehicle 100. In this embodiment, the image control device 40 is provided in front of the handlebars of the saddle-riding vehicle 100, and is provided within an instrument connectivity cluster (ICC) 90 that displays various instruments and navigation information for the saddle-riding vehicle 100 and controls wireless connections with external devices. However, the installation location of the image control device 40 is not limited thereto, and the image control device 40 may be provided within the image display device 50, or may be implemented as part of a control unit that controls the engine or brakes of the saddle-riding vehicle 100.

[0025] The turn signal 70 is composed of a signal switch 71 input by the rider, turn signal indicator lights 72 that turn on when the signal switch is input, and a relay circuit (not shown) that flashes the turn signal indicator lights. Note that this configuration is just one example, and for example, the relay circuit that controls the flashing of the turn signal indicator lights may be integrated into a control unit that controls other electrical components. In this embodiment, the signal switch 71 is provided near the left handlebar, allowing the rider to operate the signal switch 71 while holding the handlebar. The turn signal indicator lights are arranged one on each side of the headlamp of the saddle-riding type vehicle 100 (not shown), and one on each side of the tail lamp.

[0026] Furthermore, when the saddle riding vehicle 100 is turning right or left or changing lanes, either the right or left turn signal lights are turned on in response to an input from the rider. When a traffic jam occurs and the saddle riding vehicle 100 is weaving its way through traffic jammed vehicles, it is also possible to turn on hazard lights that simultaneously turn on both the left and right turn signal lights in response to an input from the rider. However, in implementing the present invention, it is not essential that the saddle riding vehicle 100 have a hazard light lighting function.

[0027] The rider assistance device 80 includes a surrounding environment detection unit 84 such as a radar, Lidar sensor, ultrasonic sensor, camera, or the like, a driving state detection device 85 that detects the vehicle speed and acceleration occurring in the vehicle, and a control unit 81 that performs driving assistance control of the vehicle based on information from the surrounding environment detection unit 84 and the driving state detection device 85. The control unit 81 in the present invention also includes an acquisition unit 82 that acquires surrounding environment information of the saddle riding type vehicle 100 based on the output of the surrounding environment detection unit 84, and an analysis unit 83 that analyzes whether or not the saddle riding type vehicle 100 is passing through obstacles based on the surrounding environment information acquired by the acquisition unit 82.

[0028] As shown in FIG. 1 , the image control device 40 provided in the saddle-riding vehicle 100 and the image display device 50 provided in the helmet 3 can exchange information via wireless communication. That is, the saddle-riding vehicle 100 is provided with a wireless communication device 61, and the image display device 50 is provided with a wireless communication device 62. The wireless communication device 61 provided in the saddle-riding vehicle 100 transmits image data output from the image control device 40 to the wireless communication device 62 provided in the image display device 50, and the wireless communication device 62 outputs image data received from the wireless communication device 61 to the image display device 50. In this way, since the image control device 40 provided in the saddle-riding vehicle 100 and the image display device 50 provided in the helmet 3 can exchange information via wireless communication, there is no need to connect the saddle-riding vehicle 100 and the helmet 3 with a cable. Because there is no cable between the saddle-riding vehicle 100 and the helmet 3, the rider can easily get on and off the saddle-riding vehicle 100 or drive the saddle-riding vehicle 100.

[0029] FIG. 3 is a block diagram of the video display control system 1.

[0030] The image control device 40 is a device that generates a composite image by combining the image data captured by the right rear camera 11, the left rear camera 12, and the center rear camera 13, and controls the transmission of the composite image data to the image display device 50 in accordance with the vehicle state. Specifically, when a signal from a turn signal 70 is detected, the composite image data is transferred to the image display device 50 via the wireless communication device 61. Furthermore, even when a signal from the turn signal 70 is detected, the image display control unit 43 controls to stop the transmission of the composite image data to the image display device if the signal from the rider assistance device indicates that the saddle-ride type vehicle is passing through other vehicles. As shown in FIG. 3, the video control device 40 includes at least a video data input unit 41, a composite video data generation unit 42, a video display control unit 43, and a storage unit 44.

[0031] The video data input unit 41 has a function of inputting the video data captured by the right rear camera 11, the left rear camera 12, and the center rear camera 13.

[0032] The composite video data generating unit 42 reads the video data input to the video data input unit 41 and uses the computer program stored in the storage unit 44 to generate a composite video. Specifically, the image data I-11 captured by the right rear camera and the image data I-13 captured by the central rear camera are combined to generate composite image data CP-1. That is, the composite image data CP-1 is image data obtained by combining images of the image capturing range 11a and the image capturing range 13a in FIG. 2 . The composite image data CP-1 is projected onto the transparent shield 5, which serves as a display unit, via the image display device 50. Therefore, for example, when a rider changes lanes of the saddle riding type vehicle 100 to an adjacent lane on the right, the rider can visually check for vehicles ahead traveling in the same lane as the saddle riding type vehicle 100, while also checking images of vehicles traveling in the adjacent lane on the right and vehicles traveling behind the saddle riding type vehicle 100 from the images projected on the transparent shield 5. This allows the rider to simultaneously check for the presence or absence of vehicles ahead, behind, and to the right rear of the saddle riding type vehicle 100 while facing forward when changing lanes to an adjacent lane on the right.

[0033] The composite video data generation unit 42 generates composite video data CP-2 by combining video data I-12 captured by the left rear camera and video data I-13 captured by the central rear camera. In this way, even when the rider changes lanes of the saddle riding type vehicle 100 to the adjacent lane on the left, the rider can simultaneously check for the presence or absence of vehicles ahead, behind, and to the left rear while keeping his or her head facing forward.

[0034] When the composite video data generating unit 42 generates the composite video data, there may be some overlapping portions between the video captured by the right rear camera 11 (or the video captured by the left rear camera 12) and the video captured by the central rear camera 13. In this case, a known method such as a reliability mapping method may be used to generate a composite video with a higher resolution.

[0035] An example of determining whether or not a rider is passing through a line of vehicles by the rider support device will be described with reference to FIGS. 3 and 7. FIG. As shown in FIG. 3, when the saddle-riding vehicle 100 is passing between the left lane L1 and the right lane L2, the acquisition unit 82 acquires ambient environment information of the saddle-riding vehicle 100 based on the output of the ambient environment detection unit 84.

[0036] 7, it is assumed that the saddle riding type vehicle 100 is currently passing through a convoy of vehicles (traveling through positions PO2' and PO3' in FIG. 7). The acquisition unit 82 acquires left relative speed information, based on the output of the surrounding environment detection unit 84, that indicates the relative speeds of left convoy vehicles V1', V3', V5', and V7', which are vehicles belonging to the left convoy, a convoy of vehicles located to the left of the driving line of the saddle riding type vehicle 100 (i.e., the left lane L1 in the example of FIG. 7), relative to the saddle riding type vehicle 100. The left convoy vehicle is preferably the vehicle that is closest in relative distance to the saddle riding type vehicle 100. This configuration improves the accuracy of analyzing whether or not a convoy of vehicles is passing through a convoy of vehicles. Furthermore, the left relative speed information may be information that indicates the average relative speed of two or more left convoy vehicles relative to the saddle riding type vehicle 100. This configuration improves the accuracy of analyzing whether or not a convoy of vehicles is passing through a convoy of vehicles. Among the vehicles positioned to the left of the travel line DL of the saddle riding type vehicle 100, vehicles whose relative distance to the saddle riding type vehicle 100 is below a reference value may be selected as left-side convoy vehicles. This configuration improves the accuracy of the analysis of whether or not a convoy is passing through vehicles. The left relative speed information may be information about a vehicle positioned to the left and in front of the saddle riding type vehicle 100. This configuration makes it possible to execute more appropriate rider assistance operations based on a future prediction of whether or not a convoy is passing through vehicles. The left relative speed information may also be information about a vehicle positioned to the left or left rear of the saddle riding type vehicle 100. Even in such cases, it is possible to execute appropriate rider assistance operations. The left relative speed information may be a speed difference in a direction parallel to the travel line of the saddle riding type vehicle 100, a differential value of the inter-vehicle distance, or any other physical quantity that can be substantially converted into these. If the left relative speed information is the speed difference in a direction parallel to the driving line DL of the saddle riding type vehicle 100, or another physical quantity that can be substantially converted to the speed difference, the accuracy of the analysis of whether or not a vehicle is passing through a line of vehicles is improved.

[0037] Furthermore, the acquisition unit 82 acquires left-side density information, which is information indicating the density of multiple left-side convoy vehicles V1', V3', V5', and V7', based on the output of the surrounding environment detection unit 84. Preferably, vehicles positioned to the left of the driving line of the saddle riding type vehicle 100 whose relative distance to the saddle riding type vehicle 100 is below a reference value are selected as left-side convoy vehicles. This configuration improves the accuracy of analyzing whether or not a convoy is passing through vehicles. Furthermore, the left-side density information is preferably information about a vehicle positioned to the left and in front of the saddle riding type vehicle 100. This configuration makes it possible to perform more appropriate rider assistance operations based on future predictions of passing through vehicles. The left-side density information may also be information about a vehicle positioned to the left or rear of the saddle riding type vehicle 100. Even in such cases, it is possible to perform appropriate rider assistance operations. The left-hand density information may be the reciprocal of the distance between two left-hand convoy vehicles, or the average value of the reciprocals of the distances between three or more left-hand convoy vehicles 201, or the number of left-hand convoy vehicles 201 located within a specified area, or the time interval between overtaking by saddle-type vehicles 100, or any other physical quantity that can be substantially converted into any of the above.

[0038] Furthermore, the acquisition unit 21 acquires right-side relative speed information, based on the output of the surrounding environment detection unit 84, that indicates the relative speeds of right-side convoy vehicles V2', V4', V6', and V8', which are vehicles belonging to a right-side convoy that is a convoy located to the right of the saddle riding vehicle 100's travel line (i.e., the right lane L2 in the example of FIG. 7), relative to the saddle riding vehicle 100. The right-side convoy vehicles are preferably vehicles that are closest in relative distance to the saddle riding vehicle 100. This configuration improves the accuracy of analyzing whether or not a convoy is passing through vehicles. The right-side relative speed information may also be information that indicates the average relative speed of two or more right-side convoy vehicles relative to the saddle riding vehicle 100. This configuration improves the accuracy of analyzing whether or not a convoy is passing through vehicles. Of the vehicles located to the right of the travel line DL of the saddle riding vehicle 100, vehicles whose relative distance to the saddle riding vehicle 100 is below a reference value are preferably selected as right-side convoy vehicles. Such a configuration improves the accuracy of the analysis of whether or not a vehicle is passing through a convoy. The right relative speed information may be information about a vehicle located to the right and in front of the saddle riding vehicle 100. Such a configuration makes it possible to execute more appropriate rider assistance operations based on future predictions of whether or not a vehicle is passing through a convoy. The right relative speed information may also be information about a vehicle located to the right of or to the right rear of the saddle riding vehicle 100. Even in such cases, it is possible to execute appropriate rider assistance operations. The right relative speed information may be a speed difference in a direction parallel to the traveling line DL of the saddle riding vehicle 100, a differential value of the inter-vehicle distance, or another physical quantity that can be substantially converted to such a value. When the right relative speed information is a speed difference in a direction parallel to the traveling line of the saddle riding vehicle 100, or another physical quantity that can be substantially converted to such a value, the accuracy of the analysis of whether or not a vehicle is passing through a convoy is improved.

[0039] Furthermore, the acquisition unit 21 acquires right-side density information, which is information indicating the density of multiple right-side convoy vehicles V2', V4', V6', and V8', based on the output of the surrounding environment detection unit 84. Preferably, vehicles positioned to the right of the saddle riding type vehicle 100 whose relative distance to the saddle riding type vehicle 100 is below a reference value are selected as right-side convoy vehicles. This configuration improves the accuracy of analyzing whether or not a convoy is passing through vehicles. Furthermore, the right-side density information is preferably information about a vehicle positioned to the right and in front of the saddle riding type vehicle 100. This configuration makes it possible to perform more appropriate rider assistance operations based on future predictions of passing through vehicles. The right-side density information may also be information about a vehicle positioned to the right or rear of the saddle riding type vehicle 100. Even in such cases, it is possible to perform appropriate rider assistance operations. The right-hand density information may be the reciprocal of the distance between two right-hand convoy vehicles, or the average value of the reciprocals of the distances between three or more right-hand convoy vehicles 301, or the number of right-hand convoy vehicles located within a specified area, or the time interval between overtaking by saddle-type vehicles 100, or any other physical quantity that can be substantially converted into any of the above.

[0040] The analysis unit 22 analyzes whether or not the saddle riding type vehicle 100 is passing through other vehicles based on the surrounding environment information acquired by the acquisition unit 82.

[0041] Specifically, the analysis unit 83 determines that a saddle-type vehicle 100 is passing through vehicles when the left relative speed information indicates a relative speed below a standard, the left density information indicates a density above the standard, the right relative speed information indicates a relative speed below the standard, and the right density information indicates a density above the standard.

[0042] <Video display control system operation> The operation of the video display control system according to the first embodiment will be described. 4 shows the operation flow of the video display control system according to the first embodiment of the present invention. Note that the order of the steps may be changed as appropriate, and other steps may be added as appropriate. FIG. 5 is a diagram showing the positional relationship between the saddle riding type vehicle 100, which is the vehicle itself, and the surroundings of the saddle riding type vehicle 100, and shows a scene to which the operation flow of the video display control system according to the first embodiment of the present invention is applied.

[0043] 4, while video display control system 1 is in operation, right rear camera 11, left rear camera 12, and central rear camera 13 capture images of shooting ranges 11a, 12a, and 13a, respectively (step S1). Then, right rear camera 11, left rear camera 12, and central rear camera 13 output shooting data indicating the captured images to video control device 40, and video data input unit 41 receives the shooting data. Here, video data captured by right rear camera 11 is designated I-11, video data captured by left rear camera 12 is designated I-12, and video data captured by central rear camera 13 is designated I-13. Video data I-11 captured by the right rear camera includes video of shooting range 11a, video data I-12 captured by left rear camera 12 includes video of shooting range 12a, and video data I-13 captured by central rear camera 13 includes video of shooting range 13a.

[0044] Next, the composite video data generator 42 combines the video data acquired from the right rear camera 11, the left rear camera 12, and the central rear camera 13 (step S2). Specifically, the composite video data generator 42 combines the video data I-11 captured by the right rear camera 11 with the video data I-13 captured by the central rear camera 13 to generate composite video data CP-1. The composite video data generator 42 also combines the video data I-12 captured by the left rear camera 12 with the video data I-13 captured by the central rear camera 13 to generate composite video data CP-2. At this time, by applying a reliability mapping method to the portions where the video data I-11 captured by the right rear camera 11 and the video data I-13 captured by the central rear camera overlap, composite video data with a low computational load can be generated. The same applies to the combination of the video data I-12 captured by the left rear camera 12 with the video data I-13 captured by the central rear camera.

[0045] Next, it is determined whether or not a signal from the direction indicator 70 has been detected (step S3). If a signal from the turn indicator 70 is not detected (no), the operation flow returns to the start point. At this time, the composite video data stored in the storage unit 44 may be deleted from the storage unit 44. In this way, it is possible to secure memory space in the storage device, and there is no need to install a storage device with a large memory capacity. If a signal from the turn indicator 70 is detected (yes), proceed to the next step.

[0046] Next, it is determined whether the signal from the detected turn indicator 70 is a right turn indicator (right) or a left turn indicator (left) (step S4). This determination may be made by detecting whether the signal switch 71 input by the rider is a right turn indicator or a left turn indicator. If the detected signal from the turn indicator 70 is a left turn indicator (left), the process proceeds to step S5, and if the detected signal from the turn indicator is a right turn indicator (right), the process proceeds to step S6.

[0047] Next, in step S5, the wireless communication device 61 transfers the composite image data CP-2 of the image data I-12 captured by the left rear camera 12 and the image data I-13 captured by the central rear camera 13 to the image display device 50. In step S6, the wireless communication device 61 transfers to the image display device 50 the composite image data CP-1 of the image data I-11 captured by the right rear camera 11 and the image data I-13 captured by the central rear camera 13.

[0048] Thereafter, it is determined whether or not the signal from the direction indicator 70 is continuously detected (step S7). If the signal from the direction indicator 70 is continuously detected (yes), if the signal from the direction indicator 70 is a left direction signal, step S4 → step S5 → step S7 → step S4 is repeated, and if the signal from the direction indicator 70 is a right direction signal, step S4 → step S6 → step S7 → step S4 is repeated. On the other hand, when the signal from the direction indicator 70 is no longer detected, the operation flow ends (E), and the transmission of the composite video data CP-1 and CP-2 to the video display device 50 is stopped.

[0049] Next, a situation in which the first embodiment of the present invention is applied will be described with reference to FIG. The saddle riding type vehicle 100 is traveling in the left lane L1 (PO1). Behind the saddle riding type vehicle 100, another saddle riding type vehicle 200 is traveling, and in front of the saddle riding type vehicle 100, a vehicle V2 is traveling in the same left lane L1. In addition, in the right lane L2, a vehicle V1 is traveling behind the saddle riding type vehicle 100, and a vehicle V3 is traveling in front of the saddle riding type vehicle 100. We will explain the application of the present invention to a series of operations (see arrows) in which the saddle-type vehicle 100 changes lanes from its current position (PO1) to the right lane L2, which is the passing lane (PO2), and then after overtaking vehicle V2 (PO3), returns to the left lane again (PO4).

[0050] First, at position PO1, the rider of the saddle-riding vehicle 100 switches on the right-turn signal switch of the turn signal 70 to change lanes to the right lane L2. When the right-turn signal is detected, composite image data CP-1 is transmitted from the image control device 40 to the image display device 50, and the composite image is projected onto a predetermined portion of the transparent shield 5 provided in front of the rider's helmet main body 4 or onto the transparent glass 5' of the eyewear 3'. The composite image is an image obtained by combining image data captured by the right rear camera 11 and the central rear camera 13. Therefore, vehicle V1 is captured by the right rear camera 11, and another saddle-riding vehicle 200 is captured by the central rear camera 13, and the combined panoramic image is projected onto a predetermined portion of the transparent shield 5 of the helmet main body 4 or onto the transparent glass 5' of the eyewear 3'. Therefore, while facing forward, the rider can directly visually confirm vehicle V2, and can simultaneously view vehicle V1 to the right rear and another saddle-riding vehicle 200 behind them using the composite image data CP-1 projected onto the transparent shield 5. In other words, if the function of the present invention is not installed, the rider would normally have to turn his / her head to check behind the saddle-type vehicle 100 or to the side, which is a blind spot, when changing lanes, but such an action is completely unnecessary.

[0051] Thereafter, the rider keeps the right turn signal switch ON until he reaches position PO2 where he completes the lane change to right lane L2, so in the operation flow of FIG. 4 , steps S4, S6, and S7 are repeated, during which time the image control device 40 continues to transmit the composite image data CP-1 to the image display device 50. When the saddle riding type vehicle 100 moves to position PO2, the lane change to right lane L2 is completed, so the rider turns the turn signal switch OFF. At this time, the operation flow of FIG. 4 proceeds from step S7 to E, and the image control device 40 ends transmission of the composite image data CP-1 to the image display device 50.

[0052] The following describes a situation in which the rider then moves the saddle-type vehicle 100 to position PO3 and changes lanes to the left lane L1. At position PO3, the rider turns on the left turn signal switch of the turn signal 70. When the left turn signal is detected, composite image data CP-2 is transmitted from the image control device 40 to the image display device 50 and projected onto a predetermined portion of the transparent shield 5 provided in front of the rider's helmet body 4 or onto the transparent glass 5' of the eyewear 3'. The composite image data CP-2 is image data obtained by combining images captured by the left rear camera 12 and the central rear camera 13. Therefore, vehicle V2 is captured by the left rear camera 12, and vehicle V1 is captured by the central rear camera 13, and the combined image is projected onto the transparent shield or the like. Therefore, while the rider remains facing forward and directly visually confirms vehicle V3, the rider can simultaneously view vehicle V2 to the left rear and vehicle V1 behind due to the composite image obtained by image processing the composite image data CP-2 projected onto the transparent shield 5. Thereafter, the rider keeps the left turn signal ON until position PO4, where the lane change to the left lane is completed, so steps S4, S5, and S7 in the operation flow are repeated, and the composite image continues to be projected onto the transparent shield. When the saddle-type vehicle 100 moves to position PO4, the lane change is completed, and the rider turns the turn signal switch OFF. Therefore, the operation flow proceeds from step S7 to E, where the transfer of the composite image data CP-2 to the image display device 50 is completed.

[0053] As described above, according to the image display control system 1 of the first embodiment of the present invention, a composite image obtained by combining an image of the right rear or left rear of the saddle riding vehicle 100 with an image of the area directly behind the saddle riding vehicle 100 in response to input of the rider's turn signal is displayed on a display unit located near the user's eyes. Therefore, while driving the saddle riding vehicle 100, the rider can easily view a panoramic image obtained by combining an image of either the right rear or left rear of the saddle riding vehicle 100 with an image of the area directly behind by barely moving their eyes or by moving their eyes a short distance from a state in front of the saddle riding vehicle 100. This allows the user to grasp the situation behind the saddle riding vehicle 100 over a wide range while driving, without neglecting to look ahead, which is the direction in which the saddle riding vehicle 100 is traveling. Furthermore, because the distance between the user's eyes and the display unit is short, the user can clearly see the image displayed on the display unit.

[0054] Furthermore, according to the image display control system 1, the composite image displayed on the image display device 50 is switched depending on the direction indicated by the turn signal 70, so that the rider can view a panoramic image of the rear view that corresponds to the driving situation through the image display device while facing forward. Furthermore, the composite image generated by the composite image data generation unit is deleted from the storage device when no signal from the turn indicator is detected, so there is no need to install a large capacity memory.

[0055] (Second embodiment) Fig. 6 shows the operation of the video display system according to the second embodiment of the present invention. As shown in Fig. 6, in the operation of the video display control system 1 according to the second embodiment of the present invention, the steps of taking images of the rear of the saddle riding type vehicle 100 by the cameras 11, 12, and 13 (step S21), combining the captured image data (step S22), detecting signals from the turn indicators (step S23), determining the type of signal from the turn indicators (step S25), and transmitting the combined image data (steps S26 and S27) are the same as the operation of the video display control system 1 according to the first embodiment of the present invention (steps S1 to S6 in Fig. 4), and are as described above.

[0056] In the video display system according to the second embodiment of the present invention, it is determined whether the saddle riding type vehicle 100 is passing through an obstacle. Specifically, after a signal from a direction indicator is detected (step S23), a signal from the rider support device is evaluated to determine whether or not the rider is passing through a narrow lane (step S24). If it is determined that the rider is passing through a narrow area (yes), the operation flow returns to the start point. As a result, the image control device does not transmit the composite image data to the display control device while the rider is passing through a narrow area, so the display of the composite image does not excessively obstruct the rider's field of vision.

[0057] On the other hand, if it is not determined in step S24 that the vehicle is passing through a narrow lane (no), the process proceeds to step S25, where it is determined whether the signal from the detected turn signal is a right or left turn signal. The subsequent steps are the same as those in the first embodiment.

[0058] Next, a situation in which the second embodiment of the present invention is applied will be described with reference to FIG.

[0059] FIG. 7 shows the saddle-type vehicle 100 traveling through a traffic jam. The saddle riding type vehicle 100 is currently at position PO1', and vehicle V1' is traveling behind the saddle riding type vehicle 100, and vehicles V3', V5', and V8' are traveling in front of the saddle riding type vehicle 100 in the left lane L1. In addition, in the right lane L2, vehicle V2' is traveling behind the saddle riding type vehicle 100, and vehicles V4', V6', and V8' are traveling in front of the saddle riding type vehicle 100. Furthermore, another saddle riding type vehicle 200 is passing through a line of vehicles and is approaching the saddle riding type vehicle 100 from behind. The application of the second embodiment of the present invention to a situation where the saddle riding type vehicle 100 switches from the current traveling in the left lane to traveling between vehicles (PO2', PO3') will be described.

[0060] First, the saddle-riding vehicle 100 is at position PO1' and is about to switch to passing through vehicles. At this time, the rider turns on the right-turn signal switch of the turn signal 70. When the right-turn signal is detected, the composite image, which is obtained by image processing the composite image data CP-1, is projected onto a predetermined portion of the transparent shield 5 provided in front of the rider's helmet body 4 or onto the transparent glass 5' of the eyewear 3'. The composite image data CP-1 is image data obtained by combining image data captured by the right rear camera 11 and the central rear camera 13. Therefore, the right rear camera 11 captures vehicle V2' and the other saddle-riding vehicle 200, and the central rear camera 13 captures vehicle V1, and the composite image is projected onto the transparent shield 5 or the like as a panoramic image. Therefore, while the rider remains facing forward and visually confirms vehicle V3', the composite image projected onto the transparent shield 5 or the like allows the rider to simultaneously view vehicle V2' to the rear right, vehicle V1' directly behind, and the other saddle-riding vehicle 200 that is passing through vehicles.

[0061] Thereafter, the rider keeps the right turn signal ON until the rider starts passing through a narrow lane, so steps S24, S25, S27, and S28 in the operation flow are repeated, and the image control device 40 continues to transmit the composite image data CP-1 to the image display device 50. When the saddle riding type vehicle 100 moves to position PO2', the rider support device 80 determines in step S24 whether the saddle riding type vehicle 100 is passing through a narrow lane. If the rider support device 80 determines that the saddle riding type vehicle 100 is passing through a narrow lane (yes), the operation flow returns to the start point (S). Therefore, as long as the rider support device 80 detects that the saddle riding type vehicle 100 is passing through a narrow lane, steps S21, S22, S23, S24, and S26 are repeated in the operation flow, and the composite image data CP-1 is not transmitted from the image control device 40 to the image display device 50. It should be noted that in step S23, it is determined whether a signal has been received from the turn indicator 70, and the turn indicator 70 signals include not only left and right direction signals but also hazard lamp signals. Therefore, depending on the vehicle type, the hazard lamp signals may continue to be output when the saddle riding type vehicle 100 is passing through a narrow lane. Even in such a case, as long as the rider assistance device 80 detects that the saddle riding type vehicle 100 is passing through a narrow lane, the composite image will not be transferred to the image display device 50. This prevents the projection of the composite image from obstructing the rider's field of vision when passing through a narrow lane.

[0062] According to the video display system of the second embodiment of the present invention, when the rider assistance device detects a vehicle passing through a convoy, the composite video is not displayed on the display screen, allowing the rider to concentrate on driving without obstructing the rider's view while passing through a convoy.

[0063] In the above-described embodiment, when passing through other vehicles starts, that is, between positions PO1' and PO2' in Figure 7, a composite image obtained by image processing the composite image data CP-1 is displayed on the display screen, so the rider can change his / her driving lane to the passing through lane while keeping his / her head forward and checking for other saddle-type vehicles 200 approaching from behind.

[0064] Furthermore, in the above-described embodiment, when the rider assistance device 80 detects a passing through a convoy of vehicles, the image control device 40 stops transmitting the composite image data, but in addition to or instead of this, when the saddle riding type vehicle 100 turns on its hazard lights while passing through a convoy of vehicles, the image control device 40 may stop transmitting the composite image data upon detecting a signal from the hazard lights. In this way, the accuracy of detecting passing through a convoy of vehicles can be improved.

[0065] In the above-described embodiment, the helmet 3 and eyewear 3' are given as examples of devices to which the image display device 50 is attached, but the present invention is not limited to these. Such devices may be, for example, goggles or eyeglasses.

[0066] Furthermore, the present invention can be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and the video display control system 1 with such modifications is also included in the technical idea of ​​the present invention. [Explanation of symbols]

[0067] 1 Image display control system, 3 helmet, 3' eyewear, 11 right rear camera, 12 left rear camera, 13 center rear camera, 40 image control device, 41 image data input unit, 42 composite image data generation unit, 43 image display control unit, 44 memory unit, 50 image display device, 51 projector, 52 processing unit, 61 wireless communication device, 62 wireless communication device, 70 turn indicator, 80 rider assistance device, 84 surrounding environment detection unit, 85 riding condition detection device.

Claims

1. an image control device (40) including a composite image data generating unit (42) that generates composite image data by synthesizing image data captured by a plurality of cameras (11, 12, 13) that capture images of the rear of a saddle-ride type vehicle (100); an image display device (50) that displays a composite image in a forward field of view of a rider of the saddle riding type vehicle by image processing the composite image data, When the image control device (40) detects a signal from a direction indicator (70) of the saddle-ride type vehicle input by the rider, it transmits the composite image data to the image display device (50); If it is determined that the saddle riding type vehicle is passing through obstacles while detecting a signal from the direction indicator (70), the image control device terminates transmission of the composite image data. Video display control system.

2. 2. The image display control system according to claim 1, wherein the plurality of cameras include a right rear camera (11) that photographs the area to the right rear of the saddle riding type vehicle, a left rear camera (12) that photographs the area to the left rear of the saddle riding type vehicle, and a central rear camera (13) that photographs the area directly behind the saddle riding type vehicle.

3. 3. The image display control system according to claim 2, wherein the composite image data generation unit (42) generates the composite image data by combining the image data captured by the left rear camera (11) or the image data captured by the right rear camera (12) with the image data captured by the central rear camera (13).

4. 2. The video display control system according to claim 1, wherein the video control device selects the composite video data to be transmitted to the video display device in accordance with a direction indicated by the direction indicator.

5. 2. The image display control system according to claim 1, wherein the image display device (50) is disposed inside a helmet worn on the head of the rider.

6. 2. The image display control system according to claim 1, wherein the image display device (50) is disposed in eyewear worn on the rider's head.

7. 7. The video display control system according to claim 1, wherein the composite video data is transmitted to the video display device via a wireless communication device.

8. acquiring video data by capturing video of the rear of the saddle-ride type vehicle using a plurality of cameras (11, 12, 13); synthesizing the video data to generate synthesized video data; detecting a signal from a turn signal (70) of the saddle-ride type vehicle as input by a rider; evaluating the signals and transmitting the composite video data to a video display device; a step of displaying the composite image in a forward field of view of the rider by image processing the composite image data; and a step of terminating transmission of the composite image data to the image display device when it is determined that the traveling state of the saddle riding type vehicle is passing through obstacles while detecting a signal from the direction indicator (70). Control method.

Images