Image processing system

Abstract

An image processing system includes an imaging device that images an area behind a motorcycle, a display device that is arranged to be located in front of a rider and that displays an image based on image data acquired by the imaging device, and a control device that is configured to perform a first control of storing first image data of a first image corresponding to a first region in an imaging range of a captured image captured by the imaging device while the motorcycle is riding, in a storage device and a second control of displaying a second image corresponding to a second region narrower than the first region in the imaging range, on the display device.

Description

Video Processing System

[0001] The present disclosure relates to video processing systems.

[0002] A technique for providing a rider with visual information about the area behind a motorcycle by displaying an image of the area behind the motorcycle on a display device has been known. For example, Japanese Patent Application Laid-Open No. 2013-60128 discloses a rear image display device that displays a portion of an image of the area behind the motorcycle on a display device.

[0003] Furthermore, Utility Model Registration No. 3227672 suggests that such a rear view display device can be linked with a recorder. Furthermore, Japanese Patent Application Laid-Open No. 2008-225976 discloses a driving status storage device that stores image data obtained by capturing an image of the area behind a motorcycle in a volatile secondary storage means such as an SDRAM, and stores only image data from the image data stored in the secondary storage means at times when the longitudinal acceleration exceeds a threshold in a non-volatile storage means such as a flash memory.

[0004] Generally, an electronic control unit (ECU) is installed in a vehicle for each control target. For example, when saving an image displayed on a display device, it is conceivable to install a display control ECU that controls displaying the image on the display device and a storage control ECU that controls saving the image data in a storage device in the vehicle.

[0005] However, in the case of motorcycles such as two-wheeled and three-wheeled vehicles, the installation space for ECUs is limited compared to vehicles, making it impossible to install many ECUs. Furthermore, increasing the number of ECUs installed requires various design changes.

[0006] In addition, the range of images that a rider wants to see while driving may differ from the range of images that they want to see when investigating, for example, an accident. In the case of an investigation, it is desirable to check as wide an area as possible around the vehicle.

[0007] An object of the present disclosure is to provide an image processing system that can display an image suitable for rearward checking on a display device without adding a control device, and can store an image of a wider range than the image to be displayed in a storage device. Note that "image" in this disclosure includes both moving images and still images.

[0008] The image processing system of a first aspect of the present disclosure includes an imaging device that images the area behind a motorcycle, a display device that is positioned in front of the rider and displays an image based on image data acquired from the imaging device, and a control device that performs first control to store, in a storage device, first image data of a first image corresponding to a first area of ​​an imaging range of an image captured by the imaging device while the motorcycle is traveling, and second control to display, on the display device, a second image corresponding to a second area that is narrower than the first area of ​​the imaging range.

[0009] In a second aspect of the video processing system of the present disclosure, in the video processing system of the first aspect of the present disclosure, the second control for displaying the second video on the display device is performed at a timing different from the first control for saving the first video data in a storage device.

[0010] An image processing system of a third aspect of the present disclosure is an image processing system of the first or second aspect of the present disclosure, further comprising at least one second imaging device that images the front or side of the motorcycle, and the control device, in the first control, associates second image data acquired from the second imaging device at the time the first image is acquired with the first image data and stores it in the storage device.

[0011] A fourth aspect of the video processing system of the present disclosure is the video processing system of the third aspect of the present disclosure, wherein the control device is further configured to perform a third control of playing back the first video data stored in the storage device and displaying the video on the display device, and the third control includes displaying an video different from the first video on the display device based on composite video data obtained by combining the first video data and the second video data stored in the storage device.

[0012] An image processing system of a fifth aspect of the present disclosure is an image processing system of any one of the first to fourth aspects of the present disclosure, wherein the control device is configured to perform a fourth control of displaying an image of a range different from the second image on the display device in response to an image switching instruction from a rider received while the second image is being displayed.

[0013] A sixth aspect of the video processing system of the present disclosure is a video processing system of any one of the first to fifth aspects of the present disclosure, in which at least one of the first video data and the second video data has been subjected to predetermined image processing including rotation correction.

[0014] A video processing system of a seventh aspect of the present disclosure is a video processing system of any one of the first to sixth aspects of the present disclosure, further comprising a microphone for collecting audio, and the control device, in the first control, associates audio data acquired from the microphone at the time the first video was acquired with the first video data and stores it in the storage device.

[0015] An image processing system of an eighth aspect of the present disclosure is an image processing system of any one of the first to seventh aspects of the present disclosure, further comprising a sensor that detects the distance and direction to an obstacle around the motorcycle, and the control device, in the first control, stores the detection data acquired from the sensor at the time the first image is acquired in the storage device in association with the first image data.

[0016] A ninth aspect of the present disclosure is an image processing system according to any one of the first to eighth aspects of the present disclosure, wherein the storage device is arranged inside the motorcycle body together with the control device.

[0017] A video processing system according to a tenth aspect of the present disclosure is the video processing system according to the ninth aspect of the present disclosure, wherein the storage device is a portable recording medium.

[0018] An image processing system of an eleventh aspect of the present disclosure is an image processing system of any one of the first to eighth aspects of the present disclosure, wherein the storage device is located outside the motorcycle body, and the control device transmits the first image data to the external storage device for storage.

[0019] A video processing system of a twelfth aspect of the present disclosure is a video processing system of any one of the first to eleventh aspects of the present disclosure, wherein the first video data is the video data itself acquired from the imaging device.

[0020] According to the image processing system of the first aspect of the present disclosure, it is possible to display images suitable for rearward checking on a display device without adding a control device, and it is possible to store images of a wider range than the displayed image in a storage device.

[0021] According to the video processing system of the second aspect of the present disclosure, video can be displayed on the display device at appropriate timing.

[0022] According to the video processing system of the third aspect of the present disclosure, more information can be obtained from a plurality of video data captured at the same time than from a single piece of video data.

[0023] According to the video processing system of the fourth aspect of the present disclosure, the stored video data can be played back in various display modes, making it possible to examine the situation around the vehicle from different angles.

[0024] According to the image processing system of the fifth aspect of the present disclosure, a second image having a wider range than the first image can be displayed to the rider even while the vehicle is moving.

[0025] According to the video processing system of the sixth aspect of the present disclosure, the video data after image processing can be displayed and saved.

[0026] According to the video processing system of the seventh aspect of the present disclosure, more information can be obtained from audio data and video data acquired at the same time than from video data alone.

[0027] According to the video processing system of the eighth aspect of the present disclosure, more information can be obtained from the sensor detection data and video data acquired at the same time than from the video data alone.

[0028] According to the video processing systems of the ninth and tenth aspects of the present disclosure, it is possible to prevent theft of storage devices in which video data is stored, which is particularly effective when the storage device is a portable storage medium.

[0029] According to the video processing system of the eleventh aspect of the present disclosure, video data can be safely stored in an external storage device.

[0030] According to the video processing system of the twelfth aspect of the present disclosure, video data containing more visual information can be saved compared to saving video data after image processing.

[0031] FIG. 1 is a schematic diagram showing an example of a configuration of a video processing system according to a first embodiment of the present disclosure. FIG. 2 is a schematic diagram showing another example of the arrangement of an imaging device. FIG. 3 is a schematic diagram showing another example of the arrangement of an imaging device. FIG. 4 is a schematic diagram showing an example of the arrangement of a display device. FIG. 5 is a block diagram showing an example of an electrical configuration of a video processing system according to a first embodiment of the present disclosure. FIG. 6 is a functional block diagram showing an example of the functional configuration of a control device according to a first embodiment of the present disclosure. FIG. 7 is a flowchart showing an example of a processing flow of a video saving and display program according to a first embodiment of the present disclosure. FIG. 8 is a flowchart showing an example of a flow of a video saving process. FIG. 9 is a schematic diagram for explaining processing of video data according to a first embodiment of the present disclosure. FIG. 10 is a schematic diagram for explaining processing of video data according to a first embodiment of the present disclosure. FIG. 11 is a flowchart showing an example of a processing flow of a video playback program according to a first embodiment of the present disclosure. FIG. 12 is a schematic diagram showing an example of a playback operation screen displayed on a display device. FIG. 13 is a schematic diagram showing an example of a display mode of a playback video. FIG. 14 is a schematic diagram showing an example of a display mode of a playback video. FIG. 15 is a schematic diagram showing an example of a display mode of a playback video. Fig. 10 is a schematic diagram showing an example of a display image according to a second embodiment of the present disclosure. Fig. 11 is a schematic diagram showing an example of a display image according to a second embodiment of the present disclosure. Fig. 12 is a schematic diagram showing an example of a display image according to a second embodiment of the present disclosure.

[0032] Exemplary embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0033] First Embodiment The image processing system of the present disclosure is an image processing system for a motorcycle, and displays an image of the area behind the vehicle to the rider. Here, "motorcycle" refers to a straddle-type vehicle such as a two-wheeled, three-wheeled, or four-wheeled motor vehicle that a rider straddles. Motorcycles include those powered by an engine and those powered by a motor. Examples of motorcycles include motorbikes, scooters, and electric scooters. The following description will be given of a case where the motorcycle is a two-wheeled motor vehicle.

[0034] (Configuration of video processing system) First, the configuration of the video processing system will be described. Fig. 1 is a schematic diagram showing an example of the configuration of a video processing system. As shown in Fig. 1, the video processing system 10 includes an imaging device 12, a display device 14, a control device 16, and an operation device 18. Although Fig. 1 shows one of each device, multiple imaging devices 12, display devices 14, and operation devices 18 may be provided.

[0035] The imaging device 12 is a device that captures images of the surroundings of the motorcycle body 11. The imaging device 12 may be, for example, a digital video camera equipped with an image sensor such as a CCD image sensor or a CMOS image sensor. The imaging device 12 may also light a pilot lamp while recording to notify following vehicles that recording is in progress.

[0036] When a single imaging device 12 is installed, as shown in Fig. 1, the imaging device 12 that images the rear of the vehicle is installed at the rear of the vehicle body 11. The imaging device 12 that images the rear of the vehicle is preferably arranged on the center line of the vehicle body 11. When a plurality of imaging devices 12 are installed, for example, as shown in Fig. 2A, in addition to the imaging device 12R that images the rear of the vehicle, an imaging device 12F that images the front of the vehicle may be installed at the front of the vehicle body 11, or as shown in Fig. 2B, an imaging device 12S that images the side of the vehicle in addition to the rear and front of the vehicle may be installed at the side of the vehicle body 11. By arranging a plurality of imaging devices, it is possible to cover all directions.

[0037] The display device 14 is a device that displays an image of the area behind the vehicle to the rider. The display unit 40 (see FIG. 4) of the display device 14 may be, for example, a liquid crystal display, an organic EL display, or a touch panel display configured by overlaying a touch panel on one of these displays. The display device 14 is installed in a position in front of the rider that is easy for the rider to see.

[0038] The display device 14 may be a display attached to the instrument panel of the vehicle body 11, or may be a retrofitted display. For example, as shown in Fig. 3, if a meter display for displaying vehicle information is located in the center of the instrument panel, the display device 14 may be located near the meter display. When locating the display device 14, the distance from the rider to the instruments (e.g., the meter display) should be approximately the same as the distance from the rider to the display device 14, making it easier for the rider to focus on the display device 14.

[0039] The control device 16 is a control function part of an electronic control unit (i.e., ECU) provided to control the display device 14. The control device 16 is installed inside the vehicle body 11, for example, by being embedded inside a seat.

[0040] The operating device 18 is a device that allows the rider to input operations. A handlebar switch or the like attached near the handlebars can be used as the operating device 18. The handlebar switch allows the rider to input operations even while riding. The operating device 18 can be attached to the left grip, for example.

[0041] (Electrical Configuration of Video Processing System) Next, the electrical configuration of the video processing system 10 will be described. Fig. 4 is a block diagram showing an example of the electrical configuration of the video processing system 10. The control device 16 includes a CPU (Central Processing Unit) 30, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 34, and a non-volatile memory 36. The ROM 32 or the non-volatile memory 36 has a "video display program" and a "video playback program" (described later) stored in advance. The CPU 30 reads out the pre-stored program and executes the program using the RAM 34 as a work area.

[0042] The image capturing device 12, the display device 14, and the operating device 18 are connected to the control device 16 by wire or wirelessly so as to be able to exchange data with the control device 16. In addition, the various sensors 20, the communication unit 22, and the storage unit 24 are also connected to the control device 16 by wire or wirelessly so as to be able to exchange data with the control device 16. The control device 16 exchanges information with each unit and controls each unit.

[0043] The sensor 20 is an inclination sensor, an acceleration sensor, etc. for detecting the inclination of the vehicle body 11. The communication unit 22 is an interface for communicating with an external device 28. The memory unit 24 is an external storage device such as an HDD (Hard Disk Drive) or a portable recording medium (e.g., a memory card such as an SD card). The memory unit 24 stores image data of various screens, which will be described later. The control device 16, together with the communication unit 22 and the memory unit 24, is installed inside the vehicle body 11 as an ECU. Note that the control device 16, the communication unit 22, and the memory unit 24 may be housed in a common case. The portable recording medium is detachable from the ECU.

[0044] The display device 14 includes a display unit 40, an operation unit 42, a sensor 44, and a speaker 46. The display unit 40 is a display for displaying images based on video data. The operation unit 42 is a switch or button for inputting operations. As described above, the display unit 40 may be a touch panel display, in which case the touch panel display also corresponds to the operation unit 42. The sensor 44 is an acceleration sensor for detecting the orientation of the display device 14. The speaker 46 is a device for outputting sound based on audio data.

[0045] The configuration described above is an example, and other components may be added, or some components may be omitted.

[0046] Here, we will briefly explain the operation of the image processing system 10. An image of the area behind the vehicle is captured by the image capturing device 12, and the obtained image data is output to the control device 16. The control device 16 processes the obtained image data and outputs it to the display device 14. The display device 14 displays an image based on the obtained image data.

[0047] The image processing system 10 also functions as a drive recorder that records image data obtained from the image capture device 12. For this reason, the control device 16 also performs processing to store the image data in a storage device inside or outside the vehicle. When multiple image capture devices are provided, the control device 16 associates multiple pieces of image data obtained from each device at the same time and stores the pieces in the storage device. Alternatively, the control device 16 may generate composite image data that represents a composite image of multiple images and store the composite image data in the storage device.

[0048] The video data may be stored in the storage unit 24, or may be transmitted to the outside via the communication unit 22 and stored in an external storage device. For example, the video data may be stored in a personal computer or smartphone of the rider or other user. The following describes a case where the video data is stored in the storage unit 24 inside the vehicle body 11.

[0049] The rider can also give various instructions via the operation device 18. For example, the image displayed on the display device 14 can be switched in response to instructions from the rider input via the operation device 18. It should be noted that the rider can input operations via the operation unit 42 of the display device 14, or via an operation screen displayed on the display unit 40 of the display device 14.

[0050] Furthermore, the image processing system 10 may record data representing the situation around the vehicle in association with the image data. Examples of data representing the situation around the vehicle include audio data and data from various sensors.

[0051] For example, the image processing system 10 may include a microphone 26 for collecting sounds around the vehicle. The microphone 26 may be, for example, a microphone built into the image capturing device 12 as shown in the figure, or may be a microphone that is attached later. Audio data obtained at the same time as the video data may be stored in the storage unit 24 in association with the video data.

[0052] The image processing system 10 may also include a sensor such as a radar (e.g., a millimeter wave sensor) or a LiDAR (Laser Imaging Detection and Ranging) sensor that detects the distance and direction to an obstacle around the vehicle. Detection data obtained at the same time as the image data may be stored in the storage unit 24 in association with the image data.

[0053] (Functional Configuration of Control Device) Next, the functional configuration of the control device 16 will be described. Fig. 5 is a functional block diagram showing an example of the functional configuration of the control device 16. The control device 16 includes an instruction receiving unit 50 that receives instructions from the rider, an image saving processing unit 52 that saves image data in a storage device, an image display processing unit 54 that displays rearward images on the display device 14, and an image playback processing unit 56 that plays back the image data saved in the storage device. Each of these functional units is realized by the CPU 30 of the control device 16 executing a "image saving and display program" or a "image playback program" described below.

[0054] (Video storage and display program) Next, the "video storage and display program" will be described. Fig. 6A is a flowchart showing an example of the processing flow of the "video storage and display program." The video storage and display program is executed by the CPU 30 of the control device 16, and is started when the engine is started by operating a switch such as the engine start switch. Here, part of the operation of the CPU 30 will be described as the operation of each functional unit shown in Fig. 5.

[0055] First, in step S100, the instruction receiving unit 50 determines whether or not an instruction to display a video has been issued. The instruction to display a video may be issued, for example, by operating the operation device 18. If an instruction to display a video has not been issued, the CPU 30 makes a negative determination in step S100, and the process proceeds to step S102. Then, in step S102, the video storage processing unit 52 executes a "video storage process."

[0056] On the other hand, if an instruction to display a video has been issued, the instruction receiving unit 50 makes a positive determination in step S100, and the process proceeds to step S104. Then, in step S104, the video storage processing unit 52 and the video display processing unit 54 execute a "video storage and display process."

[0057] That is, in the image processing system of this embodiment, saving of image data starts simultaneously with engine start and continues until the engine is stopped. Furthermore, when the rider instructs the image processing system of this embodiment to display an image, the image processing system saves the image data and displays an image of the rear on the display device 14.

[0058] Next, in step S106, the CPU 30 determines whether or not to end the routine. For example, when an instruction to stop the engine is given by operating a switch such as the engine start switch, the CPU 30 makes a positive determination in step S106 and ends the routine. On the other hand, if the routine is not to be ended, the CPU 30 makes a negative determination in step S106 and the process returns to step S100. Therefore, the processes of steps S100 to S106 are repeatedly executed until the routine is ended.

[0059] (Video Storage Processing) Next, the flow of the "video storage processing" executed in step S102 will be described with reference to FIG. 6B.

[0060] First, in step S110, the video storage processing unit 52 acquires video data from the imaging device 12. Here, the unprocessed video data acquired from the imaging device 12 is referred to as "captured video data." Next, in step S112, the video storage processing unit 52 performs image processing on the captured video data acquired from the imaging device 12 to a format suitable for storage, and generates video data for storage to be stored in the storage unit 24. Here, the video data for storage is referred to as "saved video data." Next, in step S114, the video storage processing unit 52 stores the saved video data in the storage unit 24, and ends the routine.

[0061] (Video Storage and Display Processing) Next, the flow of the "video storage and display processing" executed in step S104 will be described with reference to FIG. 6C.

[0062] First, in step S120, the video storage processing unit 52 acquires captured video data from the imaging device 12. Next, in step S122, the video storage processing unit 52 generates saved video data by image processing the captured video data acquired from the imaging device 12 into a format suitable for storage. Next, in step S124, the video storage processing unit 52 stores the saved video data in the memory unit 24.

[0063] Next, in step S126, the video display processing unit 54 cuts out a partial area of ​​the imaging range as a display range and processes the captured video data so as to horizontally flip the image of the display range, thereby generating display video data to be displayed on the display device 14. Here, the display video data is referred to as "display video data." Next, in step S128, the display control unit 56 outputs the display video data to the display device 14, causing the display device 14 to display an image based on the display video data.

[0064] 7B is a diagram showing how a partial area 64 is cropped from the imaging range of the captured image 60 shown in FIG. 7A. The imaging device 12 captures an image of a wide area behind the vehicle, but what the rider wants to see is the blind spot area directly behind the rider. However, the captured image is captured so that the blind spot area is located in the center of the imaging range. Therefore, an area 64 including the center of the imaging range is cropped and used as the display range. By cropping and displaying the partial area 64, the area that the rider wants to see can be displayed in a larger size.

[0065] 7C is a diagram showing a display image 66 based on the display image data. The display image 66 is an inverted image of the image corresponding to the area 64. If the image displayed on the display device 14 is oriented in the same direction as the mirror image reflected in the rearview mirror, it is easier for the rider to grasp the situation behind the vehicle.

[0066] Now, returning to the description of FIG. 6C , next, in step S130, the image display processing unit 54 determines whether or not to end the image display. For example, when an instruction to end the image display is given by operating the operation device 18, the image display processing unit 54 makes a positive determination in step S130 and ends the routine. For example, when an instruction to end the image display is given by the rider while riding, the image display processing unit 54 ends the image display and ends the routine. On the other hand, when the image display is not to be ended, the image display processing unit 54 makes a negative determination in step S130 and the process returns to step S120. Therefore, the processes of steps S120 to S130 are repeatedly executed until an instruction to end the image display is given.

[0067] When the video storage processing unit 52 generates the stored video data, or when the video display processing unit 54 generates the displayed video data, various types of image processing such as rotation correction, distortion correction, color correction, and noise removal may be performed.

[0068] For example, image processing for generating stored video data includes processing for removing unnecessary portions from captured video data. In the case of an imaging device 12 equipped with a wide-angle lens, distortion occurs in the peripheral portions of the image. In such a case, it is preferable for the video storage processing unit 52 to perform image processing on the captured video data so that the peripheral portions of the image are removed. By deleting some of the video data before storing it in the memory unit 24, the amount of data to be stored can be reduced.

[0069] The image storage processing unit 52 may also perform other image processing on the captured image data. For example, the image storage processing unit 52 may perform a process of rotation correction on the captured image data in accordance with the tilt of the vehicle body 11 detected by the sensor 20. The image storage processing unit 52 may also perform a process of left-right flipping the image and store the video data of the flipped image.

[0070] 6B and step S122 in Fig. 6C may be omitted, and the captured video data may be used as the stored video data as is. Since the captured video data contains more visual information, it may be possible to provide more information to the user later than in the case where only a portion of the video is extracted.

[0071] Also, in step S126 of Figure 6C, an example has been described in which the video display processing unit 54 performs image processing on the captured video data to generate display video data, but the video display processing unit 54 may also generate display video data from the saved video data that has been image processed in step S122 of Figure 6C.

[0072] (Video Playback Program) As described above, while the vehicle is traveling, the rider is shown an image that is a partial cut-out area of ​​the captured image. In the first embodiment, the rider and other viewers can also play back the stored image data, i.e., a wide range of image data, while the vehicle is stopped.

[0073] Next, the video playback program will be described. Fig. 8 is a flowchart showing an example of the processing flow of the video playback program. This video playback program is executed by the CPU 30 of the control device 16, and is started when an instruction to play back a video is given. Here, the operation of the CPU 30 will be described as the operation of the playback control unit 58 shown in Fig. 5.

[0074] An instruction to play back a video may be given, for example, by operating a playback operation screen shown in Fig. 9 displayed on the display unit 40 of the display device 14. This playback operation screen 70 includes a video display section 72 for displaying the playback video, a selection section 74 for selecting the stored video data to play back, various instruction buttons 76, a display mode selection button 78, etc.

[0075] For example, if video data of a rearward video and a forward video are recorded, the display mode can be selected from among displaying only the rearward video, displaying only the forward video, split display of the rearward video and the forward video, etc. When the viewer selects the stored video data to be played and the display mode by operating the screen and instructs playback, playback begins.

[0076] First, in step S200, the playback control unit 58 reads out the saved video data to be played back from the storage unit 24. Next, in step S202, the playback control unit 58 generates video data for displaying video in a manner according to the instruction. Here, the video data for playback display is referred to as "playback video data." Next, in step S204, the playback control unit 58 outputs the playback video data to the display device 14, causing the display device 14 to display video based on the playback video data.

[0077] As described above, the saved video is different from the displayed video, which is a clip of the captured video, in that it includes a wider range of the captured video. The saved video data is used, for example, to verify the circumstances under which an accident occurred. For this reason, it is possible to play back video of a wider range that includes more visual information.

[0078] Furthermore, it is preferable that the stored video data can be played back in a plurality of display modes so that the user can examine the situation around the vehicle from different angles. In the front-only video display mode, only a front video 80 is displayed, as shown in Fig. 10A. In the rear-only video display mode, only a rear video 82 is displayed, as shown in Fig. 10B. In the split-display display mode, a composite video 84 of the rear and front videos is displayed, as shown in Fig. 10C. The video data of the rear video and the video data of the front video are combined, and the composite video 84 is displayed based on the resulting combined video data.

[0079] Please note that the "saved video data" is an example of the "first video data of the first video" in the present disclosure, and the "display video based on the display video data" is an example of the "second video" in the present disclosure.

[0080] As described above, the video processing system according to the first embodiment can provide the following various effects.

[0081] (1) The imaging device captures a wide area behind the vehicle, but a clipped image of the area the rider wants to see from the captured image is displayed on a display device located in front of the rider while the rider is traveling, so the area the rider wants to see is displayed in a larger size, improving visibility. Furthermore, the rider can check the blind spot directly behind, making it easier to check behind. Furthermore, because it is easier to check behind, the rider's head movement can be reduced, which is expected to improve safety.

[0082] (2) The image processing system stores a wide range of image data captured by the image capture device while the vehicle is in motion, allowing playback of the data, making it possible to store image data suitable for post-accident review. In the event of an accident, this image data can be used as evidence to prove the circumstances of the accident.

[0083] In addition, the display control device (i.e., ECU) is made multifunctional, and the display control device controls the storage of captured image data in a storage device, so the image processing system can store a wide range of captured image data without adding a new ECU.

[0084] In addition, by recording video data from multiple imaging devices in association with each other, or by recording audio data and detection data from various sensors in association with video data, more information can be obtained from the saved data.

[0085] (3) The control device, into which a portable recording medium such as an SD card is inserted as a memory unit, is installed inside the vehicle body. Therefore, unlike commercially available aftermarket drive recorders, the recording medium cannot be easily removed, thereby preventing theft of the recording medium on which the video data is saved.

[0086] (4) By turning on the pilot light of the imaging device, for example, to notify following vehicles that recording is in progress, dangerous behavior such as tailgating can be deterred.

[0087] Second Embodiment In the first embodiment, the image processing system 10 plays back stored image data while the vehicle is stopped, allowing a wide range of image to be viewed, but in the second embodiment, the image processing system 10 switches images while the vehicle is moving, allowing an image of a different range to be displayed from the original image. Here, an example will be described in which the image processing system 10 switches the original image to an image of a wider range (for example, stored image) in response to a switching instruction from the rider.

[0088] In the second embodiment, the flow of the "video storage and display process" of the video storage and display program shown in Fig. 6A is different from that of the first embodiment. Therefore, a description of the same parts as in the first embodiment will be omitted, and only the differences will be described.

[0089] (Video storage and display processing) Next, the "video storage and display processing" of the second embodiment will be described. Figures 11A and 11B are flowcharts showing an example of the flow of the "video storage and display processing." Here, the operation of the CPU 30 will be described as the operation of each functional unit shown in Figure 5.

[0090] 11A, the video storage processing unit 52 acquires captured video data from the imaging device 12. Next, in step S302, the video storage processing unit 52 performs image processing on the captured video data to convert it into a format suitable for storage, thereby generating stored video data to be stored in the storage unit 24. In the following step S304, the video storage processing unit 52 stores the stored video data in the storage unit 24. Note that step S302 may be omitted.

[0091] Next, in step S306, video display processing unit 54 cuts out a partial area of ​​the imaging range as a display range, and processes the captured video data so as to horizontally flip the image of the display range, thereby generating display video data to be displayed on display device 14. In the following step S308, video display processing unit 54 outputs the display video data to display device 14, causing display device 14 to display an image based on the display video data.

[0092] The processes in steps S300 to S308 correspond to the processes in steps S100 to S108 shown in FIG. 7, so please refer to the explanation of FIG. 7 for details of each step.

[0093] Next, in step S310, the instruction receiving unit 50 determines whether or not an instruction to switch to a wider range of video has been issued. For example, if the rider operates the operation device 18 to issue an instruction to switch the video, the instruction receiving unit 50 makes a positive determination in step S310, and the process proceeds to step S314. If an instruction to switch the video has not been issued, the instruction receiving unit 50 makes a negative determination in step S310, and the process proceeds to step S312.

[0094] If an instruction to switch the image is issued, then in step S314, the image storage processing unit 52 and the image display processing unit 54 execute a "wide-range display process" to switch the displayed image to an image with a wide range. Here, a subroutine of the "wide-range display process" will be described with reference to FIG. 11B.

[0095] In step S400, the video storage processing unit 52 acquires captured video data from the imaging device 12, and in step S402, performs image processing on the captured video data to convert it into a format suitable for storage, thereby generating stored video data to be stored in the storage unit 24. In the subsequent step S404, the stored video data is stored in the storage unit 24. Note that step S402 may be omitted.

[0096] Next, in step S406, the video display processing unit 54 performs image processing on the captured video data or the saved video data so as to flip the video image horizontally, thereby generating display video data to be displayed on the display device 14. That is, in the second embodiment, image processing for cutting out a portion of the captured video range is not performed. In the following step S408, the video display processing unit 54 outputs the display video data to the display device 14, causing the display device 14 to display an image based on the display video data.

[0097] Next, in step S410, the video display processing unit 54 determines whether a predetermined time has elapsed. The predetermined time is, for example, 1 to 10 seconds. If the predetermined time has elapsed, the video display processing unit 54 makes an affirmative determination in step S410 and ends the routine. If the predetermined time has not elapsed, the video display processing unit 54 makes a negative determination in step S410 and the process returns to step S400. Therefore, the processes of steps S400 to S410 are repeatedly executed until the predetermined time has elapsed. Then, when the wide-area display process is completed, the process returns to step S300 in FIG. 11A.

[0098] Fig. 12A shows a captured image 90. Fig. 12B shows a wide-area display image 92. The display image 92 is an inverted image obtained by flipping the entire imaging range of the captured image 90 horizontally. The area 94 enclosed by a dotted line frame is the narrow range clipped out as the display range in step S306 in Fig. 11A. Fig. 12C shows a narrow-area display image 96.

[0099] When an instruction to switch the image is given by operating the operation device 18, the image displayed on the display device 14 switches from the narrow-range display image 96 to the wide-range display image 92. After a predetermined time has passed since the switch to the wide-range display image 92, the display image returns to the narrow-range display image 96. Alternatively, the image display processing unit 54 may switch from the wide-range display image 92 to the narrow-range display image 96 in response to a switch operation by the rider.

[0100] 11A. If an instruction to switch the image has not been issued, then in step S312, the image display processing unit 54 determines whether or not to end the image display. For example, if an instruction to end the image display is issued by operating the operation device 18, the image display processing unit 54 makes a positive determination in step S312 and ends the routine. On the other hand, if the image display is not to be ended, the process returns to step S300. Therefore, the processes of steps S300 to S314 are repeatedly executed until an instruction to end the image display is issued.

[0101] As described above, the image processing system according to the second embodiment not only provides the same effects as the first embodiment, but also makes it possible to check images over a wide range even while riding a motorcycle.

[0102] <Modifications> It should be noted that the configurations of the video processing systems described in the above embodiments are merely examples, and it goes without saying that the configurations may be modified within the scope of the present disclosure.

[0103] In the above embodiment, the image processing system displays an image in response to an instruction from the rider. However, the image processing system may display an image in response to the detection results of various sensors provided in the vehicle. The various sensors include a blind spot monitor (BSM) that detects the approach of a vehicle from behind the vehicle, a sensor that detects the inclination of the vehicle, a vehicle speed sensor, etc. For example, the image processing system may display an image in response to an alarm from the BSM.

[0104] In the above embodiment, an example was described in which video data storage begins simultaneously with engine start and continues until the engine is stopped, with the video being displayed in response to a rider's instruction, i.e., only when the rider wants to view the video. However, the timing of video data storage and video display are not limited to this example. The timing of video display may be the same as or different from the timing of video data storage, but it is desirable that the two be different, as described in the above embodiment. Furthermore, while it is desirable that video data be stored continuously while the motorcycle is in motion, the timing at which video data storage begins is not limited to engine start. For example, the video processing system may begin saving video data in response to a rider's instruction.

[0105] In the above embodiment, an example was described in which the display device was installed on the right or left side (as viewed from the rider) of the meter display on the instrument panel, but the display device may be installed in a position in front of the rider that is easy for the rider to see, regardless of whether a meter display is present. For example, the display device may be installed on the handlebars, in a mirror mounting portion, on top of the fuel tank, inside the cowl, or above the meter display.

[0106] In the above embodiment, an example in which a dedicated display device is provided in the video processing system has been described. However, a portable information processing device such as a smartphone or tablet may also be used as the display device. In this case, the portable information processing device executes various processes by using an installed dedicated application program in cooperation with an ECU (Electrical Control Unit). For example, a smartphone can be wirelessly connected to the communication unit 22 via Bluetooth (registered trademark), and can display images based on video data stored in the storage unit 24 on the smartphone screen (see FIG. 4).

[0107] This application claims priority from Japanese Patent Application No. 2022-207657, filed on December 23, 2022, the entire disclosure of which is incorporated herein by reference.

Claims

1. An image processing system comprising: an imaging device that images the area behind a motorcycle; a display device that is positioned in front of the rider and displays images based on image data acquired from said imaging device; and a control device that performs a first control to store in a storage device first image data of a first image corresponding to a first area of ​​the imaging range of the image captured by said imaging device while said motorcycle is traveling; and a second control to display on said display device a second image corresponding to a second area narrower than the first area of ​​the imaging range.

2. The image processing system of claim 1, wherein the control device performs the second control of displaying the second image on the display device at a timing different from the first control of saving the first image data in a storage device.

3. An image processing system as described in claim 1 or claim 2, further comprising at least one second imaging device that images the front or side of the motorcycle, wherein the control device, in the first control, stores in the storage device second image data acquired from the second imaging device at the time the first image is acquired, in association with the first image data.

4. The video processing system of claim 3, wherein the control device is further configured to perform a third control of playing back the first video data stored in the storage device and displaying the video on the display device, and the third control includes displaying on the display device an video different from the first video based on composite video data obtained by combining the first video data and the second video data stored in the storage device.

5. The image processing system according to any one of claims 1 to 4, wherein the control device is configured to perform a fourth control to cause the display device to display an image of a range different from the second image in response to an image switching instruction received from the rider while the second image is being displayed.

6. The image processing system according to any one of claims 1 to 5, wherein at least one of the first image data and the second image data has undergone predetermined image processing including rotation correction.

7. A video processing system as described in any one of claims 1 to 6, further comprising a microphone for collecting audio, wherein the control device, in the first control, associates the audio data acquired from the microphone at the time the first video was acquired with the first video data and stores it in the storage device.

8. An image processing system as described in any one of claims 1 to 7, further comprising a sensor that detects the distance and direction to an obstacle around the motorcycle, and wherein the control device, in the first control, stores in the storage device the detection data acquired from the sensor at the time the first image is acquired, in association with the first image data.

9. The image processing system according to any one of claims 1 to 8, wherein the storage device is disposed inside the body of the motorcycle together with the control device.

10. The video processing system according to any one of claims 1 to 9, wherein the storage device is a portable recording medium.

11. The image processing system according to any one of claims 1 to 10, wherein the storage device is located outside the motorcycle body, and the control device transmits the first image data to the external storage device for storage.

12. The video processing system according to any one of claims 1 to 11, wherein the first video data is the video data itself acquired from the imaging device.

Images