Video processing device and video processing system

Abstract

A video processing device for controlling a display device disposed in front of the rider of a motorcycle comprises a memory and a processor connected to the memory. The processor executes: acquiring video data from an imaging device that images the rear of the motorcycle when the motorcycle is traveling; storing the acquired video data in the memory; acquiring inclination information from a sensor that detects the inclination of the imaging device; correcting the video data so that the video displayed on the display device rotates in accordance with the acquired inclination information; and causing the display device to display a video based on the corrected video data.

Description

Image processing device and image processing system

[0001] The present disclosure relates to a video processing device and a video processing system.

[0002] 2. Description of the Related Art Conventionally, there is known a technique for providing a rider with visual information about the area behind a motorcycle by displaying an image of the area behind the motorcycle on a display device.

[0003] When turning, motorcycles and other motorbikes tilt their bodies. As the body tilts, the camera attached to the body also tilts. This can result in the display device not displaying the desired area. To solve this problem, tilt correction of captured images can be considered. For example, Japanese Patent Application Laid-Open Publication No. 2019-68343 describes a method for correcting captured images captured by a camera unit located at the rear of the vehicle using a gyro sensor or the like.

[0004] On the other hand, there is also a need to store video data captured while driving, because in the event of an accident, this video data can be used as evidence to prove the circumstances of the accident.

[0005] However, tilt correction results in the loss of part of the captured video data, which is undesirable when the video data is to be stored as evidence.

[0006] An object of the present disclosure is to provide an image processing device and an image processing system that can display images with good visibility while suppressing a decrease in the amount of information. Note that, in this disclosure, "image" includes both moving images and still images.

[0007] A first aspect of the present disclosure is an image processing device that controls a display device positioned in front of a motorcycle rider, comprising a memory and a processor connected to the memory, wherein the processor acquires image data from an imaging device that images the area behind the motorcycle while the motorcycle is traveling, stores the acquired image data in the memory, acquires tilt information from a sensor that detects the tilt of the imaging device, corrects the image data so that an image displayed on the display device rotates in accordance with the acquired tilt information, and causes the display device to display an image based on the corrected image data.

[0008] A second aspect of the present disclosure is an image processing system including an imaging device that captures an image of the area around a motorcycle, a display device that is positioned in front of a rider and displays an image based on image data acquired from the imaging device, a sensor that detects the tilt of the imaging device, and a control device that performs a first control to store the image data acquired from the imaging device in a storage device while the motorcycle is traveling, and a second control to correct the image data so that the image displayed on the display device is rotated in accordance with tilt information acquired from the sensor, and to display an image based on the corrected image data on the display device.

[0009] According to the present disclosure, it is possible to realize the display of video with good visibility while suppressing a decrease in the amount of information.

[0010] 1 is a schematic diagram showing an example of a configuration of a video processing system according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of an electrical configuration of a video processing system according to an embodiment of the present disclosure. FIG. 3 is a perspective view showing an example of a configuration of an imaging device. FIG. 4 is a schematic cross-sectional view taken along the optical axis showing an example of a configuration of an imaging device. FIG. 5 is a schematic cross-sectional view taken along the optical axis showing another example of a configuration of an imaging device. FIG. 6 is a schematic diagram illustrating alignment of an image sensor and a tilt sensor. FIG. 7 is a schematic diagram illustrating an example of an arrangement of a display device. FIG. 8 is a functional block diagram showing an example of a functional configuration of a control device according to an embodiment of the present disclosure. FIG. 9 is a flowchart illustrating an example of a processing flow of a video processing program according to an embodiment of the present disclosure. FIG. 10 is a flowchart illustrating an example of a subroutine of video saving processing. FIG. 11 is a flowchart illustrating an example of a subroutine of video saving and display processing. FIG. 12 is a diagram illustrating video tilt correction. FIG. 13 is a diagram illustrating video tilt correction. FIG. 14 is a schematic diagram illustrating an example of a displayed video. FIG. 15 is a schematic diagram illustrating an example of a saved video.

[0011] Exemplary embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0012] The image processing system of the present disclosure is an image processing system for motorcycles, and displays images of the surroundings of the vehicle (for example, rear, forward, and backward) to the rider. Here, "motorcycle" refers to a straddle-type vehicle such as a two-wheeled, three-wheeled, or four-wheeled motor vehicle that the rider straddles. Motorcycles include those powered by an engine and those powered by a motor. Examples of motorcycles include motorbikes, scooters, and electric scooters. The following description will be given assuming that the motorcycle is a two-wheeled motor vehicle.

[0013] (Configuration of the video processing system) First, the configuration of the video processing system will be described. Fig. 1 is a schematic diagram showing an example of the configuration of a video processing system. As shown in Fig. 1, the video processing system 10 includes an imaging device 12, a display device 14, a control device 16, and an operation device 18. Although Fig. 1 shows one of each device, each of the imaging device 12, the display device 14, and the operation device 18 may be provided in multiple units, or may be integrated.

[0014] The imaging device 12 is a device that captures images of the surroundings of the motorcycle body 11. The imaging device 12 may be, for example, a digital video camera equipped with an image sensor such as a CCD image sensor or a CMOS image sensor. The imaging device 12 may be, for example, a rectangular camera as shown in FIG. 3.

[0015] One or more imaging devices 12 may be installed. For example, as shown in FIG. 1 , imaging device 12R for capturing images of the rear of the vehicle may be installed at the rear of the vehicle body 11, and imaging device 12F for capturing images of the front of the vehicle may be installed at the front of the vehicle body 11. Imaging devices 12R and 12F are preferably arranged on the center line of the vehicle body 11. In this arrangement, the imaging range of the imaging device is a range of a predetermined angle to the left and right from the center line of the vehicle body, including the area directly behind the vehicle body width. The imaging device 12 may consist of only imaging device 12R, or additional imaging devices may be installed. For example, imaging devices for capturing images of the sides of the vehicle may be installed on the sides of the vehicle body 11.

[0016] The display device 14 is a device that displays an image of the area behind the vehicle to the rider. The display unit 40 (see FIG. 2) of the display device 14 may be, for example, a liquid crystal display, an organic EL display, or a touch panel display configured by overlaying a touch panel on one of these displays. The display device 14 is installed in a position in front of the rider that is easy for the rider to see.

[0017] The display device 14 may be a display attached to the instrument panel of the vehicle body 11, or may be a retrofitted display. For example, as shown in Fig. 7, if a meter display for displaying vehicle information is located in the center of the instrument panel, the display device 14 may be located inside or near the meter display. When locating the display device 14, the distance from the rider to the instruments (e.g., the meter display) should be approximately the same as the distance from the rider to the display device 14, making it easier for the rider to focus on the display device 14.

[0018] The control device 16 is a control function part of an electronic control unit (i.e., ECU) provided to control the display device 14. The control device 16 is installed inside the vehicle body 11, for example, by being embedded inside a seat.

[0019] The operating device 18 is a device that allows the rider to input operations. A handlebar switch or the like attached near the handlebars can be used as the operating device 18. The handlebar switch allows the rider to input operations even while riding. The operating device 18 can be attached to the left grip, for example.

[0020] (Electrical Configuration of Video Processing System) Next, the electrical configuration of the video processing system 10 will be described. Fig. 2 is a block diagram showing an example of the electrical configuration of the video processing system 10. The control device 16 includes a CPU (Central Processing Unit) 30, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 34, and a non-volatile memory 36. A "video processing program" (described later) is stored in advance in the ROM 32 or the non-volatile memory 36. The CPU 30 reads out the pre-stored program and executes the program using the RAM 34 as a work area.

[0021] The image capturing device 12, the display device 14, and the operating device 18 are connected to the control device 16 by wire or wirelessly so as to be able to exchange data with the control device 16. In addition, the various sensors 20, the communication unit 22, and the storage unit 24 are also connected to the control device 16 by wire or wirelessly so as to be able to exchange data with the control device 16. The control device 16 exchanges information with each unit and controls each unit.

[0022] The sensor 20 is a sensor for acquiring various types of vehicle information. For example, the sensor 20 may include a sensor such as a radar (e.g., a millimeter wave sensor) or a LiDAR (Laser Imaging Detection and Ranging) sensor that detects the distance and direction to an obstacle around the vehicle.

[0023] The communication unit 22 is an interface for communicating with an external device 28. The storage unit 24 is an external storage device such as a hard disk drive (HDD) or a portable recording medium (for example, a memory card such as an SD card). The storage unit 24 stores image data of various screens, which will be described later. The control device 16, together with the communication unit 22 and the storage unit 24, is installed inside the vehicle body 11 as an ECU. The control device 16, the communication unit 22, and the storage unit 24 may be housed in a common case. The portable recording medium is detachable from the ECU.

[0024] The imaging device 12 is equipped with a tilt sensor 26 for detecting the tilt of the imaging device 12. The tilt sensor 26 may be a two- or more-axis acceleration sensor as long as it can detect the direction of gravity. A two-axis acceleration sensor or a three-axis acceleration sensor is preferable as the tilt sensor 26 of this embodiment.

[0025] The display device 14 includes a display unit 40, an operation unit 42, a sensor 44, and a speaker 46. The display unit 40 is a display for displaying images based on video data. The operation unit 42 is a switch or button for inputting operations. As described above, the display unit 40 may be a touch panel display, in which case the touch panel display also corresponds to the operation unit 42. The sensor 44 is an acceleration sensor for detecting the orientation of the display device 14. The speaker 46 is a device for outputting sound based on audio data.

[0026] The configuration described above is an example, and other components may be added, or some components may be omitted.

[0027] (Configuration of Imaging Device) Here, a description will be given of the configuration of the imaging device 12. Fig. 3 is a perspective view showing an example of the configuration of the imaging device 12. Figs. 4 and 5 are schematic cross-sectional views taken along the optical axis showing an example of the configuration of the imaging device 12.

[0028] The imaging device 12 can take the form of a so-called rectangular camera, and includes a rectangular parallelepiped housing 50 and a mounting portion 52 for mounting the housing 50 to the vehicle body 11. The imaging device 12 may also include a pilot lamp 68 on one surface of the housing 50. During recording, the pilot lamp 68 can be turned on to notify following vehicles that recording is in progress.

[0029] The imaging device 12 includes an imaging optical system 54 such as a lens, and a board 56 on which an image sensor 58 is mounted. The imaging optical system 54 and the board 56 on which the image sensor 58 is mounted are housed within a housing 50 as a main body. A wiring cord 60 for connecting the main body to an external device extends from the housing 50.

[0030] In addition, a tilt sensor 26 is attached to the imaging device 12. By attaching the tilt sensor 26 to the imaging device 12, even if the imaging device 12 is attached at an angle relative to the vehicle body 11, the tilt of the imaging device 12 (i.e., the tilt of the image) can be detected regardless of the attachment condition of the imaging device 12 or the tilt of the vehicle body 11. This reduces the effort required of the user to attach the imaging device 12 horizontally relative to the vehicle body 11. Note that the tilt due to the attachment condition of the imaging device 12 may be corrected before capturing the image.

[0031] 4, the tilt sensor 26 is attached to the outside of the housing 50. The tilt sensor 26 is housed in a rectangular parallelepiped housing 62. A wiring cord 64 for connecting the tilt sensor 26 to an external device extends from the housing 62. The housing 62 housing the tilt sensor 26 is stacked on top of the housing 50 housing the main body of the imaging device 12, and is fixed to the housing 50 with a fixture, adhesive, or the like. The housing 62 is formed so that it protrudes forward from the front surface of the housing 50, thereby also functioning as a canopy structure.

[0032] 5, the tilt sensor 26 is attached inside the housing 50. The tilt sensor 26 is mounted on a board 66 that is different from the board 56 on which the image sensor 58 is mounted, and is housed inside the housing 50.

[0033] In either arrangement, it is preferable that the tilt sensor 26 be attached so that the horizontal direction of the tilt sensor 26 coincides with the horizontal direction of the image captured by the imaging device 12. If the tilt sensor 26 is tilted with respect to the image sensor 58, the image will be tilted even if the tilt sensor 26 indicates horizontal. This tilt of the image can be corrected by image processing, but if the horizontal direction is consistent from the beginning, correction is not necessary.

[0034] 6 , the longitudinal axis of the image sensor 58 is defined as the X-axis, the lateral axis as the Y-axis, and the optical axis direction passing through the center O as the Z-axis. The axis of the tilt sensor 26 that detects the direction of gravity is defined as the y-axis, the axis facing the optical axis direction as the z-axis, and the axis intersecting these axes as the x-axis. In this case, the X-axis of the image sensor 58 and the x-axis of the tilt sensor 26 are aligned. It is more preferable that the Z-axis of the image sensor 58 and the z-axis of the tilt sensor 26 are aligned. It is more preferable that the three-axis directions of the image sensor 58 and the three-axis directions of the tilt sensor 26 are aligned.

[0035] As shown in FIG. 4, when the tilt sensor 26 is attached to the outside of the housing 50, if the imaging device 12 is a rectangular camera, the housing 50 has a rectangular parallelepiped shape, making it easy to align the horizontal directions (i.e., axial directions) of the tilt sensor 26 and the image sensor 58.

[0036] (Operation of Image Processing System) Here, we will briefly explain the operation of the image processing system 10. An image of the area behind the vehicle is captured by the imaging device 12, and the obtained image data is output to the control device 16. The control device 16 acquires tilt information from the tilt sensor 26 of the imaging device 12, corrects the tilt of the obtained image data in accordance with the tilt information, and outputs the corrected image data to the display device 14. The display device 14 displays an image based on the obtained image data.

[0037] The image processing system 10 also functions as a drive recorder that records image data obtained from the image capture device 12. For this reason, the control device 16 also performs processing to store the image data in a storage device inside or outside the vehicle. When multiple image capture devices are provided, the control device 16 associates multiple pieces of image data obtained from each device at the same time and stores the pieces in the storage device. Alternatively, the control device 16 may generate composite image data that represents a composite image of multiple images and store the composite image data in the storage device.

[0038] The video data may be stored in the storage unit 24, or may be transmitted to the outside via the communication unit 22 and stored in an external storage device. For example, the video data may be stored in a personal computer or smartphone of the rider or other user. The following describes a case where the video data is stored in the storage unit 24 inside the vehicle body 11.

[0039] The rider can also give various instructions via the operation device 18. For example, the image displayed on the display device 14 can be switched in response to instructions from the rider input via the operation device 18. It should be noted that the rider can input operations via the operation unit 42 of the display device 14, or via an operation screen displayed on the display unit 40 of the display device 14.

[0040] Furthermore, the image processing system 10 may record data representing the situation around the vehicle in association with the video data. Examples of data representing the situation around the vehicle include audio data and detection data from various sensors. These data obtained at the same time as the video data may be stored in the storage unit 24 in association with the video data.

[0041] (Functional Configuration of Control Device) Next, the functional configuration of the control device 16 will be described. Figure 8 is a functional block diagram showing an example of the functional configuration of the control device 16. The control device 16 includes a video data acquisition unit 70 that acquires video data from the imaging device 12 that captures images of the area behind the motorcycle while the motorcycle is traveling, a video data storage unit 72 that stores the acquired video data in memory, a tilt information acquisition unit 74 that acquires tilt information from the tilt sensor 26 that detects the tilt of the imaging device 12, a tilt correction unit 76 that corrects the video data so that the video displayed on the display device is rotated in accordance with the acquired tilt information, and a display control unit 78 that displays an image based on the corrected video data on the display device 14. Each of these functional units is implemented when the CPU 30 of the control device 16 executes an "video processing program," which will be described later.

[0042] (Video Processing Program) Next, the "video processing program" will be described. Fig. 9A is a flowchart showing an example of the processing flow of the "video processing program." The video processing program is executed by the CPU 30 of the control device 16, and is started when the power is turned on by operating a power switch such as a key or a switch.

[0043] First, in step S100, the CPU 30 determines whether or not an instruction to display a video has been issued. The instruction to display a video may be issued, for example, by operating the operation device 18. If an instruction to display a video has not been issued, the CPU 30 makes a negative determination in step S100, and the process proceeds to step S102. Then, in step S102, the CPU 30 executes an "video storage process."

[0044] On the other hand, if an instruction to display a video has been issued, the CPU 30 makes an affirmative decision in step S100, and the process proceeds to step S104. Then, in step S104, the CPU 30 executes an "video storage and display process."

[0045] That is, in the image processing system of this embodiment, saving of image data begins as soon as the power is turned on, and continues until the power is turned off. Also, in the image processing system of this embodiment, when the rider instructs to display an image, the image data is saved and an image of the area behind the motorcycle is displayed on display device 14.

[0046] Next, in step S106, the CPU 30 determines whether or not to end the routine. For example, when an instruction to turn off the power is given by operating a switch such as a power switch, the CPU 30 makes an affirmative determination in step S106 and ends the routine. On the other hand, if the routine is not to be ended, the CPU 30 makes a negative determination in step S106 and the process returns to step S100. Therefore, the processes of steps S100 to S106 are repeatedly executed until the routine is ended.

[0047] (Video Storage Processing) Next, the flow of the "video storage processing" executed in step S102 will be described with reference to FIG. 9B.

[0048] First, in step S110, the CPU 30 acquires video data from the imaging device 12. Here, the unprocessed video data acquired from the imaging device 12 is referred to as "captured video data." Next, in step S112, the CPU 30 performs image processing on the captured video data acquired from the imaging device 12 to a format suitable for storage, thereby generating video data for storage in the storage unit 24. Here, the video data for storage is referred to as "saved video data." Note that image processing to a format suitable for storage may be, for example, a process of cropping the captured video without changing the angle of view. Next, in step S114, the CPU 30 saves the saved video data in the storage unit 24, and ends the routine.

[0049] (Video Storage and Display Processing) Next, the flow of the "video storage and display processing" executed in step S104 will be described with reference to FIG. 9C.

[0050] First, in step S120, the CPU 30 acquires captured video data from the imaging device 12. Next, in step S122, the CPU 30 generates saved video data by image processing the captured video data acquired from the imaging device 12 into a format suitable for storage. Next, in step S124, the CPU 30 stores the saved video data in the storage unit 24.

[0051] Next, in step S126, CPU 30 acquires tilt information from tilt sensor 26. Next, in step S128, CPU 30 corrects the captured image data so that the image is rotated in accordance with the acquired tilt information. For example, if the tilt angle is θ, CPU 30 rotates the image by angle θ in the direction opposite to the tilt direction, thereby correcting the image so that it is horizontal.

[0052] Next, in step S130, CPU 30 performs image processing on the corrected captured image data to generate display image data to be displayed on display device 14. Here, the display image data is referred to as "display image data." For example, CPU 30 may crop a partial area of ​​the captured image range as a display range and flip the image of the display range horizontally to generate display image data. Next, in step S132, CPU 30 outputs the display image data to display device 14, causing display device 14 to display an image based on the display image data.

[0053] Although the imaging device 12 captures a wide range behind the vehicle, what the rider wants to see is the blind spot directly behind the rider. Therefore, by cropping out an area including the center of the imaging range and using it as the display range, the range the rider wants to see can be displayed in a larger size. In addition, the image is flipped left and right because it is easier for the rider to grasp the situation behind the vehicle if the image displayed on the display device 14 is oriented in the same direction as the mirror image reflected in the rearview mirror.

[0054] Next, in step S134, the CPU 30 determines whether or not to end the image display. For example, if an instruction to end the image display is given by operating the operating device 18, the CPU 30 makes an affirmative determination in step S134 and ends the routine. For example, if an instruction to end the image display is given by the rider while riding, the CPU 30 ends the image display and ends the routine. On the other hand, if the image display is not to be ended, the CPU 30 makes a negative determination in step S134 and the process returns to step S120. Therefore, the processes of steps S120 to S134 are repeatedly executed until an instruction to end the image display is given.

[0055] Although the above describes an example in which the video display is terminated in response to an instruction, the video display may also be terminated when a predetermined time has elapsed. Furthermore, although the above describes an example in which a partial area of ​​the imaging range is cut out and used as the displayed video, the above may be a case in which a first video in which a first area of ​​the imaging range is cut out and displayed, and a second video in which a second area of ​​the imaging range (for example, a second area larger than the first area) is cut out and displayed, may be displayed in a switchable manner.

[0056] (Effect of Tilt Correction) The effect of tilt correction will be explained with reference to Figures 10A to 10C. Figure 10A shows a captured image 80 captured while the vehicle was tilted. A portion of the image in the center of the imaging range of the captured image 80 is cut out and used as a display image 82. Inversion is omitted here. As can be seen from the figure, because the vehicle body 11 and imaging device 12 are tilted, only the ground is displayed, and the area that the viewer wants to see, such as a following vehicle, is not displayed in the center.

[0057] Therefore, as shown in Figure 10B, captured image 80 is rotated so that the image is horizontal. After tilt correction, a portion of the corrected image 84 in the center of the captured image range is cut out and used as display image 86. Inversion is omitted here. As can be seen from Figure 10C, tilt correction allows the desired area to be displayed in the center, improving the visibility of the image.

[0058] 11A shows an example in which a rear image 82 is superimposed on a front image 90. When the vehicle body 11 and the imaging device 12 are tilted, the front image 90 and the rear image 82 will tilt in opposite directions. Therefore, when tilt correction is performed as shown in FIG. 11B, the front image 92 and the rear image 86 become horizontal, significantly improving the visibility of the images.

[0059] On the other hand, as can be seen from Figure 10B, tilt correction results in the loss of image data in the shaded portion of captured image 80. Furthermore, there is no data in the portion of corrected image 84 that does not overlap with captured image 80. In other words, the captured image data before tilt correction contains more visual information than the captured image data after correction. Therefore, if the data of corrected image 84 is saved, the amount of information will be significantly reduced compared to saving the data of captured image 80.

[0060] For this reason, in this embodiment, tilt correction is performed when displaying an image to display an image with high visibility, and when saving the image, image data that has not been tilt corrected is saved, thereby suppressing a decrease in the amount of information.

[0061] Although the above describes tilt correction, various types of image processing such as rotation correction, distortion correction, color correction, noise removal, etc. may be performed when generating each of the saved video data and the displayed video data. For example, image processing when generating saved video data may include processing to remove unnecessary parts from the captured video data and processing to add tilt information.

[0062] Although tilt correction is not performed on saved images, it is preferable to record information about the degree to which the image is tilted. Therefore, tilt information acquired at the same time may be associated with saved image data and stored in the storage unit 24. Alternatively, the image data may be processed to add tilt information, and the video data with the added tilt information may be stored. For example, as shown in FIG. 12 , the tilt of the image can be recorded by superimposing at least one of a line H representing the horizontal direction and a line V representing the vertical direction on the saved image, or by superimposing the value of the tilt angle θ on the saved image.

[0063] In the above example, the displayed image is cropped after tilt correction, but the crop range may be rotated relative to the captured image. It goes without saying that the same effect can be achieved whether the captured image or the crop range is rotated. Furthermore, the saved image data can be played back as needed using a playback device (not shown).

[0064] As described above, the video processing system according to this embodiment can provide the following various effects.

[0065] When displaying the image, tilt correction is performed to display an image with high visibility, and when saving the image, the image data without tilt correction is saved, so it is possible to display an image with good visibility while suppressing a decrease in the amount of information. In the event of an accident, the saved image data can also be used as evidence to prove the circumstances of the accident.

[0066] Furthermore, since a tilt sensor is attached to the imaging device, even if the imaging device is attached at an angle to the vehicle body, the tilt of the imaging device (i.e., the tilt of the image) can be detected regardless of how the imaging device is attached or the tilt of the vehicle body, thereby reducing the effort required of the user to attach the imaging device horizontally.

[0067] In addition, the display control device (i.e., ECU) is made multifunctional, and the display control device controls the storage of captured image data in a storage device, so that captured image data can be stored without adding a new ECU.

[0068] In addition, by recording video data from multiple imaging devices in association with each other, and by recording tilt information, audio data, and detection data from various sensors in association with video data, more information can be obtained from the saved data.

[0069] In addition, the control device, into which a portable recording medium such as an SD card is inserted as a memory unit, is installed inside the vehicle body, so unlike commercially available add-on drive recorders, the recording medium cannot be easily removed, preventing theft of the recording medium containing the video data.

[0070] In addition, dangerous behavior such as tailgating can be deterred by, for example, turning on the pilot light of the imaging device to notify following vehicles that recording is in progress.

[0071] <Modifications> It should be noted that the configurations of the video processing device and video processing system described in the above embodiments are merely examples, and it goes without saying that the configurations may be modified within the scope of the present disclosure.

[0072] In the above embodiment, the image processing system displays an image in response to an instruction from the rider. However, the image processing system may also display an image in response to the detection results of various sensors provided in the vehicle. The various sensors include a blind spot monitor (BSM) that detects the approach of a vehicle from behind the vehicle, a sensor that detects the inclination of the vehicle, a vehicle speed sensor, etc. For example, the image processing system may display an image in response to an alarm from the BSM.

[0073] In the above embodiment, an example was described in which video data saving begins simultaneously with power-on and continues until the power is turned off, with the video being displayed in response to a rider's instruction, i.e., only when the rider wants to view the video. However, the timing of video data saving and video display are not limited to this example. The timing of video display may be the same as or different from the timing of video data saving, although it is desirable that the two be different, as described in the above embodiment. Furthermore, while it is desirable that video data be saved at all times while the motorcycle is in motion, the timing at which video data saving begins is not limited to when the power is turned on. For example, the video processing system may start saving video data in response to a rider's instruction or a sensor status determination.

[0074] In the above embodiment, an example has been described in which the display device is installed to the right or left of the meter display on the instrument panel (i.e., to the right or left as seen from the rider), but the display device may be installed in a position in front of the rider that is easy to see, regardless of whether a meter display is present. For example, the display device may be installed on the handlebars, in a mirror mounting portion, on top of the fuel tank, inside the cowl, or above or integral with the meter display.

[0075] In the above embodiment, an example in which a dedicated display device is provided in the video processing system has been described. However, a portable information processing device such as a smartphone or tablet may also be used as the display device. In this case, the portable information processing device executes various processes by using an installed dedicated application program in cooperation with an ECU (Electrical Control Unit). For example, a smartphone can be wirelessly connected to the communication unit 22 via Bluetooth (registered trademark), and can display images based on video data stored in the storage unit 24 on the smartphone screen (see FIG. 2).

[0076] <Supplementary Notes> Features of the present disclosure are as follows. [Supplementary Note 1] An image processing device that controls a display device placed in front of a motorcycle rider, comprising: a memory and a processor connected to the memory, wherein the processor: acquires image data from an imaging device that images a rear of the motorcycle while the motorcycle is traveling; stores the acquired image data in the memory; acquires tilt information from a sensor that detects the tilt of the imaging device; corrects the image data so that the image displayed on the display device is rotated in accordance with the acquired tilt information; and displays an image based on the corrected image data on the display device. [Supplementary Note 2] The image processing device according to Supplementary Note 1, wherein the processor associates the tilt information acquired from the sensor at the time the image data is acquired with the image data and stores it in the memory. [Supplementary Note 3] The video processing device according to Supplementary Note 1 or Supplementary Note 2, wherein the processor processes the video data based on the tilt information acquired from the sensor at the time the video data was acquired before saving the video data so that a line representing at least one of the horizontal and vertical directions is superimposed on the video, and stores the processed video data in the memory. [Supplementary Note 4] The video processing device according to Supplementary Note 1 or Supplementary Note 2, wherein the processor processes the video data based on the tilt information acquired from the sensor at the time the video data was acquired before saving the video data so that a value representing the tilt of the video is superimposed on the video, and stores the processed video data in the memory.[Supplementary Note 5] An image processing system comprising: an image capture device that captures images of the surroundings of a motorcycle; a display device positioned in front of a rider and displaying images based on image data acquired from the image capture device; a sensor that detects the tilt of the image capture device; and a control device that performs a first control to store the image data acquired from the image capture device in a storage device while the motorcycle is traveling; and a second control to correct the image data so that the image displayed on the display device is rotated in accordance with the tilt information acquired from the sensor, and display an image based on the corrected image data on the display device. [Supplementary Note 6] The image processing system of Supplementary Note 5, wherein the sensor is attached to the outside of the housing of the image capture device. [Supplementary Note 7] The image processing system of Supplementary Note 6, wherein the housing of the image capture device is rectangular, and the sensor is housed in a second rectangular housing, and the second housing is attached to the top surface of the housing so as to be placed on the housing of the image capture device. [Supplementary Note 8] The image processing system of Supplementary Note 5, wherein the sensor is attached inside the housing of the image capture device. [Supplementary Note 9] The image processing system of any one of Supplements 6 to 8, wherein the sensor is positioned with respect to the image sensor of the imaging device so that the horizontal direction of the sensor coincides with the horizontal direction of the image. [Supplementary Note 10] The image processing system of any one of Supplements 5 to 9, wherein the control device, in the first control, stores the tilt information acquired from the sensor at the time the image data is acquired in the storage device in association with the image data. [Supplementary Note 11] The image processing system of any one of Supplements 5 to 9, wherein the imaging device includes a first imaging device that images a rear of the motorcycle, the control device, in the first control, stores first image data acquired from the first imaging device in the storage device, and the control device, in the second control, displays the first image acquired from the first imaging device.[Supplementary Note 12] The image processing system described in Supplementary Note 11, wherein the imaging device further includes a second imaging device that images the front or side of the motorcycle, and the control device, during the first control, stores the second image data acquired from the second imaging device in the storage device in association with the first image data acquired from the first imaging device, and the control device, during the second control, displays the first image acquired from the first imaging device and the second image acquired from the second imaging device on one screen.

[0077] This application claims priority from Japanese Patent Application No. 2023-120239, filed on July 24, 2023, the entire disclosure of which is incorporated herein by reference.

Claims

1. An image processing device that controls a display device positioned in front of a rider of a motorcycle, comprising a memory and a processor connected to the memory, wherein the processor performs the following operations: acquires image data from an imaging device that images the rear of the motorcycle while the motorcycle is traveling; stores the acquired image data in the memory; acquires tilt information from a sensor that detects the tilt of the imaging device; corrects the image data so that the image displayed on the display device is rotated in accordance with the acquired tilt information; and displays an image based on the corrected image data on the display device.

2. The image processing device according to claim 1, wherein the processor stores in the memory the tilt information acquired from the sensor at the time the image data was acquired in association with the image data.

3. An image processing device as described in claim 1 or claim 2, wherein, before storing the image data, the processor processes the image data so that a line representing at least one of the horizontal and vertical directions is superimposed on the image based on the tilt information obtained from the sensor at the time the image data was obtained, and stores the processed image data in the memory.

4. An image processing device as described in claim 1 or claim 2, wherein, before storing the image data, the processor processes the image data so that a value representing the tilt of the image is superimposed on the image based on the tilt information acquired from the sensor at the time the image data was acquired, and stores the processed image data in the memory.

5. An image processing system comprising: an imaging device which captures images of the surroundings of a motorcycle; a display device which is positioned in front of a rider and displays images based on image data acquired from the imaging device; a sensor which detects the tilt of the imaging device; and a control device which performs a first control to store the image data acquired from the imaging device in a storage device while the motorcycle is traveling; and a second control to correct the image data so that the image displayed on the display device is rotated in accordance with the tilt information obtained from the sensor, and to display an image based on the corrected image data on the display device.

6. The image processing system according to claim 5, wherein the sensor is attached to the outside of a housing of the imaging device.

7. The image processing system of claim 6, wherein the housing of the imaging device is rectangular, the sensor is housed in a second housing having a rectangular shape, and the second housing is attached to an upper surface of the housing so as to be placed on the housing of the imaging device.

8. The image processing system according to claim 5, wherein the sensor is attached inside a housing of the imaging device.

9. An image processing system according to any one of claims 6 to 8, wherein the sensor is positioned relative to the image sensor of the imaging device so that the horizontal direction of the sensor coincides with the horizontal direction of the image.

10. An image processing system as described in any one of claims 5 to 9, wherein, in the first control, the control device stores in the storage device the tilt information acquired from the sensor at the time the image data is acquired in association with the image data.

11. The image processing system of any one of claims 5 to 9, wherein the imaging device includes a first imaging device that images the rear of the motorcycle, the control device stores in the storage device first image data acquired from the first imaging device in the first control, and the control device displays the first image acquired from the first imaging device in the second control.

12. The image processing system of claim 11, wherein the imaging device further comprises a second imaging device that images the front or side of the motorcycle, and the control device, in the first control, associates the second image data acquired from the second imaging device with the first image data acquired from the first imaging device and stores it in the storage device, and the control device, in the second control, displays the first image acquired from the first imaging device and the second image acquired from the second imaging device on a single screen.

Images