Video transmission system, vehicle, and video transmission method

Abstract

To transmit multiple videos efficiently.SOLUTION: A video transmission system includes a first transmitter that superimposes two or more first video signals to generate a second video signal using a first video transmission standard, a second transmitter that generates a fourth video signal by superimposing a third video signal and the second video signal using a second video transmission standard different from the first video transmission standard, a first receiver that receives the fourth video signal transmitted by a single cable and separates the second video signal and the third video signal, and a second receiver that separates the two or more first video signals from the second video signal.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to video transmission. [Background technology]

[0002] There are technologies for improving the efficiency of video transmission. In this regard, for example, Patent Document 1 discloses a system in which a plurality of cameras are connected and video signals are multiplexed to achieve efficient video transmission. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-328479 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to efficiently transmit multiple videos. [Means for solving the problem]

[0005] One aspect of an embodiment of the present disclosure is a video transmission system having: a first transmitter that uses a first video transmission standard to superimpose two or more first video signals to generate a second video signal; a second transmitter that uses a second video transmission standard different from the first video transmission standard to superimpose the second video signal and a third video signal to generate a fourth video signal; a first receiver that receives the fourth video signal transmitted over a single cable and separates the second video signal from the third video signal; and a second receiver that separates the two or more first video signals from the second video signal.

[0006] One aspect of an embodiment of the present disclosure is a vehicle having a plurality of cameras that capture images outside the vehicle; a first device that uses a first video transmission standard to superimpose two or more first video signals to generate a second video signal, and that uses a second video transmission standard different from the first video transmission standard to superimpose the second video signals with a third video signal to generate a fourth video signal; and a second device that receives the fourth video signal transmitted over a single cable, separates the second video signal from the third video signal, and separates the two or more first video signals from the second video signal.

[0007] Other aspects include a method executed by the above-mentioned system or vehicle, a program for causing a computer to execute the method, or a computer-readable storage medium non-temporarily storing the program. [Effects of the Invention]

[0008] According to the present disclosure, multiple videos can be transmitted efficiently. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a vehicle system according to an embodiment. [Figure 2] Schematic diagram of a system that superimposes video signals using GVIF. [Figure 3] Schematic diagram of a system that overlays video signals using MIPI. [Figure 4] 10 shows an example of an image displayed by the in-vehicle terminal 20 when the vehicle is backing up. [Figure 5] FIG. 2 is a diagram illustrating an outline of processing performed by the video transmission system according to the first embodiment. [Figure 6] 3 is a schematic diagram of a video signal transmitted between the ECU 10 and the in-vehicle terminal 20. [Figure 7] 1 is a block diagram illustrating an example of the configuration of a vehicle 100. FIG. [Figure 8] 2 is a diagram illustrating the functional modules of the control unit 11 and the flow of data between the modules. FIG. [Figure 9] 10 shows an example of selection data transmitted by the vehicle-mounted terminal 20. [Figure 10] FIG. 2 is a diagram illustrating the functional modules of the control unit 21 and the flow of data between the modules. [Figure 11] 10 shows an example of data that the in-vehicle terminal 20 refers to when selecting a camera. [Figure 12] 4 is a flowchart of a process executed by the ECU 10 and the in-vehicle terminal 20. [Figure 13] FIG. 11 is a module configuration diagram of a control unit 11 in a second embodiment. [Figure 14] FIG. 2 is a diagram illustrating the positions of cameras installed in a vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0010] In recent years, the number of in-vehicle cameras has been increasing. Examples of in-vehicle cameras include dashcams, rear cameras, side cameras, stereo cameras for autonomous driving, and cameras for driver monitoring. FIG. 14 is a diagram illustrating cameras installed on the exterior of a vehicle. As shown in the figure, a vehicle is equipped with multiple cameras so that it can monitor multiple directions. These cameras are connected to an electronic control unit (ECU) that manages the images. Furthermore, the electronic control unit is connected to an in-vehicle device (navigation device) or the like.

[0011] A predetermined video transmission standard is used for transmitting video, such as Gigabit Video Interface (GVIF) (registered trademark) and Mobile Industry Processor Interface (MIPI) (registered trademark). Video transmission can be achieved via modules that send and receive video and cables that connect the modules together. However, as the number of cameras installed in a vehicle increases, the number of modules and cables required for sending and receiving video also increases.

[0012] One way to solve this problem is to transmit video signals in a superimposed manner, which reduces the number of modules and cables required for transmitting and receiving video.

[0013] On the other hand, increasing the number of images to be overlaid increases the amount of hardware required, which can lead to increased costs. While there are standards that allow multiple images to be overlaid using a single piece of hardware, these require uniform image resolution and frame rate, making them unsuitable for vehicles equipped with a variety of cameras. Furthermore, overlaying images can result in a decrease in bit rate, which can degrade image quality and other aspects of the image. In particular, safety-related images (e.g., images from a rear camera) must meet specified quality requirements, and degradation of image quality (such as image quality degradation) may not be permitted. The video transmission system according to the present disclosure enables the superimposed transmission of multiple videos while ensuring quality requirements.

[0014] One aspect of the present disclosure is a video transmission system. Specifically, the system includes a first transmitter that uses a first video transmission standard to superimpose two or more first video signals to generate a second video signal, a second transmitter that uses a second video transmission standard different from the first video transmission standard to superimpose the second video signal with a third video signal to generate a fourth video signal, a first receiver that receives the fourth video signal transmitted via a single cable and separates the second video signal from the third video signal, and a second receiver that separates the two or more first video signals from the second video signal.

[0015] Two or more first video signals and a third video signal can be transmitted to different devices via a single cable. When the video transmission system is applied to a vehicle, the two or more first video signals and the third video signal may be video signals generated by an in-vehicle camera. The first transmitter superimposes two or more first video signals to generate a second video signal, and the second transmitter further superimposes a third video signal on the second video signal to generate a fourth video signal.

[0016] The video transmission standard used by the first transmitter and the video transmission standard used by the second transmitter are different from each other. For example, a first transmitter may superimpose two or more first video signals using a first video transmission standard capable of superimposing multiple videos, and the resulting second video signal may be transmitted to a second transmitter, which may superimpose the second video signals and a third video signal, and the second transmitter may perform the superimposition using the second video transmission standard to generate a fourth video signal.

[0017] The first and second receivers separate the superimposed video signals in the reverse order, i.e., they separate the third video signal and the second video signal from the fourth video signal using the second video transmission standard, and then they separate the second video signal into two or more first video signals using the first video transmission standard. In this way, it becomes possible to combine a plurality of different video transmission standards and perform video superimposition and separation.

[0018] When video is superimposed and transmitted using a single video transmission standard, the following problems may arise. If the number of video signals being transmitted changes, the bandwidth allocated to each signal will change, which will affect the video quality. It is difficult to mix images with different resolutions and frame rates. On the other hand, by combining multiple different video transmission standards, it becomes possible to simultaneously transmit multiple video signals with different resolutions and frame rates while maintaining video quality requirements.

[0019] Specific embodiments of the present disclosure will be described below with reference to the accompanying drawings. Unless otherwise specified, the hardware configuration, module configuration, functional configuration, etc. described in each embodiment are not intended to limit the technical scope of the disclosure to those configurations.

[0020] (First embodiment) An overview of a vehicle system according to a first embodiment will be described with reference to Fig. 1. The vehicle system according to this embodiment includes a vehicle 100.

[0021] Vehicle 100 is an automobile equipped with a plurality of cameras, such as a rear camera, a side camera, a camera for a drive recorder, a camera for automatic driving, and a camera for monitoring the driver. The vehicle 100 may be a vehicle that allows monitoring of the outside of the vehicle from the driver's seat. The vehicle 100 may also be a vehicle that is capable of automatic or semi-automatic driving. A plurality of cameras included in the vehicle 100 are connected to the ECU 10.

[0022] The ECU 10 is an electronic control unit that processes video signals captured by multiple cameras and is also called a camera ECU. The ECU 10 is configured to be able to acquire video signals from multiple cameras, and can select and provide a video signal based on a request from other electronic control units or the in-vehicle terminal 20. Although a single ECU is illustrated in FIG. 1, the vehicle 100 may be configured to include multiple other ECUs. Examples of the other ECUs include an engine ECU, a body ECU, and the like. Examples of ECUs include a CU, an automatic driving ECU, a power control ECU, etc. The ECU 10 may be one that manages only camera images, or may be one that performs other processing (e.g., automatic driving, driver assistance, etc.) based on the camera images.

[0023] The in-vehicle terminal 20 is an information terminal mounted on the vehicle 100. The in-vehicle terminal 20 is also called an infotainment terminal, and has a function of providing information (for example, traffic information and route guidance) and entertainment (for example, music and videos) to the vehicle occupants. The in-vehicle terminal 20 may function independently, such as a car navigation device, or may have a function of linking with a smartphone or the like. The vehicle-mounted terminal 20 may also be configured to be able to communicate with a network via a communication module.

[0024] The in-vehicle terminal 20 is configured to be able to output camera images according to the status of the vehicle. For example, when the gear position is "reverse," the in-vehicle terminal 20 may receive an image corresponding to the rear camera from the ECU 10 and output it via the display. Also, when the vehicle speed of the vehicle 100 is below a certain speed and the vehicle 100 is located at an intersection with poor visibility, the in-vehicle terminal 20 may receive images corresponding to the left and right front side cameras from the ECU 10 and output them via the display. Furthermore, the in-vehicle terminal 20 may be configured to be able to output camera images based on the driver's operation. For example, when a function for monitoring the entire periphery of the vehicle is activated, the in-vehicle terminal 20 may receive images corresponding to multiple cameras from the ECU 10, synthesize the images, and generate an image from a viewpoint overlooking the vehicle 100 (hereinafter, an all-around image), which may be output via a display.

[0025] Here, problems in video transmission will be explained. Video transmission within the vehicle (for example, transmission between the ECU 10 and the in-vehicle terminal 20) can be performed using a predetermined video transmission standard. For example, when GVIF (Gigabit Video Interface) (registered trademark), which is one of the video transmission standards, is used, video signals can be transmitted over a single cable. High-speed serial transmission is possible. Figure 2(A) is a schematic diagram of a system that transmits video using GVIF. As shown in the figure, transmission can be performed by serializing the video signal using a transmitter. The serialized signal is deserialized at the receiving end, and the video signal is extracted.

[0026] On the other hand, when multiple cameras generate video signals, it becomes necessary to transmit them all at once. For example, in order to generate a panoramic video image, it is necessary to simultaneously transmit the video images from multiple cameras capturing the exterior of the vehicle. One way to achieve this is to provide multiple units (transmitters and receivers) that send and receive video signals, and multiple transmission cables, as shown in Figure 2(B). However, this method requires hardware for each camera, which increases costs.

[0027] Another method is to superimpose multiple video signals, as shown in Figure 2(C). The transmitter (serializer) used by the GVIF standard has the ability to superimpose video signals, which can be used to simultaneously transmit video signals output by multiple cameras. The transmitter transmits multiple video signals sequentially using time division. The multiple superimposed images can be separated by the receiver (deserializer). In this disclosure, "superimposing" refers to a process for sequentially transmitting multiple video signals over a single transmission path. Examples of such a process include frame interleaving, which switches the source of a video signal for each frame, and line interleaving, which switches the source of a video signal for each line.

[0028] However, this method also has a problem: the transmitter used by GVIF can only add one video signal to an existing video signal. In other words, if you want to transmit video signals generated by three cameras, you will need three transmitters and three receivers. This creates a cost problem.

[0029] On the other hand, MIPI (Mobile Industry Processor Interface), one of the video transmission standards (registered trademark), it is also possible to superimpose multiple video signals. Figure 3 is a schematic diagram of a system that superimposes video signals using MIPI. For example, a transmitter that converts multiple (up to four lanes) MIPI-standard video signals into serial signals is known.

[0030] However, when this method is used, if the number of video signals being transmitted simultaneously increases, the bandwidth per signal becomes constricted, resulting in a problem of quality degradation. FIG. 4 shows an example of an image displayed by the in-vehicle terminal 20 when the vehicle is backing up. When the vehicle is backing up, for example, as shown in FIG. 4(A), an image corresponding to a rear camera may be output. In this case, only one type of image is transmitted. On the other hand, as shown in FIG. 4(B), in addition to the image corresponding to the rear camera, images from multiple cameras may be transmitted simultaneously to generate an all-around image. In this case, four types of images are transmitted. In other words, the bandwidth allocated to one camera changes depending on the image display method. To ensure safety, it is preferable to transmit the video corresponding to the rear camera with high priority. However, with the above-mentioned method, when displaying the 360-degree view, the bandwidth of the video corresponding to the rear camera is constricted. When the bandwidth is constricted, problems such as a decrease in frame rate, deterioration in image quality, and video freezing can occur, which may pose a safety risk. Furthermore, the MIPI standard has the problem that stable transmission is not possible when images with different resolutions or frame rates are superimposed, which means that it is difficult to use a high-resolution camera only for the rear camera.

[0031] To solve this problem, the video transmission system of this embodiment superimposes multiple camera images using a MIPI standard transmitter, and then further superimposes the superimposed video signal using a GVIF standard transmitter. Figure 5 is a schematic diagram of the processing performed by the video transmission system of this embodiment. Figure 6 is a diagram explaining the video transmission band. Here, video signal 1 represents the video signal from the rear camera (that is, the video signal with high priority), while video signals 2 to 4 represent images from other cameras (for example, the front camera, side camera, etc.).

[0032] In the GVIF standard, one transmitter can superimpose a maximum of two sets of video signals, so the transmission bands are allocated as shown by reference numerals 601 and 602. That is, for video signal 1 (i.e., the video signal from the rear camera), it is possible to guarantee the minimum band shown by reference numeral 601. Furthermore, video signals superimposed according to a specific standard (MIPI) are further superimposed according to a different standard (GVIF), making it possible to mix videos with different resolutions and frame rates.

[0033] Next, each component of the vehicle 100 will be described in detail. FIG. 7 is a block diagram showing an example of the configuration of the vehicle 100 shown in FIG. 1. The vehicle 100 is configured to have an ECU 10 and an in-vehicle terminal 20. These components are connected to each other by a cable for transmitting a video signal and a cable for transmitting a control signal. In this example, the components of the vehicle 100 are the ECU 10 and the in-vehicle terminal 20. Although the terminal 20 is shown as an example, the vehicle 100 may be equipped with a plurality of electronic control units that control the vehicle, such as an engine ECU and a body ECU.

[0034] First, the ECU 10 will be described. The ECU 10 is an electronic control unit that processes video signals acquired by multiple cameras mounted on the vehicle. The ECU 10 is also called a camera ECU. The ECU 10 selects and provides video signals based on requests from other electronic control units or the in-vehicle terminal 20. In this embodiment, the ECU 10 only manages camera images, but the ECU 10 may also execute other processes (e.g., autonomous driving, driving assistance, etc.) based on the camera images. In this case, the ECU 10 may be called an ADAS-ECU or the like.

[0035] The ECU 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a video interface 14.

[0036] The control unit 11 is a calculation unit that executes a predetermined program to realize various functions of the ECU 10. The control unit 11 can be realized by, for example, a CPU or the like. The control unit 11 executes a process of selecting and transmitting video signals generated by multiple cameras based on a command received from an external source (typically, the in-vehicle terminal 20). For example, when the in-vehicle terminal 20 requests a panoramic video image showing the entire surroundings of the vehicle 100, the control unit 11 selects a camera for generating the panoramic video image and transmits the corresponding video signal to the in-vehicle terminal 20. The processing executed by the control unit 11 will be described in detail later.

[0037] The storage unit 12 is a memory device including a main storage device and an auxiliary storage device. The auxiliary storage device stores an operating system (OS), various programs, various tables, etc., and by loading the programs stored therein into the main storage device and executing them, various functions that match predetermined purposes, as will be described later, can be realized.

[0038] The communication unit 13 is an interface unit for connecting the ECU 10 and the in-vehicle terminal 20. In this embodiment, the ECU 10 and the in-vehicle terminal 20 are connected by two paths: a path for transmitting video and a path for transmitting control signals. Examples of the path for transmitting video include a twisted pair cable and a coaxial cable. In this embodiment, video from multiple channels is superimposed and transmitted, so only one cable is used. Also, an example of the path for transmitting control signals is an in-vehicle network. For example, a network using a CAN (Controller Area Network) can be used. The control signal can be transmitted via a network or Ethernet (registered trademark). The path for transmitting the control signal may be shared with other ECUs, etc. The communication unit 13 is an interface that connects these together. The communication unit 13 may include a hardware interface for transmitting video and a hardware interface (for example, a CAN controller) for transmitting control signals.

[0039] The video interface 14 is an interface unit for connecting multiple cameras mounted on a vehicle. In this embodiment, the video interface 14 receives video signals conforming to the MIPI standard (for example, MIPI CSI-2) from the cameras.

[0040] Cameras 30A to 30E are multiple cameras mounted on vehicle 100. Examples of cameras mounted on vehicle 100 include cameras for recording (drive recorders), cameras for automatic driving, cameras for safety confirmation by the driver, and cameras for monitoring the driver. Examples of camera placement locations include the front, front side, side, and rear. In the example of FIG. 7, vehicle 100 is configured with camera 30A, which is a rear camera, camera 30B, which is a front camera, and cameras C to E, which are side cameras. Rear camera 30 The front camera 30A is used to check safety when backing up the vehicle. The front camera 30B is used for driving assistance and for recording with a drive recorder. The side cameras 30C to 30E are used to check blind spots. In addition, a panoramic image can be generated based on the images captured by these cameras. These cameras are connected to the ECU1 via MIPI standards (e.g., MIPI CSI-2). It is connected to 0.

[0041] Next, the function of the control unit 11 will be described. 8 is a diagram illustrating the functional modules of the control unit 11 and the flow of data between the modules. The functional modules shown in the figure can be realized by a CPU or the like executing a program stored in a storage means such as a ROM.

[0042] First, a plurality of video signals acquired via the video interface 14 are input to the control unit 11. In this example, the video signal from the rear camera 30A is referred to as the first camera signal. The video signals from the cameras 30B to 30E are referred to as the second camera signals. In this embodiment, the first camera signal is the video signal with the highest processing priority. This is because the first camera signal includes video capturing the area behind the vehicle and should be transmitted in real time (for example, if frames are dropped while the vehicle is backing up, a pedestrian crossing behind the vehicle may not be captured). In contrast, the second camera signal is used to generate auxiliary video (all-around video), and therefore has a relatively lower priority than the first camera signal.

[0043] The first camera signal is always transmitted to the in-vehicle terminal 20, whereas the second camera signal is appropriately selected based on an instruction from the in-vehicle terminal 20. For example, when an image from a side camera is requested by the in-vehicle terminal 20, only the image signal from the side camera is selected.

[0044] The video selection unit 111 selects one of the multiple second camera signals to be transmitted. In this embodiment, the in-vehicle terminal 20 transmits data specifying a camera ("selection data" in the figure) to the video selection unit 111, and the video selection unit 111 selects a second camera signal in response to this. Figure 9(A) is an example of the selection data. For example, when the in-vehicle terminal 20 transmits selection data requesting an image for generating a panoramic image, the image selection unit 111 selects the second camera signal for generating a panoramic image. In this embodiment, the "first camera signal" and the "second camera signal selected by the video selection unit 111" are the targets of superimposition.

[0045] The image processing unit 112 adjusts the multiple second camera signals to be superimposed. For example, the image processing unit 112 performs processing to match the resolution and frame rate for each of the multiple second camera signals. For example, when superimposing video signals that comply with the MIPI standard, it is preferable to match the resolution and frame rate for all video signals in order to smooth the communication speed. The resolution may be adjusted by stretching or reducing the image, or by bleed or clipping. The processing performed by the image processing unit 112 is not limited to processing for superimposing video signals. For example, the image processing unit 112 may perform processing to adjust the frame rate of the video (for example, converting a 30 fps video signal to 29 fps) in consideration of the light emission cycle (for example, 60 Hz) of an LED traffic light.

[0046] The first superimposing unit 113 performs a process of superimposing the multiple second camera signals selected by the video selecting unit 111. As a result, the multiple video signals are converted into a single video signal including multiple lanes. Note that the selection of the second camera signals and the superimposition of the signals are performed within the video signal standard. (For example, within the range of the maximum bit rate.) The converted signal may be, for example, CSI-2.

[0047] The second convolution unit 114 performs a process of convolution of the first camera signal with the video signal convolutioned by the first convolution unit 113. This makes it possible to obtain video signals with a nested structure, as shown in Fig. 6. Video signals 1 to 4 comply with the MIPI standard, and the video signal (reference numeral 603) generated by the first convolution unit 114 complies with the GVIF standard. The video signal generated by the second superimposing unit 114 is transmitted to the vehicle-mounted terminal 20.

[0048] Next, the vehicle-mounted terminal 20 shown in FIG. 7 will be described. The in-vehicle terminal 20 is a device that provides information to the occupants of the vehicle 100, and is also called a car navigation system, an infotainment system, or a head unit. The in-vehicle terminal 20 can provide navigation and entertainment to the occupants of the vehicle. The in-vehicle terminal 20 may have a function to download traffic information, road map data, music, videos, etc. by communicating with an external network of the vehicle 100.

[0049] The in-vehicle terminal 20 can be configured as a general-purpose computer. That is, the in-vehicle terminal 20 can be configured as a computer having a processor such as a CPU or GPU, a main memory such as a RAM or ROM, and an auxiliary memory such as an EPROM, a hard disk drive, or removable media. The auxiliary memory stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, various functions that match predetermined purposes, as described below, can be realized. However, some or all of the functions may be realized by hardware circuits such as ASICs or FPGAs.

[0050] The in-vehicle terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, and an input / output unit 24.

[0051] The control unit 21 is a means for controlling the in-vehicle terminal 20. The control unit 21 is, for example, an information processing unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). It is composed of The control unit 21 provides information to the vehicle occupants. The provided information includes, for example, traffic information, navigation information, music and video, radio broadcasts, digital television broadcasts, etc. The control unit 21 outputs the information via the input / output unit 24.

[0052] Furthermore, the control unit 21 provides a driving assistance function using an in-vehicle camera. Examples of driving assistance functions using an in-vehicle camera include a function for monitoring the rear of the vehicle when backing up, and a function for monitoring traffic conditions on both sides at an intersection with poor visibility. When providing a driving assistance function using an in-vehicle camera, the control unit 21 requests the ECU 10 to transmit an image. The control unit 21 may execute a process for generating an image to be output or a process for generating a user interface screen based on the video signal transmitted from the ECU 10. The processing executed by the control unit 21 will be described in detail later.

[0053] The storage unit 22 is a means for storing information, and is configured with storage media such as RAM, a magnetic disk, a flash memory, etc. The storage unit 22 stores various programs executed by the control unit 21, data used by the programs, etc.

[0054] The communication unit 23 is an interface unit for connecting the in-vehicle terminal 20 and the ECU 10. Similar to the communication unit 13, the communication unit 23 has two interfaces: one corresponding to a path for transmitting video, and the other corresponding to a path for transmitting control signals. The interface corresponding to the path for transmitting the control signal is, for example, a twisted pair cable or a coaxial cable. The interface corresponding to the path for transmitting the control signal is, for example, a CAN (Controller Area Network).

[0055] The input / output unit 24 is a means for accepting input operations performed by the user and presenting information to the user. Specifically, it is composed of a touch panel and its control means, and a liquid crystal display and its control means. In this embodiment, the touch panel and the liquid crystal display are combined into one touch panel display. The input / output unit 24 may also have a speaker or the like for outputting audio.

[0056] Next, the function of the control unit 21 will be described. 10 is a diagram illustrating the functional modules of the control unit 21 and the flow of data between the modules. The functional modules shown in the figure can be realized by a CPU or the like executing a program stored in a storage means such as a ROM.

[0057] The selection unit 211 transmits to the ECU 10 data (selection data) that designates the camera that will capture the video. The selection unit 211 determines which camera's image is to be requested from the ECU 10 based on the driver's operation. For example, when the vehicle 100 is reversing, an image from the rear camera may be output as shown in FIG. 4(A). In this case, the rear camera is selected. When the vehicle 100 is reversing, an image from the rear camera and an all-around image may be output as shown in FIG. 4(B). In this case, all cameras for the external monitor, including the rear camera, are selected.

[0058] The camera to be used may be determined based on the driver's operation or the state of the vehicle 100. For example, when the gear position is "reverse," the camera to be used when reversing may be determined. For this reason, the in-vehicle terminal 20 may store data associating the vehicle state (mode) with the camera to be used, and generate selection data based on the data. The selection unit 211 can generate selection data based on, for example, data such as that illustrated in FIG. 11. Such data may be stored in the storage unit 22.

[0059] The first separation unit 212 receives the video signal transmitted from the ECU 10 via the communication unit 23 and separates the video signal. As described with reference to FIG. 6, the video signal transmitted and received between the ECU 10 and the in-vehicle terminal 20 conforms to the GVIF standard, and by separating the video signal, two video signals conforming to the MIPI standard (reference numerals 601 and 602 in FIG. 6) can be obtained. Of the separated video signals, the first camera signal (reference numeral 601) (i.e., the video signal generated by the rear camera) is transmitted to the information providing unit 214. The video signal (reference numeral 602 ) other than the first camera signal is sent to the second separator 213 .

[0060] Second separation unit 213 separates the video signal transmitted from first separation unit 212 and acquires a plurality of second camera signals. The plurality of second camera signals are transmitted to information provision unit 214 in the same manner as the first camera signal.

[0061] The information providing unit 214 generates a user interface screen to be provided to the vehicle occupant (driver) based on the first camera signal and the second camera signal. For example, the information providing unit 214 embeds the video captured by the camera into a predetermined graphic and outputs it. At this time, guidelines indicating the vehicle's position, predicted trajectory, etc. may be superimposed. Furthermore, a process of generating a single image (such as a panoramic video) may be performed based on the video captured by multiple cameras.

[0062] FIG. 12 is a flowchart of the process executed by the ECU 10 and the in-vehicle terminal 20. First, in step S11, the selection unit 211 determines the camera from which to capture video. This step may be executed when the driver performs a predetermined operation, or when the vehicle enters a predetermined state. The selection unit 211 determines the camera to be used based on the driver's operation or the vehicle state, and transmits data (selection data) for specifying the camera to the ECU 10.

[0063] In step S12, the ECU 10 (video selection unit 111) identifies one or more cameras from which to acquire video signals based on the selection data, and selects the video signals from the identified cameras. The selected video signals (second camera signals) are passed to the image processing unit 112. Next, in step S13, the image processing unit 112 performs preprocessing for superimposing the video signals. Specifically, the second camera signal is subjected to frame rate conversion, resolution conversion, and the like. Note that, if it is preprocessing for superimposing multiple video signals, processing other than the exemplified processing may be performed in this step. Note that, if not necessary, this step may be omitted. The second camera signal after processing by the image processing unit 112 is passed to the first superimposing unit 113.

[0064] Next, in step S14, the first superimposing unit 113 superimposes the input second camera signals to generate a single video signal. Next, in step S15, the second superimposing unit 114 superimposes the first camera signal on the second camera signal superimposed by the first superimposing unit 113. The superimposed video signal is transmitted to the in-vehicle terminal 20 via an interface for transmitting video.

[0065] In step S16, first separation unit 212 separates the received video signal to obtain a first camera signal. The separated video signal is transmitted to second separation unit 213. In step S17, second separation unit 213 separates the received video signal to obtain a second camera signal. The first camera signal and second camera signal are transmitted to information provision unit 214. The information providing unit 214 provides information to the vehicle occupant based on the received first camera signal and second camera signal.

[0066] As described above, the ECU 10 according to this embodiment superimposes video signals generated by multiple cameras in multiple stages using two different video transmission standards. This allows multiple video signals to be superimposed and transmitted while ensuring quality requirements for a given video signal. Furthermore, for video signals that require high quality, the resolution and frame rate can be set higher than for other video signals.

[0067] (Second embodiment) In the first embodiment, the video signal generated by the rear camera is transmitted with high priority. However, it may be possible to designate from outside the ECU 10 which video signal is to be given priority (i.e., which video signal is to be treated as the first camera signal). In the second embodiment, the in-vehicle terminal 20 instructs the ECU 10 on the priority of the video signal.

[0068] 13 is a module configuration diagram of ECU 10 (control unit 11) in the second embodiment. This embodiment differs from the first embodiment in that video selection unit 111 selects both the first camera signal and the second camera signal, and that selection data transmitted from in-vehicle terminal 20 includes a priority designation.

[0069] 9B is an example of selection data in the second embodiment. As shown in the figure, in this embodiment, the selection data is data that associates cameras to be used with each other by priority. In this example, the camera with the highest priority is the rear camera. That is, the first camera signal is the video signal generated by the rear camera. The second camera signal is the video signal generated by the other camera. Based on the selection data, the video selection unit 111 selects a video signal and determines the output destination for each priority. Specifically, the video signal corresponding to the camera with the highest priority is sent to the first overlay unit 114 as a first camera signal, and the other video signals are sent to the first overlay unit 113 as second camera signals. The subsequent processing is the same as in the first embodiment.

[0070] As described above, according to the second embodiment, it is possible to dynamically specify the priority of the video signals, which makes it possible to prioritize the video from the rear camera when the vehicle is backing up, and prioritize the video from the side camera when the vehicle is merging into another lane.

[0071] (Variation) The above-described embodiment is merely an example, and the present invention can be implemented by appropriately modifying it within the scope that does not deviate from the gist of the present invention. For example, the processes and means described in this disclosure can be freely combined and implemented as long as no technical contradiction occurs.

[0072] Furthermore, in the description of the embodiment, the multiple cameras output video signals that comply with the MIPI standard, but the video signals output by the cameras may also comply with other standards. In the description of the embodiment, the video signal conforming to the GVIF standard is used for transmission between the ECU 10 and the in-vehicle terminal 20, but other standards may be used for video transmission between the devices. However, it is preferable that the standard that can be adopted satisfies at least one of the following conditions:

[0073] (1) A standard that can guarantee predetermined quality requirements for at least one video signal that is superimposed and transmitted. The predetermined quality requirement is, for example, a real-time requirement, which allows the first camera signal to be transmitted in real time. (2) A standard that can always allocate a bandwidth of at least a specified value to at least one video signal that is superimposed and transmitted. This makes it possible to, for example, allocate a band equal to or greater than a predetermined value to a first camera signal, and allocate the remaining band to a second camera signal. (3) A standard that allows multiple resolutions and frame rates to be mixed.

[0074] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0075] The present disclosure can also be realized by supplying a computer program that implements the functions described in the above embodiments to a computer, and having one or more processors of the computer read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, This includes any type of disk (such as a DVD disk, Blu-ray disk, etc.), read-only memory (ROM), random-access memory (RAM), EPROM, EEPROM, magnetic card, flash memory, optical card, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0076] 100...vehicles 10 ECU 20. In-vehicle terminal 11,21 Control unit 12,22...Storage section 13,23···Communications Department 14. Video Interface 24...Input / output section 30. Camera

Claims

1. A video transmission system applied to a vehicle, a first transmitter that uses a first video transmission standard to superimpose two or more first video signals to generate a second video signal; a second transmitter that uses a second video transmission standard different from the first video transmission standard to superimpose a third video signal and the second video signal to generate a fourth video signal; a first receiver that receives the fourth video signal transmitted by a single cable and separates the second video signal and the third video signal; a second receiver for separating the two or more first video signals from the second video signal; and the third video signal is a video signal that requires real-time requirements and is generated by a rear camera of the vehicle; the first video signal is a video signal generated by a camera disposed other than the rear; a bandwidth equal to or greater than a predetermined value is allocated to the third video signal, and the remaining bandwidth is allocated to the second video signal including the two or more first video signals; the second video transmission standard is a standard that can guarantee the real-time requirement for the third video signal and that can allocate a bandwidth equal to or greater than the predetermined value to the third video signal; Video transmission system.

2. the third video signal is a single video signal, and the number of the first video signals included in the second video signal is determined based on an external designation. The video transmission system according to claim 1 .

3. a predetermined electronic control unit mounted on the vehicle has the first and second transmitters; The video transmission system according to claim 1 .

4. an in-vehicle terminal mounted on the vehicle has the first and second receivers; The video transmission system according to claim 3 .

5. the in-vehicle terminal transmits, to the electronic control unit, first data specifying two or more cameras that generate the first video signal; the first transmitter included in the electronic control unit superimposes the first video signals generated by two or more cameras designated by the first data; 5. The video transmission system according to claim 4.

6. A plurality of cameras for capturing images outside the vehicle; a first device that uses a first video transmission standard to superimpose two or more first video signals to generate a second video signal, and that uses a second video transmission standard different from the first video transmission standard to superimpose a third video signal and the second video signals to generate a fourth video signal; a second device that receives the fourth video signal transmitted by a single cable, separates the second video signal and the third video signal, and separates the two or more first video signals from the second video signal; and The third video signal is a video signal that has real-time requirements and is generated by a rear camera; the first video signal is a video signal generated by a camera disposed other than the rear; a bandwidth equal to or greater than a predetermined value is allocated to the third video signal, and the remaining bandwidth is allocated to the second video signal including the two or more first video signals; the second video transmission standard is a standard that can guarantee the real-time requirement for the third video signal and that can allocate a bandwidth equal to or greater than the predetermined value to the third video signal; vehicle.

7. the first device is an electronic control unit that manages the plurality of cameras; the second device is an in-vehicle terminal; 7. The vehicle of claim 6.

8. A video transmission method applied to a vehicle, comprising: a first step of superimposing two or more first video signals using a first video transmission standard to generate a second video signal; a second step of superimposing a third video signal and the second video signal using a second video transmission standard different from the first video transmission standard to generate a fourth video signal; a third step of transmitting the fourth video signal over a single cable; a fourth step of receiving the fourth video signal and separating the second video signal and the third video signal; a fifth step of separating the two or more first video signals from the second video signal; Including, the third video signal is a video signal that requires real-time requirements and is generated by a rear camera of the vehicle; the first video signal is a video signal generated by a camera disposed other than the rear; a bandwidth equal to or greater than a predetermined value is allocated to the third video signal, and the remaining bandwidth is allocated to the second video signal including the two or more first video signals; The second video transmission standard maintains the real-time requirements for the third video signal. a standard that can be verified and that can allocate a bandwidth equal to or greater than the predetermined value to the third video signal, Video transmission method.

9. further comprising the step of simultaneously outputting the separated third video signal and the separated two or more first video signals.

9. The video transmission method according to claim 8.

Images