Video processing apparatus, on-vehicle camera system, and video processing method

Abstract

To makes an appropriate video image to display onto a display even if there is an abnormality in a main microcomputer.SOLUTION: A video processing apparatus according to the present invention, comprises: a main microcomputer that overlaps and draws an image for assisting an operation to a first video image signal input from a camera, and converts a frame rate of the video image signal to which the image is drown to the frame rate for a display, and outputs it as a second video image signal; and a controller that outputs the second video image signal output from the main microcomputer to the display. The controller converts the frame ratio of the first video image signal into a frame rate for the display, and outputs it to the display in the case where the second video image signal is not output from the main microcomputer, or the second video image signal output from the main microcomputer is not updated.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an in-vehicle camera system, and an image processing method. [Background technology]

[0002] Conventionally, there are in-vehicle camera systems that display images of the rear of the vehicle on a display inside the vehicle. In such systems, a technology has been proposed that prevents interruption of the image by directly displaying the image input from the camera on the display when an abnormality occurs in the microcomputer (hereinafter referred to as the main microcomputer) that processes the image captured by the camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-81445 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, there is room for improvement in displaying appropriate images on the display when an abnormality occurs in the main microcomputer. The main microcomputer performs a process to convert the frame rate of the image captured by the camera into a frame rate for the display. However, when an abnormality occurs in the main microcomputer, an image with a frame rate different from the frame rate for the display may be output to the display, and the image may not be displayed correctly.

[0005] The present invention has been made in consideration of the above, and aims to provide an image processing device, an in-vehicle camera system, and an image processing method that can display appropriate images on a display even if an abnormality occurs in the main microcomputer. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the video processing device of the present invention comprises a main microcomputer that superimposes and draws an image that assists driving on a first video signal input from a camera, and converts the frame rate of the video signal on which the image is drawn to a frame rate for a display and outputs it as a second video signal, and a controller that outputs the second video signal output from the main microcomputer to a display, and when the second video signal is not output from the main microcomputer or the second video signal output from the main microcomputer is not updated, the controller converts the frame rate of the first video signal to the frame rate for the display and outputs it to the display. [Effects of the Invention]

[0007] According to the present invention, even if an abnormality occurs in the main microcomputer, an appropriate image can be displayed on the display. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of an in-vehicle camera system. [Figure 2] FIG. 2 is a block diagram of the video processing device. [Figure 3] FIG. 3 is a schematic diagram showing the signal path of a normal camera image. [Figure 4] FIG. 4 is a schematic diagram showing the signal path of the bypass camera image. [Figure 5] FIG. 5 is a flowchart showing a processing procedure executed by the video processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of an image processing device, an in-vehicle camera system, and an image processing method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.

[0010] First, an example of an installation of an in-vehicle camera system according to an embodiment will be described. Fig. 1 is a diagram showing an example of an installation of an in-vehicle camera system. As shown in Fig. 1, an in-vehicle camera system S according to an embodiment is installed in a vehicle 100.

[0011] 1, the vehicle-mounted camera system S includes an image processing device 1 and a camera C. The image processing device 1 includes a display 40, which will be described later with reference to FIG. 2, and functions as a display audio for the vehicle 100. The image processing device 1 also has a function of displaying an image captured by the camera C on the display when the vehicle 100 is in reverse gear.

[0012] The camera C is installed so as to capture images of the surroundings of the vehicle 100. The camera C captures images of the surroundings of the vehicle 100 and outputs a video signal to the video processing device 1. The example shown in FIG. 1 illustrates a case where the camera C is installed so as to capture images of the rear of the vehicle 100. Note that the vehicle-mounted camera system S may include multiple cameras C that capture images of the surroundings of the vehicle 100.

[0013] Next, an example configuration of the video processing device 1 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram of the video processing device 1. As shown in Fig. 2, the video processing device 1 includes a controller 10, a main microcomputer 20, a power supply microcomputer 30, and a display 40.

[0014] The controller 10 is, for example, a video IC and includes a memory 11. The controller 10 performs various processes for displaying the video signal input from the camera C on the display 40. As will be described later, the controller 10 also determines whether there is an abnormality in the main microcomputer 20, controls a MUX (Multiplexer) (not shown) in accordance with the determination result, and switches the signal path of the video signal input from the camera C.

[0015] In addition, when the controller 10 does not output the video signal input from the camera C to the display 40 for display, it outputs the video signal of other images, such as a navigation screen, a television screen, a DVD screen, an audio screen, etc., to the display 40 for display.

[0016] The main microcomputer 20 is, for example, an SoC (System on a Chip) and includes a memory 21. The main microcomputer 20 performs various image processing on the video signal input from the controller 10 using the memory 21, and then outputs the processed video signal to the controller 10.

[0017] Furthermore, the main microcomputer 20 generates video signals for images other than the video signal input from the camera C, such as a navigation screen, a television screen, a DVD screen, an audio screen, and the like, and outputs the signals to the controller 10.

[0018] The power supply microcomputer 30 is a microcomputer that controls the power supply of the video processing device 1. The power supply microcomputer 30 receives an ACC (accessory switch) signal from the vehicle and turns the power supplies of the controller 10 and the main microcomputer 20 ON / OFF.

[0019] The power supply microcomputer 30 receives the vehicle's reverse signal Rev, and when the reverse signal Rev is High, i.e., when the vehicle 100 is in reverse gear, causes the controller 10 and the main microcomputer 20 to execute the processing described below. More specifically, the power supply microcomputer 30 outputs a camera switching signal to the main microcomputer 20 while the reverse signal Rev is High. During the period when the main microcomputer 20 is receiving the camera switching signal, the main microcomputer 20 receives a video signal from the controller 10 and causes it to execute processing to output the video signal to the controller 10, and also causes the controller 10 to execute processing to output the video signal received from the main microcomputer 20 to the display 40 for display.

[0020] The power supply microcomputer 30 receives the vehicle's reverse signal Rev, and when the reverse signal Rev is Low, i.e., when the vehicle 100 is in a gear other than reverse, causes the main microcomputer 20 to execute a process of generating video signals for a navigation screen, television screen, DVD screen, audio screen, etc. and outputting them to the controller 10.

[0021] Furthermore, as will be described later, when the power supply microcomputer 30 receives a signal from the controller 10 notifying that there is an abnormality in the main microcomputer 20, it resets the main microcomputer 20. Note that instead of the power supply microcomputer 30 resetting the main microcomputer 20, the controller 10 that detects that there is an abnormality in the main microcomputer 20 may directly reset the main microcomputer 20.

[0022] The display 40 is, for example, a TFT (Thin Film Transistor Liquid Crystal) display. The display 40 displays an image corresponding to a video signal input from the controller 10. Note that the video processing device 1 may also display an image on an external display (for example, an external display).

[0023] As described above, the in-vehicle camera system S can display on the display 40 an image that has been image-processed from the image from the camera C when the vehicle's reverse signal Rev is High, i.e., when the vehicle 100 is in reverse gear, and can display on the display 40 an image from the navigation screen, television screen, DVD screen, audio screen, etc. when the vehicle's reverse signal Rev is Low, i.e., when the vehicle 100 is not in reverse gear.

[0024] Next, the signal path of the video signal will be explained using Figure 3. Hereinafter, the video displayed on the display 40 when the main microcomputer 20 is operating normally will be referred to as the "normal camera video," and the video displayed on the display 40 when the main microcomputer 20 is malfunctioning will be referred to as the "bypass camera video." The video signal input from the camera C to the controller 10 corresponds to the first video signal, and the video signal input from the main microcomputer 20 to the controller 10 corresponds to the second video signal.

[0025] 3 is a schematic diagram showing the signal path of a normal camera image. Note that the configuration of the image processing device 1 will be shown in a simplified form below.

[0026] 3, a video signal output from a camera C is input to a main microcomputer 20 via a controller 10. The main microcomputer 20 uses a memory 21 as a frame memory and performs a drawing and frame rate conversion process on the video signal (step S1).

[0027] For example, the main microcomputer 20 superimposes on the video signal an auxiliary image that assists in driving the vehicle 100. For example, the main microcomputer 20 draws, as an auxiliary image, a predicted driving trajectory of the vehicle 100 according to the current steering angle, a detection result of a parking space, a detection result of an obstacle, and the like.

[0028] Furthermore, the main microcomputer 20 converts the frame rate of the video signal input from the camera C into a frame rate for the display 40. For example, the main microcomputer 20 converts the 30 fps video signal input from the camera C into 60 fps for the display 40.

[0029] In this way, the main microcomputer 20 can display video signals of various frame rates on the display 40 by converting the frame rate of the video signal input from the camera C into a frame rate for the display 40. In other words, the versatility of the camera C mounted in the vehicle-mounted camera system S can be expanded.

[0030] Furthermore, if the video signal input from camera C is an interlaced signal, main microcomputer 20 may also perform so-called I / P conversion processing to convert it to a non-interlaced signal (progressive signal). This allows in-vehicle camera system S to be equipped with camera C whose video signal is an interlaced signal, even if display 40 only supports non-interlaced signals.

[0031] Then, the main microcomputer 20 outputs the video signal after various image processing to the controller 10. The controller 10 executes a fixation detection process on the video signal input from the main microcomputer 20 (step S2). Here, the video signal being fixed refers to a state in which the video signal input from the main microcomputer 20 to the controller 10 does not change.

[0032] This is because the memory 21 of the main microcomputer 20 is a frame memory, and if an abnormality occurs in the main microcomputer 20, the video signal stored in the memory 21 before the abnormality occurs continues to be output.

[0033] The controller 10 can detect an abnormality in the main microcomputer 20 by detecting a fixation of the video signal input from the main microcomputer 20. In particular, since the video processing device 1 displays the situation around the vehicle 100 on the display 40, if a fixation occurs in the video signal, a past video will be displayed on the display 40. In response to this, the controller 10 can contribute to improving the safety of the vehicle 100 by detecting a fixation of the video signal. Note that, when determining an abnormality in the main microcomputer 20, the controller 10 may determine that an abnormality has occurred if the video signal input from the main microcomputer 20 is interrupted.

[0034] Thereafter, when the controller 10 determines that the video signal is not frozen as a result of the freeze detection for the video signal, it performs a drawing process (buffering process) for displaying the video signal on the display 40 using the memory 11 as a line memory (step S3). Then, the controller 10 causes the video signal after the drawing process to be displayed on the display 40.

[0035] Next, the signal path of the video signal when an abnormality occurs in the main microcomputer 20 will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing the signal path of the bypass camera video. As shown in Fig. 4, when an abnormality occurs in the main microcomputer 20, that is, when the video signal is frozen, the controller 10 performs frame rate conversion processing on the video signal input from the camera C (step S11).

[0036] More specifically, first, when the controller 10 detects an abnormality in the main microcomputer 20, it switches the memory 11 from a line memory to a frame memory. The controller 10 switches the memory 11 from a line memory to a frame memory by increasing the capacity of the memory 11 through register settings.

[0037] The controller 10 then converts the video signal input from the camera C into a frame rate (for example, 60 fps) for the display 40 using a frame memory, and causes the display 40 to display the video signal after the frame rate conversion.

[0038] As a result, even if an abnormality occurs in the main microcomputer 20, the video processing device 1 can convert the frame rate using the controller 10, and can therefore display an appropriate video on the display.

[0039] Furthermore, the controller 10 can effectively utilize the memory 11 by utilizing the memory 11 as both a line memory and a frame memory. Note that, when there is little difference between the frame rate of the camera C and the frame rate for the display 40, the controller 10 may convert the frame rate while the memory 11 is used as a line memory. The controller 10 may also be equipped with a memory for a line memory and a memory for a frame memory. The controller 10 may also further perform fixation detection on the video signal after the frame rate has been converted by the memory 11, for example.

[0040] When switching the memory 11, the controller 10 temporarily stops outputting the video signal to the display 40, and resumes outputting the video signal after the frame rate conversion is complete. This is because if the output of the video signal is continued when switching the memory 11 from a line memory to a frame memory, the image displayed on the display 40 will be distorted.

[0041] Then, during the period when the output of the video signal from the controller 10 is stopped (the period when the video signal is muted), a black image is displayed on the display 40. In other words, the controller 10 can suppress image disturbance by temporarily stopping the output of the video signal. At this time, the controller 10 may notify the user of the change in video by sound or the like.

[0042] Then, the controller 10 causes the display 40 to display the video signal converted to the frame rate for the display 40. At this time, the controller 10 causes the display 40 to display an image that has not been subjected to image processing by the main microcomputer 20.

[0043] That is, when an abnormality occurs in the main microcomputer 20, the controller 10 causes the bypass camera image without the auxiliary image to be displayed on the display 40. This allows the controller 10 to continue displaying the image on the display 40 even when an abnormality occurs in the main microcomputer 20.

[0044] Furthermore, if the video signal input from the camera C is an interlaced signal, the controller 10 may also perform so-called I / P conversion processing, which converts the signal into a non-interlaced signal (progressive signal) using the memory 11, which is a frame memory. This allows the in-vehicle camera system S to be equipped with a camera C that outputs an interlaced video signal, even if the display 40 only supports non-interlaced signals.

[0045] In this way, if an abnormality occurs in the main microcomputer 20, the controller 10 converts the video signal input from the camera to a frame rate for the display 40 without going through the main microcomputer 20 and outputs it to the display 40.

[0046] As a result, in the image processing device 1 of the embodiment, if an abnormality occurs in the main microcontroller 20, the controller 10 performs frame conversion, so that even if an abnormality occurs in the main microcontroller 20, an appropriate image can be displayed on the display.

[0047] In addition, while the bypass camera image is being displayed on the display 40, the controller 10 outputs a video signal to the main microcomputer 20 and continues to detect whether the video signal output from the main microcomputer 20 is fixed.

[0048] When the fixation of the video signal output from the main microcomputer 20 is recovered, the controller 10 returns the video displayed on the display 40 from the bypass camera video to the normal camera video.

[0049] At this time, the controller 10 temporarily stops the video signal output to the display 40, restores the memory 11 from the frame memory to the line memory, and then resumes output of the video signal. In other words, even when switching from the bypass camera image to the normal camera image, a black screen is temporarily displayed on the display 40. This makes it possible to prevent image disturbances that may occur when switching the memory 11.

[0050] Next, a processing procedure executed by the video processing device 1 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing procedure executed by the video processing device 1. The processing described below starts when the reverse signal becomes High, and is repeatedly executed by the controller 10 while the reverse signal is High.

[0051] As shown in FIG. 5, when the reverse signal becomes High and processing begins, the controller 10 starts outputting the camera image to the display 40 instead of the image that had been output to the display 40 up until then (e.g., an image of a navigation screen, a television screen, a DVD screen, an audio screen, etc.) (step S101).

[0052] Next, the controller 10 switches the memory 11 to the line memory (step S102), and then switches to the normal camera image (step S103).

[0053] Next, the controller 10 determines whether the reverse signal Rev is High or not (step S104). If the controller 10 determines that the reverse signal Rev is High (step S104: Yes), the controller 10 determines whether the SoC (corresponding to the main microcomputer 20) is abnormal or not (step S105). That is, if the controller 10 determines that the video signal input from the main microcomputer 20 is in a fixed state, it determines that the SoC is abnormal.

[0054] When the controller 10 determines that the SoC is abnormal (step S105; Yes), it displays a black image on the display 40 (step S106), and then switches the memory 11 from a line memory to a frame memory (step S107).

[0055] Thereafter, the controller 10 switches the image displayed on the display 40 to the bypass camera image (step S108). Next, the controller 10 determines whether the reverse signal Rev is High or not (step S109), and if it is determined that the reverse signal Rev is High (step S109; Yes), it further determines whether an SoC abnormality exists (step S110).

[0056] If the controller 10 does not determine that the SoC is abnormal (step S110; No), it displays a black screen on the display 40 (step S111) and then proceeds to the processing of step S102.

[0057] Furthermore, if the controller 10 determines in the process of step S110 that an SoC abnormality has occurred (step S110; Yes), that is, if it determines that the SoC abnormality continues, the controller 10 returns to the process of step S109.

[0058] Furthermore, if controller 10 determines in steps S104 and S109 that reverse signal Rev is Low (steps S104, S109; No), it ends the output of the camera image to display 40, starts outputting to display 40 the image that was being output to display 40 before the camera image (for example, an image of a navigation screen, television screen, DVD screen, audio screen, etc.), and ends the process. Furthermore, if controller 10 determines in step S105 that there is no SoC abnormality (step S105; No), it returns to the process of step S104.

[0059] As described above, the image processing device 1 according to the embodiment includes a main microcomputer 20 that superimposes an image that assists driving on a first image signal input from a camera C, converts the frame rate of the image signal on which the image is drawn to a frame rate for the display, and outputs the second image signal; and a controller 10 that outputs the second image signal output from the main microcomputer 20 to a display 40. When the second image signal is not output from the main microcomputer 20 or the second image signal output from the main microcomputer 20 is not updated, the controller 10 converts the frame rate of the first image signal to a frame rate for the display and outputs the frame rate to the display 40.

[0060] Therefore, in the image processing device 1 according to the embodiment, even if an abnormality occurs in the main microcomputer 20, the controller 10 can convert the frame rate, so that an appropriate image can be displayed on the display 40.

[0061] In the above embodiment, the bypass camera image is displayed on the display 40 when an abnormality occurs in the main microcomputer 20, but the present invention is not limited to this. For example, when an abnormality occurs in the main microcomputer 20, the controller 10 may stop outputting the image signal to the display 40 and display a black screen on the display 40.

[0062] In addition, the video processing device 1 may be configured to be able to switch between a function of displaying bypass camera video and a function of stopping the video signal output to the display 40 when an abnormality occurs in the main microcontroller 20, by user settings, etc.

[0063] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0064] 1. Video processing device 10 Controller 11. Memory 20 Main microcomputer 21 Memory 30 Power supply microcomputer 40 Display 100 vehicles C Camera S In-vehicle camera system

Claims

1. a main microcomputer that draws a driving assist image by superimposing it on a first video signal input from a camera, converts a frame rate of the video signal on which the image is drawn into a frame rate for a display, and outputs the converted frame rate as a second video signal; a controller that outputs the second video signal output from the main microcomputer to a display; Equipped with The controller: When the second video signal is not output from the main microcomputer or the second video signal output from the main microcomputer is not updated, a frame rate of the first video signal is converted into a frame rate for the display and output to the display. Image processing device.

2. The controller: Equipped with memory, The controller: When the main microcomputer is normal, the memory is used as a line buffer, and the second video signal output from the main microcomputer is output to the display via the line buffer, and when the main microcomputer is abnormal, the memory is used as a frame buffer, and the frame rate of the first video signal input from the camera is converted to a frame rate for the display via the frame buffer and output to the display. The video processing device according to claim 1 .

3. The main microcomputer is When the first image signal input from the camera is an interlaced signal, the interlaced signal is converted into a non-interlaced signal and output as the second image signal; The controller: When an abnormality occurs in the main microcomputer, the first video signal of the interlaced signal input from the camera is converted into the non-interlaced signal and output to the display. The video processing device according to claim 1 .

4. The controller: Equipped with memory, The controller: When the main microcomputer is normal, the memory is used as a line buffer, and the second video signal, which is the non-interlaced signal output from the main microcomputer, is output to the display via the line buffer, and when the main microcomputer is abnormal, the memory is used as a frame buffer, and the first video signal, which is the interlaced signal input from the camera, is converted to the non-interlaced signal using the frame buffer and output to the display. The image processing device according to claim 3 .

5. The controller: When switching between outputting the second video signal output from the main microcomputer to the display and outputting the first video signal output from the camera to the display, the video signal being output to the display is temporarily stopped. The video processing device according to claim 1 .

6. The camera includes: It is installed to capture the surroundings of the vehicle, The controller: When an abnormality occurs in the main microcomputer, the first video signal on which an image for assisting driving is not superimposed is output to the display. The video processing device according to claim 1 .

7. A camera that captures the surroundings of the vehicle; a video processing device that converts a frame rate of a video signal input from the camera into a frame rate for a display; an in-vehicle display that displays the image input from the image processing device; An in-vehicle camera system comprising: The image processing device includes: a main microcomputer that draws a driving assist image by superimposing it on a first video signal input from a camera, converts a frame rate of the drawn video signal into a frame rate for a display, and outputs the converted frame rate as a second video signal; a controller that outputs the second video signal output from the main microcomputer to a display; Equipped with The controller: When the second video signal is not output from the main microcomputer or the second video signal output from the main microcomputer is not updated, a frame rate of the first video signal is converted into a frame rate for the display and output to the display. In-car camera system.

8. A video processing method executed by a main microcomputer and a controller, comprising: The main microcomputer, a first video signal input from a camera is superimposed with an image for assisting driving, and a frame rate of the drawn video signal is converted to a frame rate for display and output as a second video signal; Equipped with The controller: outputting the second video signal output from the main microcomputer to a display; When the second video signal is not output from the main microcomputer or the second video signal output from the main microcomputer is not updated, a frame rate of the first video signal is converted into a frame rate for the display and output to the display. A video processing method comprising:

Images