IMAGE PROCESSING SYSTEM, IMAGE PROCESSING APPARATUS, AND IMAGE PROCESSING METHOD

Abstract

Provided are an image processing system, an image processing device, and an image processing method that make it possible to reduce communication errors caused by external noise over a transmission line for image data.　According to the present invention, an image processing system comprises an onboard camera that acquires external information, an image processing device that is provided separately from the onboard camera and processes image data acquired by the onboard camera, and a transmission line that connects the onboard camera and the image processing device and transmits the image data. The image processing device has: a noise reduction filter that reduces noise over the transmission line; and a processing processor. The processing processor has: a noise source detection processing unit that determines whether objects that have been pre-registered as noise sources for the transmission line are included in the image data; and a noise reduction parameter determination unit that, when a pre-registered noise source is included in the image data, modifies parameters for the noise reduction filter in accordance with the frequency or noise level of the noise source.

Description

[Technical field]

[0001] The present invention relates to an image processing system, an image processing device, and an image processing method for correcting image data from a camera transmitted via a transmission line. [Background technology]

[0002] In recent years, the spread of in-vehicle camera devices has led to an increasing demand for various recognition functions aimed at safe driving and autonomous driving. Among these, stereo camera devices simultaneously measure visual information from images and distance information to objects, allowing for detailed understanding of various objects around the vehicle (people, cars, three-dimensional objects, road surfaces, road signs, signboards, etc.), and are said to contribute to improving safety during driving assistance.

[0003] However, as the resolution of the CMOS image sensors used in cameras increases, the amount of heat generated by the microcomputers that process the images also increases, making it difficult to house the CMOS image sensor and the image processing microcomputer in the same housing. For this reason, a possible configuration would be to separate the camera from the image processing board that houses the image processing microcomputer, and transmit image data from the camera via a transmission line. In this case, noise on the transmission line becomes a problem.

[0004] Background art of this technical field includes, for example, Patent Document 1. Patent Document 1 describes a method for reducing large unwanted radiation noise that occurs when image data signals, clock signals, and image synchronization signals transmitted from two imaging units are transmitted to an image processing unit in an in-vehicle camera device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-186574 A Summary of the Invention [Problem to be solved by the invention]

[0006] In Patent Document 1, the amount of noise on the signal line is reduced by shifting the time of signals transmitted from the two imaging units, but no consideration is given to reducing communication errors caused by external noise.

[0007] In view of the above problems, an object of the present invention is to provide an image processing system, an image processing device, and an image processing method that are capable of reducing communication errors caused by external noise on a transmission line for image data. [Means for solving the problem]

[0008] One example of the present invention is an image processing system comprising an on-board camera that acquires external world information, an image processing device that is installed separately from the on-board camera and processes image data acquired by the on-board camera, and a transmission line that connects the on-board camera and the image processing device and transmits the image data, the image processing device having a noise reduction filter that reduces noise in the transmission line and a processing processor, the processing processor having a noise source detection processing unit that determines whether or not an object that has been registered in advance as a noise source for the transmission line is included in the image data, and a noise reduction parameter determination unit that changes parameters of the noise reduction filter in accordance with the frequency or noise level of the registered noise source when the image data includes a noise source. Effect of the Invention

[0009] According to the present invention, it is possible to provide an image processing system, an image processing device, and an image processing method that are capable of reducing communication errors caused by external noise on a transmission line for image data. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a configuration block diagram of a conventional image processing system that is the premise of a first embodiment. [Diagram 2] 1 is a functional configuration block diagram of an image processing system according to a first embodiment. [Diagram 3] FIG. 11 is a functional configuration block diagram of an image processing system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the present invention will be described with reference to the drawings. EXAMPLES

[0012] In this embodiment, an image processing system using an on-board camera mounted on an automobile is taken as an example for explanation. The explanation is based on the premise that the image data acquired from the CMOS image sensor constituting the on-board camera is serially transmitted via a transmission line using a communication IC such as Serdes (Serializer / Deserializer), which is a circuit that converts serial data into parallel data and vice versa.

[0013] Fig. 1 is a block diagram showing the hardware configuration of a conventional image processing system that is the premise of this embodiment. In Fig. 1, an image processing system 100 is composed of a camera 101, an image processing device 102, and a transmission line 103 connecting them.

[0014] The camera 101 is composed of a CMOS image sensor 1011, a communication IC (Tx) 1012, and a connector unit (Tx) 1013.

[0015] The image processing device 102 is composed of a connector unit (Rx) 1021, a communication IC (Rx) 1022, and a microcomputer 1024. The microcomputer 1024 is composed of a processing processor such as a general CPU (Central Processing Unit) and a storage device, and the processing processor reads out programs and information that realize each function from the storage device, and executes, for example, recognition processing of an object captured by a camera by software processing, and performs vehicle control according to the recognition result.

[0016] In a communication method using Serdes, if left as is, the signal deteriorates due to losses on the communication path, including the board, connector, and transmission line, making it difficult to perform normal communication. For this reason, in existing technology, the communication IC (Rx) 1022, which is the communication IC on the receiving side, is equipped with equalizer functions such as a Continuous Time Linear Equalizer (hereinafter, CTLE) and a Decision Feedback Equalizer (hereinafter, DFE) to ensure communication quality, and this function is used to correct the waveform when the signal waveform deteriorates. In FIG. 1, the DFE in the communication IC (Rx) 1022 is shown as a noise reduction filter 104.

[0017] Normal correction can be handled by CTLE, but since CTLE may also amplify noise components, DFE tends to be used in addition to CTLE in high-speed communications. DFE performs correction using feedback control, but if there are radio towers or wireless communication areas around the vehicle, or if wireless communication is performed nearby, there may be cases where the vehicle is exposed to large external noise for a short period of time. If this external noise exceeds the range that the equalizer can correct, the equalizer's correction control may not be able to keep up, and a communication error may occur.

[0018] Therefore, in this embodiment, the noise source is identified from external information, and the parameters of the noise reduction filter are changed according to the noise source, thereby enabling highly reliable communication processing. Details of this embodiment will be described below.

[0019] Fig. 2 is a functional block diagram of the image processing system in this embodiment. In Fig. 2, the same functions as those in Fig. 1 are denoted by the same reference numerals, and their explanations will be omitted. Fig. 2 differs from Fig. 1 in that it uses a stereo camera, has two systems of cameras, transmission lines, and noise reduction filters, and has a signal correction processing unit 1025.

[0020] In Fig. 2, the image processing system 100 is an image processing system that performs three-dimensional object detection by processing parallax images using a stereo camera as a means for detecting nearby vehicles. Note that three-dimensional object detection can also be performed using, for example, only a monocular camera or a combination of a monocular camera and radar. Also, three or more cameras may be used.

[0021] Stereo images are captured by cameras 101 and 109. The image processing device 102 receives image data via transmission lines 103 and 110. The captured image data is sent to a recognition processing unit 1026 through noise reduction filters 104 and 111, which calculates distance information to an object by processing parallax images, and recognizes the object.

[0022] On the other hand, the noise reduction filters 104 and 111 that correct the captured image data have their correction amounts controlled by the signal correction processing unit 1025. That is, the captured image data from the noise reduction filters 104 and 111 is sent to the noise source detection processing unit 105 as external world information. The noise source detection processing unit 105 refers to the external world information and an image of an object registered in the noise source information storage unit 108 stored in advance in the memory area 107, and determines whether or not an object that is a noise source is included in the image data of the external world information. If an object is included, the noise source detection processing unit 105 acquires detailed information of the noise source, such as a frequency and a noise level corresponding to the object, registered in the noise source information storage unit 108.

[0023] The external environment information acquired by the camera includes, for example, images of steel towers, lightning, mobile phone base stations, pantographs, taxis that perform wireless communication, relay vehicles, military vehicles, and signs for high voltage power transmission, and the noise source detection processing unit 105 acquires information regarding these as noise sources.

[0024] The acquired detailed information on the noise source and the distance information to the noise source calculated by the recognition processing unit 1026 are sent to the noise reduction parameter determination unit 106. The noise reduction parameter determination unit 106 determines the amount of correction for the noise reduction filter from the sent information and corrects the parameters of the noise reduction filter. That is, the parameters of the noise reduction filter are corrected by feedforward control.

[0025] In this way, the signal correction processing unit 1025 first checks the captured image, and starts filter correction if a noise source is found in the image. If a noise source is not found in the captured image, the filter correction process is not performed. Then, the frequency and noise level of the noise source are identified from the image of the external world information, and a signal is sent to change the correction amount and correction timing of the noise reduction filter based on one or more pieces of information on the frequency, noise level, and distance to the noise source, thereby suppressing communication errors caused by external noise.

[0026] The noise reduction filter is corrected, for example, using an equalizer in the Serdes. Typically, a Serdes has two types of equalizers for filter correction: CTLE and DFE. The amount of noise correction can be set by changing the register settings for the DFE noise correction parameters. For example, the register area for setting the amount of noise correction can be rewritten using an interface provided by the Serdes, such as I2C (Inter-Integrated Circuit), to correct the preset amount of correction within a certain range in real time.

[0027] In this way, according to the present embodiment, the noise source is identified from the captured image, which is external world information, and the parameters of the noise reduction filter are changed according to the noise source, thereby suppressing the occurrence of communication errors and enabling highly reliable communication processing. This makes it possible to provide an image processing system, an image processing device, and an image processing method that are capable of reducing communication errors caused by external noise on the transmission line of image data. EXAMPLES

[0028] Fig. 3 is a functional block diagram of the image processing system in this embodiment. In Fig. 3, the same functions as those in Fig. 2 are denoted by the same reference numerals, and their explanations will be omitted. Fig. 3 differs from Fig. 2 in that vehicle speed information 112 is provided as an input to the noise reduction parameter determination unit 106.

[0029] A high vehicle speed results in a large travel distance per unit time, so as shown in Fig. 3, when the vehicle speed information indicates that the vehicle speed is high, the noise reduction parameter determination unit 106 performs processing such as shortening the interval of the noise reduction filter correction timing.

[0030] Thus, according to this embodiment, in addition to the effect of the first embodiment, the parameters of the noise reduction filter can be changed at a more optimal noise reduction filter correction timing, thereby enabling more reliable communication processing.

[0031] Although the embodiments have been described above, the present invention is not limited to the above-described embodiments and includes various modified examples. For example, the above-described embodiments have been described regarding noise reduction in the transmission of image data, but the present invention is not limited to image data and can also be applied to audio data. Furthermore, the above-described embodiments have been described in detail to easily explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. [Explanation of symbols]

[0032] 100: image processing system, 101, 109: camera, 102: image processing device, 103, 110: transmission line, 104, 111: noise reduction filter, 105: noise source detection processing unit, 106: noise reduction parameter determination unit, 107: memory area, 108: noise source information storage unit, 112: vehicle speed information, 1011: CMOS image sensor, 1012: communication IC (Tx), 1013: connector unit (Tx), 1021: connector unit (Rx), 1022: communication IC (Rx), 1024: microcomputer, 1025: signal correction processing unit, 1026: recognition processing unit

Claims

1. An image processing system including an on-board camera that acquires external environment information, an image processing device that is installed separately from the on-board camera and processes image data acquired by the on-board camera, and a transmission line that connects the on-board camera and the image processing device and transmits the image data, the image processing device includes a noise reduction filter that reduces noise in the transmission line and a processing processor; The processor includes: a noise source detection processing unit that determines whether or not an object that is registered in advance as a noise source for the transmission line is included in the image data; an image processing system comprising: a noise reduction parameter determination unit that, when the image data contains the noise source, changes parameters of the noise reduction filter in accordance with a frequency or a noise level of the noise source that has been registered in advance.

2. 2. The image processing system according to claim 1, The vehicle-mounted camera is a plurality of vehicles, and the vehicle-mounted camera has a plurality of transmission lines for transmitting image data from the plurality of vehicles, a plurality of the noise reduction filters for reducing noise in the image data from the respective plurality of transmission lines; The processor includes: Calculating distance information to an object by processing parallax images from the multiple vehicle-mounted cameras; The image processing system according to claim 1, wherein the noise reduction parameter determination unit corrects parameters of each of the plurality of noise reduction filters in accordance with information on a distance to the object.

3. 2. The image processing system according to claim 1, The image processing system according to claim 1, wherein the noise reduction parameter determination unit determines a correction timing for the parameters of the noise reduction filter by using vehicle speed information.

4. 2. The image processing system according to claim 1, The transmission of the image data is serial transmission using Serdes, 2. An image processing system according to claim 1, wherein the noise reduction filter is an equalizer in a communication IC.

5. An image processing device that inputs and processes image data from an on-board camera that acquires external information via a transmission line, A noise reduction filter for reducing noise in the transmission line and a processing processor are provided, The processor includes: a noise source detection processing unit that determines whether or not an object that is registered in advance as a noise source for the transmission line is included in the image data; an image processing device comprising: a noise reduction parameter determination unit that, when the image data contains the noise source, changes parameters of the noise reduction filter in accordance with a frequency or a noise level of the noise source that has been registered in advance.

6. 6. The image processing device according to claim 5, The vehicle-mounted cameras are multiple, and the vehicle-mounted cameras each have a multiple number of noise reduction filters that reduce noise in image data from a multiple number of transmission lines that transmit image data from the vehicle-mounted cameras, The processor includes: Calculating distance information to an object by processing parallax images from the multiple vehicle-mounted cameras; The image processing device according to claim 1, wherein the noise reduction parameter determination unit corrects parameters of each of the plurality of noise reduction filters in accordance with information on a distance to the object.

7. 6. The image processing device according to claim 5, The image processing device according to claim 1, wherein the noise reduction parameter determination unit determines a correction timing for the parameters of the noise reduction filter by using vehicle speed information.

8. An image processing method for inputting and processing image data from an on-board camera that acquires external information via a transmission line, comprising: determining whether or not an object that is registered in advance as a noise source for the transmission line is included in the image data; An image processing method characterized in that, when the image data contains the noise source, parameters of a noise reduction filter that reduces noise in the transmission line are changed in accordance with a frequency or noise level of the noise source that has been registered in advance.

9. 9. The image processing method according to claim 8, The vehicle-mounted camera and the transmission line are multiple, Calculating distance information to an object by processing parallax images from the multiple vehicle-mounted cameras; an image processing method comprising: correcting parameters of each of a plurality of noise reduction filters that reduce noise in each of the image data from a plurality of the transmission lines in accordance with distance information from the object when the image data includes the noise source.

10. 9. The image processing method according to claim 8, An image processing method comprising the steps of: determining a timing for correcting parameters of the noise reduction filter using vehicle speed information when the noise source is included in the image data.

Images