Vehicle image processing system, vehicle, and image transmission method

Abstract

The present invention relates to a vehicle image processing system, a vehicle, and an image transmission method. The vehicle image processing system includes: a plurality of first camera devices that acquire identification images for identification processing for identifying a target present in a peripheral area of the vehicle; an identification unit that performs the identification processing; and a communication unit that is communicably connected to the first camera devices and the identification unit, receives the identification images transmitted by the first camera devices, and transmits the identification images to the identification unit. The communication unit is configured to transmit the identification images to the identification unit in such a manner that the transmission timings of the identification images received from the plurality of first camera devices to the identification unit are different. The identification unit is configured to perform the identification processing using the identification images received from the communication unit.

Description

Technical Field

[0001] The present invention relates to a vehicle image processing system that sends recognition images obtained by a plurality of camera devices to a recognition unit, and the recognition unit uses the recognition images to recognize a target present in a peripheral area of the vehicle. Background Art

[0002] Conventionally, a vehicle image processing system that performs various controls based on images obtained by a camera device mounted on a vehicle has been known. A vehicle image processing system described in Japanese Unexamined Patent Application Publication No. 2016-042704 (hereinafter referred to as the "first conventional system") performs display control processing for generating a composite image obtained by synthesizing images obtained by a plurality of camera devices, and displays the composite image on a display. Hereinafter, the camera device of the first conventional system will be referred to as a "display camera device".

[0003] Moreover, a vehicle image processing system described in Japanese Unexamined Patent Application Publication No. 2019-026129 (hereinafter referred to as the "second conventional system") performs recognition processing for recognizing a target based on an image obtained by a camera device. Hereinafter, the camera device of the second conventional system will be referred to as a "recognition camera device".

[0004] A vehicle image processing system (hereinafter referred to as the "research system") is studied, and the vehicle image processing system performs the above-described recognition processing using not only an image obtained by a recognition camera device but also images obtained by a plurality of display camera devices.

[0005] The research system includes a plurality of display camera devices, a recognition camera device, a display control unit that performs the above-described display control processing, a recognition unit that performs the above-described authentication processing, and a communication unit. The communication unit is connected to the first camera device and the second camera device in a communicable manner, and is also connected to the display control unit and the recognition unit in a communicable manner.

[0006] In addition to sending "an image obtained by the recognition camera device" to the recognition unit, "images obtained by a plurality of display camera devices" are also sent to the recognition unit. Therefore, the maximum value of the usage band of the communication band (hereinafter referred to as the "recognition communication band") from the communication unit of the research system to the recognition unit is larger than the maximum value of the usage band of the recognition communication band of the second conventional system. Depending on the situation, the maximum value of the usage band may exceed the maximum communication band. In this case, the recognition unit may not be able to receive the image. Summary of the Invention

[0007] The present invention is made to address the above problems. That is, one object of the present invention is to provide a vehicle image processing system that transmits an image from a communication unit to an identification unit in such a manner that the maximum value of the used frequency band of an identification communication frequency band is reduced.

[0008] The vehicle image processing system of the present invention (hereinafter, also referred to as "the present invention system") includes: a plurality of first camera devices (34Fr and 34Rr, or 34L and 34R) that acquire identification images for identification processing for identifying objects existing in a peripheral area of the vehicle; an identification unit (24) that performs the identification processing; and a communication unit (22) that is communicably connected to the first camera devices and the identification unit, receives the identification images transmitted by the first camera devices, and transmits the identification images to the identification unit. The communication unit is configured to transmit the identification images to the identification unit in such a manner that the transmission timings of the identification images received from the plurality of first camera devices to the identification unit are different (steps 735, step 745, step 825, step 915, Figure 6 of (B) and Figure 6 of (C)), and the identification unit is configured to perform the identification processing using the identification images received from the communication unit (steps 1000 to step 1095, steps 1100 to step 1195).

[0009] According to the present invention system, the transmission timings of the identification images acquired by the plurality of first camera devices from the communication unit to the identification unit do not overlap. Thus, compared with the case where the transmission timings of these identification images are the same, the maximum value of the used frequency band of the identification communication frequency band can be reduced. As a result, the possibility that the above maximum value exceeds the maximum communication frequency band can be reduced, and thus the possibility that the identification unit reliably receives the identification images can be increased.

[0010] In one aspect of the present invention, each of the plurality of first camera devices is configured to: acquire the identification images in such a manner that the acquisition timings of the identification images are different between the first camera devices (steps 735, step 745, step 815, Figure 6 of (B) and Figure 6 of (C)); and transmit the identification images to the communication unit (step 825), and the communication unit is configured to transmit the received identification images to the identification unit when the identification images are received from the first camera devices (step 915).

[0011] According to this solution, each of the multiple first camera devices acquires an identification image at different timings and sends the identification image to the communication unit. Thereby, the transmission timings of the identification images from the communication unit to the identification unit can be made different.

[0012] In the above solution, the identification unit is configured to send an identification synchronization signal to the multiple first camera devices at different timings (step 735, step 745), and each of the multiple first camera devices is configured to acquire the identification image when receiving the identification synchronization signal (step 815).

[0013] When each of the multiple first camera devices manages the "time for determining whether it is the acquisition timing of the identification image", an error in time generated between the first camera devices may occur. The acquisition timings of the identification images of the first camera devices may become the same due to this error. According to this solution, the first camera device acquires the identification image when receiving the identification synchronization signal sent from the identification unit. Therefore, the acquisition timings of the identification images of the first camera devices can be reliably made different.

[0014] In one solution of the present invention, the vehicle image processing system further includes a display control unit (26). The display control unit (26) performs a prescribed display control process on the display image and displays the display image on which the display control process has been performed on a display (90) provided in the vehicle. Each of the multiple first camera devices is configured to be able to acquire the display image in addition to the identification image (step 830). And when each of the multiple first camera devices acquires the identification image, it assigns an identification identifier (step 820) and sends the identification image to the communication unit (step 825). When each of the multiple first camera devices acquires the display image, it assigns a display identifier (step 835) and sends the display image to the communication unit (step 840). The communication unit is also connected to the display control unit in a communicable manner. And the communication unit is configured to: when receiving either the identification image or the display image from the first camera device (step 905 is "yes"), if the image is assigned the identification identifier (step 910 is "yes"), it sends the image to the identification unit (step 915), and if the image is assigned the display identifier (step 910 is "no"), it sends the image to the display control unit (step 920).

[0015] According to this solution, the communication unit sends the identification image to the identification unit and sends the display image to the display control unit. Thus, the identification unit does not receive the display image that is not required for the identification process. Therefore, when the first camera device is configured to acquire the identification image and the display image, the maximum value of the used bandwidth of the identification communication band can be reduced.

[0016] In the above solution, the display control unit is configured to perform a process of synthesizing the display images acquired by the plurality of first camera devices (step 1210) as the display control process.

[0017] Thus, the image obtained by synthesizing the display images acquired by the plurality of first camera devices is displayed on the display, so that the driver can visually recognize the display images acquired by the plurality of first camera devices simultaneously.

[0018] In the above solution, the vehicle image processing system further includes at least one second camera device (34L, 34R) that acquires the display image. The first camera device and the second camera device are configured to share the acquisition of images of the peripheral area of the vehicle. The display control unit is configured to perform the following process (step 1210) as the display control process: generate an aerial view image of the peripheral area overlooking the vehicle based on the image obtained by synthesizing the display images acquired by the first camera device and the second camera device.

[0019] Thus, the aerial view image of the peripheral area of the vehicle overlooking the vehicle is displayed on the display, so that it is easier for the driver to visually recognize the peripheral area of the vehicle.

[0020] In one aspect of the present invention, the vehicle image processing system further includes a third camera device (32). The third camera device (32) acquires an identification image having a data capacity larger than that of the identification image acquired by the first camera device. The communication unit is connected to the third camera device in a communicable manner in addition to the first camera device and the identification unit. And the communication unit is configured to receive the identification image sent by the third camera device and send the identification image to the identification unit.

[0021] Thus, the identification unit also uses the identification image with a large data capacity acquired by the third camera device to perform the identification process, so that the target can be identified more accurately.

[0022] In one aspect of the present invention, the recognition unit is configured to: obtain a collision index value (TTC) indicating the likelihood of a collision between a target identified in the recognition process and the vehicle (step 1015); and perform at least one of alarm control (step 1025), braking control (step 1045), and avoidance control (step 1040) when the relationship between the collision index value and a specified threshold satisfies a specified condition (step 1020 is "Yes" and step 1030 is "Yes"), where the alarm control is a control for notifying the driver of the vehicle that a collision with the target may occur, the braking control is a control for decelerating the vehicle by applying a braking force to the wheels of the vehicle, and the avoidance control is a control for changing the traveling direction of the vehicle to avoid a collision between the vehicle and the target.

[0023] Accordingly, when the relationship between the collision index value and the threshold satisfies the specified condition, at least one of alarm control, braking control, and avoidance control is performed, so that the likelihood of a collision between the vehicle and the target can be reduced.

[0024] The image transmission method of the present invention is a method of receiving identification images acquired by a plurality of first camera devices (34Fr and 34Rr, or 34L and 34R) from the plurality of first camera devices and transmitting the identification images to an identification unit (24) that performs an identification process for identifying a target existing in a peripheral area of the vehicle using the identification images. Moreover, the image transmission method of the present invention includes: a step of receiving the identification images from each of the plurality of first camera devices (step 905); and a step of transmitting the identification images to the identification unit in such a manner that the transmission timings of the identification images received from the plurality of first camera devices are different (step 735, step 745, step 825, step 915, Figure 6 of (B) and Figure 6 of (C)).

[0025] Accordingly, compared with the case where the transmission timings of the identification images are the same, the maximum value of the used frequency band of the identification communication frequency band can be reduced. The possibility that the above maximum value exceeds the maximum communication frequency band can be reduced, thereby increasing the possibility that the identification unit can reliably receive the identification images.

[0026] It should be noted that in the above description, for the purpose of facilitating the understanding of the invention, the components of the invention corresponding to the following embodiments are added with the names and / or reference numerals used in these embodiments in parentheses. However, each component of the invention is not limited to the embodiments defined by the said names and / or reference numerals. Other objects, other features and attendant advantages of the present invention will be readily understood from the description of the embodiments of the present invention with reference to the following drawings. Description of the Drawings

[0027] Hereinafter, with reference to the drawings, the features, advantages and technical and industrial significance of the exemplary embodiments of the present invention will be described, wherein the same reference numerals denote the same elements, and:

[0028] Figure 1 is a schematic configuration diagram of a vehicle image processing system according to an embodiment of the present invention.

[0029] Figure 2 is Figure 1 an explanatory diagram of the shooting ranges of the front camera device, the front PVM camera device and the rear PVM camera device shown.

[0030] Figure 3 is Figure 1 an explanatory diagram of the shooting ranges of the left PVM camera device and the right PVM camera device shown.

[0031] Figure 4 is an explanatory diagram of a front collision target and an avoidance area.

[0032] Figure 5 is an explanatory diagram of an operation example of a vehicle image processing system.

[0033] Figure 6 is an explanatory diagram of the transmission timing of the recognition image and the display image of each camera device.

[0034] Figure 7 is a flowchart showing a synchronization signal transmission routine executed by the CPU of the recognition SoC.

[0035] Figure 8 is a flowchart showing an image acquisition routine executed by the CPU of each camera device.

[0036] Figure 9 is a flowchart showing an image allocation routine executed by the display Des.

[0037] Figure 10 is a flowchart showing a front safety control routine executed by the CPU of the recognition SoC.

[0038] Figure 11It is a flowchart showing the rear safety control routine executed by the CPU of the recognition SoC.

[0039] Figure 12 It is a flowchart showing the display control routine executed by the CPU of the display control SoC. Detailed implementation mode

[0040] Hereinafter, a vehicle image processing system (hereinafter referred to as "this system") 10 according to an embodiment of the present invention will be described with reference to the accompanying drawings. Figure 1 The system 10 and a vehicle VA equipped (applied) with the system 10 are shown.

[0041] As Figure 1 shown, the system 10 includes a control ECU 20, an engine ECU 40, a brake ECU 50, and a steering ECU 60. These ECUs are connected to each other via a CAN (Controller Area Network) 65 so as to be able to exchange data (communicate). ECU is an abbreviation for Electronic Control Unit. The control ECU 20 is an electronic control circuit including a plurality of SoCs (System On a chip) 24 and 26 described later. The plurality of SoCs 24 and 26 include a CPU, a ROM, a RAM, and an interface (IF), etc. The engine ECU 40, the brake ECU 50, and the steering ECU 60 are electronic control circuits having a microcomputer as a main component. The microcomputer includes a CPU, a ROM, a RAM, and an interface, etc. The CPU realizes various functions by executing instructions (routines) stored in a memory (ROM). All or several of the ECUs 40, 50, and 60 may also be integrated into one ECU.

[0042] The control ECU 20 includes a communication unit 22, a recognition SoC 24, and a display control SoC 26. The communication unit 22 is connected to a front camera device 32, a front PVM (Panoramic View Monitor) camera device 34Fr, a rear PVM camera device 34Rr, a left PVM camera device 34L, and a right PVM camera device 34R so as to be able to communicate. The recognition SoC 24 recognizes a target based on a recognition image described later input to the recognition SoC 24, and executes front safety control and rear safety control based on the target. The display control SoC 26 performs display control processing, which is a process of synthesizing a display image described later input to the display control SoC 26 and generating a PVM image (bird's-eye view image) of the peripheral area of the vehicle VA viewed from above the vehicle VA based on the synthesized image.

[0043] Hereinafter, the SoC 24 for recognition may sometimes also be referred to as the "recognition unit", and the SoC 26 for display control may be referred to as the "display control unit". Further, the PVM camera devices 34Fr, 34Rr, 34L, and 34R are referred to as "PVM camera device 34" without distinguishing them from each other.

[0044] Hereinafter, each of the camera devices 32 and 34 will be described.

[0045] As Figure 2 shown, the front camera device 32 is disposed above the front window FW in the passenger compartment of the vehicle VA. As Figure 2 shown, the front PVM camera device 34Fr is disposed at the center in the vehicle width direction of the front end portion FR of the vehicle VA. The front camera device 32 and the front PVM camera device 34Fr acquire images of the front area of the vehicle VA. The field of view angle θa of the image acquired by the front camera device 32 is smaller than the field of view angle θb of the image acquired by the front PVM camera device 34Fr. Therefore, the image acquired by the front PVM camera device 34Fr includes regions UA1 and UA2 that are not included in the image acquired by the front camera device 32.

[0046] As Figure 2 shown, the rear PVM camera device 34Rr is disposed at the center in the vehicle width direction of the rear end portion RR of the vehicle VA. The rear PVM camera device 34Rr acquires an image of the rear area of the vehicle VA. The field of view angle θc of the image acquired by the rear PVM camera device 34Rr is the same as the field of view angle θb of the image acquired by the front PVM camera device 34Fr.

[0047] As Figure 3 shown, the left PVM camera device 34L is disposed on the left side mirror LSM of the vehicle VA. The left PVM camera device 34L acquires an image of the left area of the vehicle VA. The field of view angle θd of the image acquired by the left PVM camera device 34L is the same as the field of view angle θb of the image acquired by the front PVM camera device 34Fr.

[0048] As Figure 3 shown, the right PVM camera device 34R is disposed on the right side mirror RSM of the vehicle VA. The right PVM camera device 34R acquires an image of the right area of the vehicle VA. The field of view angle θe of the image acquired by the right PVM camera device 34R is the same as the field of view angle θb of the image acquired by the front PVM camera device 34Fr.

[0049] The front camera device 32 acquires an identification image that is an image for identification processing. The front PVM camera devices 34Fr and the rear PVM camera device 34Rr acquire an identification image and a display image. The left PVM camera device 34L and the right PVM camera device 34R acquire a display image. Hereinafter, the PVM camera devices 34Fr and 34Rr that acquire an identification image and a display image may sometimes be referred to as "first camera devices". The PVM camera devices 34L and 34R that acquire a display image may sometimes be referred to as "second camera devices". The camera device 32 that acquires an identification image may sometimes be referred to as "third camera device".

[0050] Here, the difference between the identification image and the display image will be described.

[0051] The recognition SoC 24 extracts edges from the image in the recognition processing and recognizes landmarks based on the edges. When there is blurring in the image, the recognition SoC 24 may not be able to extract edges from the image. Therefore, it is desirable that the identification image is an image with less blurring. On the other hand, the display image is an image for visual recognition by the driver of the vehicle VA, so the display image may also be an image with some blurring. In order to reduce blurring, the identification image is different from the display image in that the identification image is acquired by taking a picture at a shutter speed slower than the shutter speed for taking the display image.

[0052] The engine ECU 40 is connected to a plurality of wheel speed sensors 41, an accelerator pedal operation amount sensor 42, and an engine sensor 44, and receives the detection signals from these sensors.

[0053] Each wheel speed sensor 41 is provided on the corresponding wheel (left front wheel, right front wheel, left rear wheel, or right rear wheel) of the vehicle VA, and generates a pulse signal (wheel pulse signal) every time the corresponding wheel rotates by a predetermined angle. The engine ECU 40 measures the number of pulses per unit time of the wheel pulse signals sent from each wheel speed sensor 41, and calculates the rotational speed (wheel speed) of each wheel based on the measured number of pulses. The engine ECU 40 calculates the vehicle speed Vs indicating the speed of the vehicle VA based on the wheel speeds of the respective wheels. As an example, the control ECU 20 calculates the average value of the wheel speeds of the four wheels as the vehicle speed Vs.

[0054] The accelerator pedal operation amount sensor 42 detects the operation amount of an accelerator pedal (not shown) of the vehicle VA. The engine ECU 40 receives a detection signal indicating the operation amount of the accelerator pedal from the accelerator pedal operation amount sensor 42. The accelerator pedal is an acceleration operation member for the driver to accelerate the vehicle VA by increasing the driving force generated by the drive device (in this example, an internal combustion engine) 48 of the vehicle VA.

[0055] The engine sensor 44 is a sensor that detects the operating state quantity of the internal combustion engine 48. The engine sensor 44 is a throttle opening sensor, an internal combustion engine rotational speed sensor, an intake air quantity sensor, etc.

[0056] Moreover, the engine ECU 40 is connected to engine actuators 46 such as "throttle actuators and fuel injection valves". The engine ECU 40 controls the engine actuators 46 in such a way that the target throttle opening determined based on the operation amount of the accelerator pedal and the vehicle speed Vs is consistent with the actual throttle opening. Thereby, the torque generated by the internal combustion engine 48 is changed, and the driving force of the vehicle VA is adjusted.

[0057] The brake ECU 50 is connected to the wheel speed sensor 41 and the brake pedal operation amount sensor 52, and receives the detection signals from these sensors.

[0058] The brake pedal operation amount sensor 52 detects the operation amount of a brake pedal (not shown) of the vehicle VA. The brake ECU 50 receives the detection signal indicating the operation amount of the brake pedal from the brake pedal operation amount sensor 52. The brake pedal is a deceleration operation member for the driver to operate to apply a braking force to the wheels of the vehicle VA.

[0059] The brake ECU 50 is connected to the brake actuator 54. The brake actuator 54 is a hydraulic control actuator. The brake actuator 54 is disposed in a hydraulic circuit (not shown) between "a master cylinder (not shown) that pressurizes the working oil by the stepping force of the brake pedal" and "a friction braking device (not shown) including well-known wheel cylinders provided at each wheel". The brake actuator 54 adjusts the hydraulic pressure supplied to the wheel cylinder.

[0060] The brake ECU 50 obtains a "negative braking target acceleration" based on the vehicle speed Vs and the operation amount of the brake pedal. More specifically, the larger the operation amount of the brake pedal, the smaller the value of the braking target acceleration. The brake ECU 50 drives the brake actuator 43 based on the obtained braking target acceleration, thereby controlling the hydraulic pressure of the working oil supplied to the wheel cylinder. As a result, adjusted braking forces (friction braking forces) are generated at each wheel, so that the acceleration of the vehicle VA is made consistent with the braking target acceleration.

[0061] The steering ECU 60 is a control device of a well-known electric power steering system, and is connected to the steering angle sensor 62 and the steering motor 64. The steering motor 64 is assembled in a "steering mechanism (not shown) of the vehicle VA including a steering wheel (not shown), a steering shaft (not shown) connected to the steering wheel, and a steering gear mechanism (not shown)".

[0062] The steering angle sensor 62 detects the steering angle θ of the vehicle VA. The steering ECU 60 receives a detection signal representing the steering angle θ from the steering angle sensor 62.

[0063] The steering motor 64 generates torque according to the power controlled by the steering ECU 60 such as the direction and magnitude, and applies a steering assist torque through this torque, or steers the left and right steering wheels. That is, the steering angle can be controlled using the steering motor 64. It should be noted that the above power is supplied from a battery (not shown) mounted on the vehicle VA.

[0064] The control ECU 20 is connected to the speaker 70, the first display 80, and the second display 90 in a communicable manner.

[0065] The speaker 70 is disposed inside the passenger compartment of the vehicle VA. The speaker 70 emits an alarm sound according to an alarm command from the control ECU 20.

[0066] The first display 80 is disposed inside the passenger compartment of the vehicle VA and is, for example, a multi-information display. The first display 80 displays an "alarm screen for notifying the driver that the vehicle VA may collide with an object" according to an alarm command from the control ECU 20.

[0067] The second display 90 is disposed inside the passenger compartment of the vehicle VA and is, for example, a navigation display. The second display 90 displays a PVM image transmitted from the control ECU 20.

[0068] <Forward safety control>

[0069] The recognition SoC 24 receives the recognition images (hereinafter referred to as "forward recognition images") acquired by the forward camera device 32 and the forward PVM camera device 34Fr. The recognition SoC 24 performs a forward recognition process, which is a process of recognizing an object by determining the position of the object relative to the vehicle VA based on the forward recognition image. Moreover, the recognition SoC 24 determines the moving direction of the object relative to the vehicle VA and the relative speed Vr of the object relative to the vehicle VA based on the position of the object determined in the previous forward recognition process (previous position) and the position determined in the current forward recognition process (current position).

[0070] The recognition SoC 24 determines an object that may collide with the vehicle VA as a forward collision object FOB based on the traveling direction of the vehicle VA and the moving direction and relative speed Vr of the object (refer to Figure 4), and obtain the time to collision as the time taken until the forward object FOB collides with the vehicle VA. Hereinafter, the time to collision will be referred to as "TTC". TTC is an abbreviation for Time To Collision. More specifically, the recognition SoC24 obtains the TTC by dividing the distance between the object and the vehicle VA by the relative speed Vr described above.

[0071] It should be noted that the TTC is a collision index value representing the collision possibility as the possibility of collision between the object and the vehicle VA. The smaller the value of the TTC, the higher the collision possibility.

[0072] When the TTC is equal to or less than the first threshold time T1th, the recognition SoC24 sends an alarm command to the speaker 70 and the first display 80. Such control is referred to as "alarm control".

[0073] When the TTC is equal to or less than "the second threshold time T2th which is a value smaller than the first threshold time T1th", the recognition SoC24 determines whether there is an avoidance area. The avoidance area is an area that satisfies both of the following condition 1 and condition 2.

[0074] Condition 1: The vehicle VA can avoid the forward object FOB without exceeding the own lane SL demarcated by the left white line LWL and the right white line RWL.

[0075] Condition 2: The vehicle VA can avoid the forward object FOB without being obstructed by objects other than the forward object FOB.

[0076] It should be noted that the left white line LWL and the right white line RWL are determined based on the recognition image obtained by the forward camera device 32.

[0077] In Figure 4 In the example shown, with the front center part of the vehicle VA as a reference, it overlaps with the forward object FOB on the left side, so the vehicle VA attempts to pass on the right side of the forward object FOB to avoid it. The recognition SoC24 determines whether there is an avoidance area on the right side of the forward object FOB.

[0078] More specifically, when the distance Wsp between the right end point RP of the forward object FOB and the right white line RWL is equal to or greater than the value obtained by adding the vehicle width W of the vehicle VA and the specified margin D, the recognition SoC24 determines that condition 1 is satisfied.

[0079] Moreover, when the recognition SoC24 determines that the above condition 1 is satisfied, it determines in the passing prediction area SP (refer to Figure 4.) Is there an object? The passage prediction area SP is an area predicted that the vehicle VA will pass while avoiding a front collision object FOB. When the recognition SoC24 determines that there is no object in the passage prediction area SP, it determines that condition 2 is satisfied.

[0080] The passage prediction area SP is a rectangular area that has a length Wsp in the right direction of the vehicle width direction Dy from the right end point RP of the obstacle and has a length from the front end of the vehicle VA to the point where "the length Lsp has been left behind the rear of the front collision object FOB". The length Lsp is set to about the length in the front-rear direction of the vehicle VA (the vehicle length of the vehicle VA).

[0081] In the case where there is no avoidance area, the recognition SoC24 sends a braking command including a specified negative target acceleration to the engine ECU40 and the brake ECU50. When the engine ECU40 receives the braking command, it sets the target throttle opening to "0" and controls the engine actuator 46 so that the target throttle opening coincides with the actual throttle opening. When the brake ECU50 receives the braking command, it controls the brake actuator 56 so that the target acceleration included in the braking command coincides with the actual acceleration of the vehicle VA. Such control is called "braking control".

[0082] In the case where there is an avoidance area, the recognition SoC24 obtains a target steering angle for passing through the avoidance area in such a way that the vehicle VA does not start to collide with the front collision object FOB, and sends an avoidance command including the target steering angle to the steering ECU60. When the steering ECU60 receives the avoidance command, it controls the steering motor 64 so that the target steering angle included in the avoidance command coincides with the actual steering angle. Such control is called "avoidance control".

[0083] <Rear safety control>

[0084] The recognition SoC 24 recognizes the recognition image acquired by the rear PVM camera device 34Rr behind the vehicle (hereinafter referred to as "rear recognition image"). The recognition SoC 24 performs rear recognition processing, which is a process of recognizing an object by determining the position of the object relative to the vehicle VA based on the rear recognition image. Further, the recognition SoC 24 determines the moving direction of the object relative to the vehicle VA and the relative speed Vr of the object relative to the vehicle VA based on the previous position and the current position of the above object. The recognition SoC 24 determines an object that may collide with the vehicle VA as a rear collision object ROB, and obtains the TTC of the rear collision object ROB. If the TTC is equal to or less than the first threshold time T1th, the recognition SoC 24 performs alarm control, and if the TTC is equal to or less than the second threshold time T2th, the recognition SoC 24 performs braking control.

[0085] (Outline of operation)

[0086] In the present embodiment, not only the front camera device 32 acquires a recognition image, but also the front PVM camera device 34Fr and the rear PVM camera device 34Rr acquire recognition images. These recognition images need to be sent to the recognition SoC. When the front PVM camera device 34Fr and the rear PVM camera device 34Rr acquire recognition images, the usage band of the communication band from the recognition Des220 to the recognition SoC 24 (hereinafter referred to as "recognition communication band") is larger than the case where the above PVM camera devices 34Fr and 34Rr do not acquire recognition images. In particular, if the recognition images acquired by the three camera devices 32, 34Fr, and 34Rr are sent at the same timing, the maximum value of the usage band of the recognition communication band will increase sharply temporarily. As a result, the maximum value of the usage band may exceed the maximum value of the recognition communication band, and thus the recognition SoC 24 may not be able to receive the recognition image.

[0087] Therefore, in the present embodiment, the recognition images acquired by the front PVM camera device 34Fr and the rear PVM camera device 34Rr are configured to be sent at different timings. Thereby, it is possible to prevent the maximum value of the above usage band from increasing sharply temporarily. As a result, the possibility that the recognition SoC 24 can reliably receive the recognition image can be improved.

[0088] (Example of operation)

[0089] As Figure 5As shown, each of the camera devices 32 and 34 includes an imager and a Ser (Ser is short for serializer and is sometimes also referred to as a transmission IC). The imager obtains an image (image data) by converting the intensity of light into an electrical signal. The Ser transmits the image obtained by the imager.

[0090] The communication unit 22 includes an identification Des 220 and a display Des 222. Des is short for deserializer and is sometimes also referred to as a reception IC. The identification Des 220 is communicatively connected to the Ser of the front camera device 32, the display Des 222, and the identification SoC 24. The display Des 222 is communicatively connected to the Ser of the PVM camera device 34, the identification Des 220, and the display control SoC 26.

[0091] Every time a predetermined photographing time Tp (Tp = 25 ms) elapses, the identification SoC 24 sends either the identification synchronization signal or the display synchronization signal to each of the camera devices 32 and 34. In Figure 5 it, the flow of the synchronization signal is indicated by a dotted line. The identification synchronization signal includes a shutter speed SP1 for obtaining an identification image. The display synchronization signal includes a shutter speed SP2 for obtaining a display image. When receiving the synchronization signal, each of the camera devices 32 and 34 obtains an image according to the shutter speed included in the synchronization signal.

[0092] When each PVM camera device 34 obtains an identification image, it assigns identification information (identification identifier) indicating that the image is an identification image to the identification image and transmits the identification image. Similarly, when each PVM camera device 34 obtains a display image, it assigns display information (display identifier) indicating that the image is a display image to the display image and transmits the display image. When the display Des 222 receives an image from each PVM camera device 34, if display information is assigned to the image, it transmits the image (i.e., the display image) to the display control SoC 26. On the other hand, if identification information is assigned to the image, the display Des 222 transmits the image (i.e., the identification image) to the identification Des 220. When receiving an image (i.e., the identification image) from the front camera device 32 or the display Des 222, the identification Des 220 transmits the image to the identification SoC 24.

[0093] For the front camera device 32, every time the imaging time Tp elapses, the recognition SoC 24 increments the counter N by "1", and when the counter N becomes "5", the value of the counter N is set to "1". The recognition SoC 24 sends a recognition synchronization signal to the front camera device 32 in such a way that the front camera device 32 acquires recognition images at 40 fps. That is, regardless of the value of the counter N, the recognition SoC 24 sends the recognition synchronization signal to the front camera device 32 (refer to Figure 6 (A) of

[0094] The recognition SoC 24 sends a display synchronization signal to the left PVM camera device 34L and the right PVM camera device 34R in such a way that the left PVM camera device 34L and the right PVM camera device 34R acquire display images at 30 fps. As an example, if the value of the counter N is "2", the recognition SoC 24 does not send any synchronization signal to the PVM camera devices 34L and 34R. If the value of the counter N is any one of "1", "3", and "4", the recognition SoC 24 sends a display synchronization signal to the PVM camera devices 34L and 34R (refer to Figure 6 (D) of Figure 6 (E) of

[0095] The recognition SoC 24 sends a recognition synchronization signal and a display synchronization signal to the front PVM camera device 34Fr and the rear PVM camera device 34Rr in such a way that the front PVM camera device 34Fr and the rear PVM camera device 34Rr acquire recognition images at 10 fps and acquire display images at 30 fps.

[0096] As an example, if the value of the counter N is "4", the recognition SoC 24 sends a recognition synchronization signal to the front PVM camera device 34Fr. If the value of the counter N is any one of "1" to "3", the recognition SoC 24 sends a display synchronization signal to the front PVM camera device 34Fr (refer to Figure 6 (B) of Figure 6 (C) of

[0097] Thus, in the present embodiment, the timings at which the recognition SoC 24 sends the recognition synchronization signals to the front PVM camera device 34Fr and the rear PVM camera device 34Rr are different. As a result, the timings at which the front PVM camera device 34Fr acquires the recognition images are different from the timings at which the rear PVM camera device 34Rr acquires the recognition images. Consequently, the timings at which the recognition Des 220 sends the recognition images acquired by the front PVM camera device 34Fr and the recognition images acquired by the rear PVM camera device 34Rr to the recognition SoC 24 are different.

[0098] Furthermore, the data capacity of each image acquired by the front camera device 32 is larger than the data capacity of each image acquired by each PVM camera device 34.

[0099] · Front camera device 32: 128 Mbit (= 8 Mpix × 16 bit)

[0100] · PVM camera device 34: 44.8 Mbit (= 2.8 Mpix × 16 bit)

[0101] That is, the images acquired by the front camera device 32 are approximately three times as large as the images acquired by the PVM camera device 34.

[0102] In the present embodiment, the recognition images with such a large data capacity are sent to the recognition SoC 24 at 40 fps, and the recognition images acquired by the PVM camera devices 34Fr and 34Rr are also sent to the recognition SoC 24 at 10 fps. Therefore, it is necessary to minimize the maximum value of the used frequency band of the recognition communication frequency band as much as possible. In the present embodiment, as described above, the timings at which the recognition Des 220 sends the recognition images to the recognition SoC 24 are different, so the maximum value of the above-mentioned used frequency band can be reduced.

[0103] (Specific operation)

[0104] <Synchronization signal transmission routine>

[0105] Every time the photographing time Tp elapses, the CPU of the recognition SoC 24 (hereinafter, when marked as "the first CPU", unless otherwise specified, it refers to the CPU of the recognition SoC 24.) executes the synchronization signal transmission routine shown in the flowchart in Figure 7 .

[0106] Therefore, when it reaches a specified timing, the first CPU starts processing from step 700 of Figure 7 and sequentially executes step 705 and step 710.

[0107] Step 705: The first CPU increments the value of the counter N by "1".

[0108] The value of the counter N is set to "1" in the initialization routine. When an ignition key switch (not shown) of the vehicle VA is changed from the off position to the on position, the initialization routine is executed by the first CPU.

[0109] Step 710: The first CPU determines whether the value of the counter N is "5".

[0110] When the value of the counter N is not "5" (i.e., when the value of the counter N is any one of "1" to "4"), the first CPU sequentially executes Step 715 and Step 720.

[0111] Step 715: The first CPU sends an identification synchronization signal to the front camera device 32.

[0112] Step 720: The first CPU determines whether the value of the counter N is "1" or "3".

[0113] When the value of the counter N is "1" or "3", the first CPU determines "yes" in Step 720 and proceeds to Step 725 to send a display synchronization signal to all the PVM camera devices 34. After that, the first CPU proceeds to Step 795 to temporarily end this routine.

[0114] On the other hand, when the value of the counter N is not any one of "1" and "3" at the time the first CPU enters Step 720 (i.e., when the value of the counter N is "2" or "4"), the first CPU determines "no" in this Step 720 and proceeds to Step 730. In Step 730, the first CPU determines whether the value of the counter N is "2".

[0115] When the value of the counter N is "2", the first CPU determines "yes" in Step 730 and sequentially executes Step 735 and Step 740.

[0116] Step 735: The first CPU sends an identification synchronization signal to the rear PVM camera device 34Rr.

[0117] Step 740: The first CPU sends a display synchronization signal to the front PVM camera device 34Fr.

[0118] After that, the first CPU proceeds to Step 795 to temporarily end this routine.

[0119] On the other hand, when the value of the counter N is not "2" at the time the first CPU enters Step 730 (i.e., when the value of the counter N is "4"), the first CPU determines "no" in Step 730 and sequentially executes Step 745 and Step 750.

[0120] Step 745: The first CPU sends an identification synchronization signal to the front PVM camera device 34Fr.

[0121] Step 750: The second CPU sends a display synchronization signal to the rear PVM camera device 34Rr, the left PVM camera device 34L, and the right PVM camera device 34R.

[0122] After that, the first CPU enters Step 795 to temporarily end this routine.

[0123] On the other hand, when the value of the time counter N is "5" at the time point when the first CPU enters Step 710, the first CPU determines "Yes" in Step 710 and enters Step 755. In Step 755, the first CPU sets the value of the counter N to "1" and enters Step 715.

[0124] <Image acquisition routine>

[0125] Every time a predetermined time elapses, a CPU (hereinafter, when marked as the "second CPU", unless otherwise specified, refers to the CPUs of the camera devices 32 and 34) of each camera device 32 and 34 executes the image acquisition routine shown in the flowchart in Figure 8 . It should be noted that this predetermined time is set shorter than the above-mentioned photographing time Tp.

[0126] Therefore, when it becomes a predetermined timing, the second CPU starts processing from Step 800 of Figure 8 and enters Step 805. In Step 805, the second CPU determines whether the camera device has received any one of the identification synchronization signal and the display synchronization signal during the period from the last execution of this routine to the current execution of this routine.

[0127] If the camera device has not received any one of the synchronization signals during the above period, the second CPU determines "No" in Step 805 and enters Step 895 to temporarily end this routine.

[0128] On the other hand, if the camera device has received any one of the synchronization signals during the above period, the second CPU determines "Yes" in Step 805 and enters Step 810. In Step 810, the second CPU determines whether the received synchronization signal is an identification synchronization signal.

[0129] If the received synchronization signal is an identification synchronization signal, the second CPU determines "Yes" in Step 810 and executes Steps 815 to 825.

[0130] Step 815: The second CPU causes the imager to acquire an image at the shutter speed SP1 included in the recognition synchronization signal, thereby acquiring a recognition image.

[0131] Step 820: The second CPU assigns recognition information to the acquired recognition image.

[0132] Step 825: The second CPU transmits the recognition image from Ser.

[0133] After that, the second CPU proceeds to step 895 to temporarily end this routine.

[0134] On the other hand, when the received synchronization signal is not the recognition synchronization signal, that is, when the received synchronization signal is the display synchronization signal, the second CPU determines "no" in step 810 and executes steps 830 to 840.

[0135] Step 830: The second CPU causes the imager to acquire an image at the shutter speed SP2 included in the display synchronization signal, thereby acquiring a display image.

[0136] Step 835: The second CPU assigns display information to the acquired display image.

[0137] Step 840: The second CPU transmits the display image from Ser.

[0138] After that, the second CPU proceeds to step 895 to temporarily end this routine.

[0139] <Image transmission routine>

[0140] Every time a predetermined time elapses, the display Des222 executes the image allocation routine shown in the flowchart in Figure 9 It should be noted that this predetermined time is set shorter than the above-mentioned photographing time Tp.

[0141] Therefore, when the specified timing is reached, the display Des222 starts processing from step 900 of Figure 9 and proceeds to step 905. In step 905, the display Des222 determines whether an image has been received from the PVM camera device 34 during the period from the last execution of this routine to the current execution of this routine.

[0142] When no image has been received from the PVM camera device 34 during the above period, the display Des222 determines "no" in step 905 and proceeds to step 995 to temporarily end this routine.

[0143] On the other hand, when an image is received from the PVM camera device 34 during the above period, the display Des222 determines "Yes" in step 905 and proceeds to step 910. In step 910, the display Des222 determines whether the received image has been assigned identification information.

[0144] If the received image has been assigned identification information (i.e., if the received image is an identification image), the display Des222 determines "Yes" in step 910 and proceeds to step 915. In step 915, the display Des222 sends the received image to the identification Des220 and proceeds to step 995 to temporarily end this routine.

[0145] On the other hand, if the received image has not been assigned identification information (i.e., if the received image is a display image), the display Des222 determines "No" in step 910 and proceeds to step 920. In step 920, the display Des222 sends the received image to the display control SoC26 and proceeds to step 995 to temporarily end this routine.

[0146] <Forward safety control routine>

[0147] Every time a predetermined time (4 × Tp) elapses, the first CPU executes the forward safety control routine shown in the flowchart in Figure 10 .

[0148] Therefore, when it reaches the specified timing, the first CPU starts processing from step 1000 of Figure 10 and sequentially executes step 1005 and step 1010.

[0149] Step 1005: The first CPU identifies the object existing in the front area of the vehicle VA based on the forward identification image.

[0150] Step 1010: The first CPU determines whether there is a forward collision object FOB.

[0151] If there is no forward collision object FOB, the first CPU determines "No" in step 1010 and proceeds to step 1095 to temporarily end this routine.

[0152] On the other hand, if there is a forward collision object FOB, the first CPU determines "Yes" in step 1010 and sequentially executes step 1015 and step 1020.

[0153] Step 1015: The first CPU obtains the TTC of the forward collision object FOB.

[0154] Step 1020: The first CPU determines whether the TTC is below the first threshold time T1th.

[0155] When the TTC is greater than the first threshold time T1th, the first CPU determines "No" in step 1020 and proceeds to step 1095 to temporarily end this routine.

[0156] On the other hand, when the TTC is below the first threshold time T1th, the first CPU sequentially executes step 1025 and step 1030.

[0157] Step 1025: The first CPU sends an alarm instruction to the speaker 70 and the first display 80.

[0158] Step 1030: The first CPU determines whether the TTC is below the second threshold time T2th.

[0159] When the TTC is greater than the second threshold time T2th, the first CPU determines "No" in step 1030 and proceeds to step 1095 to temporarily end this routine.

[0160] On the other hand, when the TTC is below the second threshold time T2th, the first CPU determines "Yes" in step 1030 and proceeds to step 1035.

[0161] In step 1035, the first CPU determines whether there is an avoidance area.

[0162] When there is an avoidance area, the first CPU determines "Yes" in step 1035 and proceeds to step 1040 to send an avoidance instruction to the steering ECU 60. After that, the first CPU proceeds to step 1095 to temporarily end this routine.

[0163] When there is no avoidance area, the first CPU determines "No" in step 1035 and proceeds to step 1045 to send a braking instruction to the engine ECU 40 and the brake ECU 50. After that, the first CPU proceeds to step 1095 to temporarily end this routine.

[0164] <Rear safety control routine>

[0165] Every time a specified time (100 ms = 4 × Tp) elapses, the first CPU executes the rear safety control routine shown in the flowchart in Figure 11 .

[0166] Therefore, when it reaches the specified timing, the first CPU starts processing from Figure 11 step 1100 and sequentially executes step 1105 and step 1110.

[0167] Step 1105: The first CPU identifies a target present in the rear area of the vehicle VA based on the rear recognition image.

[0168] Step 1110: The first CPU determines whether there is a rear collision target ROB.

[0169] When there is no rear collision target ROB, the first CPU determines "No" in step 1110 and proceeds to step 1195 to temporarily end this routine.

[0170] On the other hand, when there is a rear collision target ROB, the first CPU determines "Yes" in step 1110 and sequentially executes step 1115 and step 1120.

[0171] Step 1115: The first CPU obtains the TTC of the rear collision target ROB.

[0172] Step 1120: The first CPU determines whether the TTC of the rear collision target ROB is less than or equal to the first threshold time T1th.

[0173] When the TTC is greater than the first threshold time T1th, the first CPU determines "No" in step 1120 and proceeds to step 1195 to temporarily end this routine.

[0174] On the other hand, when the TTC is less than or equal to the first threshold time T1th, the first CPU determines "Yes" in step 1120 and executes step 1125 and step 1130.

[0175] Step 1125: The first CPU sends an alarm instruction to the speaker 70 and the first display 80.

[0176] Step 1130: The first CPU determines whether the TTC is less than or equal to the second threshold time T2th.

[0177] When the TTC is greater than the second threshold time T2th, the first CPU determines "No" in step 1130 and proceeds to step 1195 to temporarily end this routine.

[0178] On the other hand, when the TTC is less than or equal to the second threshold time T2th, the first CPU determines "Yes" in step 1130 and proceeds to step 1135 to send a braking instruction to the engine ECU 40 and the brake ECU 50. After that, the first CPU proceeds to step 1195 to temporarily end this routine.

[0179] <Display control routine>

[0180] Every time the photographing time Tp elapses, the CPU of the display control SoC 26 (hereinafter, when marked as the "third CPU", unless otherwise specified, it refers to the CPU of the display control SoC 26.) executes the display control routine shown by the flowchart in Figure 12 .

[0181] Therefore, when it becomes a prescribed timing, the third CPU starts processing from step 1200 of Figure 12 and enters step 1205. In step 1205, the third CPU determines whether there is a PVM camera device 34 in which the display image has not been transmitted among the PVM camera devices 34 during the period from the last execution of this routine to the current execution of this routine. In other words, such a PVM camera device 34 is a PVM camera device 34 that has not acquired the display image during the above period. Hereinafter, such a PVM camera device 34 will be referred to as a "non-acquiring PVM camera device 34". In Figure 6 the example shown, when the value of the counter N is "1" or "3", there is no non-acquiring PVM camera device 34. When the value of the counter N is "2", the rear PVM camera device 34Rr, the left PVM camera device 34L, and the right PVM camera device 34R respectively correspond to the non-acquiring PVM camera device 34. When the value of the counter N is "4", the front PVM camera device 34Fr corresponds to the non-acquiring PVM camera device 34.

[0182] When there is no non-acquiring PVM camera device 34, the third CPU determines "no" in step 1205 and sequentially executes steps 1210 to 1220.

[0183] Step 1210: The third CPU generates a PVM image based on the display image.

[0184] More specifically, the third CPU generates a stereoscopic image by projecting each pixel value of the composite image obtained by synthesizing the display images into the pixels included in a hemispherical three-dimensional surface. The bottom surface of this three-dimensional surface is centered on the vehicle VA. The pixel values of the display image and the pixels of the projected three-dimensional surface are pre-established in correspondence. The third CPU generates a PVM image representing the peripheral area when viewed directly from above the vehicle VA using the generated stereoscopic image.

[0185] It should be noted that the generation process of such a stereoscopic image itself is a well-known technique (for example, refer to Japanese Patent Laid-Open No. 2012-217000).

[0186] Step 1215: The third CPU sends the PVM image to the second display 90. When the PVM image is received, the second display 90 displays the PVM image.

[0187] Step 1220: The third CPU erases the last display image stored in the RAM of the recognition SoC 24, and stores the display image received during the above period as the new last display image in the RAM.

[0188] After that, the third CPU enters Step 1295 to temporarily end this routine.

[0189] On the other hand, when there is a non-acquired PVM camera device 34 at the time when the third CPU enters Step 1205, the third CPU determines "Yes" in Step 1205 and enters Step 1225. In Step 1225, the third CPU acquires the last display image corresponding to the non-acquired PVM camera device 34 from the RAM and enters Step 1210. It should be noted that in Step 1210, the third CPU generates a PVM image based on the display image received during the above period and the last display image acquired in Step 1225.

[0190] As described above, according to the present embodiment, the communication unit 22 makes the transmission timing of the recognition image acquired by the front PVM camera device 34Fr different from the transmission timing of the recognition image acquired by the rear PVM camera device 34Rr. Thereby, the maximum value of the used frequency band of the recognition communication frequency band can be reduced.

[0191] The present invention is not limited to the above-described embodiment and the above-described modification examples, and various modification examples can be adopted within the scope of the present invention.

[0192] (First Modification Example)

[0193] In the above-described embodiment, the recognition SoC 24 sends the recognition synchronization signal to the front PVM camera device 34Fr and the rear PVM camera device 34Rr at different timings (N = 2 or 4), but the recognition synchronization signal may be sent at the same timing (N = 1). The display Des222 of this modification example includes a buffer for temporarily storing the received image. When the display Des222 receives the recognition synchronization signal, it stores in the buffer the recognition synchronization signal that can identify from which PVM camera devices 34Fr and 34Rr it is received.

[0194] In this modified example, the recognition SoC 24 recognizes a transmission request for the recognition image acquired by the front PVM camera device 34Fr (hereinafter referred to as "first transmission request") and a transmission request for the recognition image acquired by the rear PVM camera device 34Rr (hereinafter referred to as "second transmission request") to the communication unit 22 at different timings. As an example, the recognition SoC 24 transmits the first transmission request to the communication unit 22 when the value of the counter N becomes "2", and transmits the second transmission request to the communication unit 22 when the value of the counter N becomes "4".

[0195] When the communication unit 22 receives the first transmission request, the display Des 222 acquires the recognition image acquired by the front PVM camera device 34Fr from the buffer, and transmits the recognition image to the recognition SoC 24 via the recognition Des 220. Similarly, when the communication unit 22 receives the second transmission request, the display Des 222 acquires the recognition image acquired by the rear PVM camera device 34Rr from the buffer, and transmits the recognition image to the recognition SoC 24 via the recognition Des 220. It should be noted that after transmitting the recognition image, the recognition Des 220 erases the recognition image from the buffer.

[0196] Based on the above, even when the front PVM camera device 34Fr and the rear PVM camera device 34Rr acquire recognition images at the same timing, the communication unit 22 can transmit the recognition image acquired by the front PVM camera device 34Fr and the recognition image acquired by the rear PVM camera device 34Rr at different timings.

[0197] (Second Modified Example)

[0198] In this modified example, it is also possible that the front PVM camera device 34Fr and the rear PVM camera device 34Rr only acquire display images, and the left PVM camera device 34L and the right PVM camera device 34R acquire recognition images and display images.

[0199] In the front safety control routine of this modified example, the recognition image acquired by the front camera device 32 is used as the "front recognition image". Moreover, the first CPU executes the left safety control routine and the right safety control routine instead of the rear safety control routine. The first CPU is different from the rear safety control routine in that it recognizes the target based on the recognition image acquired by the left PVM camera device 34L in the left safety control routine and based on the recognition image acquired by the right PVM camera device 34R in the right safety control routine, and other processes are the same as the rear safety control routine.

[0200] Note that the above-described embodiment can be applied as long as at least two camera devices in the PVM camera device 34 are "camera devices that acquire images for recognition and images for display". As long as the PVM camera device 34 can acquire images of the peripheral area of the vehicle VA, the number and arrangement positions of the PVM camera devices 34 are not limited to the above-described embodiment.

[0201] (Third modified example)

[0202] As a collision index value, the distance L between the target and the vehicle VA can also be used instead of TTC. The smaller the distance L, the higher the collision possibility.

[0203] Moreover, it may be that when the TTC is equal to or less than a prescribed threshold value, the recognition SoC 24 executes at least one of alarm control, avoidance control, and braking control.

[0204] Moreover, in the alarm control, the recognition SoC 24 may send an alarm instruction to either the speaker 70 or the first display 80.

[0205] Moreover, the first display 80 and the second display 90 may be a single display.

Claims

1. A vehicle image processing system, comprising: A plurality of first camera devices that acquire identification images for identification processing for identifying objects in the peripheral area of the vehicle, and the plurality of first camera devices acquire display images; An identification unit that performs the identification processing; A communication unit that is communicably connected to the first camera device and the identification unit, receives the identification image transmitted by the first camera device, and transmits the identification image to the identification unit; And A display control unit that performs a predetermined display control process on the display image and displays the display image on which the display control process has been performed on a display provided in the vehicle, The communication unit is configured to transmit the identification image to the identification unit in such a manner that the transmission timing of the identification image received from the plurality of first camera devices to the identification unit is different, The identification unit is configured to perform the identification processing using the identification image received from the communication unit, Each of the plurality of first camera devices, when acquiring the identification image, assigns an identification identifier and transmits the identification image to the communication unit, Each of the plurality of first camera devices, when acquiring the display image, assigns a display identifier and transmits the display image to the communication unit, The communication unit is also communicably connected to the display control unit, And, the communication unit is configured to: when receiving either the identification image or the display image from the first camera device, if the image is assigned the identification identifier, transmit the image to the identification unit, and if the image is assigned the display identifier, transmit the image to the display control unit.

2. The vehicle image processing system according to claim 1, Wherein, Each of the plurality of first camera devices is configured to: Acquire the identification image in such a manner that the acquisition timing of the identification image is different between the first camera devices; And Transmit the identification image to the communication unit, The communication unit is configured to, when receiving the identification image from the first camera device, transmit the identification image to the identification unit.

3. The vehicle image processing system according to claim 2, Wherein, The identification unit is configured to transmit an identification synchronization signal to the plurality of first camera devices at different timings, Each of the plurality of first camera devices is configured to acquire the identification image when receiving the identification synchronization signal.

4. The vehicle image processing system according to claim 1, Wherein, The display control unit is configured to perform a process of synthesizing the display images acquired by the plurality of first camera devices as the display control process.

5. The vehicle image processing system according to claim 4, further comprising: At least one second camera device that acquires the display image, The first camera device and the second camera device are configured to share the acquisition of images of the surrounding area of the vehicle. The display control unit is configured to perform the following processing as the display control processing: generating an overhead image of the surrounding area overlooking from above the vehicle based on an image obtained by synthesizing the display images acquired by the first camera device and the second camera device.

6. The vehicle image processing system according to any one of claims 1 to 3, further comprising: A third camera device that acquires an identification image having a data capacity larger than the data capacity of the identification image acquired by the first camera device. The communication unit is communicably connected to the third camera device in addition to the first camera device and the identification unit. And the communication unit is configured to receive the identification image transmitted by the third camera device and transmit the identification image to the identification unit.

7. The vehicle image processing system according to any one of claims 1 to 3, wherein, The identification unit is configured to: Acquire a collision index value indicating the possibility of a collision between the object identified in the identification process and the vehicle; and When the relationship between the collision index value and a specified threshold satisfies a specified condition, perform at least one of alarm control, braking control, and avoidance control, where the alarm control is a control for notifying the driver of the vehicle that a collision with the object may occur, the braking control is a control for decelerating the vehicle by applying a braking force to the wheels of the vehicle, and the avoidance control is a control for changing the traveling direction of the vehicle to avoid a collision between the vehicle and the object.

8. A vehicle comprising the vehicle image processing system according to any one of claims 1 to 7.

9. An image transmission method, receiving the identification images acquired by a plurality of first camera devices from the plurality of first camera devices, and transmitting the identification images to an identification unit that performs an identification process for identifying an object existing in the surrounding area of the vehicle using the identification images. The image transmission method includes: A step of receiving the identification image from each of the plurality of first camera devices; and A step of transmitting the identification images to the identification unit in such a manner that the transmission timings of the identification images received from the plurality of first camera devices are different. Each of the plurality of first camera devices acquires, in addition to the identification image, a display image for display on a display disposed in the vehicle. Each of the plurality of first camera devices assigns an identification identifier to the identification image when the identification image is acquired. Each of the plurality of first camera devices assigns a display identifier to the display image when the display image is acquired. The image transmission method further includes the following steps: In the case where any one of the recognition image and the display image is received from the first camera device, if the recognition identifier is assigned to the image, the image is sent to the recognition unit, and if the display identifier is assigned to the image, the image is sent to the display control unit. The display control unit performs a prescribed display control process on the display image and displays the display image on which the display control process has been performed on the display.

Images