Anomaly detection device, anomaly detection method, program, and information processing system

Abstract

The imaging unit 500 acquires a captured image by using a lens whose effective image circle is smaller than the imaging surface of the image sensor. For example, the vignetting amount calculation unit 2212 of the detection unit 221 individually calculates the area size of the vignetting region generated at the four corners in the captured image acquired by the imaging unit 500. The determination unit 2214 detects the change in the area size calculated at the time of the focus abnormality detection processing with respect to the area size calculated at the reference time, and determines that the focus abnormality is detected when the change in the area size exceeds a threshold value at any one of the four corners. This makes it possible to detect the focus abnormality with high accuracy.

Description

TECHNICAL FIELD

[0001] The present technology relates to an abnormality detection device, an abnormality detection method, a program, and an information processing system, and enables accurate detection of focus abnormality. BACKGROUND

[0002] Conventionally, for example, an information processing system using a movable body such as an automobile has been installed on the movable body to capture images of the front and other directions of the movable body and to process and analyze the obtained captured images to recognize an obstacle. Such a system cannot correctly recognize an obstacle unless the imaging device is normally operated, for example. For this reason, in Patent Literature 1, for example, disparity image data is generated from stereo image data acquired by the imaging device, and it is judged based on the disparity image data whether the imaging device is abnormal.

[0003] LIST OF CITATIONS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Laid-Open No. 2014-006243 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] Incidentally, in a case where it is judged based on disparity image data whether there is an abnormality, since the imaging device needs to generate stereo image data, the imaging device that generates image data of one viewpoint cannot detect an abnormality.

[0008] In view of the above, an object of the present technology is to provide an abnormality detection device, an abnormality detection method, a program, and an information processing system that enable accurate detection of focus abnormality.

[0009] SOLUTION TO PROBLEM

[0010] A first aspect of the present technology is

[0011] An abnormality detection device includes:

[0012] A detection unit detects a focus abnormality based on a change in a vignetting region of a captured image using an image captured by a lens having an effective image circle smaller than an imaging surface of an image sensor.

[0013] In this technique, a vignetting region is detected from an image captured by an imaging unit using a lens in which an effective image circle is smaller than an imaging surface of an image sensor, and a focus abnormality is detected based on a change in the vignetting region with respect to a reference time at the time of performing a focus abnormality detection process. For example, the detection unit detects a change in the area size of the vignetting region for each of the four corners of the captured image, and in a case where the change exceeds a vignetting determination threshold at any one of the four corners, it is determined that a focus abnormality is detected. Further, the detection unit detects the sharpness of the captured image, and in a case where the detected sharpness is lower than a preset sharpness threshold, it is determined that a focus abnormality is detected. Note that the detector can discriminate the cause of the focus abnormality based on the change in the vignetting region at the time of detection of the focus abnormality.

[0014] Further, the abnormality detection device can further include a light source unit that illuminates an object captured by the image sensor, and can further include a motion controller that controls the motion of a vehicle on which the image sensor is mounted. When the detection unit detects a focus abnormality, the motion controller can stop a driving support function of the vehicle, and in a case where the motion controller executes an automatic driving control function of the vehicle on which the image sensor is mounted, the detection unit executes a focus abnormality detection process before the motion controller starts the automatic driving of the vehicle, and when the detection unit detects a focus abnormality, the motion controller stops the automatic driving control function and switches to manual driving.

[0015] A second aspect of the present technology is

[0016] An abnormality detection method includes

[0017] An image captured by an image sensor using a lens in which an effective image circle is smaller than an imaging surface of the image sensor is used to detect a focus abnormality based on a change in a vignetting region of the captured image.

[0018] A third aspect of the present technology is

[0019] A program for causing a computer to execute a focus abnormality detection, the program including the following processes:

[0020] An image captured by an image sensor using a lens in which an effective image circle is smaller than an imaging surface of the image sensor, and

[0021] A focus abnormality is detected based on a vignetting region of the captured image.

[0022] Note that the program of the present technology is a program that can be provided by a storage medium provided in a computer-readable format to a general-purpose computer that can execute various program codes, a program provided by a storage medium such as an optical disc, a magnetic disc, or a semiconductor memory, or a program that can be provided by a communication medium such as a network. By providing such a program in a computer-readable format, a process corresponding to the program is realized on a computer.

[0023] A fourth aspect of the present technology is

[0024] An information processing system includes

[0025] An abnormality detection device, and

[0026] A server that uses a detection result of the abnormality detection device, wherein

[0027] The abnormality detection device includes

[0028] A detection unit that uses an image captured by an image sensor using a lens having an effective image circle smaller than an imaging surface of the image sensor to detect a focus abnormality based on a change in a vignetting region of the captured image, and

[0029] The server includes

[0030] A notification information generation unit that generates user notification information using a focus abnormality detection result of the detection unit. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a diagram illustrating a configuration of an information processing system using a movable body.

[0032] Figure 2 is a diagram illustrating a schematic functional configuration of a control system.

[0033] Figure 3 is a diagram illustrating a part of a configuration of a detection unit.

[0034] Figure 4 is a diagram illustrating a relationship between an amount of deviation from a focus point and an angle of view.

[0035] Figure 5 is a diagram illustrating a state of a reference time of an imaging unit.

[0036] Figure 6 is a flowchart illustrating an operation of a detection unit.

[0037] Figure 7 is a flowchart illustrating another operation of a detection unit.

[0038] Figure 8 is a diagram illustrating a case where a state of an imaging unit from a reference time changes.

[0039] Figure 9 is a diagram illustrating another case of a change in the state of the imaging unit from a reference time.

[0040] Figure 10 is a diagram illustrating a case where the optical axis of the lens in the imaging unit is shifted.

[0041] Figure 11 is a diagram illustrating a case of angular distortion of the image sensor.

[0042] Figure 12 is a diagram illustrating a case where the illumination light is not used and a case where the illumination light is used.

[0043] Figure 13 is a diagram illustrating a case where the imaging surface is divided in the vertical direction and the horizontal direction.

[0044] Figure 14 is a diagram illustrating a case where a function such as focus abnormality detection is provided in an imaging device that images the front of a vehicle.

[0045] Figure 15 is a diagram illustrating a case where a function such as focus abnormality detection is provided in an imaging device that images the rear or side of a vehicle.

[0046] Figure 16 is a diagram illustrating a configuration of a server.

[0047] Figure 17 is a flowchart illustrating an operation of the server.

[0048] Figure 18 is a diagram illustrating another configuration of the server.

[0049] Figure 19 is a flowchart illustrating another operation of the server.

[0050] Figure 20 is a diagram illustrating another partial configuration of the server.

[0051] Figure 21 is a flowchart illustrating another partial operation of the server. DETAILED DESCRIPTION

[0052] Hereinafter, a mode for implementing the present technology will be described. Note that the description will be given in the following order.

[0053] 1. Embodiment

[0054] 1-1. Configuration of vehicle

[0055] 1-2. Operation of vehicle

[0056] 1-3. Modified example

[0057] 1-4. Configuration and operation of the server

[0058] 1-5. Other configuration and operation of the server

[0059] 2. Application examples

[0060] <1. Embodiments>

[0061] Figure 1 A configuration of an information processing system using movable bodies to which the present technology can be applied is illustrated. In the information processing system 10, one or plural movable bodies, such as vehicles 20, are provided, and each vehicle 20 and a server 30 are connected through a network 40. An abnormality detection device of the present technology is provided in the vehicle 20. The abnormality detection device uses an image captured by an image sensor using a lens having an effective image circle smaller than an imaging surface of the image sensor to detect a focus abnormality based on a change in a vignetting region of the captured image.

[0062] Further, the server 30 generates user notification information using a focus abnormality detection result acquired from the vehicle 20.

[0063] <1-1. Configuration of the vehicle>

[0064] Next, the configuration of the vehicle 20 will be described. The vehicle 20 includes an imaging unit, and performs a driving support function or an automatic driving control function using an image captured by the imaging unit. Further, the vehicle 20 detects a focus abnormality in the imaging unit and notifies a passenger, a driving control unit, or the like of the vehicle of a focus abnormality detection result. Further, the vehicle 20 notifies the server 30 of log information including at least the focus abnormality detection result or identification information of the vehicle, or the like. For example, the server 30 generates user notification information based on the log information notified from the vehicle 20, and provides a service for notifying an administrator or the like of the vehicle 20.

[0065] Figure 2 is a diagram illustrating a schematic functional configuration of a control system in a vehicle. The control system 200 of the vehicle 20 includes an input unit 201, a data acquisition unit 202, a communication unit 203, an in-vehicle device 204, an output controller 205, an output unit 206, a drive train controller 207, a drive train system 208, a body line controller 209, a body line system 210, a storage unit 211, and an automatic driving controller 213. The input unit 201, the data acquisition unit 202, the communication unit 203, the output controller 205, the drive train controller 207, the body line controller 209, the storage unit 211, and the automatic driving controller 213 are connected to each other through a communication network 212. Note that the units of the control system 200 can be directly connected to each other without interposing the communication network 212.

[0066] Note that, hereinafter, in a case where each unit of the control system 200 communicates through the communication network 212, the description of the communication network 212 will be omitted. For example, in a case where the input unit 201 and the automatic driving controller 213 communicate with each other through the communication network 212, it is simply described that the input unit 201 and the automatic driving controller 213 communicate with each other.

[0067] The input unit 201 includes a device by which a passenger inputs various data, instructions, and the like. For example, the input unit 201 includes an operation device such as a touch panel, a button, and a switch, and an operation device that enables input by a method other than manual operation, such as voice and a gesture, and the like. Further, for example, the input unit 201 can be a remote control device using infrared rays, radio waves, and the like, or an external connection device corresponding to the operation of the control system 200. For example, the input unit 201 generates an input signal based on data and instructions input by the passenger, and supplies the input signal to the units of the control system 200.

[0068] The data acquisition unit 202 includes various sensors that acquire data used for processing of the control system 200, and supplies the acquired data to the units of the control system 200. The data acquisition unit 202 includes an imaging unit 500 for grasping a situation outside the vehicle. The imaging unit 500 captures an image using a lens having an effective image circle smaller than an imaging surface of an image sensor. Further, in order to detect a situation outside the vehicle, the data acquisition unit 202 can include, for example, an environmental sensor for detecting weather and the like, and an environmental information detection sensor for detecting an object around the vehicle. The environmental sensor includes, for example, a raindrop sensor, a fog sensor, a sunlight sensor, a snow sensor, and the like. The environmental information detection sensor includes, for example, an ultrasonic sensor, a radar, a light detection and ranging, a laser imaging detection and ranging (LiDAR), a sonar, and the like.

[0069] Further, the data acquisition unit 202 includes, for example, a sensor for detecting a state of the vehicle, and the like. For example, the data acquisition unit 202 includes a gyro sensor, an acceleration sensor, an inertial measurement unit (IMU), and a sensor for detecting an accelerator pedal operation amount, a brake pedal operation amount, a steering wheel steering angle, a number of revolutions of an engine, a number of revolutions of a motor, a rotational speed of a wheel, and the like. Also, the data acquisition unit 202 includes various sensors for detecting a current position of the vehicle. For example, the data acquisition unit 202 includes a global navigation satellite system (GNSS) receiver that receives a GNSS signal from a GNSS satellite, and the like.

[0070] Further, the data acquisition unit 202 can include various sensors for detecting information about the inside of the vehicle. For example, the data acquisition unit 202 includes an imaging device that captures an image of the driver, a biometric sensor that detects biometric information of the driver, and a microphone that collects sound inside the vehicle, and the like. For example, the biometric sensor is provided on a seat surface, a steering wheel, or the like, and detects biometric information of an occupant seated on the seat or a driver holding the steering wheel.

[0071] For example, the communication unit 203 communicates with the in-vehicle device 204 and various devices outside the vehicle, such as the server 30, to transmit data supplied from the units of the control system 200 and supply received data to the units of the control system 200. Note that the communication protocol supported by the communication unit 203 is not particularly limited, and the communication unit 203 can support multiple types of communication protocols.

[0072] For example, the communication unit 203 performs wireless communication with the in-vehicle device 204 by wireless LAN, Bluetooth (registered trademark), near field communication (NFC), wireless USB (WUSB), or the like. Further, for example, the communication unit 203 performs wired communication with the in-vehicle device 204 by universal serial bus (USB), high-definition multimedia interface (HDMI) (registered trademark), mobile high-definition link (MHL), or the like through a connection terminal (and, if necessary, a cable) not shown.

[0073] Further, the communication unit 203 communicates with the server 30 through a network (for example, the Internet, a cloud network, or a company's dedicated network). Also, for example, the communication unit 203 can be configured to perform communication with a terminal, a wireless station, or the like in the vicinity of the vehicle, vehicle-to-vehicle communication, road-to-vehicle (vehicle-to-infrastructure) communication, vehicle-to-home communication, and pedestrian-to-vehicle communication.

[0074] The in-vehicle device 204 includes, for example, a device possessed by an occupant, an information device provided in the vehicle, a navigation device that searches for a route, and the like.

[0075] The output controller 205 controls output of various information to an occupant of the vehicle or to the outside of the vehicle. For example, the output controller 205 generates an output signal including at least one of visual information (for example, image data) or auditory information (for example, voice data), and supplies the output signal to the output unit 206 to control output of the visual information and the auditory information from the output unit 206. For example, the output controller 205 supplies an output signal including an aerial view image, a panoramic image, or the like to the output unit 206. Further, the output controller 205 supplies an output signal including voice data or the like to the output unit 206, including a warning sound, a warning message, or the like for a danger such as a collision, a contact, an entry into a dangerous zone, or the like.

[0076] The output unit 206 includes a device capable of outputting visual information or auditory information to an occupant of the vehicle or outside of the vehicle. For example, the output unit 206 includes a display device, an instrument panel, a sound speaker, a headphone, a wearable device, a projector, a lamp, and the like. The display device included in the output unit 206 can be, in addition to a device having a normal display, for example, a device that displays visual information in the field of view of the driver, such as a head-up display, a transmissive display, or a device having an augmented reality (AR) display function.

[0077] The drive train controller 207 controls the drive train system 208 by generating various control signals and supplying them to the drive train system 208. Further, for example, the drive train controller 207 supplies control signals to units other than the drive train system 208 as necessary to notify these units of the control state of the drive train system 208.

[0078] The drive train system 208 includes various devices related to the drive train of the vehicle. For example, the drive train system 208 includes a drive force generating device for generating a drive force of an internal combustion engine, a drive motor, or the like, a drive force transmission mechanism for transmitting the drive force to a wheel, a steering mechanism for adjusting a steering angle, a brake device for generating a braking force, an anti-lock brake system (ABS), an electronic stability control (ESC), an electric power steering device, and the like.

[0079] The body line controller 209 controls the body line system 210 by generating various control signals and supplying them to the body line system 210. Further, for example, the body line controller 209 supplies control signals to units other than the body line system 210 as necessary to notify these units of the control state of the body line system 210.

[0080] The body line system 210 includes various body line devices mounted on the vehicle body. For example, the body line system 210 includes a keyless entry system, a smart key system, a power window device, a power seat, a steering wheel, an air conditioner, various lamps (for example, a headlamp, a tail lamp, a brake lamp, a flasher, a fog lamp, and the like), and the like.

[0081] The storage unit 211 includes, for example, a magnetic storage device such as a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, and the like. The storage unit 211 stores various programs, data, and the like used by the units of the control system 200. For example, the storage unit 211 stores map data such as a three-dimensional high-precision map (like a dynamic map), a global map that is less precise than the high-precision map but covers a wide range, and a local map including information around the vehicle, and the like.

[0082] Further, the storage unit 211 stores log information of the vehicle 20 in which the control system 200 is provided. The log information includes identification information of the vehicle 20 (for example, a chassis number or a vehicle number, an initial registration year such as a year in which the first user starts to use), identification information of the imaging unit 500 provided in the data acquisition unit 202 (for example, a part number or a serial number), a use start year of the imaging unit 500, a focus abnormality detection result of the imaging unit 500, a time and a position at which an abnormality is detected, an image captured at the time of abnormality detection, and the like. Further, the log information can include a cause of the focus abnormality. Further, the log information can include information of an administrator or the like of the vehicle 20, a discrimination result of the cause of the focus abnormality, and the like. Also, the log information can include environmental information of the outside and the inside of the vehicle (for example, an in-vehicle temperature, an outside temperature, or weather at the time of use of the vehicle 20), use state information of the vehicle (a travel distance, a use start time, a use end time, and the like), failure state information of each unit (for example, a failure occurrence state or a repair state), and the like. The log information can include a threshold value for discriminating the sharpness of the image captured by the imaging unit 500 (a sharpness threshold value) and a threshold value for discriminating a change in vignetting (a vignetting discrimination threshold value). Note that the log information can include information detected by the detection unit 221, a discrimination result, and the like described later, and information calculated using such information. Further, the log information can include a history of informing an abnormality state to a driver or a vehicle occupant, and can include accident information (for example, a type or a damage situation of an accident).

[0083] The automatic driving controller 213 performs an automatic driving control function, a driving support function, and the like.

[0084] The automatic driving controller 213 performs, for example, a coordinated control aimed at realizing a function of an advanced driver assistance system (ADAS), including, for example, collision avoidance or impact mitigation of a vehicle, follow-up travel based on a vehicle distance, vehicle speed maintenance travel, vehicle collision warning, vehicle lane departure warning, and the like, and a coordinated control aimed at performing automatic driving in which a vehicle autonomously travels without depending on an operation of a driver. Further, the automatic driving controller 213 can perform a driving support function instead of an automatic driving control function. The automatic driving controller 213 includes a detection unit 221, a self-positioning unit 222, a situation analysis unit 223, a planning unit 224, and a motion controller 225.

[0085] When the automatic driving function control and the driving support function are executed, the automatic driving controller 213 determines whether or not to execute the automatic driving function control and the driving support function on the basis of the automatic driving level control information. In a case where the above functions can be executed, the automatic driving controller 213 executes these functions. In a case where there is a function that should not be executed, the automatic driving controller 213 does not execute the function that should not be executed. The automatic driving level control information can be set in advance, or can be received from a server or the like outside the vehicle.

[0086] The detection unit 221 detects various kinds of information necessary for controlling automatic driving. The detection unit 221 performs a detection process of information on the outside and the surrounding environment of the vehicle on the basis of data or signals from the units of the control system 200. The detection unit 221 performs a recognition process, a tracking process, and a distance detection process on other vehicles, people, obstacles, buildings, roads, traffic lights, traffic signs, road markings, and the like as the detection process of information on the outside of the vehicle, and performs a detection process of weather, temperature, humidity, brightness, a road surface state, and the like as the detection process of the surrounding environment.

[0087] Further, the detection unit 221 performs an internal information detection process on the basis of data or signals from the units of the control system 200. The detection unit 221 performs an authentication process and a recognition process of a driver, a driver state detection process, an occupant detection process, an in-vehicle environment detection process, and the like as the internal information detection process. The detection unit 221 detects a physical condition, a degree of alertness, a degree of concentration, a degree of fatigue, a line-of-sight direction, and the like in the driver state detection process, and detects a temperature, a humidity, a brightness, an odor, and the like in the in-vehicle environment detection process.

[0088] Further, the detection unit 221 performs a vehicle state detection process on the basis of data or signals from the units of the control system 200. The detection unit 221 detects a speed, an acceleration, a steering angle, presence / absence and content of an abnormality, a state of a driving operation, a position and a tilt of an electric seat, a state of a door lock, a state of other in-vehicle devices, and the like as the vehicle state detection process.

[0089] Moreover, the detection unit 221 performs a focus abnormality detection process on the imaging unit 500 provided in the data acquisition unit 202. Note that details of the focus abnormality detection process will be described later.

[0090] The detection unit 221 supplies data indicating the focus abnormality detection result to the self-positioning unit 222, the situation analysis unit 223, the motion controller 225, and the like. Further, the detection unit 221 causes the storage unit 211 to store information on the focus abnormality detection result and the detection result obtained by performing various detection processes on the basis of data or signals from each unit of the control system 200.

[0091] The self-positioning unit 222 performs estimation processing of the position, orientation, etc. of the vehicle based on data or signals from the detection unit 221, the situation analysis unit 223 described later, etc., and supplies data indicating the estimation result to the situation analysis unit 223.

[0092] The situation analysis unit 223 performs analysis processing of the vehicle and the surrounding situation. The situation analysis unit 223 performs analysis processing of various maps stored in the storage unit 211 using data or signals from the units of the control system 200 such as the detection unit 221 and the self-positioning unit 222 as necessary, and constructs maps including information necessary for automatic driving processing.

[0093] Based on the constructed maps and data or signals from each unit of the control system 200 such as the detection unit 221 and the self-positioning unit 222, the situation analysis unit 223 performs recognition processing on, for example, the position and state of a traffic light around the vehicle, the content of a traffic regulation around the vehicle, and a traffic rule such as a lane on which the vehicle can travel.

[0094] Further, the situation analysis unit 223 performs recognition processing of the vehicle situation, the situation around the vehicle, the situation of the driver of the vehicle, etc. based on data or signals from each unit of the control system 200 such as the detection unit 221 and the self-positioning unit 222, the constructed maps, etc. For example, as the recognition processing of the vehicle situation, the situation analysis unit 223 performs recognition processing of, for example, the position, orientation, and movement (e.g., speed, acceleration, and moving direction) of the vehicle, and the presence or absence and content of an abnormality. Further, as the recognition processing of the situation around the vehicle, the situation analysis unit 223 performs recognition processing of, for example, the type and position of a surrounding stationary object, the type, position, and motion (e.g., speed, acceleration, moving direction, etc.) of a surrounding moving object, the configuration and situation of a road surface of a surrounding road, environmental weather, temperature, humidity, and brightness, etc. Also, as the recognition processing of the situation of the driver of the vehicle, etc., the situation analysis unit 223 performs recognition processing of, for example, the physical condition, the degree of alertness, the degree of concentration, the degree of fatigue, the movement of the line of sight, the driving operation, etc.

[0095] Further, the situation analysis unit 223 performs prediction processing of the vehicle situation, the situation around the vehicle, the situation of the driver, and the like, based on the recognition processing results regarding the recognition processing results of the traffic rules, the situation of the vehicle, the situation around the vehicle, the situation of the driver, and the like. As the prediction processing of the vehicle situation, the situation analysis unit 223 performs prediction processing of, for example, the behavior of the vehicle, the occurrence of an abnormality, the distance that can be traveled, and the like. As the prediction processing of the situation around the vehicle, the situation analysis unit 223 performs prediction processing of, for example, the behavior of a moving object around the vehicle, the change in the signal state, the change in the environment such as the weather, and the like. Further, as the prediction processing of the situation of the driver, the situation analysis unit 223 performs prediction processing of, for example, the behavior of the driver, the physical condition, and the like. The situation analysis unit 223 supplies the constructed map and data indicating the results of the recognition processing and the prediction processing to the planning unit 224.

[0096] The planning unit 224 plans a route to the destination based on data or signals from each unit of the control system 200, such as the situation analysis unit 223. For example, the planning unit 224 plans a route from the current position to a specified destination based on the global map, taking into account, for example, situations such as traffic congestion, accidents, traffic regulations, construction, and the like, and the physical condition of the driver. Further, in order to safely travel the planned route within the planned time, the planning unit 224 plans actions such as starting, stopping, the direction of travel, the travel lane, the travel speed, overtaking, and the like. Also, the planning unit 224 plans the motion of the vehicle to achieve the planned actions such as acceleration, deceleration, the travel trajectory, and the like. The planning unit 224 supplies data indicating the planned motion of the vehicle to the motion controller 225.

[0097] The motion controller 225 controls the motion of the vehicle. The motion controller 225 calculates control target values of the drive force generation device or the brake device to achieve the motion of the vehicle planned by the planning unit 224, such as acceleration, deceleration, or sudden stopping, and control target values of the steering mechanism for achieving travel to the destination, lane change, and the like, and supplies a control command indicating the calculated control target values to the powertrain controller 207. Further, the motion controller 225 performs processing of detecting an emergency situation such as a collision, contact, entry into a dangerous zone, an abnormality of the driver, or an abnormality of the vehicle, based on the detection results of the detection unit 221, and in the case where it is detected that an emergency situation has occurred, the motion controller 225 plans the motion of the vehicle to avoid the emergency situation such as sudden stopping or sudden turning, and supplies a control command for performing the planned motion of the vehicle to the powertrain controller 207. Further, in the case where the detection unit 221 determines a focusing abnormality, the motion controller 225 can stop the drive support function and the automatic driving control function and switch to manual driving.

[0098] Figure 3A part of the configuration of the detection unit is illustrated. The detection unit 221 includes an abnormality detection unit 2210 that detects a focus abnormality of the imaging unit 500 provided in the data acquisition unit 202. The abnormality detection unit 2210 includes a signal processing unit 2211, a vignetting amount calculation unit 2212, and a determination unit 2214. Further, the abnormality detection unit 2210 can also include a failsafe detection unit 2213.

[0099] The signal processing unit 2211 performs signal processing on the image signal generated by the imaging unit 500 of the data acquisition unit 202, so that the vignetting amount can be accurately calculated. For example, when the signal level of the image signal is small, the boundary of the vignetting region becomes unclear, and the vignetting amount cannot be accurately calculated. Thus, the signal processing unit 2211 adjusts the signal level of the image signal generated by the imaging unit 500, so that the vignetting amount can be accurately calculated. Further, the signal processing unit 2211 can remove noise from the image signal, so that the vignetting amount is not affected by the noise. The signal processing unit 2211 outputs the processed image signal to the vignetting amount calculation unit 2212 and the failsafe detection unit 2213.

[0100] The vignetting amount calculation unit 2212 discriminates the vignetting region from the image signal supplied from the signal processing unit 2211, and calculates the area size of the vignetting region as the vignetting amount. The vignetting region is, for example, a continuous pixel region from each of the four corners having a signal level smaller than a region threshold value set in advance for each corner, and the area (or the number of pixels) of the vignetting region is set as the vignetting amount. The vignetting amount calculation unit 2212 outputs the calculated vignetting amount to the determination unit 2214.

[0101] The failsafe detection unit 2213 detects whether or not to perform a failsafe operation based on the image signal supplied from the signal processing unit 2211. For example, when the sharpness of the captured image is reduced due to a focus abnormality or the like in the imaging unit 500 of the data acquisition unit 202, it becomes difficult to distinguish a peripheral subject or the like. Therefore, the failsafe detection unit 2213 detects the sharpness based on the image signal supplied from the signal processing unit 2211, and in a case where the sharpness is lower than a preset sharpness threshold value (in a case where the image is blurred), performs a warning or a motion control so as not to perform a driving support function, an automatic driving control function, or the like based on the image signal supplied from the signal processing unit 2211. For example, a high-frequency component and an edge region are extracted from the image signal supplied from the signal processing unit 2211, and the intensity of the high-frequency component of a predetermined edge region size is calculated as the sharpness. Note that another method can be used to calculate the sharpness. The failsafe detection unit 2213 outputs the comparison result of the sharpness with the preset sharpness threshold value as a failsafe detection result to the determination unit 2214. Note that a threshold value transmitted from a server outside the vehicle can be used as the sharpness threshold value.

[0102] The determination unit 2214 determines whether the imaging unit 500 of the data acquisition unit 202 is normal based on the vignetting amount calculated by the vignetting amount calculation unit 2212. When the change in the vignetting amount calculated by the vignetting amount calculation unit 2212 from the vignetting amount at the time of the focus abnormality detection processing with respect to the vignetting amount at the reference time (for example, at the time of alignment during manufacturing, at the time of initial use start, or at the time of repair completion) does not exceed the vignetting determination threshold value, the determination unit 2214 determines that the imaging unit 500 is normal, whereas when the change in the vignetting amount exceeds the vignetting determination threshold value, the determination unit 2214 determines that the imaging unit 500 is abnormal. For example, when the change in the vignetting amount at any one of the four corners of the captured image exceeds the vignetting determination threshold value, the determination unit 2214 determines that a focus abnormality is detected. The vignetting determination threshold value for determining the change in the vignetting amount is set in advance. Note that the threshold value transmitted from the server outside the vehicle can be used as the vignetting determination threshold value.

[0103] Further, in the case where the fail-safe detection unit 2213 is also included, the determination unit 2214 determines whether the imaging unit 500 of the data acquisition unit 202 is normal using not only the vignetting amount but also the detection result of the fail-safe detection unit 2213. In the case where the detection result of the fail-safe detection unit 2213 indicates that the sharpness is lower than the preset sharpness threshold value, the determination unit 2214 determines that the imaging unit 500 is abnormal regardless of the vignetting amount. The determination unit 2214 outputs the determination result to the motion controller 225. Note that the determination unit 2214 can discriminate the cause of the abnormality based on the change in the vignetting amount.

[0104] The determination unit 2214 determines whether the focus deviation is within a normal range or abnormal, and outputs the determination result. The determination result can include information indicating whether the determination is based on the sharpness or the vignetting amount, or can include the sharpness value, the sharpness threshold value, each vignetting amount value, or the vignetting amount threshold value.

[0105] <1-2. Operation of the vehicle>

[0106] Figure 4 The relationship between the amount of deviation from the focal point and the angle of view is illustrated. The imaging unit has a characteristic that the angle of view increases as the distance between the lens and the image sensor increases and the angle of view decreases as the distance decreases. Further, as the distance from the focal point increases, the resolution decreases and the spatial resolution (MTF) decreases. Thus, the present technology detects a focus abnormality based on the change in the angle of view. Specifically, the imaging unit 500 used in the data acquisition unit 202 generates a vignetting region around the imaging face of the image sensor by using a lens having a small effective image circle with respect to the size of the image sensor. Further, the detector determines a focus abnormality based on the change in the vignetting region (vignetting amount).

[0107] Figure 5 Fig. 1 illustrates a state of a reference time of an imaging unit. Figure 5 Fig. 1 illustrates a state of a reference time of an imaging unit. Fig. 1 illustrates a state of a reference time of an imaging unit.

[0108] Fig. 1 illustrates a state of a reference time of an imaging unit. Figure 5 Fig. 1 illustrates a state of a reference time of an imaging unit. Figure 5 Fig. 1 illustrates a state of a reference time of an imaging unit. Fig. 1 illustrates a state of a reference time of an imaging unit.

[0109] Fig. 1 illustrates a state of a reference time of an imaging unit. Fig. 1 illustrates a state of a reference time of an imaging unit.

[0110] Fig. 1 illustrates a state of a reference time of an imaging unit. Figure 6 Fig. 1 illustrates a state of a reference time of an imaging unit. Figure 6 Fig. 1 illustrates a state of a reference time of an imaging unit. Fig. 1 illustrates a state of a reference time of an imaging unit.

[0111] Fig. 1 illustrates a state of a reference time of an imaging unit. Fig. 1 illustrates a state of a reference time of an imaging unit.

[0112] In step ST1, the detection unit determines whether or not the change in the amount of right upper vignetting is within a threshold value. The detection unit 221 calculates the area size Sru of the right upper vignetting region as the amount of right upper vignetting, and determines whether or not the calculated area size Sru is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Sru-s of the reference time. If the calculated change in the right upper vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST2, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST6.

[0113] In step ST2, the detection unit determines whether or not the change in the amount of left upper vignetting is within a threshold value. The detection unit 221 calculates the area size Slu of the left upper vignetting region as the amount of left upper vignetting, and determines whether or not the calculated area size Slu is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Slu-s of the reference time. If the calculated change in the left upper vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST3, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST6.

[0114] In step ST3, the detection unit determines whether or not the change in the amount of right lower vignetting is within a threshold value. The detection unit 221 calculates the area size Srd of the right lower vignetting region as the amount of right lower vignetting, and determines whether or not the calculated area size Srd is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Srd-s of the reference time. If the calculated change in the right lower vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST4, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST6.

[0115] In step ST4, the detection unit determines whether or not the change in the amount of left lower vignetting is within a threshold value. The detection unit 221 calculates the area size Sld of the left lower vignetting region as the amount of left lower vignetting, and determines whether or not the calculated area size Sld is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Sld-s of the reference time. If the calculated change in the left lower vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST5, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST6.

[0116] In step ST5, the detection unit determines that the imaging unit 500 is normal. The detection unit 221 generates a determination result indicating that the focus deviation of the imaging unit 500 of the data acquisition unit 202 is within a normal range, and outputs the determination result to the motion controller 225. Thus, the automated driving controller 213 determines whether or not the driving support function or the automated driving control function of the vehicle 20 using the captured image acquired by the imaging unit 500 of the data acquisition unit 202 can be executed. Then, in a case where the driving support function and the automated driving function can be executed, the automated driving controller 213 executes the driving support function and the automated driving function.

[0117] In step ST6, the detection unit determines that the imaging unit 500 is abnormal. The detection unit 221 generates a determination result indicating that the focus deviation of the imaging unit 500 of the data acquisition unit 202 is outside the normal range and is abnormal, and outputs the determination result to the motion controller 225. Thus, the automated driving controller 213 stops the driving support function and the automated driving control function using the captured image acquired by the imaging unit 500 of the data acquisition unit 202.

[0118] In this way, the detection unit 221 can accurately detect the focusing abnormality (focus deviation) based on the change in the vignetting region in the captured image of one viewpoint.

[0119] Next, an operation in a case where the fail-safe detection unit is provided will be described. Figure 7 is a flowchart illustrating another operation of the detection unit 221 of the control system 200. Note that, Figure 7 illustrates an operation in a case where the fail-safe detection unit 2213 is provided, and this operation is also executed at, for example, the startup of the imaging unit 500 or before the automated driving of the vehicle 20 is started, similarly to Figure 6 illustrated in.

[0120] In step ST10, the detection unit determines whether or not the sharpness of the captured image is equal to or greater than a preset threshold value. The detection unit 221 calculates the sharpness of the captured image acquired by the imaging unit 500 of the data acquisition unit 202. If the calculated sharpness is equal to or greater than the preset sharpness threshold value, the detection unit 221 proceeds to step ST11, and if the calculated sharpness is less than the preset sharpness threshold value (if the image is blurred), the detection unit 221 proceeds to step ST16.

[0121] In step ST11, the detection unit determines whether or not the change in the amount of right upper vignetting is within a threshold value. The detection unit 221 calculates the area size Sru of the right upper vignetting region as the amount of right upper vignetting, and determines whether or not the calculated area size Sru is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Sru-s of the reference time. If the calculated change in the right upper vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST12, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST16.

[0122] In step ST12, the detection unit determines whether or not the change in the amount of left upper vignetting is within a threshold value. The detection unit 221 calculates the area size Slu of the left upper vignetting region as the amount of left upper vignetting, and determines whether or not the calculated area size Slu is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Slu-s of the reference time. If the calculated change in the left upper vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST13, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST16.

[0123] In step ST13, the detection unit determines whether or not the change in the amount of right lower vignetting is within a threshold value. The detection unit 221 calculates the area size Srd of the right lower vignetting region as the amount of right lower vignetting, and determines whether or not the calculated area size Srd is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Srd-s of the reference time. If the calculated change in the right lower vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST14, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST16.

[0124] In step ST14, the detection unit determines whether or not the change in the amount of left lower vignetting is within a threshold value. The detection unit 221 calculates the area size Sld of the left lower vignetting region as the amount of left lower vignetting, and determines whether or not the calculated area size Sld is a change within the vignetting determination threshold value (e.g., within a range of ±Ts%) with respect to the area size Sld-s of the reference time. If the calculated change in the left lower vignetting region is within the vignetting determination threshold value, the detection unit 221 proceeds to step ST15, and if the change exceeds the vignetting determination threshold value, it proceeds to step ST16.

[0125] In step ST15, the detection unit determines that the imaging unit 500 is normal. The detection unit 221 generates a determination result indicating that the focus deviation of the imaging unit 500 of the data acquisition unit 202 is within a normal range, and outputs the determination result to the motion controller 225. Thus, the automated driving controller 213 determines whether or not the driving support function or the automated driving control function of the vehicle 20 using the captured image acquired by the imaging unit 500 of the data acquisition unit 202 can be executed. Then, in a case where the driving support function and the automated driving function can be executed, the automated driving controller 213 executes the driving support function and the automated driving function.

[0126] In step ST16, the detection unit determines that the imaging unit 500 is abnormal. The detection unit 221 generates a determination result indicating that the focus deviation of the imaging unit 500 of the data acquisition unit 202 is outside the normal range and is abnormal, and outputs the determination result to the motion controller 225. Thus, the automated driving controller 213 stops the driving support function and the automated driving control function of the vehicle 20 using the captured image acquired by the imaging unit 500 of the data acquisition unit 202.

[0127] As described above, in a case where the sharpness of the captured image is determined to be equal to or greater than the preset sharpness threshold based on the image data of one viewpoint, the detection unit 221 can accurately detect the focus abnormality based on the change in the vignetting region in the captured image. Further, in a case where the sharpness of the captured image is lower than the preset sharpness threshold (in a case of image blur), the driving support function based on the captured image with low sharpness is stopped. Thus, the vehicle 20 can prevent the automated driving or the like from being executed in a case where the surrounding situation cannot be accurately grasped based on the captured image.

[0128] Figure 8 A case where the state of the imaging unit changes from a reference time is illustrated. Figure 8 The (a) part illustrates a case where the interval between the lens 501 and the image sensor 502 becomes narrower than at the reference time due to a change in time, an external force, or the like. In this case, as illustrated in the (a) part of FIG. 6, the image sensor 502 is closer to the lens 501 than the focus position of the lens 501. Figure 8 As illustrated in the (b) part of FIG. 6, the image sensor 502 is closer to the lens 501 than the focus position of the lens 501. As illustrated in the (b) part of FIG. 6, the image sensor 502 is closer to the lens 501 than the focus position of the lens 501. Figure 8As shown in section (c), the effective image circle IMC is smaller than that at the reference time, and the area size of the vignetting regions at the four corners is larger than that at the reference time. That is, the area size Sru-n of the upper right vignetting region is larger than the area size Sru-s. Similarly, the area size Slu-n of the upper left vignetting region is larger than the area size Slu-s, the area size Srd-n of the lower right vignetting region is larger than the area size Srd-s, and the area size Sld-n of the lower left vignetting region is larger than the area size Sld-s. Therefore, when the area size of the vignetting regions at the four corners becomes larger than the area size at the reference time and any change in the vignetting region exceeds the vignetting determination threshold, the detection unit 221 determines that the imaging unit 500 is abnormal.

[0129] Figure 9 The illustration shows another scenario where the imaging unit changes state from the reference time. Figure 9 Part (a) illustrates a case where the distance between lens 501 and image sensor 502 becomes wider than the reference time due to changes in time, external forces, etc. In this situation, as... Figure 9 As shown in section (b), the image sensor 502 is located further away from the lens 501 than the focusing position of the lens 501. Figure 9 As shown in section (c), the effective image circle IMC is larger than the reference time, and the area size of the vignetting regions at the four corners is smaller than that at the reference time. That is, the area size Sru-f of the upper right vignetting region is smaller than the area size Sru-s. Similarly, the area size Slu-f of the upper left vignetting region is smaller than the area size Slu-s, the area size Srd-f of the lower right vignetting region is smaller than the area size Srd-s, and the area size Sld-f of the lower left vignetting region is smaller than the area size Sld-s. Therefore, when the area size of the vignetting regions at the four corners becomes larger than the area size at the reference time, and the change in any of the vignetting regions exceeds the vignetting determination threshold, the detection unit 221 determines that the imaging unit 500 is abnormal.

[0130] Figure 10 The illustration shows the displacement of the optical axis of the lens within the imaging unit. Figure 10 Part (a) illustrates a situation where the optical axis of lens 501 is tilted relative to image sensor 502 due to changes in time, external forces, etc. In this case, as... Figure 10 As shown in section (b), the focusing position of the lens moves from the center of the image sensor 502, and as... Figure 10 As shown in section (c), the effective image circle IMC shifts from its position at the reference time, and the area size of the vignetting regions at the four corners changes. For example, in Figure 10In the case of part (c), the region size Sru-t of the upper right vignetting region is smaller than the region size Sru-s, and the region size Srd-t of the lower right vignetting region is smaller than the region size Srd-s. Furthermore, the region size Slu-t of the upper left vignetting region is larger than the region size Slu-s, and the region size Sld-t of the lower left vignetting region is smaller than the region size Sld-s. Therefore, when the region size of the vignetting region at any of the four corners becomes larger or smaller than the size at the reference time and the change in the vignetting region exceeds the vignetting determination threshold, the detection unit 221 determines that the imaging unit 500 is abnormal.

[0131] It should be noted that, although Figure 10 The illustration shows the optical axis tilt of lens 501, but when the image sensor is tilted, the area size of the vignetting region at the four corners is as follows: Figure 10 The change is shown in section (c). Therefore, the detection unit 221 can similarly detect focusing abnormalities caused by the tilt of the image sensor.

[0132] Moreover, the detection unit 221 can similarly detect focusing anomalies caused by deformation of the corners of the image sensor. Figure 11 The illustration shows the angular distortion of the image sensor. For example... Figure 11 As shown in section (a), the image sensor 502 is located at the focus position of the lens 501, and the lower right corner is distorted. In this case, as... Figure 11 As shown in section (b), the effective image circle IMC is substantially equal to the reference time, and the region size of the vignetting region corresponding to the deformed corner is different from the reference time. That is, the region size Sru-w of the upper right vignetting region is substantially equal to the region size Sru-s, the region size Slu-w of the upper left vignetting region is substantially equal to the region size Slu-s, and the region size Sld-w of the lower left vignetting region is substantially equal to the region size Sld-s. Furthermore, the region size Srd-w of the lower right vignetting region is greater than the region size Sld-s. Therefore, when the change in the lower right vignetting region exceeds the vignetting determination threshold, the detection unit 221 determines that the imaging unit 500 is abnormal.

[0133] According to the present technology as described above, the detection unit can detect focusing abnormalities caused by changes in the positional relationship between the lens and the image sensor due to changes in the fixing state between the lens holder 503 and the base 504, deformation of the image sensor, etc. (which are caused by time-varying changes or external forces depending on the usage environment) based on changes in the vignetting region. Furthermore, the detection unit can detect focusing abnormalities where the sharpness of the captured image acquired by the image sensor is reduced due to lens degradation, etc.

[0134] For example, the detection unit 221 can easily and accurately detect focusing abnormalities caused by lens degradation due to changes in the positional relationship between the lens and the image sensor, or due to accidental impacts applied to the imaging unit 500, the use of the imaging unit 500 in a temperature environment not covered by the warranty, the exposure of the imaging unit 500 to chemicals, or accidental malfunctions, without using spatial resolution measurement charts, a wide workspace for anomaly detection, or a conversion lens for reducing the workspace.

[0135] Furthermore, since the change in the vignetting region corresponds to the cause of the focus anomaly as described above, the detection unit 221 can determine the cause of the anomaly based on the change in the vignetting region. For example, as Figure 8 As shown, when the area size of the vignetting region at the four corners is large, the detection unit 221 determines that the narrowing of the distance between the lens and the image sensor is the cause of the anomaly. Furthermore, the detection unit 221 can send a notification about the determined cause from the output unit 206 to the user of the vehicle 20 or similar entities. In this way, if a notification about the cause of the anomaly is sent based on the change in the vignetting region, service centers or similar entities can easily perform the work of eliminating the abnormal state.

[0136] Incidentally, in dark conditions around the vehicle, even if the signal level of the image signal is adjusted by the signal processing unit 2211, the detection unit 221 has difficulty determining the boundaries of the vignetting region in the captured image. Therefore, the vehicle 20 may include a light source unit LT that emits illumination light, so that the imaging unit 500 can capture a bright image. Figure 12 The illustration shows the case without illumination and the case with illumination. In dark conditions surrounding the vehicle, the detection unit 221 has difficulty calculating the area size of the vignetting region because, as... Figure 12 As shown in section (a), the captured image is dark unless illumination is used. However, if the detection unit 221 illuminates the subject OB using the light source unit LT, as... Figure 12 As shown in part (b), the vignetting region outside the effective image circle IMC can be easily identified, such as... Figure 12 As shown in section (c). Therefore, even in darkness around the vehicle, the detection unit 221 can accurately detect focusing abnormalities.

[0137] It should be noted that, as in this technology, when a lens with an effective image circle smaller than the image sensor size is used to generate a vignetting region around the imaging surface of the image sensor, the detection unit may have difficulty calculating the region size of the vignetting region at the four corners when the vignetting region is continuous. In this case, the detection unit may pre-divide the imaging surface of the image sensor and calculate the region size of the vignetting region for each divided region. Figure 13The case where the imaging surface is divided in the vertical direction and the horizontal direction with the optical axis position as a reference is illustrated. Note that, in Figure 13 , the distinction positions are indicated by dotted lines. In this way, by dividing the imaging surface into four sections and calculating the area size of the vignetting region for each section, the detection unit can calculate the area size of the vignetting region at the upper right, the upper left, the lower right, and the lower left even when the vignetting region is continuous.

[0138] Further, for example, Figure 6 and 7 the operations shown in Figure 6 and 7 may be performed each time a predetermined time elapses from the start, or can be performed when the temperature change exceeds a predetermined value or the weather changes.

[0139] Further, in the present technology, by using a lens whose effective image circle is smaller than the image sensor size, a vignetting region is generated at the end portion of the imaging surface of the image sensor. Thus, in the case where various types of image processing such as subject recognition are performed, for example, using the image acquired by the imaging unit 500, the detection unit can cut out and use an image of a predetermined region excluding the vignetting region located at the center, for example, to perform various types of image processing without being affected by vignetting.

[0140] <1-3. Modified example>

[0141] Incidentally, although the above-described embodiment has explained a case where the focus abnormality is detected by the automatic driving controller 213 based on the image data acquired by the imaging unit 500 provided in the data acquisition unit 202 of the vehicle 20, the detection of the focus abnormality or the like can be performed by an imaging device that images the front of the vehicle, such as a front sensing camera (FSC).

[0142] Figure 14 The case where a function such as focus abnormality detection is provided in an imaging device that images the front of the vehicle is illustrated.

[0143] The imaging device 60f includes an optical system 61, an imaging unit 62, an image processing unit 63, a recognition processing unit 64, a calculation processing unit 65, an information generation unit 67, a drive control unit 68, an abnormality detection unit 69, and an output unit 70.

[0144] The optical system 61 includes the lens 501 shown in Figure 5 and forms an optical image indicating the front of the vehicle and the periphery of the vehicle on the imaging unit 62. Note that, as Figure 5As shown in the figure, the optical system 61 is attached to the imaging unit 62 using the lens holder 503 and the fixing member 505.

[0145] Imaging unit 62 includes, for example Figure 5 The image sensor 502 and the driving circuit for driving the image sensor 502, etc., are shown, and an image signal indicating an optical image is generated. The technology of this application is suitable for imaging devices using optical systems where the vertical and horizontal dimensions of the imaging area in the image sensor are, for example, 4.32 mm and 8.64 mm (1 / 1.7 type), the number of pixels is several megapixels or more (particularly 7 megapixels or more), and the allowable range of focus position deviation is within ± a few μm (e.g., ±3 μm). Furthermore, the technology of this application is suitable for imaging devices using image sensors with a higher pixel density than a 1 / 1.7 type image sensor with 7 megapixels (e.g., an area per pixel less than 6.1 (μm × μm)), and optical systems with an allowable range of focus position deviation within ± a few μm (e.g., ±3 μm). The imaging unit 62 outputs the generated image signal to the image processing unit 63 and the anomaly detection unit 69.

[0146] The image processing unit 63 performs processing to make it easier to identify objects in the captured image acquired by the imaging unit 62. For example, the image processing unit 63 performs image processing such as automatic exposure control, automatic white balance adjustment, and high dynamic range synthesis, and outputs the processed image to the recognition processing unit 64.

[0147] The recognition processing unit 64 uses the processed image output from the image processing unit 63 to perform object recognition processing and detects, for example, vehicles, pedestrians, obstacles, traffic lights, traffic signs, road lanes, and curbs of sidewalks. The recognition processing unit 64 outputs the recognition results to the computation processing unit 65.

[0148] The computational processing unit 65 calculates object information about the object detected by the recognition processing unit 64. The computational processing unit 65 calculates, for example, the object's shape, distance to the object, the object's direction of movement, and its speed as object information. Furthermore, the computational processing unit 65 uses multiple temporally consecutive captured images when calculating dynamic object information.

[0149] The information generation unit 67 generates driving control information that includes the driving content necessary for the vehicle. Based on the recognition results of the recognition processing unit 64 and the object information calculated by the calculation processing unit 65, the information generation unit 67 determines the driving content to be performed by the vehicle and generates driving control information that includes the driving content.

[0150] Examples of the content of driving of the vehicle include a change in speed (acceleration and deceleration), a change in the direction of travel, and the like. As specific examples, the information generation unit 67 judges that it is necessary to decelerate in a case where the vehicle distance to the preceding vehicle is small, and judges that it is necessary to change the direction of travel to the center of the lane in a case where the vehicle is likely to deviate from the lane.

[0151] The information generation unit 67 transmits the drive control information to the drive control unit 68. Note that the information generation unit 67 can generate information other than the drive control information. For example, the information generation unit 67 can detect the brightness of the surrounding environment from the processed image and generate illumination control information for turning on the headlamp of the vehicle when the surrounding environment is dark.

[0152] The drive control unit 68 generates a drive control signal on the basis of the drive control information and outputs the drive control signal from the output unit 70. For example, the drive control unit 68 can accelerate the vehicle by a drive force generation mechanism, decelerate the vehicle by a brake mechanism, and change the direction of travel of the vehicle by a steering mechanism.

[0153] The configuration of the abnormality detection unit 69 is similar to that of the abnormality detection unit 2210 shown in Figure 3 The abnormality detection unit 69 performs signal processing such as signal level adjustment, noise removal, and the like on the image signal generated by the imaging unit 62, whereby the amount of vignetting can be accurately calculated. Further, the abnormality detection unit 69 discriminates the vignetting region from the image signal and calculates the region size of the vignetting region as the amount of vignetting. The abnormality detection unit 69 judges whether the imaging unit 60f is normal on the basis of the calculated amount of vignetting, and judges that the imaging unit 60f is normal when the calculated amount of vignetting indicates that the change in the amount of vignetting with respect to the amount of vignetting at the reference time does not exceed the vignetting determination threshold value in the case of focus abnormality, and judges that the imaging unit 60f is abnormal when the calculated amount of vignetting exceeds the vignetting determination threshold value.

[0154] Further, the abnormality detection unit 69 detects the sharpness on the basis of the image signal generated by the imaging unit 62, and in a case where the sharpness is lower than a preset sharpness threshold value (in a case of image blur), performs warning or operation control so as not to perform a driving support function, an automatic driving control function, or the like on the basis of the captured image acquired by the imaging unit 62.

[0155] Moreover, the abnormality detection unit 69 judges whether the imaging unit 62 is normal using the amount of vignetting and the fail-safe detection result by using the result of comparison between the sharpness and the sharpness threshold value as the fail-safe detection result, and judges that the imaging unit 62 is abnormal when it indicates that the sharpness is lower than the preset sharpness threshold value. The abnormality detection unit 69 outputs the judgment result to the output unit 70.

[0156] The output unit 70 outputs the drive control signal generated by the drive control unit 68 and the determination result of the abnormality detection unit 69 to another functional block of the vehicle. Further, the output unit 70 can output the image signal generated by the imaging unit 62 or the image signal processed by the image processing unit 63 to another functional block.

[0157] Further, the detection of the focus abnormality or the like is not limited to the imaging device that images the front of the vehicle, and can be provided in the imaging device that images the rear or the side of the vehicle.

[0158] Figure 15 A case where a function such as focus abnormality detection is provided in the imaging device that images the rear or the side of the vehicle is illustrated.

[0159] The imaging device 60sb includes the optical system 61, the imaging unit 62, the image processing unit 63, the recognition processing unit 64, the second calculation processing unit 66, the abnormality detection unit 69, and the output unit 70. Further, the configurations of the optical system 61, the imaging unit 62, the image processing unit 63, the recognition processing unit 64, the abnormality detection unit 69, and the output unit 70 are similar to those of the above-described imaging device 60f.

[0160] The second calculation processing unit 66 calculates object information on the object detected by the recognition processing unit 64. The second calculation processing unit 66 calculates a distance to the object as the object information based on, for example, the positional relationship between the detected object and the vehicle, and outputs the distance through the output unit 70.

[0161] The output unit 70 outputs the object information calculated by the second calculation processing unit 66 and the determination result of the abnormality detection unit 69 to another functional block of the vehicle. Further, the output unit 70 can output the image signal generated by the imaging unit 62 or the image signal processed by the image processing unit 63 to another functional block.

[0162] Note that, in a case where the drive control of the vehicle is performed on the basis of the object included in the captured image of the rear or the side of the vehicle, the imaging device 60sb can include the information generation unit 67 and the drive control unit 68.

[0163] Further, although the log information is stored in the storage unit 211 of the vehicle 20 in the above-described embodiment, the log information can be stored in the imaging device (60f or 60sb).

[0164] As described above, even if the imaging device is provided separately from the vehicle 20, the present technology can perform an operation similar to that of the above-described embodiment and obtain a similar effect.

[0165] <1-4. Configuration and operation of server>

[0166] Next, the configuration and operation of the server will be described. Figure 16 The configuration of the server is illustrated. The server 30 includes a communication unit 301, a notification information generation unit 302, a user information database 305, and a notification unit 306.

[0167] The communication unit 301 communicates with the vehicle 20, acquires log information stored in the storage unit 211 of the vehicle 20, and outputs the log information to the notification information generation unit 302.

[0168] The notification information generation unit 302 generates notification information based on the result of the judgment of the vehicle performance from the log information acquired from the vehicle 20 or the like. The notification information generation unit 302 judges whether or not the predetermined performance is maintained based on the log information acquired from the vehicle 20 or the like. When the notification information generation unit 302 judges that the predetermined performance cannot be maintained due to the occurrence of an abnormality, the frequency of occurrence of an abnormality, or the like, the notification information generation unit 302 generates notification information for notifying a vehicle administrator or the like and outputs the notification information to the notification unit 306. Note that the administrator includes the owner of the vehicle and the manager, a management company, or the like in the case where the vehicle is shared.

[0169] The user information database 305 stores user information about the administrator or the like of each vehicle, for example, information indicating the name, address, contact address, or the like of the administrator or the like.

[0170] The notification unit 306 acquires information about the administrator or the like of the vehicle 20 whose performance is judged by the notification information generation unit 302 from the user information database 305, and notifies the notification information generated by the notification information generation unit 302 to the contact person indicated by the acquired user information. For example, the notification unit 306 performs processing for transmitting an email indicating the occurrence of a focus abnormality or the like to the administrator of the vehicle 20 or giving an audio notification. When the log information includes information indicating the contact address or the like of the administrator, the notification unit 306 can use the contact address indicated by the log information.

[0171] Figure 17 is a flowchart illustrating the operation of the server. In step ST21, the server acquires log information. The communication unit 301 of the server 30 communicates with the vehicle 20 to acquire information generated by the vehicle 20 and proceeds to step ST22. Note that in the acquisition of the log information, for example, the server 30 can request the vehicle 20 for log information every time a predetermined period elapses, or the server 30 can acquire log information provided from the vehicle 20 every time a predetermined period elapses or an abnormality occurs.

[0172] In step ST22, the server performs performance judgment processing. The notification information generating unit 302 of the server 30 judges the performance of the vehicle 20 based on the log information acquired from the vehicle 20 and proceeds to step ST23.

[0173] In step ST23, the server judges whether or not notification is necessary. The notification information generating unit 302 of the server 30 judges whether or not notification of the manager of the vehicle or the like is necessary based on the result of the performance judgment of step ST22. When the notification information generating unit 302 judges that the predetermined performance is maintained in the vehicle 20, the notification information generating unit 302 judges that notification is unnecessary and returns to step ST21. Alternatively, when the notification information generating unit 302 judges that the predetermined performance cannot be maintained in the vehicle 20, the notification information generating unit 302 judges that notification is necessary and proceeds to step ST24.

[0174] In step ST24, the server generates notification information. The notification information generating unit 302 of the server 30 generates notification information to give a notification that the predetermined performance cannot be maintained or to give a notification of an abnormal part or the like and proceeds to step ST25.

[0175] In step ST25, the server performs notification processing. The notification unit 306 of the server 30 performs processing of notifying the manager of the vehicle or the like that acquired the log information in step ST21 of the notification information generated in step ST24.

[0176] In this way, if the result of the performance judgment is notified to the manager of the vehicle 20 or the like, when an abnormality in focusing or the like occurs, not only the occupant of the vehicle 20 but also the manager of the vehicle 20 or the like can grasp the abnormality. Therefore, it is possible to reliably take measures to prevent occurrence of the abnormality. Further, by using the server 30 at the manufacturer of the vehicle 20, the service center or the like, it is easy to grasp the situation of the vehicle, and it is also possible to quickly provide appropriate service.

[0177] <1-5. Other configuration and operation of the server>

[0178] Next, another configuration and operation of the server will be described, for example, in the case where the server is used in an insurance company that subscribes an insurance contract for a vehicle. Figure 18 Another configuration of the server is illustrated. The server 30 includes a communication unit 301, an accident information database 303, a notification information generating unit 304, a user information database 305, and a notification unit 306.

[0179] The communication unit 301 communicates with the vehicle 20, acquires the log information stored in the storage unit 211 of the vehicle 20, and outputs the log information to the notification information generating unit 304.

[0180] For example, the accident information database 303 stores the kind of the accident vehicle, the abnormality occurrence state of the accident vehicle, the accident occurrence rate for each kind of the type of the accident vehicle and the occurred abnormality, the accident kind, the damage situation, and the like as the accident information.

[0181] The notification information generating unit 304 calculates the insurance premium of the vehicle 20 from which the log information is acquired, based on the log information acquired from the vehicle 20 and the accident information stored in the accident information database 303. The notification information generating unit 304 generates notification information indicating the calculated insurance premium, and outputs the notification information to the notification unit 306. For example, the notification information generating unit 304 calculates the insurance premium using at least one of the accident occurrence rate in the case where the focus abnormality occurs, the accident type, the damage situation, and the like, the duration of the state in which the focus abnormality occurs, the degree of abnormality, and the like. For example, in the case where the focus abnormality does not occur or in the case where the abnormality occurrence is promptly handled and the duration of the abnormality state is short, the notification information generating unit 304 reduces the insurance premium. Further, for example, in the case where the state in which the abnormality occurs is left for a long time, in the case where the accident occurrence rate is high when the focus abnormality occurs, in the case where there are many serious accidents, and the like, the notification information generating unit 304 can increase the insurance premium.

[0182] The user information database 305 stores user information about the contractor or the like who subscribes to the insurance contract for each vehicle, such as information indicating the name, the address, the contact address, and the like of the contractor.

[0183] The notification unit 306 acquires the information about the insurer of the vehicle 20 for which the insurance premium has been calculated by the notification information generating unit 304 from the user information database 305, and notifies the notification information to the contact person indicated by the acquired user information. For example, the notification unit 306 performs a process for transmitting an email indicating that the insurance premium is changed due to the occurrence of the focus abnormality or the like or giving an audio notification to the insurer.

[0184] Note that the calculation of the insurance premium and the notification of the insurance premium can be performed, for example, at the timing of a predetermined period before the expiration of the insurance contract. In the case where an event in which the insurance premium cannot be reduced based on the log information (or an event in which the insurance premium increases) occurs, the notification of the insurance premium can be promptly performed.

[0185] Figure 19 is a flowchart illustrating another operation of the server. In step ST31, the server acquires log information. The communication unit 301 of the server 30 communicates with the vehicle 20 to acquire the information generated by the vehicle 20, and proceeds to step ST32. Note that in the acquisition of the log information, for example, the server 30 can request the log information to the vehicle 20 every time a predetermined period elapses, or the server 30 can acquire the log information provided from the vehicle 20 every time a predetermined period elapses or when an abnormality occurs.

[0186] In step ST32, the server executes insurance fee calculation processing. The notification information generating unit 304 of the server 30 calculates the insurance fee based on the log information acquired from the vehicle 20 and the accident information stored in the database as described above, and proceeds to step ST33.

[0187] In step ST33, the server executes notification processing. The notification unit 306 of the server 30 executes processing of notifying the manager or the like of the vehicle that acquired the log information in step ST31 of the insurance fee calculated in step ST32.

[0188] In this way, if the insurance fee calculation result is notified to the contractor or the like who has concluded an insurance contract with the vehicle 20 that is a contract vehicle, the insurance fee is notified to the contractor or the like in the case where the focus abnormality occurs. Therefore, it is possible to take measures promptly against the occurrence of the abnormality, so that the insurance fee does not increase.

[0189] Incidentally, the server 30 can generate the automatic driving level control information used by the automatic driving controller 213, and the threshold value (the sharpness threshold value or the vignetting determination threshold value) used by the detection unit 221 of the automatic driving controller 213 or the abnormality detection unit 69 of the imaging device 60f (60sb) using the log information accumulated in the information database.

[0190] Figure 20 Another partial configuration of the server is illustrated. As described above, the communication unit 301 communicates with the vehicle 20 and acquires the log information (which can include the accident information) from the vehicle. Note that the communication unit 301 can acquire the accident information or the like from an external database (for example, a database of an insurance company or the like). Further, the communication unit 301 transmits the automatic driving level control information and the appropriate threshold value (the sharpness threshold value or the vignetting determination threshold value) to be described later to each vehicle.

[0191] The information database 311 stores information of each vehicle, such as the type of the vehicle, the initial registration year of the vehicle, the type of the in-vehicle camera (for each model), the in-vehicle camera usage years, the threshold value (the sharpness threshold value or the vignetting determination threshold value), the presence or absence of the accident, and the information of each accident vehicle (such as the occurrence state of the abnormality in the accident vehicle, the type of the accident vehicle for each already occurred abnormality, the accident occurrence rate, the accident type, the damage situation, and the like).

[0192] The safety determination unit 312 discriminates the safety of the threshold value based on the information of the information stored in the information database 311. For example, it is determined whether the threshold value (the sharpness threshold value or the vignetting determination threshold value) set by statistically processing the information on the vehicle that did not cause the accident, the information on the vehicle that caused the accident, and the accident information is appropriate in terms of safety.

[0193] When the security determination unit 312 determines that the security is insufficient, the security determination unit 312 generates instruction information for lowering the level of automatic driving allowed in the vehicle (including instruction information for disallowing the automatic driving function ( / ADAS function)) as the automatic driving level control information, and transmits the instruction information to the vehicle through the communication unit 301. Further, when the security determination unit 312 determines that the security is sufficient, the security determination unit 312 generates automatic driving permission information as the automatic driving level control information, and transmits the automatic driving permission information to the vehicle through the communication unit 301.

[0194] Note that the security determination unit 312 can calculate an appropriate threshold value for each of the vehicle type, the type of the on-vehicle camera, and the usage years of the on-vehicle camera, for example, by the statistical processing described above. Further, in a case where the calculated appropriate threshold value is different from the currently set threshold value, the security determination unit 312 transmits the appropriate threshold value to each vehicle through the communication unit 301.

[0195] Figure 21 is a flowchart illustrating another part of the operation of the server. In step ST41, the server acquires log information and accident information. The communication unit 301 of the server 30 communicates with the vehicle 20 to acquire the log information and the accident information generated in the vehicle 20, the information database 311 stores the acquired log information and the accident information, and the process proceeds to step ST42. Note that, in the acquisition of the log information, for example, the server 30 can request the log information from the vehicle 20 every time a predetermined period elapses, or the server 30 can acquire the log information provided from the vehicle 20 every time a predetermined period elapses or an abnormality occurs. Further, in the acquisition of the accident information, for example, the accident information provided from the vehicle 20 at the time of an accident occurrence is acquired by the server 30.

[0196] In step ST42, the server performs a security determination process. The security determination unit 312 of the server 30 discriminates the security of the threshold value on the basis of the log information and the accident information stored in the information database 311, generates the automatic driving level control information according to the discrimination result, and proceeds to step ST43. For example, when the security determination unit 312 determines that the set threshold value (the sharpness threshold value or the veiling glare determination threshold value) is insufficient in terms of security, the security determination unit 312 generates the automatic driving level control information indicating an instruction to lower the level of automatic driving allowed in the vehicle. Further, when the security determination unit 312 determines that the set threshold value is sufficient in terms of security, the security determination unit 312 generates the automatic driving level control information indicating the automatic driving permission information. Further, the security determination unit 312 calculates an appropriate threshold value by the statistical processing as described above, and proceeds to step ST43.

[0197] In step ST43, the server executes information transmission processing. The communication unit 301 of the server 30 executes processing of transmitting the automatic driving level control information generated in step ST42 to the vehicle 20. Further, in a case where the security determination unit 312 determines that the calculated appropriate threshold value is different from the currently set threshold value, the security determination unit 312 transmits the calculated appropriate threshold value to the vehicle or the imaging device through the communication unit 301.

[0198] As described above, since the server 30 transmits the automatic driving level control information generated based on the log information and the accident information acquired from the vehicle 20 and the calculated appropriate threshold value to the vehicle 20 and the imaging devices 60f, 60sb, the vehicle 20 can perform operation control more safely.

[0199] <2. Application Examples>

[0200] The technology according to the present disclosure can be applied to various fields using an imaging unit. For example, the technology according to the present disclosure can be implemented as a device mounted not only on a vehicle as described above but also on a movable body such as a two-wheeled vehicle, a bicycle, a personal mobility device, an airplane, a drone, a ship, and a robot. Further, the present disclosure can also be applied to a device used in a factory production process, a device used in the field of construction, the field of monitoring, the field of medical care, and the like.

[0201] The series of processes described in the specification can be executed by a configuration of hardware, software, or a combination of both. In a case where the processes are executed by software, a program in which a process sequence is recorded is installed in a memory of a computer incorporated in a dedicated hardware to be executed. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various types of processes.

[0202] For example, the program can be recorded in advance on a hard disk, a solid state drive (SSD), or a read only memory (ROM) as a recording medium. Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a compact disk read only memory (CD-ROM), a magneto optical (MO) disk, a digital versatile disk (DVD), a Blu-ray disk (BD) (registered trademark), a magnetic disk, or a semiconductor memory card. Such a removable recording medium can be provided as so-called package software.

[0203] In addition to installing the program from the removable recording medium to the computer, the program can be transmitted from a download site to the computer in a wireless manner or a wired manner through a network such as a local area network (LAN) or the Internet. The computer can receive the program transmitted in this manner and install it on a recording medium such as a built-in hard disk.

[0204] Note that the effects described in the specification are merely examples and the effects are not limited thereto. Additional effects can be obtained that are not described herein. Furthermore, the present technology should not be construed as being limited to the embodiments of the technology described above. Embodiments of the present technology disclose the present technology in a form of examples, and it is obvious to those skilled in the art that modifications and replacements can be made to the embodiments without departing from the gist of the present technology. In other words, the scope of the claims should be considered to determine the gist of the present technology.

[0205] Further, the abnormality detection apparatus of the present technology can also have the following configuration.

[0206] (1) An abnormality detection apparatus comprising:

[0207] a detection unit that uses an image captured by an image sensor using a lens having an effective image circle smaller than an imaging surface of the image sensor to detect a focus abnormality based on a change in a vignetting region of the captured image.

[0208] (2) The abnormality detection apparatus according to (1), wherein the detection unit determines that the focus abnormality is detected in a case where a change in the vignetting region at the time of the focus abnormality detection processing with respect to a vignetting region at a reference time exceeds a vignetting determination threshold.

[0209] (3) The abnormality detection apparatus according to (2), wherein the detection unit detects a change in a region size of the vignetting region for each of four corners of the captured image, and determines that the focus abnormality is detected in a case where the change exceeds the vignetting determination threshold at any one of the four corners.

[0210] (4) The abnormality detection apparatus according to (3), wherein the detection unit determines a cause of the focus abnormality based on the change in the vignetting region at the time of the detection of the focus abnormality.

[0211] (5) The abnormality detection apparatus according to any one of (1) to (4), wherein the detection unit detects a sharpness of the captured image, and determines that the focus abnormality is detected in a case where the detected sharpness is lower than a preset sharpness threshold.

[0212] (6) The abnormality detection apparatus according to any one of (1) to (5), further comprising

[0213] a light source unit that illuminates a subject captured by the image sensor.

[0214] (7) The abnormality detection apparatus according to any one of (1) to (6), further comprising an imaging unit that captures an image using a lens having an effective image circle smaller than an imaging surface of the image sensor.

[0215] (8) The abnormality detection device according to any one of (1) to (7), further comprising a motion controller that controls a motion of a vehicle on which the image sensor is mounted, wherein

[0216] When the detection unit detects the focus abnormality, the motion controller stops a drive support function of the vehicle.

[0217] (9) The abnormality detection device according to any one of (1) to (7), further comprising a motion controller that executes an automatic drive control function of a vehicle on which the image sensor is mounted, wherein

[0218] When the detection unit detects the focus abnormality, the motion controller stops the automatic drive control function and switches to manual drive.

[0219] (10) The abnormality detection device according to (9), wherein the detection unit executes the focus abnormality detection processing before the motion controller starts the automatic drive of the vehicle.

[0220] LIST OF REFERENCE NUMERALS

[0221] 10 information processing system

[0222] 20 vehicle

[0223] 30 server

[0224] 40 network

[0225] 60f, 60sb imaging device

[0226] 61 optical system

[0227] 62 imaging unit

[0228] 63 image processing unit

[0229] 64 recognition processing unit

[0230] 65 calculation processing unit

[0231] 66 second calculation processing unit

[0232] 67 information generation unit

[0233] 68 drive control unit

[0234] 69 abnormality detection unit

[0235] 70 output unit

[0236] 200 control system

[0237] 201 input unit

[0238] 202 data acquisition unit

[0239] 203 communication unit

[0240] 204 in-vehicle device

[0241] 211 storage unit

[0242] 212 communication network

[0243] 213 automated driving controller

[0244] 221 detection unit

[0245] 222 self-localization unit

[0246] 223 situation analysis unit

[0247] 224 planning unit

[0248] 225 motion controller

[0249] 301 communication unit

[0250] 302, 304 notification information generation unit

[0251] 303 accident information database

[0252] 305 user information database

[0253] 306 notification unit

[0254] 311 information database

[0255] 312 safety determination unit

[0256] 500 imaging unit

[0257] 501 lens

[0258] 502 image sensor

[0259] 502a imaging surface

[0260] 503 lens holder

[0261] 504 base

[0262] 505 fixing member

[0263] 2210 abnormality detection unit

[0264] 2211 signal processing unit

[0265] 2212 vignetting amount calculation unit

[0266] 2213 fail-safe detection unit

[0267] 2214 judging unit

Claims

1. An anomaly detection device, comprising: The detection unit uses an image captured by an image sensor using a lens with an effective image circle smaller than the imaging plane of the image sensor to detect focus anomalies based on changes in the vignetting region of the captured image. as well as A motion controller that performs autonomous driving control functions for vehicles equipped with image sensors, wherein When the detection unit detects a focusing abnormality, the motion controller stops the automatic driving control function and switches to manual driving.

2. The anomaly detection device according to claim 1, wherein... If the change in the vignetting region relative to the vignetting region at the reference time exceeds the vignetting determination threshold during the focus anomaly detection process, the detection unit determines that a focus anomaly has been detected.

3. The anomaly detection device according to claim 2, wherein... The detection unit detects changes in the size of the vignetting region at each of the four corners of the captured image, and determines that a focus anomaly is detected if the change exceeds the vignetting determination threshold at any of the four corners.

4. The anomaly detection device according to claim 3, wherein... The detection unit determines the cause of the focus anomaly based on the change in the vignetting area during focus anomaly detection.

5. The anomaly detection device according to claim 1, wherein... The detection unit detects the sharpness of the captured image, and determines that a focus abnormality has been detected if the detected sharpness is lower than a preset sharpness threshold.

6. The anomaly detection device according to claim 1 further includes a light source unit, which illuminates the subject captured by the image sensor.

7. The anomaly detection device according to claim 1 further includes an imaging unit, wherein the imaging unit uses a lens with an effective image circle smaller than the imaging surface of the image sensor to capture an image.

8. The anomaly detection device according to claim 1 further includes a motion controller for controlling the movement of a vehicle equipped with an image sensor, wherein... When the detection unit detects a focusing abnormality, the motion controller stops the vehicle's driving support functions.

9. The anomaly detection device according to claim 8, wherein... Before the motion controller begins autonomous driving of the vehicle, the detection unit performs focus anomaly detection processing.

10. An anomaly detection method, comprising: Using images captured by a lens with an effective image circle smaller than the image sensor's imaging plane, focus anomalies are detected based on changes in the vignetting region of the captured image; and To perform autonomous driving control functions for vehicles equipped with image sensors, among which When a focus anomaly is detected, the autopilot control function is stopped and manual driving is switched.

11. A computer program product comprising a program that causes a computer to perform focus anomaly detection, the program performing the following processes when executed by a processor: Images are captured by an image sensor using a lens whose effective image circle is smaller than the imaging plane of the image sensor; Detecting focus anomalies based on the vignetting region of the captured image; as well as To perform autonomous driving control functions for vehicles equipped with image sensors, among which When a focus anomaly is detected, the autopilot control function is stopped and manual driving is switched.

12. An information processing system, comprising: Anomaly detection equipment; as well as The server uses the detection results from an anomaly detection device, wherein... Anomaly detection equipment includes The detection unit uses an image captured by an image sensor using a lens with an effective image circle smaller than the imaging plane of the image sensor to detect focus anomalies based on changes in the vignetting region of the captured image. as well as A motion controller that performs autonomous driving control functions for vehicles equipped with image sensors, wherein When the detection unit detects a focus anomaly, the motion controller stops the automatic driving control function and switches to manual driving. The server includes The notification information generation unit uses the focus anomaly detection result of the detection unit to generate user notification information.

13. The information processing system according to claim 12, wherein... The notification information generation unit generates performance information to show administrators of devices equipped with image sensors the performance of the device as determined by the focus anomaly detection results, as user notification information.

14. The information processing system according to claim 12, wherein... The notification information generation unit generates information to show the administrator of a vehicle equipped with an image sensor the insurance premium calculated using the focus anomaly detection results, as a user notification.

15. The information processing system according to claim 12, wherein... The server includes A safety determination unit generates autonomous driving level control information and a threshold for the detection of focus anomalies by using log information including the focus anomaly detection results of the detection unit.

Images